The Application of Real-time Emotion Recognition in Video Conferencing

Weikun Lin

doi:10.5281/zenodo.13926257

Authors

Weikun Lin Shandong University of Science and Technology

DOI:

https://doi.org/10.5281/zenodo.13926257

ARK:

https://n2t.net/ark:/40704/JCTAM.v1n4a10

Disciplines:

Computer Science

Subjects:

Deep Learning

References:

27

Keywords:

Real-time Emotion Recognition, Video Conferencing, User Experience, Technical Challenges, Deep Learning, Multi-modal Analysis, Privacy and Security, Edge Computing

Abstract

Video conferencing has become a crucial tool for global remote collaboration, but existing platforms have limitations in capturing and conveying participants' emotions. Real-time emotion recognition technology, by combining computer vision, deep learning, and temporal analysis, can automatically analyze and identify emotional changes in participants, addressing this gap. This paper first introduces the core technical processes of emotion recognition, including face detection, feature extraction, and emotion classification, with a focus on the technical details of using Convolutional Neural Networks (CNNs) for feature extraction and classification algorithms.[1] To enhance the system's temporal dynamics, the paper also presents methods for capturing emotional changes based on Long Short-Term Memory networks (LSTMs) or Temporal Convolutional Networks (TCNs). In particular, we use a Softmax classifier for probability estimation of emotions, coupled with temporal analysis methods for real-time engagement assessment in meetings. Furthermore, this paper discusses the system implementation under an edge computing and cloud collaboration framework to optimize real-time performance and computational efficiency while proposing security strategies focused on privacy protection, such as homomorphic encryption and federated learning. Through specific application examples, this paper demonstrates the significant role of real-time emotion recognition technology in video conferencing and its potential impact on future remote communication modes.

Author Biography

Weikun Lin, Shandong University of Science and Technology

Software Engineering, Shandong University of Science and Technology, Shandong, China.

References

Ba, D., & Zeng, L. (2021). A review of deep learning for emotion recognition from facial expressions. Journal of Visual Communication and Image Representation, 78, 103034. https://doi.org/10.1016/j.jvcir.2021.103034

Chen, Y., Liu, M., & Zhang, J. (2020). Real-time emotion recognition from facial expressions in video conferences. Proceedings of the International Conference on Multimedia and Expo (ICME), 1-6. https://doi.org/10.1109/ICME49207.2020.00128

D’Mello, S. K., & Kory, J. (2015). A review of affective computing: From emotion recognition to social signal processing. ACM Computing Surveys, 47(3), 1-36. https://doi.org/10.1145/2723158

Zhao, G., Song, C., & Wu, B. (2024). 3D Integrated Circuit (3D IC) Technology and Its Applications. Journal of Industrial Engineering and Applied Science, 2(4), 60–65.

Wu, B., Song, C., & Zhao, G. (2024). Applications of Heterogeneous Integration Technology in Chip Design. Journal of Industrial Engineering and Applied Science, 2(4), 66–72.

Song, C., Wu, B., & Zhao, G. (2024). Optimization of Semiconductor Chip Design Using Artificial Intelligence. Journal of Industrial Engineering and Applied Science, 2(4), 73–80.

Song, C., Wu, B., & Zhao, G. (2024). Applications of Novel Semiconductor Materials in Chip Design. Journal of Industrial Engineering and Applied Science, 2(4), 81–89.

Li, W. (2024). Transforming Logistics with Innovative Interaction Design and Digital UX Solutions. Journal of Computer Technology and Applied Mathematics, 1(3), 91-96.

Li, W. (2024). User-Centered Design for Diversity: Human-Computer Interaction (HCI) Approaches to Serve Vulnerable Communities. Journal of Computer Technology and Applied Mathematics, 1(3), 85-90.

Hossain, M. S., & Muhammad, G. (2020). Emotion recognition using deep learning approach: A review. IEEE Access, 8, 23906-23921. https://doi.org/10.1109/ACCESS.2020.2977602

Liu, Y., & Wang, Q. (2020). Facial expression recognition: A comprehensive review. Computer Vision and Image Understanding, 192, 102896. https://doi.org/10.1016/j.cviu.2019.102896

Zhao, Y., Zhang, Z., & Chen, L. (2019). A survey of emotion recognition methods: From the perspectives of images and videos. Journal of Visual Communication and Image Representation, 60, 53-68. https://doi.org/10.1016/j.jvcir.2019.07.016

Ko, B. C. (2018). A brief review of facial emotion recognition based on visual information. Sensors, 18(2), 401. https://doi.org/10.3390/s18020401

Li, S., Deng, W., & Du, J. (2019). Reliable crowdsourcing and deep locality-preserving learning for unconstrained facial expression recognition. IEEE Transactions on Image Processing, 28(1), 356-370. https://doi.org/10.1109/TIP.2018.2866741

Picard, R. W. (1997). Affective computing. MIT Press.

Li, W. (2024). The Impact of Apple’s Digital Design on Its Success: An Analysis of Interaction and Interface Design. Academic Journal of Sociology and Management, 2(4), 14–19.

Chen, Q., & Wang, L. (2024). Social Response and Management of Cybersecurity Incidents. Academic Journal of Sociology and Management, 2(4), 49–56.

Song, C. (2024). Optimizing Management Strategies for Enhanced Performance and Energy Efficiency in Modern Computing Systems. Academic Journal of Sociology and Management, 2(4), 57–64.

Chen, Q., Li, D., & Wang, L. (2024). Blockchain Technology for Enhancing Network Security. Journal of Industrial Engineering and Applied Science, 2(4), 22–28.

Chen, Q., Li, D., & Wang, L. (2024). The Role of Artificial Intelligence in Predicting and Preventing Cyber Attacks. Journal of Industrial Engineering and Applied Science, 2(4), 29–35.

Chen, Q., Li, D., & Wang, L. (2024). Network Security in the Internet of Things (IoT) Era. Journal of Industrial Engineering and Applied Science, 2(4), 36–41.

Li, D., Chen, Q., & Wang, L. (2024). Cloud Security: Challenges and Solutions. Journal of Industrial Engineering and Applied Science, 2(4), 42–47.

Li, D., Chen, Q., & Wang, L. (2024). Phishing Attacks: Detection and Prevention Techniques. Journal of Industrial Engineering and Applied Science, 2(4), 48–53.

Song, C., Zhao, G., & Wu, B. (2024). Applications of Low-Power Design in Semiconductor Chips. Journal of Industrial Engineering and Applied Science, 2(4), 54–59.

Tariq, U., Afzal, S., & Akhtar, U. (2022). Real-time emotion recognition in video conferencing using deep neural networks. Multimedia Tools and Applications, 81(15), 22213-22233. https://doi.org/10.1007/s11042-022-12716-8

Zhao, G., & Pietikäinen, M. (2007). Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(6), 915-928. https://doi.org/10.1109/TPAMI.2007.1110

Zhang, X., Yin, L., Cohn, J. F., Canavan, S., Reale, M., Horowitz, A., Liu, P., & Girard, J. M. (2014). BP4D-spontaneous: A high-resolution spontaneous 3D dynamic facial expression database. Image and Vision Computing, 32(10), 692-706. https://doi.org/10.1016/j.imavis.2014.06.002