A Comprehensive Framework for Multimodal Sensor Fusion in Intelligent Manufacturing: Innovations, Interpretability, and Real-world Applications
DOI:
https://doi.org/10.5281/zenodo.13905495ARK:
https://n2t.net/ark:/40704/JCTAM.v1n4a05Disciplines:
Computer ScienceSubjects:
Deep LearningReferences:
19Keywords:
Multimodal Sensor Fusion, Smart Manufacturing, Explainable AI (XAI), Industry 4.0, Predictive Maintenance, Fault Detection, Attention Mechanisms, Deep Learning, Hybrid Fusion, Random Forest Classifier, Data Integration, Machine Learning, Feature Extraction, Real-Time Processing, Condition MonitoringAbstract
This paper presents the novel work of developing an intelligent manufacturing framework based on multimodal sensor integration and computer vision. In this paper, we propose a hybrid fusion method that includes both early and late fusion with attention mechanisms to select the most important sensor data. Our system gathers visual, thermal, acoustic, and vibration data and offers accurate and interpretable predictions for fault identification, process enhancement, and product quality. (Liu et al. 2024) We meet the challenge of the opacity of AI systems by using explainable AI methods to help the user comprehend the results of the model. It shows that the proposed system is accurate, efficient, scalable, and can be applied to various types of data. Examples from the industry present real-life experiences and issues that may be encountered when implementing our system in different manufacturing contexts. It presents a new paradigm shift in smart manufacturing systems through the enhancement of efficiency, reliability, and interpretability for future research and industrial development.
References
Liu, Z., Xiao, G., Liu, H., & Wei, H. (2024). Multi-Sensor Measurement and Data Fusion. IEEE Transactions on Instrumentation and Measurement, 73, 234-245. https://ieeexplore.ieee.org/abstract/document/9693406?casa_token=40VH55MYDtEAAAAA:5M5RuueqkCua6ONFCVjNeNl8V0TlZ4-giZ8VQzK0U6CXh_TefMxhQaCBotzrZXsBrmaDEjnq
Diez-Olivan, A., Del Ser, J., Galar, D., & Sierra, B. (2019). Data Fusion and Machine Learning for Industrial Prognosis: Trends and Perspectives Towards Industry 4.0. Journal of Manufacturing Processes, 40, 123-134. https://www.sciencedirect.com/science/article/pii/S1566253518304706?casa_token=Bi4TM627bvIAAAAA:FIhyFRBhR4K3o7_Ly5thLLoHChk7UYVPCZTvMDEK5aPxTYUJQuAPOI_Dn3ZHRIRz1TUbIaQ_
Helwig, Nikolai, Pignanelli, Eliseo, and Schtze, Andreas. (2018). Condition monitoring of hydraulic systems. UCI Machine Learning Repository. https://doi.org/10.24432/C5CW21.
Vergara, Alexander. (2013). Gas Sensor Array Drift at Different Concentrations. UCI Machine Learning Repository. https://doi.org/10.24432/C5MK6M.
Buscema, M., Terzi, S., & Tastle, W. (2010). Steel Plates Faults. UCI Machine Learning Repository. https://doi.org/10.24432/C5J88N
Zhang, Z., Farnsworth, M., Song, B., Tiwari, D., & Tiwari, A. (2024). Deep Transfer Learning With Self-Attention for Industry Sensor Fusion Tasks. IEEE Transactions on Industrial Informatics. https://ieeexplore.ieee.org/abstract/document/9813440
Sensor Data Fusion in Industry 4.0. (2024). Farnell. https://si.farnell.com/sensor-data-fusion-in-industry-40-trc-ar
Goyal, D., & Pabla, B. S. (2015). Condition-based maintenance of machine tools—A review. CIRP Journal of Manufacturing Science and Technology, 10, 24-35. https://doi.org/10.1016/j.cirpj.2015.05.002
Kang, H. S., Lee, J. Y., Choi, S., Kim, H., Park, J. H., Son, J. Y., ... & Do Noh, S. (2016). Smart manufacturing: Past research, present findings, and future directions. International Journal of Precision Engineering and Manufacturing-Green Technology, 3(1), 111-128. https://doi.org/10.1007/s40684-016-0015-5
Kusiak, A. (2019). Fundamentals of smart manufacturing: A multi-thread perspective. Annual Reviews in Control, 47, 214-220. https://doi.org/10.1016/j.arcontrol.2019.02.001
Liu, Y., Li, S., Liu, C., & Wang, H. (2021). Explainable AI for intelligent manufacturing: A survey. AI Open, 2(1), 105-123. https://doi.org/10.1002/aiopen.76
Monostori, L., Kádár, B., Bauernhansl, T., Kondoh, S., Kumara, S., Reinhart, G., ... & Ueda, K. (2016). Cyber-physical systems in manufacturing. CIRP Annals, 65(2), 621-641. https://doi.org/10.1016/j.cirp.2016.06.005
Tao, F., Cheng, J., Qi, Q., Zhang, M., Zhang, H., & Sui, F. (2018). Digital twin-driven product design, manufacturing, and service with big data. The International Journal of Advanced Manufacturing Technology, 94(9-12), 3563-3576. https://doi.org/10.1007/s00170-017-0233-1
Wuest, T., Weimer, D., Irgens, C., & Thoben, K. D. (2016). Machine learning in manufacturing: advantages, challenges, and applications. Production & Manufacturing Research, 4(1), 23-45. https://doi.org/10.1080/21693277.2016.1192517
Yin, S., Ding, S. X., Xie, X., & Luo, H. (2014). A review on basic data-driven approaches for industrial process monitoring. IEEE Transactions on Industrial Electronics, 61(11), 6418-6428. https://doi.org/10.1109/TIE.2014.2301773
Zheng, P., Wang, H., Sang, Z., Zhong, R. Y., Liu, Y., Liu, C., ... & Mubarok, K. (2018). Smart manufacturing systems for Industry 4.0: Conceptual framework, scenarios, and future perspectives. Frontiers of Mechanical Engineering, 13(2), 137-150. https://doi.org/10.1007/s11465-018-0499-5
Zhong, R. Y., Xu, X., Klotz, E., & Newman, S. T. (2017). Intelligent manufacturing in the context of industry 4.0: a review. Engineering, 3(5), 616-630. https://doi.org/10.1016/J.ENG.2017.05.015
Shao, Y., Yuan, X., Zhang, C., Song, Y., & Xu, Q. (2020). A Novel Fault Diagnosis Algorithm for Rolling Bearings Based on One-Dimensional Convolutional Neural Network and INPSO-SVM. Applied Sciences, 10(12), 4303. https://doi.org/10.3390/app10124303
Qiu, S., Cui, X., Ping, Z., Shan, N., Li, Z., Bao, X., & Xu, X. (2023). Deep Learning Techniques in Intelligent Fault Diagnosis and Prognosis for Industrial Systems: A Review. Sensors, 23(3), 1305. https://doi.org/10.3390/s23031305
Downloads
Published
How to Cite
Issue
Section
ARK
License
Copyright (c) 2024 The author retains copyright and grants the journal the right of first publication.
This work is licensed under a Creative Commons Attribution 4.0 International License.