A Review of Multimodal Perception and Closed-Loop Task Execution for Human-Robot Collaboration Robots Oriented Toward Intelligent Assembly

Main Article Content

Zhuoyuan Li
Faculty of Mechanical and Electrical Engineering, Kunming University of Science and Technology, Kunming 650500, China

DOI: https://doi.org/10.70267/aitia.2026102118

Keywords

human-robot collaboration robots, intelligent assembly, multimodal perception, task allocation, quality feedback, digital twin, embodied intelligence

Abstract

With the increasing demand for flexible manufacturing, especially for multi-variety and small-batch production, human-robot collaboration has become more widely used in intelligent assembly. Many studies have investigated multimodal perception, task alloca tion, safety control, and quality inspection, and considerable progress has been made in these areas. Even so, several problems are still difficult to address. Information exchange between different technical modules is often not well connected, task under standing is still limited, and feedback from the assembly process is not fully incorporated into decision making. This review discusses recent research from the perspective of a complete "perception-decision-execution-feedback" process. The main topics inc lude operator action and intention recognition, perception of part states, task allocation under human-related constraints, safe trajectory planning, and quality feedback. Rather than considering these techniques separately, attention is given to how they interact during collaborative assembly. Current studies suggest that the main challenge is no longer improving the accuracy of a single perception or control module. A more important issue is how to connect multi-source perception, task understanding, safety constraints, and quality feedback into a stable closed-loop framework. Future work is expected to pay more attention to cross-station adaptation with limited training samples, practical and verifiable use of vision - language-action models, digital twin-based safety validation, and skill updating based on assembly deviations. It is hoped that this review can provide useful ideas for future research and system development in intelligent human-robot collaborative assembly.

Abstract 6 | PDF Downloads 4

References

  • [1] Keshvarparast, A., Battini, D., Battaia, O., & Pirayesh, A. (2024). Collaborative robots in manufacturing and assembly systems: literature review and future research agenda. Journal of Intelligent Manufacturing, 35(5), 2065-2118.
  • [2] Lu, Y., Zheng, H., Chand, S., Xia, W., Liu, Z., Xu, X.,... & Bao, J. (2022). Outlook on human-centric manufacturing towards Industry 5.0. Journal of manufacturing systems, 62, 612-627.
  • [3] Zhang, C., Wang, Z., Zhou, G., Chang, F., Ma, D., Jing, Y.,... & Zhao, D. (2023). Towards new-generation human-centric smart manufacturing in Industry 5.0: A systematic review. Advanced Engineering Informatics, 57, 102121.
  • [4] Vianello, L., Ivaldi, S., Aubry, A., & Peternel, L. (2024). The effects of role transitions and adaptation in human–cobot collaboration. Journal of Intelligent Manufacturing, 35(5), 2005-2019.
  • [5] Wang, B., Zheng, L., Wang, Y., & Qi, Z. (2025). LLM-based multi-agent task planning for human-robot collaborative assembly balancing operator experience and efficiency. Journal of Manufacturing Systems, 82, 1020-1045.
  • [6] Suryoputro, M. R., Zhang, T., & Kiridena, S. (2025). Ergonomics and safety in human–collaborative robot interaction: A review of literature for manufacturing industries. International Journal of Industrial Ergonomics, 110, 103837.
  • [7] Liu, C., Qian, Y., Tang, D., Zhu, H., Pang, J., & Cai, Q. (2026). From insight to action: Embodied multi- agent system integrating vision language model for digital twin-assisted human-robot collaborative assembly. Journal of Manufacturing Systems, 85, 531-556.
  • [8] Liu, C., Song, J., Tang, D., Wang, L., Zhu, H., & Cai, Q. (2026). From insight to autonomous execution: VLM-enhanced embodied agents towards digital twin-assisted human-robot collaborative assembly. Robotics and Computer-Integrated Manufacturing, 98, 103176.
  • [9] Petzoldt, C., Harms, M., & Freitag, M. (2023). Review of task allocation for human-robot collaboration in assembly. International Journal of Computer Integrated Manufacturing, 36(11), 1675-1715.
  • [10] Kheirabadi, M., Keivanpour, S., Chinniah, Y. A., & Frayret, J. M. (2023). Human-robot collaboration in assembly line balancing problems: Review and research gaps. Computers & Industrial Engineering, 186, 109737.
  • [11] Fathi, M., Sepehri, A., Ghobakhloo, M., Iranmanesh, M., & Tseng, M. L. (2024). Balancing assembly lines with industrial and collaborative robots: Current trends and future research directions. Computers & industrial engineering, 193, 110254.
  • [12] Angleraud, A., Ekrekli, A., Samarawickrama, K., Sharma, G., & Pieters, R. (2024). Sensor-based human– robot collaboration for industrial tasks. Robotics and computer-integrated manufacturing, 86, 102663.
  • [13] Zhang, Y., Jiang, X., Qi, X., Cui, E., & Fu, H. (2026). Few-shot assembly action recognition in smart manufacturing: A cross-domain metric framework. Advanced Engineering Informatics, 74, 104610.
  • [14] Wang, T., Liu, Z., Wang, L., Li, M., & Wang, X. V. (2024). Data-efficient multimodal human action recognition for proactive human–robot collaborative assembly: A cross-domain few-shot learning approach. Robotics and Computer-Integrated Manufacturing, 89, 102785.
  • [15] Dong, L., Hu, T., Sun, T., Li, J., & Ma, S. (2025). RGB video and inertial sensing fusion method for human action recognition in human-robot collaborative manufacturing. Journal of Manufacturing Systems, 83, 216-234.
  • [16] Matin, A., Islam, M. R., Wang, X., & Huo, H. (2024). Robust multimodal approach for assembly action recognition. Procedia Computer Science, 246, 4916-4925.
  • [17] Gao, Q., Liu, Z., Hou, M., Sa, G., & Tan, J. (2026). Multimodal action recognition in human–robot collaborative assembly: A contrastive semantic query approach. Robotics and Computer-Integrated Manufacturing, 98, 103163.
  • [18] Ding, P., Zhang, J., Zhang, P., & Lyu, Y. (2025). Large Language Model-powered Operator Intention Recognition for Human-Robot Collaboration. IFAC-PapersOnLine, 59(10), 2280-2285.
  • [19] Fan, J., & Zheng, P. (2024). A vision-language-guided robotic action planning approach for ambiguity mitigation in human–robot collaborative manufacturing. Journal of Manufacturing Systems, 74, 1009 - 1018.
  • [20] He, J., Zhang, C., Wang, Y., Zheng, P., & Zhang, J. (2026). Intention recognition and task allocation in human–robot collaborative assembly based on adaptive networks and DT belief space-GA. Journal of Manufacturing Systems, 84, 20-39.
  • [21] Hui, J., Zhang, Y., Ding, K., Guo, L., Chen, C. H., & Wang, L. (2024). A multi-stage approach for desired part grasping under complex backgrounds in human-robot collaborative assembly. Advanced Engineering Informatics, 62, 102778.
  • [22] Zheng, H., Xia, W., & Xu, X. (2025). A human-robot collaborative assembly framework with quality checking based on real-time dual-hand action segmentation. Robotics and Computer-Integrated Manufacturing, 94, 102976.
  • [23] Geiser, A., Fuhrmann, A., Hailu, M., Hashem, S., Benfer, M., & Lanza, G. (2026). Automated Vision-Based Quality Control of Robotic Assembly in HMLV Manufacturing. Procedia CIRP, 139, 198-203.
  • [24] Alessio, A., Aliev, K., & Antonelli, D. (2024). Multicriteria task classification in human-robot collaborative assembly through fuzzy inference. Journal of Intelligent Manufacturing, 35(5), 1909-1927.
  • [25] Kiyokawa, T., Shirakura, N., Wang, Z., Yamanobe, N., Ramirez-Alpizar, I. G., Wan, W., & Harada, K. (2023). Difficulty and complexity definitions for assembly task allocation and assignment in human–robot collaborations: A review. Robotics and Computer-Integrated Manufacturing, 84, 102598.
  • [26] Weyel, J., Kalmbacher, B., Beinke, T., Schultheis, M., Mü ller, T., Milde, S.,... & Hartmann, F. (2025). Dynamic Human-Cobot Task Allocation: Productivity versus Balance in Manufacturing. IFAC-PapersOnLine, 59(26), 295-300.
  • [27] Zhu, Y., Chen, S., Zhang, C., Piao, Z., & Yang, G. (2024). Development of adaptive safety constraint by predicting trajectories of closest points between human and co-robot. Journal of Intelligent Manufacturing, 35(3), 1197-1206.
  • [28] Zhang, S., Huang, H., Huang, D., Yao, L., Wei, J., & Fan, Q. (2022). Subtask-learning based for robot self-assembly in flexible collaborative assembly in manufacturing. The International Journal of Advanced Manufacturing Technology, 120(9), 6807-6819.
  • [29] Gao, J., Wu, K., Li, W., Glogowski, P., & Kuhlenkö tter, B. (2026). Human-centered safe trajectory planning for human–robot collaborative assembly of electric vehicle battery copper busbar. CIRP Journal of Manufacturing Science and Technology, 68, 84-102.
  • [30] Ma, D., Zhang, C., Xu, Q., & Zhou, G. (2026). Large and small-scale models’ fusion-driven proactive robotic manipulation control for human-robot collaborative assembly in industry 5.0. Robotics and Computer-Integrated Manufacturing, 97, 103078.
  • [31] Ding, K., Mao, Q., Zhang, Y., Zhang, Y., Zheng, P., & Wang, L. (2026). Review and perspectives on multimodal perception, mutual cognition, and embodied execution for human–robot collaboration in Industry 5.0. Robotics and Computer-Integrated Manufacturing, 101, 103280.
  • [32] Mü ller, M., Ruppert, T., Jazdi, N., & Weyrich, M. (2024). Self-improving situation awareness for human– robot-collaboration using intelligent Digital Twin. Journal of Intelligent Manufacturing, 35(5), 2045-2063.

Article Sidebar

PDF
Published
Sep 13, 2026
Conference Proceedings Volume
Vol. 16 (2026): Proceedings of the 2026 International Conference on AI and Technological Innovation Applications (AITIA 2026)
Section
Articles
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

Li, Zhuoyuan. “A Review of Multimodal Perception and Closed-Loop Task Execution for Human-Robot Collaboration Robots Oriented Toward Intelligent Assembly”. Exploring Science Academic Conference Series, vol. 16, Sept. 2026, pp. 102-18, https://doi.org/10.70267/aitia.2026102118.