VR&AR Combined Manual Operation Instruction System on Industry Products: A Case Study

Currently, manual operations done by human forces are widely performed and repeated on various industrial products during manufacturing process. In recent years, AR technology has been researched as an intuitive and efficient way to assist manual operations including assembly, disassembly and maintenance. This paper presented a VR&AR combined manual operation instruction system (VAMOIS) to assist technicians take manual operation instructions under mixed reality scene, both work site and virtual instruction contents are displayed synchronously and in real-time. The system developed virtual hands tutor in VR based instruction planning subsystem to guide the trainee move parts and use tools in correct pose. Dynamic position registration and natural occlusion method are proposed to improve the visual effect of the AR instruction system. Experimental results show that our system improves manual operation efficiency with better user experience compared with traditional training methods.