3D computer graphics based interface to real microscopic worlds for μ-robot telemanipulation and position control

The goal of our project is to control the position in 3D-space of a microrobot (μ-robot) and manipulate microsystems aided by a real-time 3D computer graphics (virtual reality). The first part of our project was to develop a real time virtual environment man-machine interface to guide the robot and to evaluate measurement techniques to verify its position in the region of interest (workspace). In the article we focus in detail on the vision system and on the virtual reality interface of the complex system. Basically the user interacts with the virtual 3D microscope and sees the μ-robot as if he is looking through a real microscope. He is able to simulate the assembly of the missing parts beforehand in order to verify the assembly manipulation steps such as measuring, moving the table to the right position or performing the manipulation. Micro-manipulation in the form of a teleoperation is then performed by the robot-unit and the position is controlled by vision. First results have shown that a guided manipulation with a precision of 1 μm is feasible