Assessing human-robot performances in complex situations with unit task tests

This paper presents a test battery method to investigate (relative) operational upsides and downsides of robot innovations. This method was applied to test the teleoperated Generaal robot and the NIFTi robot that could be operated in manual and semi-automatic mode for navigation and situation sensing. The unit tasks of our test battery proved to be discriminatory to assess differences between the robots. Most often the performance with the Generaal was faster than the semi-autonomous NIFTi robot, whereas the semi-autonomous NIFTi robot had sometimes less collisions. To lower workload for operators there is a clear need for a semi-autonomous robot, but the current version needs improvement on the planning and path-finding to better perform on speed.