A Universal Approach for the Intuitive Control of Mobile Robots using an AR/VR-based Interface
Mobile robots are used in a large field of scenarios,
like exploring contaminated areas, repairing oil rigs under water,
finding survivors in collapsed buildings, etc. Currently, there is no
unified intuitive user interface (UI) to control such complex mobile
robots. As a consequence, some scenarios are done without the
exploitation of experience and intuition of human teleoperators.
A novel framework has been developed to embed a flexible and
modular UI into a complete 3-D virtual reality simulation system.
This new approach wants to access maximum benefits of human
operators. Sensor information received from the robot is prepared for
an intuitive visualization. Virtual reality metaphors support the
operator in his decisions. These metaphors are integrated into a real
time stereo video stream. This approach is not restricted to any
specific type of mobile robot and allows for the operation of different
robot types with a consistent concept and user interface.
[1] E. Freund and J. Rossmann, "Projective Virtual Reality: Bridging the
gap between Virtual Reality and Robotics," in IEEE Transactions on
Robotics and Automation, Vol. 15, No. 3, pages 411-422, June 1999.
[2] J. Rossmann, P. Krahwinler, and C. Schlette, "Navigation of mobile
robots in natural environments: Using sensor fusion in forestry," in The
2nd International Multi-Conference on Engineering and Technological
Innovation (IMETI), Orlando, USA, Vol. 3, pages 5-9, July 2009.
[3] J. Rossmann, M. Schluse, and C. Schlette, "The virtual forest: Robotics
and simulation technology as the basis for new approaches to the
biological and the technical production in the forest," in The 13th World
Multi-Conference on Systemics,Cybernetics and Informatics (WMSCI ),
Orlando, USA, Vol. 2, pages 33-38, July 2009.
[4] S. Jeschke, T. Aach, L. Eckstein, J. Rossmann, and B. Rumpe,
"RescueCars - Autonomous Service Vehicles in Rescue Scenarios,"
RWTH Aachen University, Germany, Tech. Rep., Feb. 2010.
[5] J. Casper and R. R. Murphy, "Human-Robot Interactions During the
Robot-Assisted Urban Search and Rescue Response at the World Trade
Center," in IEEE Transactions on Systems, Man, and Cybernetics, Vol.
33, No. 3, pages 367-385, June 2003.
[6] A. Davids, "Urban Search and Rescue Robots: from Tragedy to
Technology," in IEEE Intelligent Systems, Vol. 17, Issue 2, pages 81-83,
Mar. 2002.
[7] http://www.robocuprescue.org/, visited May 2010.
[8] M. Montemerlo, S. Thrun, D. Kooler, and B. Wegbreit, "FastSLAM: A
Factored Solution to the Simultaneous Localization and Mapping
Problem," in Proceedings of the AAAI National Conference on Artificial
Intelligence, Edmonton, Canada, 2002.
[9] J. Suthakorn, S. S. H. Shah, S. Jantarajit, W. Onprasert, et al., "On the
Design and Development of A Rough Terrain Robot for Rescue
Missions," in Proceedings of the IEEE International Conference on
Robotics and Biomimetics (ROBIO), Bangkok, Thailand, pages 1830-
1835, Feb. 2009.
[10] M. Wang and J. N. K. Lui, "A Novel Teleoperation Paradigm for
Human-Robot Interaction," in Proceedings of the IEEE Conference on
Robotics, Automation and Mechatronics (RAM), Singapore, pages 13-
18, Dec. 2004.
[11] H. A. Yanco and J. Drury, "-Where am I?- Acquiring situation
awareness using a remote robot platform," in IEEE International
Conference on Systems, Man and Cybernetics (SMC), Vol. 3, pages
2835-2840, Oct. 2004.
[12] M. Backer, R. Casey, B. Keyes, and H. A. Yanco, "Improved Interfaces
for Human-Robot Interaction in Urban Search and Rescue," in IEEE
International Conference on Systems, Man and Cybernetics (SMC), Vol.
3, pages 2960-2965, Oct. 2004.
[13] J. L. Durry, J. Scholz, and H. A. Yanco, "Applying CSCW and HCI
techniques to human-robot interaction," in Proceedings CHI 2004
Workshop on Shaping Human-Robot Interaction, Vienna, Austria, pages
13-16, Apr. 2004.
[14] Y. D. Kim, Y. G. Kim, S. H. Lee, J. H. Kang, et al., "Portable Fire
Evacuation Guide Robot System," in The IEEE/RSJ International
Conference on Intelligent Robots and Systems (IROS), St. Louis, USA,
pages 2789-2794, Oct. 2009.
[15] R. Murai, K. Ito, and F. Matsuno, "An intuitive Human-Robot Interface
for Rescue Operation of a 3D Snake Robot - The Intuitive Human-Robot
Interface Aims for Non-Professional Operators" in Proceedings of the
12th IASTED International Conference on Robotics and Applications
(RA), Honolulu, USA, pages 138-143, Aug. 2006.
[16] D. Tamura, A. Fujino, and K. Ito, "A Rescue Robot designed for Ease of
Use: Development of Exploration System using Behavior of Bombycid,"
in Artificial Life and Robotics (AROB), Vol. 13, No. 1, pages 251-254,
Dec. 2008.
[17] J. Borenstein, M. Hansen, and H. Nguyen, "The OmniTread OT-4
Serpentine Robot for Emergencies and Hazardous Environments,"
International Joint Topical Meeting: Sharing Solutions for Emergencies
and Hazardous Environments, Salt Lake City, Utah, USA, Feb. 12-15,
2006.
[18] J. Baker and J. Borenstein, "The Joysnake - A Haptic Operator Console
for High-Degree-of-Freedom Robots," International Joint Topical
Meeting: Sharing Solutions for Emergencies and Hazardous
Environments, Salt Lake City, Utah, USA, Feb. 12-15, 2006.
[19] J. Rossmann, H. Ruf, and C. Schlette, "Model-based programming 'by
Demonstration' - Fast setup of robot systems (ProDemo)," in T. Kröger
and F. M. Wahl (Eds.): "Advances in Robotics Research - Theory,
Implementation, Application", German Workshop on Robotics, TU
Braunschweig, Germany, pages 159-168, June 2009.
[20] Festo Robotino Manual, http://www.festo-didactic.com/ov3/
media/customers/1100/544305_robotino_deen.pdf, visited May 2010.
[1] E. Freund and J. Rossmann, "Projective Virtual Reality: Bridging the
gap between Virtual Reality and Robotics," in IEEE Transactions on
Robotics and Automation, Vol. 15, No. 3, pages 411-422, June 1999.
[2] J. Rossmann, P. Krahwinler, and C. Schlette, "Navigation of mobile
robots in natural environments: Using sensor fusion in forestry," in The
2nd International Multi-Conference on Engineering and Technological
Innovation (IMETI), Orlando, USA, Vol. 3, pages 5-9, July 2009.
[3] J. Rossmann, M. Schluse, and C. Schlette, "The virtual forest: Robotics
and simulation technology as the basis for new approaches to the
biological and the technical production in the forest," in The 13th World
Multi-Conference on Systemics,Cybernetics and Informatics (WMSCI ),
Orlando, USA, Vol. 2, pages 33-38, July 2009.
[4] S. Jeschke, T. Aach, L. Eckstein, J. Rossmann, and B. Rumpe,
"RescueCars - Autonomous Service Vehicles in Rescue Scenarios,"
RWTH Aachen University, Germany, Tech. Rep., Feb. 2010.
[5] J. Casper and R. R. Murphy, "Human-Robot Interactions During the
Robot-Assisted Urban Search and Rescue Response at the World Trade
Center," in IEEE Transactions on Systems, Man, and Cybernetics, Vol.
33, No. 3, pages 367-385, June 2003.
[6] A. Davids, "Urban Search and Rescue Robots: from Tragedy to
Technology," in IEEE Intelligent Systems, Vol. 17, Issue 2, pages 81-83,
Mar. 2002.
[7] http://www.robocuprescue.org/, visited May 2010.
[8] M. Montemerlo, S. Thrun, D. Kooler, and B. Wegbreit, "FastSLAM: A
Factored Solution to the Simultaneous Localization and Mapping
Problem," in Proceedings of the AAAI National Conference on Artificial
Intelligence, Edmonton, Canada, 2002.
[9] J. Suthakorn, S. S. H. Shah, S. Jantarajit, W. Onprasert, et al., "On the
Design and Development of A Rough Terrain Robot for Rescue
Missions," in Proceedings of the IEEE International Conference on
Robotics and Biomimetics (ROBIO), Bangkok, Thailand, pages 1830-
1835, Feb. 2009.
[10] M. Wang and J. N. K. Lui, "A Novel Teleoperation Paradigm for
Human-Robot Interaction," in Proceedings of the IEEE Conference on
Robotics, Automation and Mechatronics (RAM), Singapore, pages 13-
18, Dec. 2004.
[11] H. A. Yanco and J. Drury, "-Where am I?- Acquiring situation
awareness using a remote robot platform," in IEEE International
Conference on Systems, Man and Cybernetics (SMC), Vol. 3, pages
2835-2840, Oct. 2004.
[12] M. Backer, R. Casey, B. Keyes, and H. A. Yanco, "Improved Interfaces
for Human-Robot Interaction in Urban Search and Rescue," in IEEE
International Conference on Systems, Man and Cybernetics (SMC), Vol.
3, pages 2960-2965, Oct. 2004.
[13] J. L. Durry, J. Scholz, and H. A. Yanco, "Applying CSCW and HCI
techniques to human-robot interaction," in Proceedings CHI 2004
Workshop on Shaping Human-Robot Interaction, Vienna, Austria, pages
13-16, Apr. 2004.
[14] Y. D. Kim, Y. G. Kim, S. H. Lee, J. H. Kang, et al., "Portable Fire
Evacuation Guide Robot System," in The IEEE/RSJ International
Conference on Intelligent Robots and Systems (IROS), St. Louis, USA,
pages 2789-2794, Oct. 2009.
[15] R. Murai, K. Ito, and F. Matsuno, "An intuitive Human-Robot Interface
for Rescue Operation of a 3D Snake Robot - The Intuitive Human-Robot
Interface Aims for Non-Professional Operators" in Proceedings of the
12th IASTED International Conference on Robotics and Applications
(RA), Honolulu, USA, pages 138-143, Aug. 2006.
[16] D. Tamura, A. Fujino, and K. Ito, "A Rescue Robot designed for Ease of
Use: Development of Exploration System using Behavior of Bombycid,"
in Artificial Life and Robotics (AROB), Vol. 13, No. 1, pages 251-254,
Dec. 2008.
[17] J. Borenstein, M. Hansen, and H. Nguyen, "The OmniTread OT-4
Serpentine Robot for Emergencies and Hazardous Environments,"
International Joint Topical Meeting: Sharing Solutions for Emergencies
and Hazardous Environments, Salt Lake City, Utah, USA, Feb. 12-15,
2006.
[18] J. Baker and J. Borenstein, "The Joysnake - A Haptic Operator Console
for High-Degree-of-Freedom Robots," International Joint Topical
Meeting: Sharing Solutions for Emergencies and Hazardous
Environments, Salt Lake City, Utah, USA, Feb. 12-15, 2006.
[19] J. Rossmann, H. Ruf, and C. Schlette, "Model-based programming 'by
Demonstration' - Fast setup of robot systems (ProDemo)," in T. Kröger
and F. M. Wahl (Eds.): "Advances in Robotics Research - Theory,
Implementation, Application", German Workshop on Robotics, TU
Braunschweig, Germany, pages 159-168, June 2009.
[20] Festo Robotino Manual, http://www.festo-didactic.com/ov3/
media/customers/1100/544305_robotino_deen.pdf, visited May 2010.
@article{"International Journal of Electrical, Electronic and Communication Sciences:62310", author = "Juergen Rossmann and Andre Kupetz and Roland Wischnewski", title = "A Universal Approach for the Intuitive Control of Mobile Robots using an AR/VR-based Interface", abstract = "Mobile robots are used in a large field of scenarios,
like exploring contaminated areas, repairing oil rigs under water,
finding survivors in collapsed buildings, etc. Currently, there is no
unified intuitive user interface (UI) to control such complex mobile
robots. As a consequence, some scenarios are done without the
exploitation of experience and intuition of human teleoperators.
A novel framework has been developed to embed a flexible and
modular UI into a complete 3-D virtual reality simulation system.
This new approach wants to access maximum benefits of human
operators. Sensor information received from the robot is prepared for
an intuitive visualization. Virtual reality metaphors support the
operator in his decisions. These metaphors are integrated into a real
time stereo video stream. This approach is not restricted to any
specific type of mobile robot and allows for the operation of different
robot types with a consistent concept and user interface.", keywords = "3-D simulation system, augmented reality,teleoperation of mobile robots, user interface.", volume = "4", number = "8", pages = "1263-8", }
