An AR/VR Based Approach Towards the Intuitive Control of Mobile Rescue Robots
An intuitive user interface for the teleoperation of mobile rescue robots is one key feature for a successful exploration of inaccessible and no-go areas. Therefore, we have developed a novel framework to embed a flexible and modular user interface into a complete 3-D virtual reality simulation system. Our approach is based on a client-server architecture to allow for a collaborative control of the rescue robot together with multiple clients on demand. Further, it is important that the user interface is not restricted to any specific type of mobile robot. Therefore, our flexible approach allows for the operation of different robot types with a consistent concept and user interface. In laboratory tests, we have evaluated the validity and effectiveness of our approach with the help of two different robot platforms and several input devices. As a result, an untrained person can intuitively teleoperate both robots without needing a familiarization time when changing the operating robot.
[1] E. Freund and J. Rossmann, "Projective Virtual Reality: Bridging the
gap between Virtual Reality and Robotics," in IEEE Transactions on
Robotics and Automation, Vol. 15, No. 3, pages 411-422, June 1999
[2] J. Roßmann, P. Krahwinler, and C. Schlette, "Navigation of mobile
robots in natural environments: Using sensor fusion in forestry," in
The 2nd International Multi-Conference on Engineering and
Technological Innovation (IMETI), Orlando, USA, Vol. 3, pages 5-9,
July 2009.
[3] J. Roßmann, M. Schluse, and C. Schlette, "The virtual forest: Robotics
and simulation technology as the basis for new approaches to the
biological and the technical production in the forest," in The 13th
World Multi-Conference on Systemics, Cybernetics and Informatics:
WMSCI 2009, Orlando, USA, Vol. 2, pp. 33-38, July 2009.
[4] S. Jeschke, T. Aach, L. Eckstein, J. Rossmann, and B. Rumpe,
"RescueCars - Autonomous Service Vehicles in Rescue Scenarios,"
RWTH Aachen University, Germany, Tech. Rep., Feb. 2010.
[5] J. Casper and R.R. Murphy, "Human-Robot Interactions During the
Robot-Assisted Urban Search and Rescue Response at the World
Trade Center," in IEEE Transactions on Systems, Man, and
Cybernetics, Vol. 33, No. 3, pages 367-385, Jun. 2003.
[6] A. Davids, "Urban Search and Rescue Robots: from Tragedy to
Technology," in IEEE Intelligent Systems, Vol. 17, Issue 2, pages 81-
83, Mar. 2002.
[7] http://www.robocuprescue.org/, visited Jun. 2010.
[8] M. Montemerlo, S. Thrun, D. Kooler, and B. Wegbreit, "FastSLAM:
A Factored Solution to the Simultaneous Localization and Mapping
Problem," in Proceedings of the AAAI National Conference on
Artificial Intelligence, Edmonton, Canada, 2002.
[9] J. Suthakorn, S.S.H. Shah, S. Jantarajit, W. Onprasert, et al., "On the
Design and Development of A Rough Terrain Robot for Rescue
Missions," in Proceedings of the IEEE International Conference on
Robotics and Biomimetics, Bangkok, Thailand, pages 1830-1835, Feb.
2009.
[10] M. Wang and J.N.K. Lui, "A Novel Teleoperation Paradigm for
Human-Robot Interaction," in Proceedings of the 2004 IEEE
Conference on Robotics, Automation and Mechatronics, Singapore,
pages 13-18, Dec. 2004.
[11] H.A. Yanco and J. Drury, "-Where am I?- Acquiring situation
awareness using a remote robot platform," in IEEE International
Conference on Systems, Man and Cybernetics, Vol. 3, pages 2835-
2840, Oct. 2004
[12] M. Backer, R. Casey, B. Keyes, and H.A. Yanco, "Improved
Interfaces for Human-Robot Interaction in Urban Search and Rescue,"
in IEEE International Conference on Systems, Man and Cybernetics,
Vol. 3, pages 2960-2965, Oct. 2004.
[13] J.L. Durry, J. Scholz, and H.A. Yanco, "Applying CSCW and HCI
techniques to human-robot interaction," in Proceedings CHI 2004
Workshop on Shaping Human-Robot interaction, Vieanna, USA,
pages 13-16, Apr. 2004.
[14] Y.D. Kim, Y.G. Kim, S.H Lee, J.H. Kang, et al., "Portable Fire
Evacuation Guide Robot System," in The 2009 IEEE/RSJ
International Conference on Intelligent Robots and Systems, St. Louis,
USA, pages 2789-2794, Oct. 2009.
[15] http://www.bluetooth.com/Bluetooth/Technology/, visited Jun. 2010.
[16] R. Muraui, K. Ito, and F. Matsuno, "An intuitive Human-Robot
Interface for Rescue Operation of a 3D Snake Robot the Intuitive
Human-Robot Interface Aims for Non-Professional Operators" in
Proceedings of the 12th IASTED International Conference Robotics
and Applications, Honolulu, USA, pages 138-143, Aug. 2006.
[17] D. Tamura, A. Fujino, and K. Ito, "A Rescue Robot designed for ease
of Use: Development of Exploration System using Behavior of
Bombycid," in Artificial life and Robotics, Vol. 13, No. 1, pages 251-
254, Dec. 2008.
[18] G. E. Krasner and S. T. Pope, "A cookbook for using the model-view
controller user interface paradigm in Smalltalk-80," in Journal of
Object-Oriented Programming, Vol. 1, Issue 3, pages 26-49,
Sept. 1988.
[19] http://www.festo-didactic.com/int-en/learning-systems/education-andresearch-
robots-robotino/ visited Jun. 2010.
[1] E. Freund and J. Rossmann, "Projective Virtual Reality: Bridging the
gap between Virtual Reality and Robotics," in IEEE Transactions on
Robotics and Automation, Vol. 15, No. 3, pages 411-422, June 1999
[2] J. Roßmann, P. Krahwinler, and C. Schlette, "Navigation of mobile
robots in natural environments: Using sensor fusion in forestry," in
The 2nd International Multi-Conference on Engineering and
Technological Innovation (IMETI), Orlando, USA, Vol. 3, pages 5-9,
July 2009.
[3] J. Roßmann, M. Schluse, and C. Schlette, "The virtual forest: Robotics
and simulation technology as the basis for new approaches to the
biological and the technical production in the forest," in The 13th
World Multi-Conference on Systemics, Cybernetics and Informatics:
WMSCI 2009, Orlando, USA, Vol. 2, pp. 33-38, July 2009.
[4] S. Jeschke, T. Aach, L. Eckstein, J. Rossmann, and B. Rumpe,
"RescueCars - Autonomous Service Vehicles in Rescue Scenarios,"
RWTH Aachen University, Germany, Tech. Rep., Feb. 2010.
[5] J. Casper and R.R. Murphy, "Human-Robot Interactions During the
Robot-Assisted Urban Search and Rescue Response at the World
Trade Center," in IEEE Transactions on Systems, Man, and
Cybernetics, Vol. 33, No. 3, pages 367-385, Jun. 2003.
[6] A. Davids, "Urban Search and Rescue Robots: from Tragedy to
Technology," in IEEE Intelligent Systems, Vol. 17, Issue 2, pages 81-
83, Mar. 2002.
[7] http://www.robocuprescue.org/, visited Jun. 2010.
[8] M. Montemerlo, S. Thrun, D. Kooler, and B. Wegbreit, "FastSLAM:
A Factored Solution to the Simultaneous Localization and Mapping
Problem," in Proceedings of the AAAI National Conference on
Artificial Intelligence, Edmonton, Canada, 2002.
[9] J. Suthakorn, S.S.H. Shah, S. Jantarajit, W. Onprasert, et al., "On the
Design and Development of A Rough Terrain Robot for Rescue
Missions," in Proceedings of the IEEE International Conference on
Robotics and Biomimetics, Bangkok, Thailand, pages 1830-1835, Feb.
2009.
[10] M. Wang and J.N.K. Lui, "A Novel Teleoperation Paradigm for
Human-Robot Interaction," in Proceedings of the 2004 IEEE
Conference on Robotics, Automation and Mechatronics, Singapore,
pages 13-18, Dec. 2004.
[11] H.A. Yanco and J. Drury, "-Where am I?- Acquiring situation
awareness using a remote robot platform," in IEEE International
Conference on Systems, Man and Cybernetics, Vol. 3, pages 2835-
2840, Oct. 2004
[12] M. Backer, R. Casey, B. Keyes, and H.A. Yanco, "Improved
Interfaces for Human-Robot Interaction in Urban Search and Rescue,"
in IEEE International Conference on Systems, Man and Cybernetics,
Vol. 3, pages 2960-2965, Oct. 2004.
[13] J.L. Durry, J. Scholz, and H.A. Yanco, "Applying CSCW and HCI
techniques to human-robot interaction," in Proceedings CHI 2004
Workshop on Shaping Human-Robot interaction, Vieanna, USA,
pages 13-16, Apr. 2004.
[14] Y.D. Kim, Y.G. Kim, S.H Lee, J.H. Kang, et al., "Portable Fire
Evacuation Guide Robot System," in The 2009 IEEE/RSJ
International Conference on Intelligent Robots and Systems, St. Louis,
USA, pages 2789-2794, Oct. 2009.
[15] http://www.bluetooth.com/Bluetooth/Technology/, visited Jun. 2010.
[16] R. Muraui, K. Ito, and F. Matsuno, "An intuitive Human-Robot
Interface for Rescue Operation of a 3D Snake Robot the Intuitive
Human-Robot Interface Aims for Non-Professional Operators" in
Proceedings of the 12th IASTED International Conference Robotics
and Applications, Honolulu, USA, pages 138-143, Aug. 2006.
[17] D. Tamura, A. Fujino, and K. Ito, "A Rescue Robot designed for ease
of Use: Development of Exploration System using Behavior of
Bombycid," in Artificial life and Robotics, Vol. 13, No. 1, pages 251-
254, Dec. 2008.
[18] G. E. Krasner and S. T. Pope, "A cookbook for using the model-view
controller user interface paradigm in Smalltalk-80," in Journal of
Object-Oriented Programming, Vol. 1, Issue 3, pages 26-49,
Sept. 1988.
[19] http://www.festo-didactic.com/int-en/learning-systems/education-andresearch-
robots-robotino/ visited Jun. 2010.
@article{"International Journal of Information, Control and Computer Sciences:60580", author = "Jürgen Roßmann and André Kupetz and Roland Wischnewski", title = "An AR/VR Based Approach Towards the Intuitive Control of Mobile Rescue Robots", abstract = "An intuitive user interface for the teleoperation of mobile rescue robots is one key feature for a successful exploration of inaccessible and no-go areas. Therefore, we have developed a novel framework to embed a flexible and modular user interface into a complete 3-D virtual reality simulation system. Our approach is based on a client-server architecture to allow for a collaborative control of the rescue robot together with multiple clients on demand. Further, it is important that the user interface is not restricted to any specific type of mobile robot. Therefore, our flexible approach allows for the operation of different robot types with a consistent concept and user interface. In laboratory tests, we have evaluated the validity and effectiveness of our approach with the help of two different robot platforms and several input devices. As a result, an untrained person can intuitively teleoperate both robots without needing a familiarization time when changing the operating robot.
", keywords = "Teleoperation of mobile robots, augmented reality, user interface, virtual reality.", volume = "4", number = "11", pages = "1748-8", }
