3D Spatial Interaction with the Wii Remote for Head-Mounted Display Virtual Reality
This research investigates the design of a low-cost 3D
spatial interaction approach using the Wii Remote for immersive
Head-Mounted Display (HMD) virtual reality. Current virtual reality
applications that incorporate the Wii Remote are either desktop
virtual reality applications or systems that use large screen displays.
However, the requirements for an HMD virtual reality system differ
from such systems. This is mainly because in HMD virtual reality,
the display screen does not remain at a fixed location. The user views
the virtual environment through display screens that are in front of
the user-s eyes and when the user moves his/her head, these screens
move as well. This means that the display has to be updated in realtime
based on where the user is currently looking. Normal usage of
the Wii Remote requires the controller to be pointed in a certain
direction, typically towards the display. This is too restrictive for
HMD virtual reality systems that ideally require the user to be able to
turn around in the virtual environment. Previous work proposed a
design to achieve this, however it suffered from a number of
drawbacks. The aim of this study is to look into a suitable method of
using the Wii Remote for 3D interaction in a space around the user
for HMD virtual reality. This paper presents an overview of issues
that had to be considered, the system design as well as experimental
results.
[1] M. Katzourin, D. Ignatoff, L. Quirk, J.J. LaViola Jr. and O.C. Jenkins,
"Swordplay: Innovating game development through VR," IEEE
Computer Graphics and Applications, vol. 26, no. 6, 2006, pp. 15-18.
[2] D.A. Bowman, J. Chen, C.A. Wingrave, J. Lucas, A. Ray, N.F. Polys,
Q. Li, Y. Haciahmetoglu, J.S. Kim, S. Kim, S. Boehringer and T. Ni,
"New directions in 3D user interfaces," The International Journal of
Virtual Reality, vol. 5, no. 2, pp. 3-14, 2006.
[3] S. Seedharam, E.S. Zurita and B. Plimmer, "3D input for 3D worlds," in
Proc. of Australasian Computer-Human Interaction Conference
(OzCHI 2007), Adelaide, Australia, 2007, pp. 227-230.
[4] J.J. LaViola Jr., "Bringing VR and spatial 3D interaction to the masses
through video games," IEEE Computer Graphics and Applications, vol.
28, no. 5, 2008, pp. 10-15.
[5] J.C. Lee, "Hacking the Nintendo Wii Remote," Pervasive Computing,
vol. 7, no. 3, 2008, pp. 39-45.
[6] WiiLi.org, http://www.wiili.org/
[7] Wiibrew.org, http://wiibrew.org/
[8] J.C. Lee, http://www.cs.cmu.edu/~johnny/projects/wii/
[9] B. Peek, http://blogs.msdn.com/coding4fun/archive/2007/03/14/
1879033.aspx
[10] T. Schlömer, B. Poppinga, N. Henze and S. Boll, "Gesture recognition
with a Wii controller," in Proc. of the 2nd International Conference on
Tangible and Embedded Interaction (TEI- 08), Bonn, Germany, 2008,
pp. 11-14.
[11] S.J. Castellucci, I.S. MacKenzie, "UniGest: Text entry using three
degrees of motion," in Proc. of ACM CHI 2008, Florence, Italy, 2008,
pp. 3549-3554.
[12] M. Lapping-Carr, O.C. Jenkins, D.H. Grollman, J.N. Schowertfeger and
T.R. Hinkle, "Wiimote interfaces for lifelong robot learning," in Proc.
of AAAI Symposium on using AI to Motivate Greater Participation in
Computer Science, Palo Alto, C.A., 2008.
[13] A. Shirai, E. Geslin and S. Richir, "WiiMedia: Motion analysis methods
and applications using a consumer video game controller," in Proc. of
ACM SIGGRAPH Sandbox Symposium 2007, San Diego, C.A., 2007,
pp. 133-140.
[14] L.Gallo, G.D. Pietro and I. Marra, "3D interaction with volumetric
medical data: experiencing the Wiimote." in Proc. of the 1st
International Conference on Ambient Medai and Systems, Quebec,
Canada, 2008, article no. 14.
[15] M. Tamai, W. Wu, K. Nahrstedt, K. Yasumoto, "A view control
interface for 3D tele-immersive environments," in Proc. of IEEE
International Conference on Multimedia and Expo, Hannover,
Germany, 2008, pp. 1101-1104.
[16] B. Bruegge, C. Teschner, P. Lachenmaier, E. Fenzl, D. Schmidt and S.
Bierbaum, "Pinocchio: Conducting a virtual symphony orchestra," in
Proc. of the International Conference of Advances in Computer
Entertainment Technology (ACE), Salzburg, Austria, 2007, pp. 294-
295.
[17] H.J. Lee, H. Kim, G. Gupta and A. Mazalek, "WiiArts: Creating
collaborative art experience with WiiRemote Interaction," in Proc. of
the 2nd International Conference on Tangible and Embedded
Interaction (TEI-08), Bonn, Germany, 2008, pp. 33-36.
[18] S.Hay, J. Newman and R. Harle, "Optical tracking using commodity
hardware," in IEEE International Symposium on Mixed and Augmented
Reality 2008, Cambridge, UK, 2008, pp. 159-160.
[19] L. Deligiannidis and J. Larkin, "Navigating inexpensively and
wirelessly," in Proc. of Human System Interaction (HSI-08), Krakow,
Poland, 2008, pp. 165-169.
[20] S. Attygalle, M. Duff, T. Rikakis and J. He, "Low-cost, at-home
assessment system with Wii Remote based motion capture," in Proc. of
Virtual Rehabilitation 2008, Vancouver, Canada, 2008, pp. 168-174.
[21] T. Schou and H.J. Gardner, "A Wii Remote, a game engine, five sensor
bars and a virtual reality theatre," in Proc. of Australasian Computer-
Human Interaction Conference (OzCHI 2007), Adelaide, Australia,
2007, pp. 231-234.
[22] P. Bourke, http://local.wasp.uwa.edu.au/~pbourke/projection/Wii/
[23] Nintendo, http://www.nintendo.com/wii/what/controllers
[24] H. Godbersen, "Virtual environments for anyone," IEEE Multimedia,
vol. 15, no. 3, 2008, pp. 90-95.
[25] Y.W. Chow, "The Wii Remote as an input device for 3D interaction in
immersive head-mounted display virtual reality," in Proc. IADIS
International Conference Gaming 2008: Design for Engaging
Experience and Social Interaction, Amsterdam, The Netherlands, 2008,
pp. 85-92.
[26] J.J. LaViola, "A discussion of cybersickness in virtual environment,"
ACM SIGCHI Bulletin, vol. 32, no. 1, pp. 47-56.
[27] D. A. Bowman, E. Kruijff, J.J. LaViola Jr. and I. Poupyrev, 3D User
Interfaces: Theory and Practice, Addison Wesley, 2004.
[28] G. Welch and E. Foxlin, "Motion tracking: No silver bullet, but a
respectable arsenal," IEEE Computer Graphics and Applications, vol.
22, no. 6, 2002, pp. 24-38.
[29] Nintendo, http://e3.nintendo.com/pressrelease/
[30] C.A. Wingrave, D.A. Bowman and N. Ramakrishnan, "Towards
preference in virtual environment interfaces," in Proc. of the
Eurographics Workshop on Virtual Environments, Barcelona, Spain,
2002, pp. 63-72.
[31] G. Welch, G. Bishop, L. Vicci, S. Brumback, K. Keller and D. Colucci,
"High-performance wide-area optical tracking: the HiBall tracking
system," Presence: Teleoperators and Virtual Environments, vol. 10,
no. 1, pp. 1-21.
[32] B. Rasco, http://www.gamasutra.com/view/feature/1494/
wheres_the_wiimote_using_kalman_.php
[1] M. Katzourin, D. Ignatoff, L. Quirk, J.J. LaViola Jr. and O.C. Jenkins,
"Swordplay: Innovating game development through VR," IEEE
Computer Graphics and Applications, vol. 26, no. 6, 2006, pp. 15-18.
[2] D.A. Bowman, J. Chen, C.A. Wingrave, J. Lucas, A. Ray, N.F. Polys,
Q. Li, Y. Haciahmetoglu, J.S. Kim, S. Kim, S. Boehringer and T. Ni,
"New directions in 3D user interfaces," The International Journal of
Virtual Reality, vol. 5, no. 2, pp. 3-14, 2006.
[3] S. Seedharam, E.S. Zurita and B. Plimmer, "3D input for 3D worlds," in
Proc. of Australasian Computer-Human Interaction Conference
(OzCHI 2007), Adelaide, Australia, 2007, pp. 227-230.
[4] J.J. LaViola Jr., "Bringing VR and spatial 3D interaction to the masses
through video games," IEEE Computer Graphics and Applications, vol.
28, no. 5, 2008, pp. 10-15.
[5] J.C. Lee, "Hacking the Nintendo Wii Remote," Pervasive Computing,
vol. 7, no. 3, 2008, pp. 39-45.
[6] WiiLi.org, http://www.wiili.org/
[7] Wiibrew.org, http://wiibrew.org/
[8] J.C. Lee, http://www.cs.cmu.edu/~johnny/projects/wii/
[9] B. Peek, http://blogs.msdn.com/coding4fun/archive/2007/03/14/
1879033.aspx
[10] T. Schlömer, B. Poppinga, N. Henze and S. Boll, "Gesture recognition
with a Wii controller," in Proc. of the 2nd International Conference on
Tangible and Embedded Interaction (TEI- 08), Bonn, Germany, 2008,
pp. 11-14.
[11] S.J. Castellucci, I.S. MacKenzie, "UniGest: Text entry using three
degrees of motion," in Proc. of ACM CHI 2008, Florence, Italy, 2008,
pp. 3549-3554.
[12] M. Lapping-Carr, O.C. Jenkins, D.H. Grollman, J.N. Schowertfeger and
T.R. Hinkle, "Wiimote interfaces for lifelong robot learning," in Proc.
of AAAI Symposium on using AI to Motivate Greater Participation in
Computer Science, Palo Alto, C.A., 2008.
[13] A. Shirai, E. Geslin and S. Richir, "WiiMedia: Motion analysis methods
and applications using a consumer video game controller," in Proc. of
ACM SIGGRAPH Sandbox Symposium 2007, San Diego, C.A., 2007,
pp. 133-140.
[14] L.Gallo, G.D. Pietro and I. Marra, "3D interaction with volumetric
medical data: experiencing the Wiimote." in Proc. of the 1st
International Conference on Ambient Medai and Systems, Quebec,
Canada, 2008, article no. 14.
[15] M. Tamai, W. Wu, K. Nahrstedt, K. Yasumoto, "A view control
interface for 3D tele-immersive environments," in Proc. of IEEE
International Conference on Multimedia and Expo, Hannover,
Germany, 2008, pp. 1101-1104.
[16] B. Bruegge, C. Teschner, P. Lachenmaier, E. Fenzl, D. Schmidt and S.
Bierbaum, "Pinocchio: Conducting a virtual symphony orchestra," in
Proc. of the International Conference of Advances in Computer
Entertainment Technology (ACE), Salzburg, Austria, 2007, pp. 294-
295.
[17] H.J. Lee, H. Kim, G. Gupta and A. Mazalek, "WiiArts: Creating
collaborative art experience with WiiRemote Interaction," in Proc. of
the 2nd International Conference on Tangible and Embedded
Interaction (TEI-08), Bonn, Germany, 2008, pp. 33-36.
[18] S.Hay, J. Newman and R. Harle, "Optical tracking using commodity
hardware," in IEEE International Symposium on Mixed and Augmented
Reality 2008, Cambridge, UK, 2008, pp. 159-160.
[19] L. Deligiannidis and J. Larkin, "Navigating inexpensively and
wirelessly," in Proc. of Human System Interaction (HSI-08), Krakow,
Poland, 2008, pp. 165-169.
[20] S. Attygalle, M. Duff, T. Rikakis and J. He, "Low-cost, at-home
assessment system with Wii Remote based motion capture," in Proc. of
Virtual Rehabilitation 2008, Vancouver, Canada, 2008, pp. 168-174.
[21] T. Schou and H.J. Gardner, "A Wii Remote, a game engine, five sensor
bars and a virtual reality theatre," in Proc. of Australasian Computer-
Human Interaction Conference (OzCHI 2007), Adelaide, Australia,
2007, pp. 231-234.
[22] P. Bourke, http://local.wasp.uwa.edu.au/~pbourke/projection/Wii/
[23] Nintendo, http://www.nintendo.com/wii/what/controllers
[24] H. Godbersen, "Virtual environments for anyone," IEEE Multimedia,
vol. 15, no. 3, 2008, pp. 90-95.
[25] Y.W. Chow, "The Wii Remote as an input device for 3D interaction in
immersive head-mounted display virtual reality," in Proc. IADIS
International Conference Gaming 2008: Design for Engaging
Experience and Social Interaction, Amsterdam, The Netherlands, 2008,
pp. 85-92.
[26] J.J. LaViola, "A discussion of cybersickness in virtual environment,"
ACM SIGCHI Bulletin, vol. 32, no. 1, pp. 47-56.
[27] D. A. Bowman, E. Kruijff, J.J. LaViola Jr. and I. Poupyrev, 3D User
Interfaces: Theory and Practice, Addison Wesley, 2004.
[28] G. Welch and E. Foxlin, "Motion tracking: No silver bullet, but a
respectable arsenal," IEEE Computer Graphics and Applications, vol.
22, no. 6, 2002, pp. 24-38.
[29] Nintendo, http://e3.nintendo.com/pressrelease/
[30] C.A. Wingrave, D.A. Bowman and N. Ramakrishnan, "Towards
preference in virtual environment interfaces," in Proc. of the
Eurographics Workshop on Virtual Environments, Barcelona, Spain,
2002, pp. 63-72.
[31] G. Welch, G. Bishop, L. Vicci, S. Brumback, K. Keller and D. Colucci,
"High-performance wide-area optical tracking: the HiBall tracking
system," Presence: Teleoperators and Virtual Environments, vol. 10,
no. 1, pp. 1-21.
[32] B. Rasco, http://www.gamasutra.com/view/feature/1494/
wheres_the_wiimote_using_kalman_.php
@article{"International Journal of Information, Control and Computer Sciences:50469", author = "Yang-Wai Chow", title = "3D Spatial Interaction with the Wii Remote for Head-Mounted Display Virtual Reality", abstract = "This research investigates the design of a low-cost 3D
spatial interaction approach using the Wii Remote for immersive
Head-Mounted Display (HMD) virtual reality. Current virtual reality
applications that incorporate the Wii Remote are either desktop
virtual reality applications or systems that use large screen displays.
However, the requirements for an HMD virtual reality system differ
from such systems. This is mainly because in HMD virtual reality,
the display screen does not remain at a fixed location. The user views
the virtual environment through display screens that are in front of
the user-s eyes and when the user moves his/her head, these screens
move as well. This means that the display has to be updated in realtime
based on where the user is currently looking. Normal usage of
the Wii Remote requires the controller to be pointed in a certain
direction, typically towards the display. This is too restrictive for
HMD virtual reality systems that ideally require the user to be able to
turn around in the virtual environment. Previous work proposed a
design to achieve this, however it suffered from a number of
drawbacks. The aim of this study is to look into a suitable method of
using the Wii Remote for 3D interaction in a space around the user
for HMD virtual reality. This paper presents an overview of issues
that had to be considered, the system design as well as experimental
results.", keywords = "3D interaction, head-mounted display, virtual reality, Wii remote", volume = "3", number = "2", pages = "280-7", }
