The purpose of this study is to design a portable virtual
piano. By utilizing optical fiber gloves and the virtual piano software
designed by this study, the user can play the piano anywhere at any
time. This virtual piano consists of three major parts: finger tapping
identification, hand movement and positioning identification, and
MIDI software sound effect simulation. To play the virtual piano, the
user wears optical fiber gloves and simulates piano key tapping
motions. The finger bending information detected by the optical fiber
gloves can tell when piano key tapping motions are made. Images
captured by a video camera are analyzed, hand locations and moving
directions are positioned, and the corresponding scales are found. The
system integrates finger tapping identification with information about
hand placement in relation to corresponding piano key positions, and
generates MIDI piano sound effects based on this data. This
experiment shows that the proposed method achieves an accuracy rate
of 95% for determining when a piano key is tapped.
[1] D. J. Sturman and D. Zeltzer, "A Survey of Glove- based Input", IEEE
Computer Graphics and Applications, vol.14, no.1 pp. 30-39, 1994.
[2] G. C. Sun, Self-balancing Control and Manipulation of a Glove Puppet
Robot on a Two-wheel Mobile Platform, Master's thesis, Institute of
Electrical Control Engineering, National Chiao Tung University, 2009.
[3] J. S. Hu, J. J. Wang, and G. Q. Sun, "The Glove Puppet Robot: X-puppet,"
IEEE/RSJ 2008 International Conference on Intelligent Robots and
Systems (IROS -08), Nice, France, 2008.
[4] M. C. Su, C. H. Hung, C. K. Yang, H. T. Yang, Z. J. Huang, Y. X. Zhao,
and D. Y. Huang, 2005, "Design of Optical Fiber Glove and Its
Application," in 2005 13th National Conference on Fuzzy Theory and
Applications, Kaohsing, Taiwan.
[5] J. M. Hollerbach, "Anthropomorphic Robot and Human Interactions", in
Proc. of 1st. Intl. Symposium on Humanoid Robots, 1996.
[6] S. S. Fels and G. E. Hinton, "Glove-Talk: a neural network interface
between a Data-Glove and a speech synthesizer", IEEE Trans. On Neural
Networks, Vol. 1, pp. 2-8, 1993.
[7] M. C. Su, Y. X. Zhao, C. H. Hung, and M. Chunag, "A Fuzzy Rule-Based
Approach to Hand Shape Recognition Systems," in The Joint Conference
on AI, Fuzzy System, and Grey System, Taiwan, Dec. 2003.
[8] Y. X. Zhao, A Portable Speaking-Aid System for Taiwanese Sign
Language, Master's thesis, Department of Electrical Engineering,
Tamkang University, 2002.
[9] C. H. Hung, A Trajectory-based Approach to Gesture Recognition,
Master's thesis, Department of Computer Science and Information,
Engineering, National Central University, 2006.
[10] M. C. Su, Y. X. Zhao, H. Huang, and H. F. Chen, "A Fuzzy Rule-Based
Approach to Recognizing 3D Arm Movements," IEEE Trans. on Neural
Systems and Rehabilitation Engineering, vol.9, no. 2, pp. 191-201, June
2001.
[11] M. C. Su, C. H. Hung, and Y. X. Zhao, "A SOMART System for Gesture
Recognition," WSEAS Transactions on Computers, vol. 5, pp. 2764-2771,
2006.
[12] V. Tartter and K. Knowlton, "Perception of sign language from an array
of 27 moving spots," Nature, vol. 2, pp. 676-678, 1981.
[13] J. Weissmann and R. Salomon, "Gesture recognition for virtual reality
applications using data gloves and neural networks," Proceeding of the
International Joint Conference on Neural Networks, Vol.3, pp.2043-2046,
1999.
[14] G. Burdea, S. Deshpande, B. Liu, N. Langrana, and D. Gomez, "A Virtual
Reality-Base System for Hand Diagnosis and Rehabilitation," IEEE
Transaction on Rehabilitation Engineering, vol. 6, no. 2, pp. 229-240,
1997.
[15] V. Popescu, G. Burdea, and M. Bouzit, "Virtual Reality Simulation for a
Haptic Glove," IEEE Computer Animation Proceedings, Geneva,
Switzerland, pp. 195-200, 1999.
[16] V. Popescu, and G. Burdea, "A Virtual-Reality-Base Telerehabilitation
System with Force Feedback," IEEE Transactions on Information
Technology in Biomedicine, vol. 4, Issue: 1, pp. 45-51, March 2000.
[17] Power Glove, http://en.wikipedia.org/wiki/Power_Glove.
[18] 5DT Data Glove & CyberGlove,
http://www.vrlogic.com/html/datagloves.html.
[19] S. Vance, L. Migachyov, and W. Fu, I.Hajjar, "Fingerless glove for
Interacting with data processing system", United States Patent, Patten No:
6,304,840, Oct.16, 2001.
[1] D. J. Sturman and D. Zeltzer, "A Survey of Glove- based Input", IEEE
Computer Graphics and Applications, vol.14, no.1 pp. 30-39, 1994.
[2] G. C. Sun, Self-balancing Control and Manipulation of a Glove Puppet
Robot on a Two-wheel Mobile Platform, Master's thesis, Institute of
Electrical Control Engineering, National Chiao Tung University, 2009.
[3] J. S. Hu, J. J. Wang, and G. Q. Sun, "The Glove Puppet Robot: X-puppet,"
IEEE/RSJ 2008 International Conference on Intelligent Robots and
Systems (IROS -08), Nice, France, 2008.
[4] M. C. Su, C. H. Hung, C. K. Yang, H. T. Yang, Z. J. Huang, Y. X. Zhao,
and D. Y. Huang, 2005, "Design of Optical Fiber Glove and Its
Application," in 2005 13th National Conference on Fuzzy Theory and
Applications, Kaohsing, Taiwan.
[5] J. M. Hollerbach, "Anthropomorphic Robot and Human Interactions", in
Proc. of 1st. Intl. Symposium on Humanoid Robots, 1996.
[6] S. S. Fels and G. E. Hinton, "Glove-Talk: a neural network interface
between a Data-Glove and a speech synthesizer", IEEE Trans. On Neural
Networks, Vol. 1, pp. 2-8, 1993.
[7] M. C. Su, Y. X. Zhao, C. H. Hung, and M. Chunag, "A Fuzzy Rule-Based
Approach to Hand Shape Recognition Systems," in The Joint Conference
on AI, Fuzzy System, and Grey System, Taiwan, Dec. 2003.
[8] Y. X. Zhao, A Portable Speaking-Aid System for Taiwanese Sign
Language, Master's thesis, Department of Electrical Engineering,
Tamkang University, 2002.
[9] C. H. Hung, A Trajectory-based Approach to Gesture Recognition,
Master's thesis, Department of Computer Science and Information,
Engineering, National Central University, 2006.
[10] M. C. Su, Y. X. Zhao, H. Huang, and H. F. Chen, "A Fuzzy Rule-Based
Approach to Recognizing 3D Arm Movements," IEEE Trans. on Neural
Systems and Rehabilitation Engineering, vol.9, no. 2, pp. 191-201, June
2001.
[11] M. C. Su, C. H. Hung, and Y. X. Zhao, "A SOMART System for Gesture
Recognition," WSEAS Transactions on Computers, vol. 5, pp. 2764-2771,
2006.
[12] V. Tartter and K. Knowlton, "Perception of sign language from an array
of 27 moving spots," Nature, vol. 2, pp. 676-678, 1981.
[13] J. Weissmann and R. Salomon, "Gesture recognition for virtual reality
applications using data gloves and neural networks," Proceeding of the
International Joint Conference on Neural Networks, Vol.3, pp.2043-2046,
1999.
[14] G. Burdea, S. Deshpande, B. Liu, N. Langrana, and D. Gomez, "A Virtual
Reality-Base System for Hand Diagnosis and Rehabilitation," IEEE
Transaction on Rehabilitation Engineering, vol. 6, no. 2, pp. 229-240,
1997.
[15] V. Popescu, G. Burdea, and M. Bouzit, "Virtual Reality Simulation for a
Haptic Glove," IEEE Computer Animation Proceedings, Geneva,
Switzerland, pp. 195-200, 1999.
[16] V. Popescu, and G. Burdea, "A Virtual-Reality-Base Telerehabilitation
System with Force Feedback," IEEE Transactions on Information
Technology in Biomedicine, vol. 4, Issue: 1, pp. 45-51, March 2000.
[17] Power Glove, http://en.wikipedia.org/wiki/Power_Glove.
[18] 5DT Data Glove & CyberGlove,
http://www.vrlogic.com/html/datagloves.html.
[19] S. Vance, L. Migachyov, and W. Fu, I.Hajjar, "Fingerless glove for
Interacting with data processing system", United States Patent, Patten No:
6,304,840, Oct.16, 2001.
@article{"International Journal of Electrical, Electronic and Communication Sciences:60375", author = "Yu-Xiang Zhao and Chien-Hsing Chou and Mu-Chun Su and Yi-Zeng Hsieh", title = "Portable Virtual Piano Design", abstract = "The purpose of this study is to design a portable virtual
piano. By utilizing optical fiber gloves and the virtual piano software
designed by this study, the user can play the piano anywhere at any
time. This virtual piano consists of three major parts: finger tapping
identification, hand movement and positioning identification, and
MIDI software sound effect simulation. To play the virtual piano, the
user wears optical fiber gloves and simulates piano key tapping
motions. The finger bending information detected by the optical fiber
gloves can tell when piano key tapping motions are made. Images
captured by a video camera are analyzed, hand locations and moving
directions are positioned, and the corresponding scales are found. The
system integrates finger tapping identification with information about
hand placement in relation to corresponding piano key positions, and
generates MIDI piano sound effects based on this data. This
experiment shows that the proposed method achieves an accuracy rate
of 95% for determining when a piano key is tapped.", keywords = "virtual piano, portable, identification, optical fibergloves.", volume = "4", number = "7", pages = "1065-4", }
