Humanoid Personalized Avatar Through Multiple Natural Language Processing
There has been a growing interest in implementing humanoid avatars in networked virtual environment. However, most existing avatar communication systems do not take avatars- social backgrounds into consideration. This paper proposes a novel humanoid avatar animation system to represent personalities and facial emotions of avatars based on culture, profession, mood, age, taste, and so forth. We extract semantic keywords from the input text through natural language processing, and then the animations of personalized avatars are retrieved and displayed according to the order of the keywords. Our primary work is focused on giving avatars runtime instruction from multiple natural languages. Experiments with Chinese, Japanese and English input based on the prototype show that interactive avatar animations can be displayed in real time and be made available online. This system provides a more natural and interesting means of human communication, and therefore is expected to be used for cross-cultural communication, multiuser online games, and other entertainment applications.
[1] T. Capin, L. Pandzic, N. M. Thalmann, and D. Thalmann, Avatars in
Networked Virtual Environments, John Wiley & Sons Ltd, England,
1999.
[2] H. Tang, Y. Fu, J. L. Tu, M. Hasegawa-Johnson, and T. S. Huang,
"Humanoid audio-visual avatar with emotive text-to-speech synthesis,"
IEEE Trans. Mul., vol. 10, no. 6, pp. 969-981, October 2008.
[3] Y. Fu, R. Li, T. S. Huang, and M. Danielsen, "Real-time multimodal
human-avatar interaction," IEEE Trans. CSVT, vol. 18, no. 4, pp. 467-
477, April 2008.
[4] X. L. Yang, D. C. Petriu, T. E. Whalen, and E. M. Petriu, "Hierarchical
animation control of avatars in 3-D virtual environments," IEEE Trans.
Instrumentation and Measurement, vol. 54, no. 3, pp. 1333-1341, June
2005.
[5] J. Hou, Y. Koine, and Y. Aoki, "Ontology-based social avatar language,"
Journal of Signal Processing, vol. 7, no. 5, pp. 393-403, Septemper 2003.
[6] T. Noma, L. W. Zhao, and N. Badler, "Design of a virtual human
presenter," IEEE Comput. Graph. Appl., vol. 20, no. 4, pp. 79-85, July
2000.
[7] J. Cassell, H. Vilhjalmsson, and T. Bickmore, "BEAT: the behavior
expression animation toolkit," in Proc. 28th Int. Conf. Computer Graphics
and Interactive Techniques, Los Angeles, USA, Aug. 2001, pp. 477-486.
[8] Z. X. Yang, L. Li, and D. Zhang, "Embodiment of text based on cirtual
robotic avatar," in Proc. IEEE Int. Conf. Robotics and Biomimetics,
Sanya, China, Dec. 2007, pp. 1285 - 1289.
[9] W. Steptoe and A. Steed, "High-fidelity avatar eye-representation," in
Proc. IEEE Virtual Reality Conf., Nevada, USA, Mar. 2008, pp. 111-114.
[10] Z. Sakr and C. Sudama, "A curvilinear avatar with avatar collision
detection scheme in collaborative virtual environments," in Proc. IEEE
Instrumentation and Measurement Technology Conf., Victoria
Vancouver Island, Canada, May 2008, pp. 1027 - 1030.
[11] Y. Ishii and T. Watanabe, "An embodied avatar mediated communication
system with VirtualActor for human interaction analysis," in Proc. 16th
IEEE Int. Conf. Robot and Human Interactive Communication, Jeju,
Korea, Aug. 2007, pp. 37-42.
[12] S. Piotr and K. Bozena, "Personalized avatar animation for virtual
reality," in Proc. 1st Int. Conf. Information Technology, Gdansk, Poland,
May 2008, pp. 1-4.
[13] S. Rusdorf, G. Brunnett, M. Lorenz, and T. Winkler, "Real-time
interaction with a humanoid avatar in an immersive table tennis
simulation," IEEE Trans. Visualization and Computer Graphics, vol. 13,
no. 1, pp. 15-25, January-February 2007.
[14] C. Lee, H. Lee, and K. Oh, "Real-time image-based 3D avatar for
immersive game." in Proc. 7th ACM SIGGRAPH Int. Conf. Virtualreality
Continuum and Its Applications in Industry, Singapore, Dec. 2008,
pp. 1-2.
[15] N. Ahmed, E. Aguiar, C. Theobalt, M. Magnor, and H. P. Seidel,
"Automatic generation of personalized human avatars from multi-view
video." in Proc. the ACM Symposium Virtual Reality Software and
Technology, Monterey, USA, Nov. 2005, pp. 257-260.
[16] The Humanoid Animation Group-s. (Online). Available:
http://www.hanim.org/.
[17] N. M. Thalmann, H. Seo, and F. Cordier, "Automatic modeling of
animatable virtual humans-a survey," in Proc. 4th Int. Conf. 3-D Digital
Imaging and Modeling, Banff, Canada, Oct. 2003, pp. 2-10.
[18] I. J. Pelczer, F. Cabiedes, and F. Gamboa, "Emotions and interactive
agents," in Proc. IEEE Int. Conf. VECIMS, La Coru├▒a, Spain, Jul. 2006,
pp.184-187,.
[19] Globe Warp-s web site (Online). Available:
http://chat.globewarp.or.jp/index.html
[20] T. Wang, X. Li, and J. Shi, "An avatar-based approach to 3D user
interface design for children," in Proc. IEEE Symposium 3D User
Interfaces, North Carolina, USA, Mar. 2007, pp. 155-161,.
[21] Sony-s 3-D Chat web site (Online). Available: http://www.sonet.
ne.jp/paw
[22] Y. J. Oh, K. H. Park, and Z. Bien, "Body motion editor for sign language
avatar," in Proc. Int. Conf. Control, Automation and Systems, Seoul,
Korea, Oct. 2007, pp. 1752-1757.
[1] T. Capin, L. Pandzic, N. M. Thalmann, and D. Thalmann, Avatars in
Networked Virtual Environments, John Wiley & Sons Ltd, England,
1999.
[2] H. Tang, Y. Fu, J. L. Tu, M. Hasegawa-Johnson, and T. S. Huang,
"Humanoid audio-visual avatar with emotive text-to-speech synthesis,"
IEEE Trans. Mul., vol. 10, no. 6, pp. 969-981, October 2008.
[3] Y. Fu, R. Li, T. S. Huang, and M. Danielsen, "Real-time multimodal
human-avatar interaction," IEEE Trans. CSVT, vol. 18, no. 4, pp. 467-
477, April 2008.
[4] X. L. Yang, D. C. Petriu, T. E. Whalen, and E. M. Petriu, "Hierarchical
animation control of avatars in 3-D virtual environments," IEEE Trans.
Instrumentation and Measurement, vol. 54, no. 3, pp. 1333-1341, June
2005.
[5] J. Hou, Y. Koine, and Y. Aoki, "Ontology-based social avatar language,"
Journal of Signal Processing, vol. 7, no. 5, pp. 393-403, Septemper 2003.
[6] T. Noma, L. W. Zhao, and N. Badler, "Design of a virtual human
presenter," IEEE Comput. Graph. Appl., vol. 20, no. 4, pp. 79-85, July
2000.
[7] J. Cassell, H. Vilhjalmsson, and T. Bickmore, "BEAT: the behavior
expression animation toolkit," in Proc. 28th Int. Conf. Computer Graphics
and Interactive Techniques, Los Angeles, USA, Aug. 2001, pp. 477-486.
[8] Z. X. Yang, L. Li, and D. Zhang, "Embodiment of text based on cirtual
robotic avatar," in Proc. IEEE Int. Conf. Robotics and Biomimetics,
Sanya, China, Dec. 2007, pp. 1285 - 1289.
[9] W. Steptoe and A. Steed, "High-fidelity avatar eye-representation," in
Proc. IEEE Virtual Reality Conf., Nevada, USA, Mar. 2008, pp. 111-114.
[10] Z. Sakr and C. Sudama, "A curvilinear avatar with avatar collision
detection scheme in collaborative virtual environments," in Proc. IEEE
Instrumentation and Measurement Technology Conf., Victoria
Vancouver Island, Canada, May 2008, pp. 1027 - 1030.
[11] Y. Ishii and T. Watanabe, "An embodied avatar mediated communication
system with VirtualActor for human interaction analysis," in Proc. 16th
IEEE Int. Conf. Robot and Human Interactive Communication, Jeju,
Korea, Aug. 2007, pp. 37-42.
[12] S. Piotr and K. Bozena, "Personalized avatar animation for virtual
reality," in Proc. 1st Int. Conf. Information Technology, Gdansk, Poland,
May 2008, pp. 1-4.
[13] S. Rusdorf, G. Brunnett, M. Lorenz, and T. Winkler, "Real-time
interaction with a humanoid avatar in an immersive table tennis
simulation," IEEE Trans. Visualization and Computer Graphics, vol. 13,
no. 1, pp. 15-25, January-February 2007.
[14] C. Lee, H. Lee, and K. Oh, "Real-time image-based 3D avatar for
immersive game." in Proc. 7th ACM SIGGRAPH Int. Conf. Virtualreality
Continuum and Its Applications in Industry, Singapore, Dec. 2008,
pp. 1-2.
[15] N. Ahmed, E. Aguiar, C. Theobalt, M. Magnor, and H. P. Seidel,
"Automatic generation of personalized human avatars from multi-view
video." in Proc. the ACM Symposium Virtual Reality Software and
Technology, Monterey, USA, Nov. 2005, pp. 257-260.
[16] The Humanoid Animation Group-s. (Online). Available:
http://www.hanim.org/.
[17] N. M. Thalmann, H. Seo, and F. Cordier, "Automatic modeling of
animatable virtual humans-a survey," in Proc. 4th Int. Conf. 3-D Digital
Imaging and Modeling, Banff, Canada, Oct. 2003, pp. 2-10.
[18] I. J. Pelczer, F. Cabiedes, and F. Gamboa, "Emotions and interactive
agents," in Proc. IEEE Int. Conf. VECIMS, La Coru├▒a, Spain, Jul. 2006,
pp.184-187,.
[19] Globe Warp-s web site (Online). Available:
http://chat.globewarp.or.jp/index.html
[20] T. Wang, X. Li, and J. Shi, "An avatar-based approach to 3D user
interface design for children," in Proc. IEEE Symposium 3D User
Interfaces, North Carolina, USA, Mar. 2007, pp. 155-161,.
[21] Sony-s 3-D Chat web site (Online). Available: http://www.sonet.
ne.jp/paw
[22] Y. J. Oh, K. H. Park, and Z. Bien, "Body motion editor for sign language
avatar," in Proc. Int. Conf. Control, Automation and Systems, Seoul,
Korea, Oct. 2007, pp. 1752-1757.
@article{"International Journal of Information, Control and Computer Sciences:62843", author = "Jin Hou and Xia Wang and Fang Xu and Viet Dung Nguyen and Ling Wu", title = "Humanoid Personalized Avatar Through Multiple Natural Language Processing", abstract = "There has been a growing interest in implementing humanoid avatars in networked virtual environment. However, most existing avatar communication systems do not take avatars- social backgrounds into consideration. This paper proposes a novel humanoid avatar animation system to represent personalities and facial emotions of avatars based on culture, profession, mood, age, taste, and so forth. We extract semantic keywords from the input text through natural language processing, and then the animations of personalized avatars are retrieved and displayed according to the order of the keywords. Our primary work is focused on giving avatars runtime instruction from multiple natural languages. Experiments with Chinese, Japanese and English input based on the prototype show that interactive avatar animations can be displayed in real time and be made available online. This system provides a more natural and interesting means of human communication, and therefore is expected to be used for cross-cultural communication, multiuser online games, and other entertainment applications.
", keywords = "personalized avatar, mutiple natural luanguage processing, social backgrounds, anmimation, human computer interaction", volume = "3", number = "11", pages = "2715-6", }
