The paper describes a Chinese shadow play animation
system based on Kinect. Users, without any professional training, can
personally manipulate the shadow characters to finish a shadow play
performance by their body actions and get a shadow play video
through giving the record command to our system if they want. In our
system, Kinect is responsible for capturing human movement and
voice commands data. Gesture recognition module is used to control
the change of the shadow play scenes. After packaging the data from
Kinect and the recognition result from gesture recognition module,
VRPN transmits them to the server-side. At last, the server-side uses
the information to control the motion of shadow characters and video
recording. This system not only achieves human-computer interaction,
but also realizes the interaction between people. It brings an
entertaining experience to users and easy to operate for all ages. Even
more important is that the application background of Chinese shadow
play embodies the protection of the art of shadow play animation.
[1] Ugur Güdükbay, Fatih Erol, Nezih Erdoğan. Tradition offers artistic
possibilities for new media technologies:An animation system for shadow
theatre, actes / proceedings isea2000 – 10/12/2000 – salle/room 300.
[2] Yi-Bo Zhu, Chen-Jia Li, I-Fan Shen, Kwan-Liu Ma, Aleksander Stompel,
A New Form of Traditional Art – Visual Simulation of Chinese Shadow
Play. Proceeding SIGGRAPH '03, ACM SIGGRAPH 2003 Sketches &
Applications. 2003:1-1.
[3] GAO Lu-jing, CAI jian-ping, Analysis and Implementation of Features of
Character’s Action in Shadowgraph Animation. Computer Engineering
and Design, 2010. 2335-2342
[4] S.W. Hsu, T.Y. Li. Generating Secondary Motions in Shadow Play
Animations with Motion Planning Techniques. Proceedings of
SIGGRAPH 2005 conference on Sketches & applications, 2005, Article
No. 69,1 pages.
[5] S.W. Hsu, T.Y. Li. Planning Character Motions for Shadow Play
Animations. Proceedings of Computer Animation and Social Agents,
2005:1-6.
[6] Totilo, Stephen (January 7, 2010). "Natal Recognizes 31 Body Parts, Uses
Tenth of Xbox 360 "Computing Resources"". Kotaku, Gawker Media.
Retrieved November 25, 2010.
http://en.wikipedia.org/wiki/Kinect#Technology
[7] L. Leite and V. Orvalho, “Shape your body: control a virtual silhouette
using body motion” CHI '12 Extended Abstracts on Human Factors in
Computing Systems: 1913-1918
[8] Seth Hunter, Pattie Maes. Designing digital puppetry systems: guidelines
and best practices. Extended Abstracts on Human Factors in Computing
Systems. 2013
[9] Robert Held, Ankit Gupta, Brian Curless, Maneesh Agrawala. 3D
puppetry: a kinect-based interface for 3D animation. Proceedings of the
25th annual ACM symposium on User interface software and technology,
Oct., 2012.
[10] Bradley Wesson, Brett Wilkinson. Evaluating organic 3D sculpting using
natural user interfaces with the Kinect, Proceedings of the 25th Australian
Computer-Human Interaction Conference, Nov., 2013.
[11] Kam Lai, Konrad, J., Ishwar, P. A gesture-driven computer interface
using Kinect, published byImage Analysis and Interpretation (SSIAI),
2012 IEEE. Pages: 185 – 188.
[12] Karthik Nandakumar, Kong Wah Wan, Siu Man Alice Chan, Wen Zheng
Terence Ng, Jian Gang Wang, Wei Yun Yau. A multi-modal gesture
recognition system using audio, video, and skeletal joint data,
Proceedings of the 15th ACM on International conference on multimodal
interaction, Dec.,2013
[13] Tsai-Te Chu, Chung-Yen Su. A Kinect-based Handwritten Digit
Recognition for TV Remote Controller, published byIntelligent Signal
Processing and Communications, 4-7 Nov. 2012.Pages: 414 – 419
[14] Anderson, D., Bailey, C. and Skubic, M. (2004) Hidden Markov Model
symbol recognition for sketch-based interfaces.AAAI Fall Symposium.
Menlo Park, CA: AAAI Press, 15-21.
[15] M. Black, A. Jepson. A probabilistic framework for matching temporal
trajectories: Condensation-based recognition of gestures and expressions.
in Proc. European Conf. on Computer Vision(ECCV’98), 1998
[16] M. J. Black, A. D. Jepson. Recognition temporal trajectories using the
Condensation algorithm. in Proc. IEEE Int’1 Conf. on Automated Face
and Gesture Recognition(FG’98), Japan, 1998: 16~21
[17] Asma Ben Hadj Mohamed, Thierry VAL, Laurent Andrieux,
Abdennaceur KACHOUR. Assisting people with disabilities through
Kinect sensors into a smart house, International Conference on Computer
Medical Application,Jan.2013.Pages:1-5
[18] Foley, J., V.L. Wallace, and P. Chan,The Human Factors of Computer
Graphics Interaction Techniques. IEEE Computer Graphics and
Application, 1984.4 (11): p. 13-48.
[19] Russell M. Taylor II, Thomas C. Hudson, Adam Seeger, Hans Weber,
Jeffrey Juliano, Aron T. Helser. VRPN: A Device-Independent,
Network-Transparent VR Peripheral System. Proceedings of VRST 2001,
ACM Virtual Reality Software and Technology. 2001.
[20] http://depts.washington.edu/aimgroup/proj/dollar/
[21] Jacob O.Wobbrock, Andrew D.Wilson and Yang Li.(2007) Gestures
without libraries, toolkits or training: a $1 recognizer for user interface
prototypes. Proc. UIST ’07. New York: ACM Press, 159-168.
[22] Paul Doliotis, Alexandra Stefan, Christopher McMurrough, David
Eckhard, Vassilis Athitsos. Comparing gesture recognition accuracy
using color and depth information, Proceedings of the 4th International
Conference on PErvasive Technologies Related to Assistive
Environments, May, 2011.
[23] Jaemin Soh, Yeongjae Choi, Youngmin Park, Hyun S. Yang.
User-friendly 3D object manipulation gesture using Kinect, Nov., 2013.
[1] Ugur Güdükbay, Fatih Erol, Nezih Erdoğan. Tradition offers artistic
possibilities for new media technologies:An animation system for shadow
theatre, actes / proceedings isea2000 – 10/12/2000 – salle/room 300.
[2] Yi-Bo Zhu, Chen-Jia Li, I-Fan Shen, Kwan-Liu Ma, Aleksander Stompel,
A New Form of Traditional Art – Visual Simulation of Chinese Shadow
Play. Proceeding SIGGRAPH '03, ACM SIGGRAPH 2003 Sketches &
Applications. 2003:1-1.
[3] GAO Lu-jing, CAI jian-ping, Analysis and Implementation of Features of
Character’s Action in Shadowgraph Animation. Computer Engineering
and Design, 2010. 2335-2342
[4] S.W. Hsu, T.Y. Li. Generating Secondary Motions in Shadow Play
Animations with Motion Planning Techniques. Proceedings of
SIGGRAPH 2005 conference on Sketches & applications, 2005, Article
No. 69,1 pages.
[5] S.W. Hsu, T.Y. Li. Planning Character Motions for Shadow Play
Animations. Proceedings of Computer Animation and Social Agents,
2005:1-6.
[6] Totilo, Stephen (January 7, 2010). "Natal Recognizes 31 Body Parts, Uses
Tenth of Xbox 360 "Computing Resources"". Kotaku, Gawker Media.
Retrieved November 25, 2010.
http://en.wikipedia.org/wiki/Kinect#Technology
[7] L. Leite and V. Orvalho, “Shape your body: control a virtual silhouette
using body motion” CHI '12 Extended Abstracts on Human Factors in
Computing Systems: 1913-1918
[8] Seth Hunter, Pattie Maes. Designing digital puppetry systems: guidelines
and best practices. Extended Abstracts on Human Factors in Computing
Systems. 2013
[9] Robert Held, Ankit Gupta, Brian Curless, Maneesh Agrawala. 3D
puppetry: a kinect-based interface for 3D animation. Proceedings of the
25th annual ACM symposium on User interface software and technology,
Oct., 2012.
[10] Bradley Wesson, Brett Wilkinson. Evaluating organic 3D sculpting using
natural user interfaces with the Kinect, Proceedings of the 25th Australian
Computer-Human Interaction Conference, Nov., 2013.
[11] Kam Lai, Konrad, J., Ishwar, P. A gesture-driven computer interface
using Kinect, published byImage Analysis and Interpretation (SSIAI),
2012 IEEE. Pages: 185 – 188.
[12] Karthik Nandakumar, Kong Wah Wan, Siu Man Alice Chan, Wen Zheng
Terence Ng, Jian Gang Wang, Wei Yun Yau. A multi-modal gesture
recognition system using audio, video, and skeletal joint data,
Proceedings of the 15th ACM on International conference on multimodal
interaction, Dec.,2013
[13] Tsai-Te Chu, Chung-Yen Su. A Kinect-based Handwritten Digit
Recognition for TV Remote Controller, published byIntelligent Signal
Processing and Communications, 4-7 Nov. 2012.Pages: 414 – 419
[14] Anderson, D., Bailey, C. and Skubic, M. (2004) Hidden Markov Model
symbol recognition for sketch-based interfaces.AAAI Fall Symposium.
Menlo Park, CA: AAAI Press, 15-21.
[15] M. Black, A. Jepson. A probabilistic framework for matching temporal
trajectories: Condensation-based recognition of gestures and expressions.
in Proc. European Conf. on Computer Vision(ECCV’98), 1998
[16] M. J. Black, A. D. Jepson. Recognition temporal trajectories using the
Condensation algorithm. in Proc. IEEE Int’1 Conf. on Automated Face
and Gesture Recognition(FG’98), Japan, 1998: 16~21
[17] Asma Ben Hadj Mohamed, Thierry VAL, Laurent Andrieux,
Abdennaceur KACHOUR. Assisting people with disabilities through
Kinect sensors into a smart house, International Conference on Computer
Medical Application,Jan.2013.Pages:1-5
[18] Foley, J., V.L. Wallace, and P. Chan,The Human Factors of Computer
Graphics Interaction Techniques. IEEE Computer Graphics and
Application, 1984.4 (11): p. 13-48.
[19] Russell M. Taylor II, Thomas C. Hudson, Adam Seeger, Hans Weber,
Jeffrey Juliano, Aron T. Helser. VRPN: A Device-Independent,
Network-Transparent VR Peripheral System. Proceedings of VRST 2001,
ACM Virtual Reality Software and Technology. 2001.
[20] http://depts.washington.edu/aimgroup/proj/dollar/
[21] Jacob O.Wobbrock, Andrew D.Wilson and Yang Li.(2007) Gestures
without libraries, toolkits or training: a $1 recognizer for user interface
prototypes. Proc. UIST ’07. New York: ACM Press, 159-168.
[22] Paul Doliotis, Alexandra Stefan, Christopher McMurrough, David
Eckhard, Vassilis Athitsos. Comparing gesture recognition accuracy
using color and depth information, Proceedings of the 4th International
Conference on PErvasive Technologies Related to Assistive
Environments, May, 2011.
[23] Jaemin Soh, Yeongjae Choi, Youngmin Park, Hyun S. Yang.
User-friendly 3D object manipulation gesture using Kinect, Nov., 2013.
@article{"International Journal of Information, Control and Computer Sciences:68768", author = "Bo Wan and Xiu Wen and Lingling An and Xiaoling Ding", title = "Interactive Shadow Play Animation System", abstract = "The paper describes a Chinese shadow play animation
system based on Kinect. Users, without any professional training, can
personally manipulate the shadow characters to finish a shadow play
performance by their body actions and get a shadow play video
through giving the record command to our system if they want. In our
system, Kinect is responsible for capturing human movement and
voice commands data. Gesture recognition module is used to control
the change of the shadow play scenes. After packaging the data from
Kinect and the recognition result from gesture recognition module,
VRPN transmits them to the server-side. At last, the server-side uses
the information to control the motion of shadow characters and video
recording. This system not only achieves human-computer interaction,
but also realizes the interaction between people. It brings an
entertaining experience to users and easy to operate for all ages. Even
more important is that the application background of Chinese shadow
play embodies the protection of the art of shadow play animation.
", keywords = "Gesture recognition, Kinect, shadow play animation, VRPN.", volume = "9", number = "1", pages = "127-6", }
