We introduce a new interactive 3D simulator of ocular motion and expressions suitable for: (1) character animation applications to game design, film production, HCI (Human Computer Interface), conversational animated agents, and virtual reality; (2) medical applications (ophthalmic neurological and muscular pathologies: research and education); and (3) real time simulation of unconscious cognitive and emotional responses (for use, e.g., in psychological research). Using state-of-the-art computer animation technology we have modeled and rigged a physiologically accurate 3D model of the eyes, eyelids, and eyebrow regions and we have 'optimized' it for use with an interactive and web deliverable platform. In addition, we have realized a prototype device for realtime control of eye motions and expressions, including unconsciously produced expressions, for application as in (1), (2), and (3) above. The 3D simulator of eye motion and ocular expression is, to our knowledge, the most advanced/realistic available so far for applications in character animation and medical pedagogy.
[1] F. Parke, "Parameterized models of human faces". PhD Thesis,
University of Utah, 1974.
[2] S. Platt and N. Badler, "Animating facial expressions". Computer
Graphics (Siggraph- 81 Proc.), 1981, pp.279-288.
[3] K. Waters, "A muscle model for animating three dimensional facial
expressions". Computer Graphics (Siggraph -87 Proc.), 1987, pp.17-
24.
[4] P. Kalra, A. Mangili, N. Magnenat Thalmann and D. Thalmann,
"Simulation of Facial Muscle Actions Based on Rational Free Form
Deformations". Computer Graphics Forum, 1992, Vol. 11, Issue 3, pp.
59-69.
[5] S. P. Lee, J. B. Badler, and N. I. Badler. "Eyes Alive". ACM
Transactions on Graphics, 2002, 21 (3), pp. 637-644.
[6] N. Adamo-Villani, G. Beni, "Keyboard Encoding of Facial
Expressions". IEEE Proceedings of IV04 - 8th International Conference
on Information Visualization, London, 2004, pp. 324-328.
[7] J. Cassell, H. Vilhjalmsson, and T. Bickmore, "BEAT: the Behavior
Expression Animation Toolkit" Siggraph 2001 Proc.
[8] J. Cassel, O. Torees, S. Prevost, "Turn taking vs. discourse structure:
how best to model multimodal conversation". In Machine
Conversations, Wilks eds, pp. 143-154.
[9] The Polar Express Production Notes (Online). Available:
http://polarexpressmovie.warnerbros.com/movie_prodnotes.html
[10] Online Interactive Patient Simulators, New York Emergency Room.
http://www.nyerrn.com/simulators.htm
[11] R. Lasslo, Eye Simulation Application. (Online). Available:
http://cim.ucdavis.edu/Eyes/version1/eyesim.htm
[12] Eye Simulator/Virtual Patient Simulator. (Online). Available:
http://cim.ucdavis.edu/EyeRelease/Interface/TopFrame.htm
[13] R. Cole, J. Movellan, J. Gratch. NSF Workshop proposal on
Perceptive Animated Interfaces and Virtual Humans, 2002.
[14] A. Bayklor, Y. Kim. "Pedagogical Agent Design: The Impact of Agent
Realism, Gender, Ethnicity, and Instructional Role". Proc. of
International Conference on Intelligent Tutoring Systems, 2004.
[15] K. Moreno, B. Klettke , K. Nibbaragandla, A. C. Graesser. "Perceived
Characteristics and Pedagogical Efficacy of Animated Conversational
Agents". Proc. of Intelligent Tutoring Systems : 6th International
Conference, France, 2002.
[16] T. Cacioppo, L. Tassinary. Principles of psychophysiology: physical,
social, and inferential elements. Cambridge University Press, 1990.
[17] E. Hecht. Optics. 2nd Ed. Addison Wesley, 1987.
[18] P Ekman, W Friesen, J Hager. Facial Action Coding System: A
Technique for the Measurement of Facial Movement. Consulting
Psychologists Press, Palo Alto, CA, 1978.
[19] E. Ackerman. Considerable material on vision from a medical point of
view. Biophysical Science, Prentice-Hall, 1962.
[20] F. Delacerda. Some interesting details about accommodation of the eye.
Collision Avoidance, Flight Training, May 1991.
[21] L.Stark, P.M. Sherman. A servoanalytic study of consensual pupil reflex
to light. Yale University, 1956.
[22] Game Controller: Belkin Nostromo Speedpad n52. (Online). Available:
http://www.tomshardware.com/game/200403061/
[23] Mathsigner (Online). Available :http://www.tech.purdue.edu/cgt/I3/
[24] N. Adamo-Villani, J. Doublestein, Z. Martin. "The MathSigner: an
Interactive Learning Tool for American Sign Language K-3
Mathematics". IEEE Proceedings of IV04 - 8th International
Conference on Information Visualization, London, 2004, pp.713-716.
[25] N. Adamo-Villani, J. Doublestein, Z. Martin. "Sign Language for K-8
Mathematics by 3D Interactive Animation". Journal of Educational
Technology Systems, Vol. 33, No. 3, pp. 241 - 257, 2004-2005.
[26] Fakespace systems (Online). Available: www.fakespace.com
[27] CAL3D (Online). Available: http://cal3d.sourceforge.net/
[28] Open Scene Graph (Online). Available: http://www.openscenegraph.org
[29] Virtual Juggler (Online). Available: http://www.vrjuggler.org
[30] Intersense, Inc. (Online). Available: http://www.isense.com/
[1] F. Parke, "Parameterized models of human faces". PhD Thesis,
University of Utah, 1974.
[2] S. Platt and N. Badler, "Animating facial expressions". Computer
Graphics (Siggraph- 81 Proc.), 1981, pp.279-288.
[3] K. Waters, "A muscle model for animating three dimensional facial
expressions". Computer Graphics (Siggraph -87 Proc.), 1987, pp.17-
24.
[4] P. Kalra, A. Mangili, N. Magnenat Thalmann and D. Thalmann,
"Simulation of Facial Muscle Actions Based on Rational Free Form
Deformations". Computer Graphics Forum, 1992, Vol. 11, Issue 3, pp.
59-69.
[5] S. P. Lee, J. B. Badler, and N. I. Badler. "Eyes Alive". ACM
Transactions on Graphics, 2002, 21 (3), pp. 637-644.
[6] N. Adamo-Villani, G. Beni, "Keyboard Encoding of Facial
Expressions". IEEE Proceedings of IV04 - 8th International Conference
on Information Visualization, London, 2004, pp. 324-328.
[7] J. Cassell, H. Vilhjalmsson, and T. Bickmore, "BEAT: the Behavior
Expression Animation Toolkit" Siggraph 2001 Proc.
[8] J. Cassel, O. Torees, S. Prevost, "Turn taking vs. discourse structure:
how best to model multimodal conversation". In Machine
Conversations, Wilks eds, pp. 143-154.
[9] The Polar Express Production Notes (Online). Available:
http://polarexpressmovie.warnerbros.com/movie_prodnotes.html
[10] Online Interactive Patient Simulators, New York Emergency Room.
http://www.nyerrn.com/simulators.htm
[11] R. Lasslo, Eye Simulation Application. (Online). Available:
http://cim.ucdavis.edu/Eyes/version1/eyesim.htm
[12] Eye Simulator/Virtual Patient Simulator. (Online). Available:
http://cim.ucdavis.edu/EyeRelease/Interface/TopFrame.htm
[13] R. Cole, J. Movellan, J. Gratch. NSF Workshop proposal on
Perceptive Animated Interfaces and Virtual Humans, 2002.
[14] A. Bayklor, Y. Kim. "Pedagogical Agent Design: The Impact of Agent
Realism, Gender, Ethnicity, and Instructional Role". Proc. of
International Conference on Intelligent Tutoring Systems, 2004.
[15] K. Moreno, B. Klettke , K. Nibbaragandla, A. C. Graesser. "Perceived
Characteristics and Pedagogical Efficacy of Animated Conversational
Agents". Proc. of Intelligent Tutoring Systems : 6th International
Conference, France, 2002.
[16] T. Cacioppo, L. Tassinary. Principles of psychophysiology: physical,
social, and inferential elements. Cambridge University Press, 1990.
[17] E. Hecht. Optics. 2nd Ed. Addison Wesley, 1987.
[18] P Ekman, W Friesen, J Hager. Facial Action Coding System: A
Technique for the Measurement of Facial Movement. Consulting
Psychologists Press, Palo Alto, CA, 1978.
[19] E. Ackerman. Considerable material on vision from a medical point of
view. Biophysical Science, Prentice-Hall, 1962.
[20] F. Delacerda. Some interesting details about accommodation of the eye.
Collision Avoidance, Flight Training, May 1991.
[21] L.Stark, P.M. Sherman. A servoanalytic study of consensual pupil reflex
to light. Yale University, 1956.
[22] Game Controller: Belkin Nostromo Speedpad n52. (Online). Available:
http://www.tomshardware.com/game/200403061/
[23] Mathsigner (Online). Available :http://www.tech.purdue.edu/cgt/I3/
[24] N. Adamo-Villani, J. Doublestein, Z. Martin. "The MathSigner: an
Interactive Learning Tool for American Sign Language K-3
Mathematics". IEEE Proceedings of IV04 - 8th International
Conference on Information Visualization, London, 2004, pp.713-716.
[25] N. Adamo-Villani, J. Doublestein, Z. Martin. "Sign Language for K-8
Mathematics by 3D Interactive Animation". Journal of Educational
Technology Systems, Vol. 33, No. 3, pp. 241 - 257, 2004-2005.
[26] Fakespace systems (Online). Available: www.fakespace.com
[27] CAL3D (Online). Available: http://cal3d.sourceforge.net/
[28] Open Scene Graph (Online). Available: http://www.openscenegraph.org
[29] Virtual Juggler (Online). Available: http://www.vrjuggler.org
[30] Intersense, Inc. (Online). Available: http://www.isense.com/
@article{"International Journal of Information, Control and Computer Sciences:55869", author = "Nicoletta Adamo-Villani and Gerardo Beni and Jeremy White", title = "3D Simulator of Ocular Motion and Expression", abstract = "We introduce a new interactive 3D simulator of ocular motion and expressions suitable for: (1) character animation applications to game design, film production, HCI (Human Computer Interface), conversational animated agents, and virtual reality; (2) medical applications (ophthalmic neurological and muscular pathologies: research and education); and (3) real time simulation of unconscious cognitive and emotional responses (for use, e.g., in psychological research). Using state-of-the-art computer animation technology we have modeled and rigged a physiologically accurate 3D model of the eyes, eyelids, and eyebrow regions and we have 'optimized' it for use with an interactive and web deliverable platform. In addition, we have realized a prototype device for realtime control of eye motions and expressions, including unconsciously produced expressions, for application as in (1), (2), and (3) above. The 3D simulator of eye motion and ocular expression is, to our knowledge, the most advanced/realistic available so far for applications in character animation and medical pedagogy.
", keywords = "3D animation, HCI, medical simulation, ocularmotion and expression.", volume = "1", number = "8", pages = "2476-6", }
