Designing and Evaluating Pedagogic Conversational Agents to Teach Children

In this paper, the possibility of children studying by using an interactive learning technology called Pedagogic Conversational Agent is presented. The main benefit is that the agent is able to adapt the dialogue to each student and to provide automatic feedback. Moreover, according to Math teachers, in many cases students are unable to solve the problems even knowing the procedure to solve them, because they do not understand what they have to do. The hypothesis is that if students are helped to understand what they have to solve, they will be able to do it. Taken that into account, we have started the development of Dr. Roland, an agent to help students understand Math problems following a User-Centered Design methodology. The use of this methodology is proposed, for the first time, to design pedagogic agents to teach any subject from Secondary down to Pre-Primary education. The reason behind proposing a methodology is that while working on this project, we noticed the lack of literature to design and evaluate agents. To cover this gap, we describe how User-Centered Design can be applied, and which usability techniques can be applied to evaluate the agent.

Authors:

• Silvia Tamayo-Moreno
• Diana Pérez-Marín

Keywords:

• Pedagogic conversational agent
• human-computer interaction
• user-centered design
• natural language interface.

References:

[1] Narayanan, S., & Potamianos, A. (2002). Creating Conversational Interfaces for Children, IEEE Transactions On Speech And Audio Processing. Vol. 10, no 2, 65-78.
[2] Johnson, W., Rickel, J., and Lester, J. (2000). Animated Pedagogical Agents: Face-to-Face Interaction in Interactive Learning Environments, Journal of Artificial Intelligence in Education 11, 47-78.
[3] Graesser, A., D’Mello, S., Craig, S., Witherspoon, A., Sullins, J., McDaniel, B., & Gholson, B. (2008). The relationship between affective states and dialog patterns during interactions with AutoTutor, Journal of Interactive Learning Research, 19 (2), 293-302.
[4] Dzikovska, M.O., Isard, A., & Bell, P. (2011). Adaptive Intelligent Tutorial Dialogue in the BEETLE II System, Proceedings of the Artificial Intelligence in Education Conference. Vol. 6738 LNCS, 621.
[5] Lane, H.C., Noren, D., Auerbach, D., Birch, M., & Swartout, W. (2011). Intelligent Tutoring Goes to the Museum in the Big City: A Pedagogical Agent for Informal Science Education, Proceedings of the Artificial Intelligence in Education Conference. Vol. 7926 LNCS, 309-318.
[6] Abascal, J., Aedo, I., Cañas, J., Gea, M., Gil, A.B., Lorés, J,; Martínez, A.B., Ortega, M., Valero, P., Vélez, M. (2001). La Interacción Persona-Ordenador, Jesús Lorés, Asociación Interacción Persona Ordenador (AIPO).
[7] ISO 9241-210. (2010). Ergonomics of human-system interaction -- Part 210: Human-centred design for interactive systems. ISO.org
[8] Sauro, J. (2012). http://www.measuringusability.com/blog/effective-he.php. Last visit: January 25th, 2017.
[9] Pérez-Marín, D. & Salmerón, L. (2014). When Human-Computer Interaction meets Blended Learning and Heuristics are not Enough: Evaluating the usability with Field Observation and Interviews. In Press, Nova Publisihing.
[10] Alsumait, A. & Al-Osaimi, A. (2009). Usability Heuristics Evaluation for Child E-learning Applications. 11th International Conference on Information Integration and Web-based applications & Services, pp. 425-430.
[11] Dix, A., Finlay, J., Abowd, G., & Beale, R. (2004). Human-Computer Interaction, Prentice Hall.
[12] Tamayo-Moreno, S. & Pérez-Marín, D. Adapting the design and the use methodology of a Pedagogical Conversational Agent of Secondary Education to Childhood Education. In Press, IEEE 2016 International Symposium on Computers in Education (SIIE).