Predicting the Three Major Dimensions of the Learner-s Emotions from Brainwaves
This paper investigates how the use of machine learning techniques can significantly predict the three major dimensions of learner-s emotions (pleasure, arousal and dominance) from brainwaves. This study has adopted an experimentation in which participants were exposed to a set of pictures from the International Affective Picture System (IAPS) while their electrical brain activity was recorded with an electroencephalogram (EEG). The pictures were already rated in a previous study via the affective rating system Self-Assessment Manikin (SAM) to assess the three dimensions of pleasure, arousal, and dominance. For each picture, we took the mean of these values for all subjects used in this previous study and associated them to the recorded brainwaves of the participants in our study. Correlation and regression analyses confirmed the hypothesis that brainwave measures could significantly predict emotional dimensions. This can be very useful in the case of impassive, taciturn or disabled learners. Standard classification techniques were used to assess the reliability of the automatic detection of learners- three major dimensions from the brainwaves. We discuss the results and the pertinence of such a method to assess learner-s emotions and integrate it into a brainwavesensing Intelligent Tutoring System.
[1] Aha, D., & Kibler, D. Instance-based learning algorithms. Machine
Learning, 6, 37-66, 1991
[2] Anderson T. W. An Introduction to Multivariate Statistical Analysis, 3rd
ed. Wiley, New York, 2003.
[3] Bear, M. F., Connors, B. W., & Paradiso, M. A. Neuroscience:
Exploring the Brain, second ed. Lippincott Williams & Williams,
Baltimore, MD, 2001.
[4] Bellifemine, F., A. Poggi, & G. Rimassa, "JADE - A FIPA-compliant
Agent Framework", PAAM '99, London, UK, 1999, pp. 97ÔÇö108, 1999.
[5] Breazeal, C. Designing sociable robots .Cambridge: MIT Press, 2003.
[6] Breiman L. Bagging predictors. Machine Learning, 24(2):123-140,
1996.
[7] Cantor, D. S. An overview of quantitative EEG and its applications to
neurofeedback. In Introduction to Quantitative EEG and Neurofeedback,
J. R. Evans and A. Abarbanel, Eds. Academic Press, ch. 1, pp. 3-27,
1999.
[8] Conati C., Probabilistic assessment of user's emotions in educational
games. Journal of Applied Artificial Intelligence, 16, 555-575, 2002.
[9] Conati C., How to evaluate models of user affect?. Proceedings of ADS
04, Tutorial and Research Workshop on Affective Dialogue Systems.
Kloster Irsee, Germany, June 2004. p. 288-300, 2004.
[10] Craig, S.D., Graesser, A. C.,Sullins, J., & Gholson, B., Affect and
learning: An exploratory look into the role of affect in learning. Journal
of Educational Media, 29, 241-250, 2004.
[11] D'Mello, S.K., S.D. Craig, B. Gholson, S. Franklin, R.W. Picard, & A.C.
Graesser, "Integrating Affect Sensors in an Intelligent Tutoring System."
/In Affective Interactions: The Computer in the Affective Loop
Workshop at 2005 International conference on Intelligent User
Interfaces,/ AMC Press, New York, pp. 7-13, 2005.
[12] De Vicente, A., & Pain, H., Informing the detection of students'
motivational state : An empirical study. In S.A. Cerri, G. Gouarderes,
and F. Paraguacu (Eds Proceedings of the sixth international conference
on intelligent tutoring systems (pp.933-943). Berlin, Germany: Springer,
2002.
[13] Fan, C., Sarrafzadeh, A., Overmyer, S., Hosseini, H. G., Biglari-Abhari,
M., & Bigdeli, A. A fuzzy approach to facial expression analysis in
intelligent tutoring systems. In Antonio Méndez-Vilas and J.A.Mesa
González(Eds.) Advances in Technology-based Education: Towards a
Knowledge-based Society Vol 3. (pp. 1933-1937). Badajoz, Spain: Junta
De Extremadura, 2003.
[14] Fisher R. A., The use of multiple measurements in taxonomic problems.
Annals of Eugenics, 7:179-188, 1936
[15] Harter, S., A new self-report scale of intrinsic versus extrinsic
orientation in the classroom: Motivation and informational components.
Developmental Psychology, 17, 300-312, 1981.
[16] Hastie T., A. Buja, & R. Tibshirani. Penalized discriminant analysis.
Annals of Statistics, 23:73-102, 1995.
[17] Hastie T., R. Tibshirani, and A. Buja. Flexible discriminant analysis by
optimal scoring. J. American Statistical Association, 89:1255-1270,
1994.
[18] Heraz, A., Razaki, R. & Frasson, C., Using machine learning to predict
learner emotional state from brainwaves. 7th IEEE conference on
Advanced Learning Technologies: ICALT 2007, Niigata, Japan, (In
Press).
[19] Kort, B., Reilly, R., & Picard, R., An affective model of interplay
between emotions and learning: Reengineering educational pedagogyÔÇö
building a learning companion. In T. Okamoto, R. Hartley, Kinshuk, &
J. P. Klus (Eds.), Proceedings IEEE International Conference on
Advanced Learning Technology: Issues, Achievements and Challenges
(pp. 43-48). Madison, Wisconsin: IEEE Computer Society, 2001.
[20] Lang, P. J., Behavioral treatment and bio-behavioral assessment:
Computer applications. In J. B. Sidowski, J. H. Johnson, & T. A.
Williams (Eds.), Technology in mental health care delivery) systems (pp.
119-137). Norwood, NJ: Ablex, 1980.
[21] Lang, P.J., Bradley, M.M., & Cuthbert, B.N. International affective
picture system (IAPS): Affective ratings of pictures and instruction
manual. Technical Report A-6. University of Florida, Gainesville, FL,
2005.
[22] Lepper, M. R., & Woolverton, M., The wisdom of practice: Lessons
learned from the study of highly effective tutors. In J. Aronson (Ed.),
Improving academic achievement: Impact of psychological factors on
education (pp. 135-158). Orlando, FL: Academic Press, 2002.
[23] Lester, J. C., Towns, S.G. & FitzGerald, P.J., Achieving affective
impact: visual emotive communication in lifelike pedagogical agents.
The International Journal of Artificial Intelligence in Education, 10(3-4),
278-291, 1999.
[24] Litman, D. J., & Forbes-Riley, K. Predicting student emotions in
computer-human tutoring dialogues. In Proceedings of the 42nd annual
meeting of the association for computational linguistics (pp. 352-359).
East Stroudsburg, PA: Association for Computational Linguistics, 2004.
[25] Mehrabian, A., & Russell, J. A., An approach to environmental
psychology. Cambridge, MA: MIT Press, 1974.
[26] Miserandino, M., Children who do well in school: Individual differences
in perceived competence and autonomy in above-average children.
Journal of Educational Psychology, 88, 203-214, 1996.
[27] Picard, R. W., Affective computing. Cambridge: MIT Press, 1997.
[28] Quinlan, R., C4.5: Programs for Machine Learning. San Mateo, CA:
Morgan Kaufmann Publishers, 1993.
[29] Robson C. Real word research: A resource for social scientist and
practitioner researchers. Oxford: Blackwell, 1993.
[30] Snow, R., Corno, L., & Jackson, D., Individual differences in affective
and cognitive functions. In D. C. Berliner & R. C. Calfee (Eds.),
Handbook of educational psychology (pp. 243-310). New York:
Macmillan, 1996.
[31] Stipek, D., Motivation to Learn: From Theory to Practice 3rd edition.
Boston: Allyn and Bacon, 1998.
[32] Tellegen, A., Structures of mood and personality and their relevance to
assessing anxiety, with an emphasis on self-report. In A. H. Tuma & J.
D. Maser (Eds.), Anxiety and the anxiety disorders (pp. 681-706).
Hillsdale, NJ: Erlbaum, 1985.
[33] Wang, N., Johnson, W.L., Mayer, R., Rizzzo, P., Shaw, E., & Collins,
H., The politeness effect: Pedagogical agents and learning gains. In
Looi, C., McCalla, G., Bredeweg, B., & Breuker, J. (Eds.), Artificial
intelligence in education (pp. 686ÔÇö693). Amsterdam: IOS Press, 2005.
[34] Witten, I.H., and E. Frank, Data Mining: Practical Machine Learning
Tools and Techniques with Java Implementations. Morgan Kaufmann,
San Francisco, 2005.
[35] Wundt, W., Gundriss der Psychologie [Outlines of psychology]. Leipzig,
Germany: Entgelmann, 1896.
[36] Youden. W. J. How to evaluate accuracy. Materials Research and
Standards, ASTM, 1961.
[1] Aha, D., & Kibler, D. Instance-based learning algorithms. Machine
Learning, 6, 37-66, 1991
[2] Anderson T. W. An Introduction to Multivariate Statistical Analysis, 3rd
ed. Wiley, New York, 2003.
[3] Bear, M. F., Connors, B. W., & Paradiso, M. A. Neuroscience:
Exploring the Brain, second ed. Lippincott Williams & Williams,
Baltimore, MD, 2001.
[4] Bellifemine, F., A. Poggi, & G. Rimassa, "JADE - A FIPA-compliant
Agent Framework", PAAM '99, London, UK, 1999, pp. 97ÔÇö108, 1999.
[5] Breazeal, C. Designing sociable robots .Cambridge: MIT Press, 2003.
[6] Breiman L. Bagging predictors. Machine Learning, 24(2):123-140,
1996.
[7] Cantor, D. S. An overview of quantitative EEG and its applications to
neurofeedback. In Introduction to Quantitative EEG and Neurofeedback,
J. R. Evans and A. Abarbanel, Eds. Academic Press, ch. 1, pp. 3-27,
1999.
[8] Conati C., Probabilistic assessment of user's emotions in educational
games. Journal of Applied Artificial Intelligence, 16, 555-575, 2002.
[9] Conati C., How to evaluate models of user affect?. Proceedings of ADS
04, Tutorial and Research Workshop on Affective Dialogue Systems.
Kloster Irsee, Germany, June 2004. p. 288-300, 2004.
[10] Craig, S.D., Graesser, A. C.,Sullins, J., & Gholson, B., Affect and
learning: An exploratory look into the role of affect in learning. Journal
of Educational Media, 29, 241-250, 2004.
[11] D'Mello, S.K., S.D. Craig, B. Gholson, S. Franklin, R.W. Picard, & A.C.
Graesser, "Integrating Affect Sensors in an Intelligent Tutoring System."
/In Affective Interactions: The Computer in the Affective Loop
Workshop at 2005 International conference on Intelligent User
Interfaces,/ AMC Press, New York, pp. 7-13, 2005.
[12] De Vicente, A., & Pain, H., Informing the detection of students'
motivational state : An empirical study. In S.A. Cerri, G. Gouarderes,
and F. Paraguacu (Eds Proceedings of the sixth international conference
on intelligent tutoring systems (pp.933-943). Berlin, Germany: Springer,
2002.
[13] Fan, C., Sarrafzadeh, A., Overmyer, S., Hosseini, H. G., Biglari-Abhari,
M., & Bigdeli, A. A fuzzy approach to facial expression analysis in
intelligent tutoring systems. In Antonio Méndez-Vilas and J.A.Mesa
González(Eds.) Advances in Technology-based Education: Towards a
Knowledge-based Society Vol 3. (pp. 1933-1937). Badajoz, Spain: Junta
De Extremadura, 2003.
[14] Fisher R. A., The use of multiple measurements in taxonomic problems.
Annals of Eugenics, 7:179-188, 1936
[15] Harter, S., A new self-report scale of intrinsic versus extrinsic
orientation in the classroom: Motivation and informational components.
Developmental Psychology, 17, 300-312, 1981.
[16] Hastie T., A. Buja, & R. Tibshirani. Penalized discriminant analysis.
Annals of Statistics, 23:73-102, 1995.
[17] Hastie T., R. Tibshirani, and A. Buja. Flexible discriminant analysis by
optimal scoring. J. American Statistical Association, 89:1255-1270,
1994.
[18] Heraz, A., Razaki, R. & Frasson, C., Using machine learning to predict
learner emotional state from brainwaves. 7th IEEE conference on
Advanced Learning Technologies: ICALT 2007, Niigata, Japan, (In
Press).
[19] Kort, B., Reilly, R., & Picard, R., An affective model of interplay
between emotions and learning: Reengineering educational pedagogyÔÇö
building a learning companion. In T. Okamoto, R. Hartley, Kinshuk, &
J. P. Klus (Eds.), Proceedings IEEE International Conference on
Advanced Learning Technology: Issues, Achievements and Challenges
(pp. 43-48). Madison, Wisconsin: IEEE Computer Society, 2001.
[20] Lang, P. J., Behavioral treatment and bio-behavioral assessment:
Computer applications. In J. B. Sidowski, J. H. Johnson, & T. A.
Williams (Eds.), Technology in mental health care delivery) systems (pp.
119-137). Norwood, NJ: Ablex, 1980.
[21] Lang, P.J., Bradley, M.M., & Cuthbert, B.N. International affective
picture system (IAPS): Affective ratings of pictures and instruction
manual. Technical Report A-6. University of Florida, Gainesville, FL,
2005.
[22] Lepper, M. R., & Woolverton, M., The wisdom of practice: Lessons
learned from the study of highly effective tutors. In J. Aronson (Ed.),
Improving academic achievement: Impact of psychological factors on
education (pp. 135-158). Orlando, FL: Academic Press, 2002.
[23] Lester, J. C., Towns, S.G. & FitzGerald, P.J., Achieving affective
impact: visual emotive communication in lifelike pedagogical agents.
The International Journal of Artificial Intelligence in Education, 10(3-4),
278-291, 1999.
[24] Litman, D. J., & Forbes-Riley, K. Predicting student emotions in
computer-human tutoring dialogues. In Proceedings of the 42nd annual
meeting of the association for computational linguistics (pp. 352-359).
East Stroudsburg, PA: Association for Computational Linguistics, 2004.
[25] Mehrabian, A., & Russell, J. A., An approach to environmental
psychology. Cambridge, MA: MIT Press, 1974.
[26] Miserandino, M., Children who do well in school: Individual differences
in perceived competence and autonomy in above-average children.
Journal of Educational Psychology, 88, 203-214, 1996.
[27] Picard, R. W., Affective computing. Cambridge: MIT Press, 1997.
[28] Quinlan, R., C4.5: Programs for Machine Learning. San Mateo, CA:
Morgan Kaufmann Publishers, 1993.
[29] Robson C. Real word research: A resource for social scientist and
practitioner researchers. Oxford: Blackwell, 1993.
[30] Snow, R., Corno, L., & Jackson, D., Individual differences in affective
and cognitive functions. In D. C. Berliner & R. C. Calfee (Eds.),
Handbook of educational psychology (pp. 243-310). New York:
Macmillan, 1996.
[31] Stipek, D., Motivation to Learn: From Theory to Practice 3rd edition.
Boston: Allyn and Bacon, 1998.
[32] Tellegen, A., Structures of mood and personality and their relevance to
assessing anxiety, with an emphasis on self-report. In A. H. Tuma & J.
D. Maser (Eds.), Anxiety and the anxiety disorders (pp. 681-706).
Hillsdale, NJ: Erlbaum, 1985.
[33] Wang, N., Johnson, W.L., Mayer, R., Rizzzo, P., Shaw, E., & Collins,
H., The politeness effect: Pedagogical agents and learning gains. In
Looi, C., McCalla, G., Bredeweg, B., & Breuker, J. (Eds.), Artificial
intelligence in education (pp. 686ÔÇö693). Amsterdam: IOS Press, 2005.
[34] Witten, I.H., and E. Frank, Data Mining: Practical Machine Learning
Tools and Techniques with Java Implementations. Morgan Kaufmann,
San Francisco, 2005.
[35] Wundt, W., Gundriss der Psychologie [Outlines of psychology]. Leipzig,
Germany: Entgelmann, 1896.
[36] Youden. W. J. How to evaluate accuracy. Materials Research and
Standards, ASTM, 1961.
@article{"International Journal of Information, Control and Computer Sciences:60199", author = "Alicia Heraz and Claude Frasson", title = "Predicting the Three Major Dimensions of the Learner-s Emotions from Brainwaves", abstract = "This paper investigates how the use of machine learning techniques can significantly predict the three major dimensions of learner-s emotions (pleasure, arousal and dominance) from brainwaves. This study has adopted an experimentation in which participants were exposed to a set of pictures from the International Affective Picture System (IAPS) while their electrical brain activity was recorded with an electroencephalogram (EEG). The pictures were already rated in a previous study via the affective rating system Self-Assessment Manikin (SAM) to assess the three dimensions of pleasure, arousal, and dominance. For each picture, we took the mean of these values for all subjects used in this previous study and associated them to the recorded brainwaves of the participants in our study. Correlation and regression analyses confirmed the hypothesis that brainwave measures could significantly predict emotional dimensions. This can be very useful in the case of impassive, taciturn or disabled learners. Standard classification techniques were used to assess the reliability of the automatic detection of learners- three major dimensions from the brainwaves. We discuss the results and the pertinence of such a method to assess learner-s emotions and integrate it into a brainwavesensing Intelligent Tutoring System.
", keywords = "Algorithms, brainwaves, emotional dimensions, performance.", volume = "1", number = "7", pages = "2152-7", }
