Investigating Student Behavior in Adopting Online Formative Assessment Feedback
In this paper we describe one critical research
program within a complex, ongoing multi-year project (2010 to 2014
inclusive) with the overall goal to improve the learning outcomes for
first year undergraduate commerce/business students within an
Information Systems (IS) subject with very large enrolment. The
single research program described in this paper is the analysis of
student attitudes and decision making in relation to the availability of
formative assessment feedback via Web-based real time conferencing
and document exchange software (Adobe Connect). The formative
assessment feedback between teaching staff and students is in respect
of an authentic problem-based, team-completed assignment. The
analysis of student attitudes and decision making is investigated via
both qualitative (firstly) and quantitative (secondly) application of the
Theory of Planned Behavior (TPB) with a two statistically-significant
and separate trial samples of the enrolled students. The initial
qualitative TPB investigation revealed that perceived self-efficacy,
improved time-management, and lecturer-student relationship
building were the major factors in shaping an overall favorable
student attitude to online feedback, whilst some students expressed
valid concerns with perceived control limitations identified within the
online feedback protocols. The subsequent quantitative TPB
investigation then confirmed that attitude towards usage, subjective
norms surrounding usage, and perceived behavioral control of usage
were all significant in shaping student intention to use the online
feedback protocol, with these three variables explaining 63 percent of
the variance in the behavioral intention to use the online feedback
protocol. The identification in this research of perceived behavioral
control as a significant determinant in student usage of a specific
technology component within a virtual learning environment (VLE)
suggests that VLEs could now be viewed not as a single, atomic
entity, but as a spectrum of technology offerings ranging from the
mature and simple (e.g., email, Web downloads) to the cutting-edge
and challenging (e.g., Web conferencing and real-time document
exchange). That is, that all VLEs should not be considered the same.
The results of this research suggest that tertiary students have the
technological sophistication to assess a VLE in this more selective
manner.
[1] Freebody, P., & Luke, A. “Literacies programs: Debates and demands in
cultural context.” Prospect: Australian Journal of TESOL, 5(7), 1990,
pp. 7-16.
[2] Freebody, P., & Luke, A. “Further notes on the Four Resources Model”.
1999 Available online at: http: //www.readingonline.org/research/
lukefreebody.html (last accessed 1st September 2010).
[3] Clutterbuck, P., Seamons, O., & Rowlands, T. “Enhancing Information
Systems Literacy Education via the Four Resources Model”.
Proceedings of the Sixth Education in a Changing Environment (ECE)
Conference, Salford. 2011, UK.
[4] Clutterbuck, P., Seamons, O., & Rowlands, T. “Enhancing Information
Systems Literacy Education via the Four Resources Literacy Education
Model – Further Analysis”, International Journal of Arts & Sciences,
5(2), 2012, pp. 177-194.
[5] Wilson, B. G. “Constructivist Learning Environments: Case Studies in
Instructional Design”, Educational Technology Publications, 1996,
Englewood Cliffs, NJ.
[6] Piccoli, G., Ahmad, R., Ives, B. “Web-Based Virtual Learning
Environments: A Research Framework and a Preliminary Assessment of
Effectiveness in Basic IT Skills Training”, MIS Quarterly, Vol. 25. No.
4, 2001, pp. 402-426.
[7] Keeves, J. P. (1994). “Assessment in schools, methods of assessment”.
In Husen, Torsten, Postlethwaite, & T. Neville (Eds.), 2nd ed, The
international encyclopedia of education, vol. 1, 1994, pp. 362-370.
Oxford: Pergamon Press.
[8] Reeves, T. C., & Hedberg, J. G. “Evaluation strategies for open and
distributed learning environments”. In C. Spratt, & P. Lajbcygier (Eds.),
E-Learning technologies and evidence based assessment approaches,
2009, pp. 234-253, New York: Information Science Reference.
[9] Oosterhof, A., Conrad, R. M., & Ely, D. P. “Assessing learners online”,
2008, New Jersey: Pearson.
[10] Hargreaves, E. “Assessment”. In G. McCulloch, & D. Crook (Eds.), The
Routledge international encyclopedia of education, 2008, pp. 37-38,
New York: Routledge.
[11] Vonderwell, S., Liang, X., & Alderman, K. “Asynchronous discussions
and assessment in online learning”. Journal of Research on Technology
in Education, 39(3), 2007, pp. 309-328.
[12] Black, P., & Wiliam, D. “Developing the theory of formative
assessment”. Educational Assessment, Evaluation and Accountability (formerly the Journal of Personnel Evaluation in Education), 21(1),
2009, pp. 5-31.
[13] Pachler, N., Daly, C., Mor, Y., & Mellar, H. “Formative e-assessment:
Practitioner cases”. Computers & Education, 54, 2010, pp. 715-721.
[14] Gikandi, J. W., Morrow, D., & Davis, N. E. “Online formative
assessment in higher education: A review of the literature”. Computers
& Education, 57, 2011, pp. 2333-2351.
[15] Hyman, P. “In the Year of Disruptive Education”. Communications of
the ACM. 55(12), 2012, 20-22.
[16] Kaufmann, P. B., & Mohan, J. “Video use and higher education: Options
for the future”, 2009, New York University. Available at:
http://library.nyu.edu/about/Video_Use_in_Higher_Education.pdf
[17] Moran, M., Seaman, J., & Tinti-Kane, H. “Teaching, learning, and
sharing: How today’s higher education faculty use social media”, 2011,
Available from: http://www.pearsonlearningsolutions.com/educators/
pearson-social-media-survey-2011-bw.pdf.
[18] Gardner, H. “Multiple intelligences”. New Horizons, 2006, New York:
Basic Books.
[19] Corporation for Public Broadcasting. “Television goes to school: the
impact of video on student learning in formal education”, 2004,
Available from: http://www.dcmp.org/caai/nadh173.pdf.
[20] Holtzblatt, M., & Tschakert, N. “Expanding your accounting classroom
with digital video technology”. Journal of Accounting Education, 29,
2011, pp. 100-121.
[21] Merriam, B. “Qualitative research and case study applications in
education”, 1998, San Francisco: Jossey-Bass.
[22] Travers, M. “Qualitative Research through Case Studies”, 2001, Sage
Publications, London.
[23] Marshall, C., Rossman, G. “Designing Qualitative Research”, 1989,
Sage Publications, California.
[24] Glaser, B.G., Strauss, A.L. “The Discovery of grounded theory:
strategies for qualitative research”. Aldine Publishing Company, 1967,
Chicago.
[25] Ajzen, I. “The Theory of Planned Behavior”, Organizational Behavior
and Human Decision Processes, Vol. 50, 1991, pp. 179-211.
[26] Kline, R.B. “Principles and Practice of Structural Equation Modeling,
2nd ed., 2005, Guilford Press, New York, NY.
[27] Fornell, C. and Larcker, D.F. “Evaluating structural equation models
with unobservable variables and measurement error”, Journal of
Marketing Research, Vol. 28, No. 1, 1981, pp. 39-50.
[28] Hair, J.F. Jr, Black, W.C., Babin, B.J., Anderson, R.E. and Tatham, R.L.
“Multivariate Data Analysis, 6th ed., 2006, Prentice-Hall International,
Englewood Cliffs, NJ.
[29] Nunnally, J.C. and Bernstein, I.H. “Psychometric Theory, 1994,
McGraw-Hill, New York, NY.
[30] Armitage, C.J. and Conner, M. “Efficacy of the Theory of Planned
Behavior, a meta-analysis review”, British Journal of Social Psychology,
Vo. 40, No. 4, 2001, pp. 471-499.
[31] Compeau, D.R. and Higgins, C.A. “Computer self-efficacy:
development of a measure and initial test”, MIS Quarterly, Vol. 19, No.
2, 1995, pp. 189-211.
[32] Teo, T., and Lee, C. B. “Explaining the intention to use technology
among student teachers – an application of the Theory of Planned
Behavior (TPB)”, Campus Wide Information Systems, 2010, Vol. 27,
No. 2, pp. 60-67.
[33] Larreamendy-Joerns, J., & Leinhardt, G. “Going the distance with online
education”, Review of Educational Research, 76(4), 2006, pp. 567-605.
[1] Freebody, P., & Luke, A. “Literacies programs: Debates and demands in
cultural context.” Prospect: Australian Journal of TESOL, 5(7), 1990,
pp. 7-16.
[2] Freebody, P., & Luke, A. “Further notes on the Four Resources Model”.
1999 Available online at: http: //www.readingonline.org/research/
lukefreebody.html (last accessed 1st September 2010).
[3] Clutterbuck, P., Seamons, O., & Rowlands, T. “Enhancing Information
Systems Literacy Education via the Four Resources Model”.
Proceedings of the Sixth Education in a Changing Environment (ECE)
Conference, Salford. 2011, UK.
[4] Clutterbuck, P., Seamons, O., & Rowlands, T. “Enhancing Information
Systems Literacy Education via the Four Resources Literacy Education
Model – Further Analysis”, International Journal of Arts & Sciences,
5(2), 2012, pp. 177-194.
[5] Wilson, B. G. “Constructivist Learning Environments: Case Studies in
Instructional Design”, Educational Technology Publications, 1996,
Englewood Cliffs, NJ.
[6] Piccoli, G., Ahmad, R., Ives, B. “Web-Based Virtual Learning
Environments: A Research Framework and a Preliminary Assessment of
Effectiveness in Basic IT Skills Training”, MIS Quarterly, Vol. 25. No.
4, 2001, pp. 402-426.
[7] Keeves, J. P. (1994). “Assessment in schools, methods of assessment”.
In Husen, Torsten, Postlethwaite, & T. Neville (Eds.), 2nd ed, The
international encyclopedia of education, vol. 1, 1994, pp. 362-370.
Oxford: Pergamon Press.
[8] Reeves, T. C., & Hedberg, J. G. “Evaluation strategies for open and
distributed learning environments”. In C. Spratt, & P. Lajbcygier (Eds.),
E-Learning technologies and evidence based assessment approaches,
2009, pp. 234-253, New York: Information Science Reference.
[9] Oosterhof, A., Conrad, R. M., & Ely, D. P. “Assessing learners online”,
2008, New Jersey: Pearson.
[10] Hargreaves, E. “Assessment”. In G. McCulloch, & D. Crook (Eds.), The
Routledge international encyclopedia of education, 2008, pp. 37-38,
New York: Routledge.
[11] Vonderwell, S., Liang, X., & Alderman, K. “Asynchronous discussions
and assessment in online learning”. Journal of Research on Technology
in Education, 39(3), 2007, pp. 309-328.
[12] Black, P., & Wiliam, D. “Developing the theory of formative
assessment”. Educational Assessment, Evaluation and Accountability (formerly the Journal of Personnel Evaluation in Education), 21(1),
2009, pp. 5-31.
[13] Pachler, N., Daly, C., Mor, Y., & Mellar, H. “Formative e-assessment:
Practitioner cases”. Computers & Education, 54, 2010, pp. 715-721.
[14] Gikandi, J. W., Morrow, D., & Davis, N. E. “Online formative
assessment in higher education: A review of the literature”. Computers
& Education, 57, 2011, pp. 2333-2351.
[15] Hyman, P. “In the Year of Disruptive Education”. Communications of
the ACM. 55(12), 2012, 20-22.
[16] Kaufmann, P. B., & Mohan, J. “Video use and higher education: Options
for the future”, 2009, New York University. Available at:
http://library.nyu.edu/about/Video_Use_in_Higher_Education.pdf
[17] Moran, M., Seaman, J., & Tinti-Kane, H. “Teaching, learning, and
sharing: How today’s higher education faculty use social media”, 2011,
Available from: http://www.pearsonlearningsolutions.com/educators/
pearson-social-media-survey-2011-bw.pdf.
[18] Gardner, H. “Multiple intelligences”. New Horizons, 2006, New York:
Basic Books.
[19] Corporation for Public Broadcasting. “Television goes to school: the
impact of video on student learning in formal education”, 2004,
Available from: http://www.dcmp.org/caai/nadh173.pdf.
[20] Holtzblatt, M., & Tschakert, N. “Expanding your accounting classroom
with digital video technology”. Journal of Accounting Education, 29,
2011, pp. 100-121.
[21] Merriam, B. “Qualitative research and case study applications in
education”, 1998, San Francisco: Jossey-Bass.
[22] Travers, M. “Qualitative Research through Case Studies”, 2001, Sage
Publications, London.
[23] Marshall, C., Rossman, G. “Designing Qualitative Research”, 1989,
Sage Publications, California.
[24] Glaser, B.G., Strauss, A.L. “The Discovery of grounded theory:
strategies for qualitative research”. Aldine Publishing Company, 1967,
Chicago.
[25] Ajzen, I. “The Theory of Planned Behavior”, Organizational Behavior
and Human Decision Processes, Vol. 50, 1991, pp. 179-211.
[26] Kline, R.B. “Principles and Practice of Structural Equation Modeling,
2nd ed., 2005, Guilford Press, New York, NY.
[27] Fornell, C. and Larcker, D.F. “Evaluating structural equation models
with unobservable variables and measurement error”, Journal of
Marketing Research, Vol. 28, No. 1, 1981, pp. 39-50.
[28] Hair, J.F. Jr, Black, W.C., Babin, B.J., Anderson, R.E. and Tatham, R.L.
“Multivariate Data Analysis, 6th ed., 2006, Prentice-Hall International,
Englewood Cliffs, NJ.
[29] Nunnally, J.C. and Bernstein, I.H. “Psychometric Theory, 1994,
McGraw-Hill, New York, NY.
[30] Armitage, C.J. and Conner, M. “Efficacy of the Theory of Planned
Behavior, a meta-analysis review”, British Journal of Social Psychology,
Vo. 40, No. 4, 2001, pp. 471-499.
[31] Compeau, D.R. and Higgins, C.A. “Computer self-efficacy:
development of a measure and initial test”, MIS Quarterly, Vol. 19, No.
2, 1995, pp. 189-211.
[32] Teo, T., and Lee, C. B. “Explaining the intention to use technology
among student teachers – an application of the Theory of Planned
Behavior (TPB)”, Campus Wide Information Systems, 2010, Vol. 27,
No. 2, pp. 60-67.
[33] Larreamendy-Joerns, J., & Leinhardt, G. “Going the distance with online
education”, Review of Educational Research, 76(4), 2006, pp. 567-605.
@article{"International Journal of Business, Human and Social Sciences:69093", author = "Peter Clutterbuck and Terry Rowlands and Owen Seamons", title = "Investigating Student Behavior in Adopting Online Formative Assessment Feedback ", abstract = "In this paper we describe one critical research
program within a complex, ongoing multi-year project (2010 to 2014
inclusive) with the overall goal to improve the learning outcomes for
first year undergraduate commerce/business students within an
Information Systems (IS) subject with very large enrolment. The
single research program described in this paper is the analysis of
student attitudes and decision making in relation to the availability of
formative assessment feedback via Web-based real time conferencing
and document exchange software (Adobe Connect). The formative
assessment feedback between teaching staff and students is in respect
of an authentic problem-based, team-completed assignment. The
analysis of student attitudes and decision making is investigated via
both qualitative (firstly) and quantitative (secondly) application of the
Theory of Planned Behavior (TPB) with a two statistically-significant
and separate trial samples of the enrolled students. The initial
qualitative TPB investigation revealed that perceived self-efficacy,
improved time-management, and lecturer-student relationship
building were the major factors in shaping an overall favorable
student attitude to online feedback, whilst some students expressed
valid concerns with perceived control limitations identified within the
online feedback protocols. The subsequent quantitative TPB
investigation then confirmed that attitude towards usage, subjective
norms surrounding usage, and perceived behavioral control of usage
were all significant in shaping student intention to use the online
feedback protocol, with these three variables explaining 63 percent of
the variance in the behavioral intention to use the online feedback
protocol. The identification in this research of perceived behavioral
control as a significant determinant in student usage of a specific
technology component within a virtual learning environment (VLE)
suggests that VLEs could now be viewed not as a single, atomic
entity, but as a spectrum of technology offerings ranging from the
mature and simple (e.g., email, Web downloads) to the cutting-edge
and challenging (e.g., Web conferencing and real-time document
exchange). That is, that all VLEs should not be considered the same.
The results of this research suggest that tertiary students have the
technological sophistication to assess a VLE in this more selective
manner.
", keywords = "Formative assessment feedback, virtual learning
environment, theory of planned behavior, perceived behavioral
control.", volume = "9", number = "1", pages = "328-8", }
