Do C-Test and Cloze Procedure Measure what they Purport to be Measuring? A Case of Criterion-Related Validity
This article investigated the validity of C-test and Cloze test which purport to measure general English proficiency. To provide empirical evidence pertaining to the validity of the interpretations based on the results of these integrative language tests, their criterion-related validity was investigated. In doing so, the test of English as a foreign language (TOEFL) which is an established, standardized, and internationally administered test of general English proficiency was used as the criterion measure. Some 90 Iranian English majors participated in this study. They were seniors studying English at a university in Tehran, Iran. The results of analyses showed that there is a statistically significant correlation among participants- scores on Cloze test, C-test, and the TOEFL. Building on the findings of the study and considering criterion-related validity as the evidential basis of the validity argument, it was cautiously deducted that these tests measure the same underlying trait. However, considering the limitations of using criterion measures to validate tests, no absolute claims can be made as to the construct validity of these integrative tests.
[1] J. W. Oller, Language tests at School: A Pragmatic Approach,
London: Longman, 1979.
[2] A. D. Cohen, "On taking tests: what the students report,"
Language Testing, vol. 1, pp. 70-81, 1984.
[3] C. J. Weir, Communicative Language Testing, Hemel
Hempstead: Prentice Hall, Inc., 1998.
[4] J. D. Brown, "Correlational study of four methods for scoring
Cloze tests," Unpublished master-s thesis, University of
California, USA, 1978.
[5] J. W. Oller, and J. Jonz, Eds. Cloze and Coherence, Cranbury,
NY: Bucknell University Press, 1994.
[6] M. R. Sampson, and L. D. Briggs, "A new technique for Cloze
scoring: A semantically consistent method," Clearing House, vol.
57, no. 4, pp. 177-79, 1983.
[7] M. Pikulski, and A. W. Tobin, "Cloze procedure as an informal
assessment technique," In J. J. Rikulsky and T. Shanahan, Eds.
Approaches to the informal evaluation of reading, Newark:
International Reading Association, 1982.
[8] J. C. Alderson, "The effect on the Cloze test of changes in
deletion frequency," Journal of Research in Reading, vol. 2, pp.
108-18, 1979.
[9] A. G. Sciarone, and J. J. Schrool, "The Cloze test: or why small
isn-t always beautiful," Language Learning, vol. 39, no. 3, 415-
38, 1989.
[10] M. Soudek, and L. Soudek, "Cloze after thirty years: New uses in
language teaching," ELT Journal, Vol. 37, no. 4, pp. 335-40,
1983.
[11] J. C. Alderson, "The effect of certain methodological variables on
Cloze test performance and its implications for the use of Cloze
procedure in EFL testing," Paper presented at the fifth
International Congress of Applied Linguistics, Montreal, 1978.
[12] J. C. Alderson, "The Cloze Procedure and Proficiency in English
as a Foreign Language," In J. W. Jr. Oller, Ed. Issues in Language
Testing Research, Rowly, Mass: Newbury House, 1983, pp. 205-
217.
[13] J. D. Brown, "A closer look at Cloze: validity and reliability," In J.
W. Oller, Ed. Issues in Language Testing Research, Rowley, Mass:
Newbury House, 1983, pp. 237-250.
[14] J. B. Heaton, Writing English Language Tests. London: Longman,
1990.
[15] J. W. Oller, "Cloze tests of second language proficiency and what
they measure," .Language Learning, vol. 23, pp. 101-105, 1973.
[16] P. Irvine, P. Atai, and J.W. Oller, "Cloze, dictation, and the Test of
English as a Foreign Language," Language Learning, vol. 24, pp.
245-52, 1974.
[17] J. B. Stubbs, and G. R. Tucker, "The Cloze test as a measure of
English proficiency," The Modern Language Journal, vol. 58, pp.
239-41, 1974.
[18] K. G. Aitken, "Using Cloze procedure as an overall language
proficiency test," TESOL Quarterly, vol. 11, pp. 59-68, 1977.
[19] C. Klein-Braley, and U. Raatz, "A survey of research on the C-test,"
Language Testing, vol. 1, no. 2, pp. 134-46, 1984.
[20] U. Raatz, and C. Klein-Braley, "Ein neuer Ansatz zur Messung der
Sprachleistung. Der C-test: Theorie und Praxis," In R. Horn, K.
Ingencamp, and R. Jager, Eds. Tests und Trends, Jahrbuch der
Padagogischen Diaanostik. Weinheim: Beltz: 1983, 107-38.
[21] C. J. Weir, Understanding and Developing Language Tests, Prentice
Hall, Inc., 1993.
[22] A. Cohen, M. Segal, and R. Weiss, "The C-test in Hebrew,"
Language Testing, vol. 1, pp. 221-225, 1984.
[23] R. Grotjahn, "Test validation and cognitive psychology: Some
methodological considerations," Language Testing, vol. 3, pp. 159-
185, 1986.
[24] R. Grotjahn, "How to construct and evaluate a C-test: A discussion
of some problems and some statistical analyses," In: R. Grotjahn, C.
Klein-Braley, and D. K. Stevenson, Eds. Taking their Measure: The
Validity and Validation of Language Tests, Brockmeyer, Bochum,
Germany, 1987, pp. 219- 253.
[25] R. Grotjahn, Ed. "Der C-test: Theoretische Grundlagen und
praktische Anwendungen," AKS-Verlag, Bochum, 2002.
[26] U. Feldmann, and B. Stemmer, "Thin____ aloud a____ retrospective
da____ in c-te___ taking: Diffe______languages - diff_____
learners - sa____ approaches?" In: C. Faerch, C. Kasper, Eds.
Introspection in Second Language Research, Clevedon: Multilingual
Matters, 1987.
[27] C. Cleary, "The C-test in English: Left-hand deletions," RELC
Journal, vol. 19, pp. 26-38, 1988.
[28] C. Chappelle, and R. Abraham, "Cloze method: What difference
does it make?" Language Testing, vol. 7, pp. 121-146, 1990.
[29] M. Hood, "The C-test: A viable alternative to the use of the cloze
procedure in testing?" In L. Arena, Ed. Language Proficiency.
Plenum Press, New York, 1990, pp. 173-189.
[30] Z. Dornyei, and L. Katona, "Validation of C-test among
Hungarian EFL learner," Language Testing, vol. 9, pp. 187-206,
1992.
[31] T. Kamimoto, "An inquiry into what a C-test measures," Fukuoka
Women-s Jr. College Studies, vol. 44, pp. 67-79, 1992.
[32] T. Kamimoto, "Tailoring the test to fit the students: Improvement
of the C-test through classical item analysis," Fukuoka Women-s
Junior College Studies, vol. 30, pp. 47-61, 1993.
[33] A. Jafarpur, "Is C-test superior to Cloze?" Language Testing, vol.
12, pp. 194-216, 1995.
[34] A. Jafarpur, "Can the C-test be improved with classical item
analysis?" System, vol. 27, pp. 79-89, 1999.
[35] G. Sigott, J. Kobrel, "Deletion patterns and C-test difficulty
across languages," In: R. Grotjahn, Ed. Der C-test, Theoretische
grundlagen und praktische anwwndungen, vol. 3, pp. 159-172,
Brockmeyer, Bochum, 1996.
[36] C. Klein-Braley, "C-tests in the context of reduced redundancy
testing: An appraisal," Language Testing, vol. 14, pp. 47-84,
1997.
[37] C. Ikeguchi, "Do different C-tests discriminate proficiency levels
of EL2 learners?" JALT Testing & Evaluation SIG Newsletter,
vol. 2, pp. 3-8, 1998.
[38] E. Babaii, and H. Ansary, "The C-test: a valid operationalization
of reduced redundancy principle?" vol. 29, pp. 209-219, 2001.
[39] A. J. Hastings, "The Focal Skills Approach: an Assessment," In F.
Ekman, et al., Eds. Second Language acquisition Theory and
Pedagogy, Hillsdale, NJ: Lawrence Erlbaum, 1995, pp. 29-43.
[40] R. J. Mislevy, "Test theory reconceived," Journal of Educational
Measurement, vol. 33, no. 4, pp. 379-416, 1996.
[41] L. F. Bachman, Statistical Analysis for Language Assessment,
Cambridge: Cambridge University Press, 2003.
[42] A. Hughes, Testing for language teachers, 1st ed. Cambridge:
Cambridge University Press, 1989.
[43] G. Henning, A guide to language testing. Cambridge, MA: Newbury
House, 1987.
[44] L. F. Bachman, Fundamental Considerations in Language Testing,
Oxford: Oxford University Press, 1990.
[45] P. Groot, "Language testing in research and education: The need for
standards," In J. De Jong, Ed. Standardization in Language Testing,
AILA Review, vol. 7, pp. 9-23, 1990.
[46] M. T. Kane, "An argument-based approach to validity,"
Psychological Bulletin, vol. 122, no. 3, pp. 527-35, 1992.
[47] E. Hatch, and H. Farhady, Research Design and Statistics for
Applied Linguistics. Rowley, Massachusetts: Newbury House, 1982.
[1] J. W. Oller, Language tests at School: A Pragmatic Approach,
London: Longman, 1979.
[2] A. D. Cohen, "On taking tests: what the students report,"
Language Testing, vol. 1, pp. 70-81, 1984.
[3] C. J. Weir, Communicative Language Testing, Hemel
Hempstead: Prentice Hall, Inc., 1998.
[4] J. D. Brown, "Correlational study of four methods for scoring
Cloze tests," Unpublished master-s thesis, University of
California, USA, 1978.
[5] J. W. Oller, and J. Jonz, Eds. Cloze and Coherence, Cranbury,
NY: Bucknell University Press, 1994.
[6] M. R. Sampson, and L. D. Briggs, "A new technique for Cloze
scoring: A semantically consistent method," Clearing House, vol.
57, no. 4, pp. 177-79, 1983.
[7] M. Pikulski, and A. W. Tobin, "Cloze procedure as an informal
assessment technique," In J. J. Rikulsky and T. Shanahan, Eds.
Approaches to the informal evaluation of reading, Newark:
International Reading Association, 1982.
[8] J. C. Alderson, "The effect on the Cloze test of changes in
deletion frequency," Journal of Research in Reading, vol. 2, pp.
108-18, 1979.
[9] A. G. Sciarone, and J. J. Schrool, "The Cloze test: or why small
isn-t always beautiful," Language Learning, vol. 39, no. 3, 415-
38, 1989.
[10] M. Soudek, and L. Soudek, "Cloze after thirty years: New uses in
language teaching," ELT Journal, Vol. 37, no. 4, pp. 335-40,
1983.
[11] J. C. Alderson, "The effect of certain methodological variables on
Cloze test performance and its implications for the use of Cloze
procedure in EFL testing," Paper presented at the fifth
International Congress of Applied Linguistics, Montreal, 1978.
[12] J. C. Alderson, "The Cloze Procedure and Proficiency in English
as a Foreign Language," In J. W. Jr. Oller, Ed. Issues in Language
Testing Research, Rowly, Mass: Newbury House, 1983, pp. 205-
217.
[13] J. D. Brown, "A closer look at Cloze: validity and reliability," In J.
W. Oller, Ed. Issues in Language Testing Research, Rowley, Mass:
Newbury House, 1983, pp. 237-250.
[14] J. B. Heaton, Writing English Language Tests. London: Longman,
1990.
[15] J. W. Oller, "Cloze tests of second language proficiency and what
they measure," .Language Learning, vol. 23, pp. 101-105, 1973.
[16] P. Irvine, P. Atai, and J.W. Oller, "Cloze, dictation, and the Test of
English as a Foreign Language," Language Learning, vol. 24, pp.
245-52, 1974.
[17] J. B. Stubbs, and G. R. Tucker, "The Cloze test as a measure of
English proficiency," The Modern Language Journal, vol. 58, pp.
239-41, 1974.
[18] K. G. Aitken, "Using Cloze procedure as an overall language
proficiency test," TESOL Quarterly, vol. 11, pp. 59-68, 1977.
[19] C. Klein-Braley, and U. Raatz, "A survey of research on the C-test,"
Language Testing, vol. 1, no. 2, pp. 134-46, 1984.
[20] U. Raatz, and C. Klein-Braley, "Ein neuer Ansatz zur Messung der
Sprachleistung. Der C-test: Theorie und Praxis," In R. Horn, K.
Ingencamp, and R. Jager, Eds. Tests und Trends, Jahrbuch der
Padagogischen Diaanostik. Weinheim: Beltz: 1983, 107-38.
[21] C. J. Weir, Understanding and Developing Language Tests, Prentice
Hall, Inc., 1993.
[22] A. Cohen, M. Segal, and R. Weiss, "The C-test in Hebrew,"
Language Testing, vol. 1, pp. 221-225, 1984.
[23] R. Grotjahn, "Test validation and cognitive psychology: Some
methodological considerations," Language Testing, vol. 3, pp. 159-
185, 1986.
[24] R. Grotjahn, "How to construct and evaluate a C-test: A discussion
of some problems and some statistical analyses," In: R. Grotjahn, C.
Klein-Braley, and D. K. Stevenson, Eds. Taking their Measure: The
Validity and Validation of Language Tests, Brockmeyer, Bochum,
Germany, 1987, pp. 219- 253.
[25] R. Grotjahn, Ed. "Der C-test: Theoretische Grundlagen und
praktische Anwendungen," AKS-Verlag, Bochum, 2002.
[26] U. Feldmann, and B. Stemmer, "Thin____ aloud a____ retrospective
da____ in c-te___ taking: Diffe______languages - diff_____
learners - sa____ approaches?" In: C. Faerch, C. Kasper, Eds.
Introspection in Second Language Research, Clevedon: Multilingual
Matters, 1987.
[27] C. Cleary, "The C-test in English: Left-hand deletions," RELC
Journal, vol. 19, pp. 26-38, 1988.
[28] C. Chappelle, and R. Abraham, "Cloze method: What difference
does it make?" Language Testing, vol. 7, pp. 121-146, 1990.
[29] M. Hood, "The C-test: A viable alternative to the use of the cloze
procedure in testing?" In L. Arena, Ed. Language Proficiency.
Plenum Press, New York, 1990, pp. 173-189.
[30] Z. Dornyei, and L. Katona, "Validation of C-test among
Hungarian EFL learner," Language Testing, vol. 9, pp. 187-206,
1992.
[31] T. Kamimoto, "An inquiry into what a C-test measures," Fukuoka
Women-s Jr. College Studies, vol. 44, pp. 67-79, 1992.
[32] T. Kamimoto, "Tailoring the test to fit the students: Improvement
of the C-test through classical item analysis," Fukuoka Women-s
Junior College Studies, vol. 30, pp. 47-61, 1993.
[33] A. Jafarpur, "Is C-test superior to Cloze?" Language Testing, vol.
12, pp. 194-216, 1995.
[34] A. Jafarpur, "Can the C-test be improved with classical item
analysis?" System, vol. 27, pp. 79-89, 1999.
[35] G. Sigott, J. Kobrel, "Deletion patterns and C-test difficulty
across languages," In: R. Grotjahn, Ed. Der C-test, Theoretische
grundlagen und praktische anwwndungen, vol. 3, pp. 159-172,
Brockmeyer, Bochum, 1996.
[36] C. Klein-Braley, "C-tests in the context of reduced redundancy
testing: An appraisal," Language Testing, vol. 14, pp. 47-84,
1997.
[37] C. Ikeguchi, "Do different C-tests discriminate proficiency levels
of EL2 learners?" JALT Testing & Evaluation SIG Newsletter,
vol. 2, pp. 3-8, 1998.
[38] E. Babaii, and H. Ansary, "The C-test: a valid operationalization
of reduced redundancy principle?" vol. 29, pp. 209-219, 2001.
[39] A. J. Hastings, "The Focal Skills Approach: an Assessment," In F.
Ekman, et al., Eds. Second Language acquisition Theory and
Pedagogy, Hillsdale, NJ: Lawrence Erlbaum, 1995, pp. 29-43.
[40] R. J. Mislevy, "Test theory reconceived," Journal of Educational
Measurement, vol. 33, no. 4, pp. 379-416, 1996.
[41] L. F. Bachman, Statistical Analysis for Language Assessment,
Cambridge: Cambridge University Press, 2003.
[42] A. Hughes, Testing for language teachers, 1st ed. Cambridge:
Cambridge University Press, 1989.
[43] G. Henning, A guide to language testing. Cambridge, MA: Newbury
House, 1987.
[44] L. F. Bachman, Fundamental Considerations in Language Testing,
Oxford: Oxford University Press, 1990.
[45] P. Groot, "Language testing in research and education: The need for
standards," In J. De Jong, Ed. Standardization in Language Testing,
AILA Review, vol. 7, pp. 9-23, 1990.
[46] M. T. Kane, "An argument-based approach to validity,"
Psychological Bulletin, vol. 122, no. 3, pp. 527-35, 1992.
[47] E. Hatch, and H. Farhady, Research Design and Statistics for
Applied Linguistics. Rowley, Massachusetts: Newbury House, 1982.
@article{"International Journal of Business, Human and Social Sciences:61258", author = "Masoud Saeedi and Mansour Tavakoli and Shirin Rahimi Kazerooni and Vahid Parvaresh", title = "Do C-Test and Cloze Procedure Measure what they Purport to be Measuring? A Case of Criterion-Related Validity", abstract = "This article investigated the validity of C-test and Cloze test which purport to measure general English proficiency. To provide empirical evidence pertaining to the validity of the interpretations based on the results of these integrative language tests, their criterion-related validity was investigated. In doing so, the test of English as a foreign language (TOEFL) which is an established, standardized, and internationally administered test of general English proficiency was used as the criterion measure. Some 90 Iranian English majors participated in this study. They were seniors studying English at a university in Tehran, Iran. The results of analyses showed that there is a statistically significant correlation among participants- scores on Cloze test, C-test, and the TOEFL. Building on the findings of the study and considering criterion-related validity as the evidential basis of the validity argument, it was cautiously deducted that these tests measure the same underlying trait. However, considering the limitations of using criterion measures to validate tests, no absolute claims can be made as to the construct validity of these integrative tests.
", keywords = "Integrative testing, C-test, Cloze test, theTOEFL, Validity.", volume = "5", number = "2", pages = "210-10", }
