Ensuring the Context Validity of English Reading Tests for Academic Purposes (EAP) in Oman

Abstract

Students entering academic programmes are frequently overwhelmed by the demand for extensive reading and comprehension of information derived from multiple and contrasting sources. This entails both careful and expeditious reading. The latter has been generally neglected in research and has not been the focus of many tests. Both types of reading were investigated in this study through a validation process of the summative English reading test for academic purposes taken at the end of the Foundation Programme in Oman. In particular, context validity was established through focusing on the interaction between the linguistic demands and task setting parameters and also the cognitive processes through which the students engaged with the test tasks. To establish the context validity of the test, this study adopted Khalifa and Weir’s (2009) model which not only embraced the complex and multi-componential nature of reading but also provided a workable validation framework. A multi-strategy approach was adopted. A natural experiment utilising Verbal Protocol Analysis captured the cognitive processes through which students engaged in reading. Automated analysis software and opinions of expert judges were used to compare test passages with text extracts drawn from first year academic courses. Correlation tests and factor analysis revealed these cognitive processes and established the robustness of the Khalifa and Weir (2009) model, which was thus validated in a second language context. Passages in the foundation tests were found to be generally representative of academic texts although certain features such as abstractness were under-represented.