Assessing Content Validity and Thematic Trends in CSS English Literature Question Papers: A Critical Study

Abstract

This study examines the effectiveness of English literature question papers of Pakistan’s Central Superior Services (CSS) examination from 2019 to 2023. The analysis is limited to literature papers, excluding exams from other disciplines. Designing assessments is crucial to ensuring test validity. Particularly, literature exams depend on the clarity and alignment of tasks. A qualitative research approach is utilized to evaluate the effectiveness of tests. The study is based on the Test Usefulness Model of Bachman and Palmer’s (1996) and Shohamy’s (2020) critical viewpoint on language testing, which focuses on reliability, validity, authenticity, and syllabus alignment. The study uses 50 exam papers, 10 from each year, obtained from the CSS website. The results indicate issues such as unclear instructions, inconsistent tasks, uneven difficulty levels, and misalignment of the syllabus. Language testing is an underexplored area of the CSS exam. Resultantly, clearer tasks and regular syllabus updates are required to improve exam design.