Expert validation of a qualitative interview protocol for esl instructors’ digital assessment literacy in Malaysian higher education: A Delphi study

Ameera Irdena Binti Suyansah and Noraini binti Said and Esther binti Jawing and Wardatul Akmam Binti Din (2025) Expert validation of a qualitative interview protocol for esl instructors’ digital assessment literacy in Malaysian higher education: A Delphi study. International Journal of Research and Innovation in Social Science (IJRISS), 9. pp. 2605-2618. ISSN 2454-6186

Text FULLTEXT.pdf Restricted to Registered users only Download (518kB) | Request a copy

Abstract

The integration of technology into educational assessment is a critical component of modern pedagogy, yet many educators lack proficiency in digital assessment methodologies. This study addresses the need for a validated instrument to explore the digital assessment literacy (DAL) of English as a Second Language (ESL) instructors in the Malaysian higher education context. The primary objective was to validate a qualitative interview protocol designed to investigate instructors' assessment practices and their level of technology integration. The protocol consists of two instruments, one guided by the Teacher Assessment Literacy in Practice (TALiP) framework and the other by the Substitution, Augmentation, Modification, and Redefinition (SAMR) model. A Delphi method was employed to achieve consensus on the instrument's content validity. Six experts in the fields of digital assessment and ESL, holding PhDs with over five years of relevant experience, were selected to evaluate the interview items. Inter-rater reliability was calculated using the Fleiss' Kappa statistic. The results indicated an "almost perfect" level of agreement among the expert validators, with a Kappa value of 0.81 for the TALiP instrument and 0.83 for the SAMR instrument. Based on expert feedback, several items were refined for clarity, including the separation of double-barreled questions and the rephrasing of complex terminology, which increased the total number of items in the TALiP protocol from 10 to 15. The study concludes that the validated interview protocol is a robust and reliable instrument for exploring the multifaceted dimensions of DAL among ESL instructors. This tool can be effectively utilized by researchers and educational institutions to inform the design of targeted professional development programs aimed at enhancing digital assessment practices in higher education.

Item Type:	Article
Keyword:	Digital Assessment Literacy, Instrument Validation, ESL Studies, Delphi Method, Teacher Assessment Literacy
Subjects:	L Education > LB Theory and practice of education > LB5-3640 Theory and practice of education > LB1555-1602 Elementary or public school education P Language and Literature > P Philology. Linguistics > P1-1091 Philology. Linguistics > P87-96 Communication. Mass media
Department:	FACULTY > Faculty of Education and Sport Studies
Depositing User:	JUNAINE JASNI -
Date Deposited:	31 Oct 2025 17:28
Last Modified:	31 Oct 2025 17:28
URI:	https://eprints.ums.edu.my/id/eprint/45603

Actions (login required)

View Item