Developing an online adaptive testing system for word reading ability

Abstract

During the first years of primary education, language development plays a significant role, as children are unable to perform well in school without suitable reading ability. A paper-and-pencil (PP) diagnostic test system was developed by József Nagy to diagnose children’s word reading ability in the 1990s. With the spread of computers and applications of computer-based (CB) assessment, a number of new opportunities were introduced, making it possible to renew the paper-and-pencil test system by incorporating a number of advantages of online testing, such as objective administration and scoring, rapid response time, immediate feedback, cheaper data collection and adaptive testing. The aim of this paper was to explore and quantify the advantages of electronic testing and study the media effect by making detailed comparisons of test results delivered by different media. In this paper we: (1) outline the original diagnostic assessment system for word reading ability; (2) compare the achievement of children in Years 1 to 5 in PP and CB modes; (3) outline options for improving the system; and (4) represent the renewed online adaptive assessment system for word reading ability. The original PP test system was converted to a CB format. The online test system was delivered via the eDia platform, and data collection was carried out via the Internet using computer facilities available at schools. In the pilot sample 154 primary school children were involved between Years 1–5. A partial credit model was used to scale the items. According to the results, the online version of the test system was suitable for assessing the children’s word reading ability. The person separation reliability of the test was 0.88. The difficulty parameters of the items covered a large scale (349–855), which made the test appropriate for a wide range of abilities. The research represented a promising step towards more precise educational assessment using computerised adaptive testing among young children.