Measuring the effectiveness of browsing the internet among sixth-to eleventh-grade students

Abstract

Information and communication technologies (ICTs) have become increasingly common in community settings, at school and at home; thus, ICT literacy has had a growing importance in the well-being of societies around the world. ICT literacy cannot be defined primarily as the mastery of technical skills. The focus of the present study is the measurement of its most fundamental component in the 21st century, the effectiveness of an information search in reallife scenarios, using a third-generation test in an educational context. The sample consisted of sixth- to eleventh-grade students (N=10,064) from 150 primary and secondary schools in Hungary. A complex online platform, eDia, was used to support the entire assessment process from third-generation item writing through test delivery and analysing the data to providing automatic feedback. Cronbach’s α was used to describe the reliability of the 44-item instrument, the functions of item response theory were used to visualize the appropriateness of the test from the point of view of difficulty, and structural equation modelling was employed to check the dimensionality of the measured construct. The internal consistency of the test was high (α=,904). Students’ mean performance varied between 41,6% (SD=16,13) for Grade 6 and 72,4% (SD=16.1) for Grade 11. Across all grades, development of the accessing information component of ICT literacy was significant. On average, students’ ability levels grew by one-third standard deviation per year, suggesting a more rapid development than had been experienced in other abilities not explicitly improved at school. The development showed a regular developmental tendency and could be identified with a logistic curve. According to item/person maps, the test had the right range of task difficulty for measuring sixth- to eleventh-grade students’ ICT literacy in an educational context. Results from structural equation models (CFI=0,939, TLI=0,935, RMSEA=0,037) confirmed that the accessing information component of ICT literacy can be described as a two-dimensional construct, distinguishing a search for easily accessible information and one requiring more complex operations. The results suggested that the improved third-generation instrument, which measures the assessing information component of ICT literacy within a broad age range within a 40-minute period, is suited to a national educational environment.