The effectiveness and changeof exploration strategies in an interactive problem-solving environment Logfile analyses

Abstract

Improvements in educational assessment technology have now made it possible to record and analyse not only students’ answers, but also the actions they take during testing. Through the contextual data in logfiles, we can reconstruct everything the studentswere thinking in completing each item on the test. Based on contextual data collected during the research on both the developmental level and development of the interactive problem-solving strategies used by students in Grades 6–8 in the study(n=2226), we analysed the effective and less effective mapping and problem-solving strategies they employedin solving interactive problems. In phase one, we constructed a mathematical model based on theoretically good strategies that led to the solving of minimally complex problems. This model provided an overview that made it possible toanalyse the combination of steps and strategies used.Withthis coding procedure and model, we were able to obtaina precise picture of students’ successful and less successful problem-solving strategies. We have thus succeeded in expanding the range of problem-solving strategies identified thus far in the literature, where the vary-one-thing-at-a-time (VOTAT) strategy has been viewedas the only correct strategy in minimally complexproblem solving. The reliability indices of the test increasesignificantly if we base our determinations not only on students’ specific answers (α=0.72), but also on an analysis of the characteristic contextual data that enable us to reconstruct the problem-solving process more accurately (α=0.91). Despite the fact that an understanding and mapping of minimally complex systems may be achieved with a number of problem-solving strategies, the use of thestrategy of isolated variables, which is grouped amongthe VOTAT strategies–strategies that are discussed with near exclusivity in the literature –has provedthe most effective. The more complex the system that underlies the problem, the greater the gap in performance between the more and less conscious VOTAT strategy users. The analyses elucidatethe students’ relative achievements,which wereclose overall, though they had taken different paths with problem-solving strategies of varying effectiveness. The results have contributed to a more thorough understanding of strategies that are and can be used in an interactive problem-solving environment, and they have shed light once again on the increasingly urgent task of shifting from instruction grounded almost completely in the transmission of knowledgeto an expansion of problem-based teaching that also brings thinking skills into focus.