This poster presents standardisation, reliability, and validity evidence for a digital cognitive and metacognitive dynamic assessment tool for students in grades 2–9 (N=218). The tool integrates analogical reasoning and working memory in a dynamic pre–post design, attention regulation via Response Time (RT)-interference, and a 12-item metacognitive self-report scale. Reliability was good to excellent (self-report α=.864, ω=.866; analogical reasoning pretest split-half=.907; working-memory pretest split-half=.809; RT-interference split-half=.873). Criterion validity against teacher ratings showed coherent patterns. Static measures (pre-test scores) correlated moderately with mathematics and language (analogical reasoning–mathematics ρ=.367; working memory–mathematics ρ=.266; analogical reasoning–language ρ=.334; working memory–language ρ=.335; all p<.001). Dynamic responsiveness measures (RG-residual gain calculated on pretest–posttest differences) showed a differentiated pattern. Working-memory responsiveness correlated significantly with mathematics (ρ=.227, p=.002) and language (ρ=.316, p<.001), whereas analogical-reasoning responsiveness was weaker (mathematics ρ=.131, p=.079; language ρ=.187, p=.012). Surprisingly, RT-interference showed weak, non-significant associations with mathematics (ρ=−.116, p=.136) and language (ρ=−.149, p=.054). Stepwise models indicated that metacognitive self-report added unique explanatory value beyond cognitive performance for mathematics (ΔR²≈.09; β≈.29; p<.001), language (ΔR²≈.07; β≈.23; p<.001), and support needs (ΔAIC=−7.4; OR≈.62). Overall, combining dynamic cognitive measures with metacognitive self-report yields more informative assessment profiles than cognitive performance alone. Across grades, findings support educational interpretation and practical decision-making.
Extended Summary:Dynamic assessment (DA) captures students' responsiveness to cognitive strategy support beyond static ability indicators (Embretson, 1992; Grigorenko & Sternberg, 1998; Resing et al., 2012), while metacognition research reveals complex relationships with academic achievement (Dent & Koenka, 2016; Ohtani & Hisasaka, 2018). Schools require psychometrically sound screening tools providing nuanced, learning-relevant profiles rather than single performance indices. This poster reports standardisation, reliability, and validity of a digital cognitive and metacognitive screening tool for grades 2–9.
The tool integrates four components: analogical reasoning and working memory in dynamic pre–post format, attention regulation via response-time (RT) interference, and a 12-item metacognitive self-report scale targeting planning, monitoring, and learning-related self-regulatory beliefs. The design views learning as jointly shaped by cognitive capacity, responsiveness to support, and metacognitive regulation—theoretically related but empirically distinguishable.
Standardisation used a stratified sample of 218 students in grades 2–9 from Swedish compulsory schools. Trained special education teachers collected data, establishing year-grade–specific norms for all performance-based measures. Reliability analyses showed good to excellent internal consistency and split-half estimates. Metacognitive self-report demonstrated high internal consistency (α = .864; ω = .866). Analogical reasoning pretest showed strong split-half reliability (.907), working-memory pretest acceptable to good reliability (.809), and attention regulation high reliability (.873), supporting its use as a stable regulatory indicator.
Criterion validity was examined against teacher ratings of curriculum goal attainment in mathematics and language (Swedish). Correlations examined pre-test scores (static cognitive level) and residual gain scores (responsiveness to brief strategy instruction). Static measures showed coherent, theoretically expected patterns: analogical reasoning and working memory pre-tests were moderately related to academic outcomes (analogical reasoning–mathematics ρ = .367, p < .001; working memory–mathematics ρ = .266, p < .001; analogical reasoning–language ρ = .334, p < .001; working memory–language ρ = .335, p < .001).
For responsiveness measures (RG-residual gain from pre–post differences), a differentiated pattern emerged. Working-memory responsiveness correlated significantly with mathematics (ρ = .227, p = .002) and language (ρ = .316, p < .001), suggesting students benefiting more from brief working-memory strategy support tend to perform better academically. Analogical-reasoning responsiveness showed weaker associations with mathematics (ρ = .131, p = .079) and modest association with language (ρ = .187, p = .012). RT-interference showed weak, non-significant associations with academic ratings (mathematics ρ = −.116, p = .136; language ρ = −.149, p = .054), suggesting attention regulation as operationalised here is largely dissociable from academic performance, though operationalisation limitations cannot be excluded.
To evaluate metacognitive self-report's added value beyond cognitive performance, stepwise models were estimated. Metacognitive self-report contributed unique explanatory value beyond analogical reasoning and working memory for mathematics (ΔR² ≈ .09; β ≈ .29; p < .001) and language (ΔR² ≈ .07; β ≈ .23; p < .001), and for identified support needs, substantially improving model fit (ΔAIC = −7.4; OR ≈ .62). The stronger effect for mathematics suggests metacognitive self-regulation may be particularly important for mathematical learning.
Pre-test measures provided reliable, substantively meaningful baselines, underscoring the importance of distinguishing initial level from learning responsiveness in dynamic assessment. Combining dynamic cognitive measures with metacognitive self-report yields more informative learner profiles than cognitive performance alone. The differential responsiveness pattern suggests dynamic indicators capture distinct learning responsiveness aspects. This multidimensional screening approach supports distinctions between current cognitive level, responsiveness to brief instructional support, and metacognitive self-regulation, with direct relevance for educational interpretation, early identification of support needs, and pedagogical planning within a psychometrically robust standardisation and validation framework.