Developing New Indices to Measure Digital Technology Access and Familiarity

The National Assessment of Educational Progress (NAEP), or the nation’s report card, is in the process of transitioning from a paper-based to a digitally based assessment. This transition brings up important questions: Are all children are ready for the transition—and would any of them would be disadvantaged by it? If students’ access to, and familiarity with, technology correspond to better performance on the digital NAEP assessment, results could differ from the paper-based assessment, which has been used to assess educational and learning trends since 1990 for math and 1992 for reading.

To investigate these issues, AIR developed a new set of survey items related to digital technology for the 2015 administration of the NAEP math and reading assessment. New items on the survey that aimed to measure access to and familiarity with digital technology, as well as self-efficacy in working with digital technology, were given to fourth-, eighth-, and twelfth-graders. Additional survey items about more-advanced uses of digital technology and familiarity with digital concepts were given just to eighth- and twelfth-graders.
 

Key Lessons Learned

• Traditionally underserved populations (Black and Hispanic students, national school lunch program-eligible students, English language learners, and students with disabilities) were more likely to indicate familiarity with digital technology through use of tablets and, to a lesser extent, computers than their counterparts—just the opposite from what was hypothesized. Yet, as hypothesized, these subpopulations generally reported lower digital self-efficacy and less familiarity with digital concepts.

• However, student access, familiarity, and self-efficacy with digital technology did not differentiate student performance on the digital mode compared to the paper mode.