Advancing Research on Equitable AI Education Through a Focus on Implementation: Insights from a Middle School Computer Vision Module Beta-Test

Abstract

Part of a university initiative supporting responsible AI for social empowerment and education, the project-based RAICA (Responsible AI for Computational Action) curriculum supports middle/high school learners and novice AI literacy teachers use AI creatively for good. This paper offers a rare example of design-based implementation research (DBIR) in AI education across widely varied contexts, provides fine grain implementation data that contributes to a foundation for evaluating effectiveness and expanding access. We present a novel approach to analyzing fidelity of implementation data from RAICA’s computer vision module beta-test. Twelve educators working with ~282 students across nine pilot sites in four countries used a bespoke fidelity of implementation data collection tool (pre-made comment prompts in a Google Docs version of the teacher guide) to provide 236 qualitative responses about AI literacy and responsible design activities, plus 111 ordinal ratings of embedded teacher supports. Analyses revealed that while the curriculum was generally implemented as designed, educators frequently made modifications. Although most changes produced practical insights for improved curriculum design, others helped the design team anticipate and prevent changes that could obscure learning objectives and hinder outcomes. We discuss the pedagogical, design, and research implications of these findings for effective AI teaching/learning in diverse settings.