- https://arxiv.org/abs/2601.20245
- Source: arxiv
TL;DR
A randomized experiment shows that AI assistance during learning impairs conceptual understanding and debugging ability in novice developers, with the interaction pattern mattering more than AI presence itself.
Summary
Novice developers learning asynchronous programming were split into AI-assisted and unassisted groups in a randomized experiment. Despite expectations of productivity gains, AI use impaired conceptual understanding, code reading, and debugging without delivering significant efficiency improvements on average. Six distinct interaction patterns emerged: three involving active cognitive engagement preserved learning outcomes, while passive reliance degraded them. The core finding is that immediate performance gains from AI can mask long-term skill atrophy.
Key contributions
- Conducts a randomized controlled experiment measuring AI’s effect on skill formation in novice programmers.
- Introduces a taxonomy of six interaction patterns distinguishing active from passive AI engagement.
- Demonstrates that passive AI reliance leads to measurable skill atrophy in foundational programming competencies.
- Shows that productivity metrics alone are insufficient — comprehension and retention must be measured separately.
- Establishes that the quality of human-AI interaction predicts learning outcomes better than AI presence alone.
When to cite
- When arguing that AI tools in education require careful design to avoid degrading foundational skills.
- When supporting claims that efficiency metrics are misleading proxies for learning or competence.
- When discussing the risks of cognitive offloading or skill atrophy in AI-assisted workflows.
- When evaluating how interaction design shapes the long-term effects of AI assistance on skill development.