Get stories like this delivered straight to your inbox. Sign up for The 74 Newsletter
This summer, I ran a one-week artificial intelligence pilot at the Chicago headquarters of Forging Opportunities for Refugees in America, a nonprofit dedicated to helping refugees succeed in school. Most of the students were still building their literacy skills, and several of the 22 newcomers, ranging in age from 11 to 17, had arrived from Afghanistan, Burma, Malaysia and Syria after years of interrupted schooling. These are the kids the education system tends to write off and expect the least from.
On the second day, one of the teens was fed up. Gemini was broken, she told me. It kept asking her questions instead of answering them. She was adamant that ChatGPT was a better tool, and that we should use it instead.
What she didn’t know was that I had told Gemini to behave that way, to question her about her daily reading instead of just handing her answers. Then she scrolled up and read the instructions I had written into the prompt, and her annoyance turned to understanding. She wasn’t fighting a broken tool; she was being forced to pause and think.
The next day, she scored 100% on a quiz covering the prior day’s reading. She started leaning into our daily lessons, taking the time to answer questions thoughtfully instead of rushing through them. During our final session, she was one of the last students to turn in her quiz, making her mom wait a few extra minutes while she finished. Unheard of, especially over summer break.
That moment showed what’s possible when AI tools are built to make students actively think rather than think for them.
RelatedAI Tutors Are Praising Instead of Teaching. Here’s Why That’s Hurting Students
Over the course of a week, I gave the students a series of short readings about how their own brains work, and each day they used AI in a different way to deepen their understanding. Some days, I had the tool summarize the reading and told them to simply use that summary to build their understanding of the material. Other days, the tool had them engage in a Socratic Q&A. And some days it made them write their own question first, letting their own curiosity lead the process.
Then I quizzed them on what they remembered, both after their study session that same day and again a day later.
The results were fascinating. On the same-day quiz, students performed about the same, regardless of whether the AI tool had summarized the reading or questioned them. But a day later, the students who had engaged in the Socratic Q&A remembered more, scoring, on average, 83% on a quiz about the material they had reasoned through, versus 67% on the material the AI had summarized for them. More than two-thirds of the students said the version of AI that questioned them helped them learn more than the one that just summarized their readings.
Throughout the week, the students’ perception of their own skill in using AI as a study partner rose sharply. But when I lined those ratings up against what they could actually remember, the two had almost nothing to do with each other. How much students’ confidence in using AI grew had almost nothing to do with how much they had retained. Feeling like you have learned is not the same as actually learning.
This should worry anyone who cares about education. AI is everywhere. Students carry these tools in their pockets. School districts are slated to spend hundreds of millions of dollars on AI ed tech contracts in the coming years, without a clear idea of what they should be evaluating those contracts on. My week-long pilot program taught me that AI tools cannot be judged by student engagement or satisfaction alone. Success cannot be gauged by whether students finish a problem set, or whether they enjoy a particular lesson. It must be measured by what a student retains after the tool disappears.
RelatedA Step-by-Step Guide to AI in Schools — How Much to Use and When
My students told me one more thing worth highlighting. They want to learn to use AI as a thinking partner, but no one has taught them how. When they play around with AI to get answers, they often can’t obtain explanations at their own grade and language level. They told me it is sometimes harder to get what they need from these tools than to study on their own. The students who could benefit most from personalized learning get the least usable output, perpetuating the very inequity these tools are supposed to close.
That is why AI literacy must be taught in schools. Kids with tech-savvy parents are already learning to use AI well; the rest are typing in prompts and hoping for the best. School is the one place that can close that gap.
Did you use this article in your work?
We’d love to hear how The 74’s reporting is helping educators, researchers, and policymakers. Tell us how