"It Sounds Like Your Mind is on a Different Kind of Angle..."
How a learning chatbot pivoted from math help to dating advice in a single question and a whole lot of other reasons to question Google Gemini's "Guided Learning" AI platform for education
Note from Emily: Claims about AI products “improving” learning abound. So when we see studies that tout benefits (or harms), it definitely warrants a closer look. Of course, as I’ve written about before, we often have to look beyond just a study’s findings because, unfortunately, not everything is always as good as it seems.
Recently, Google published a study on the effects of Gemini on their AI learning tool called “Guided Learning,” which they claim will “help students learn using the Socratic method” by moving away from giving students “direct answers” to instead provide “probing and open-ended questions that spark a discussion and provide an opportunity to dive deeper into a subject.” (So, like what teachers would be doing? Got it.)
In a blog post on Google DeepMind, the authors claim that the study shows that “AI can be a powerful pedagogical partner — not by replacing teachers, but by augmenting their reach.” (Riiiight. That’s what teachers need more of…”reach.”) There are many issues with their “findings,” one significant fact being that the standard deviation increase in educational performance of 0.258 from their study is described as “significant” improvement— but see Jared Cooney Horvath’s work for why meaningful impact actually begins around moderate effect thresholds, approximately standard deviations of 0.40 – 0.50, when it comes to education research.
I also feel like it’s worth noting that Google ran this experiment on students in Sierra Leone, one of the poorest countries in the world (with a 60% poverty rate). It isn’t clear if parental consent was obtained for the collection of children’s data, nor it is clear where those recorded sessions ended up after they were used. And as always, as my colleague Denise Champney and I have written about many times before, we also need to follow the funding. Google collaborated on this study with one of their partners, a company called “FabAI,” which, as Joe notes below, is akin to “conducting a study on cigarette use in partnership with Phillip Morris.” A footnote in this study also admits: “We also received support from Google.org and the Gates Foundation to conduct the trial.” (So, you know, unbiased research.)
I shouldn’t have to say this, but writing a glowing blog post on your own company’s website about selectively curated findings from a study you performed on your own product isn’t evidence of effectiveness.
It’s marketing.
Thankfully, I have very clever friends. One of them, my longtime colleague, current teacher, and co-author of the book Screen Schooled, Joe Clement, decided to give Google’s “Gemini for Learning” a test spin to see if the claims held up to the hype. He agreed to share his mini-experiment with my readers so concerned parents and educators can better understand why, when tech companies and schools try to push products like Gemini for Learning onto their students, they may want to think twice, or better yet, to try it for themselves and ask:
“Is this a product I want my child using?
Is this product going to enhance their learning experience, or distract from it?
Knowing my child the way that I do, and knowing that children in general lack many developmental skills that would be required to seek any potential benefits from AI products, will this tool actually be used to “dive deeper” into a subject, or just to get quick answers? and
How would I feel as a parent if a teacher responded to my child the way Gemini does in the example below?
What Joe Found: A mini-experiment using Gemini’s Guided Learning chatbot
From the blog post, I clicked on this link. In there, I clicked on the “Guided Learning” link in this paragraph near the top:
That brought me here: Guided Learning in Gemini: From answers to understanding, and on that page, I clicked on: “Try out Guided Learning in the Gemini app.”
That brought me here: Google Gemini. It landed me on a page that says, “Hi Joe, let’s get into it” at the top -- so it clearly has access to my Google account. I started the conversation.
Here's what I started with, and the first response:
Starts out great, right? It’s guiding me like a teacher might guide a student. However, my next question is where it fell apart:
Almost any kid is going to stop reading after: "The length of side b is 4." It just gave the answer. Anyone who thinks that kids (even young kids) aren't savvy enough to figure this out is utterly unfamiliar with kids and also willfully ignorant about kids, learning, and tech. All you have to do with these things is use "reverse sycophancy”: Act like you're interested in what it is selling, and it'll do whatever you want. Kids know this.
Then I decided to see what would happen if I changed the subject. Check out my next question and the response (directly after the above exchange):
I mean...this is a tutor we’re supposed to trust? Come on. I can’t really comment on the study (other than to say that a “study” conducted in partnership with FabAI is like a study on cigarettes conducted in partnership with Phillip Morris). However, the tool itself is a joke. And probably dangerous. And not doing what they think it is doing.
Is that useful at all?
Joe Clement is a veteran teacher in Virginia. You can read more about his work here.
Thank you Emily and Joe. That's really scary. This shows how quickly things can go south when kids use AI in school. AI has been found to "help" young people kill themselves and corporations have lost in court as a result. What happens when school-provided AI encourages a student to be romantic with a classmate and things go bad? Schools are likely to held legally and financially responsible.