Let's start up something new!
I've been busy with teaching my AI class at the University of Zürich, taking advantage of being in central Europe by visiting various places for fun or for work. Strassbourg, Munich, St. Gallen, Brugg in the past few weeks, with Graz (Austria), London, Cambridge, Basel, and a few others still to come. (In other words, I'm a bit busy.)
I'm constantly asked questions like "What's the one thing I need to know to make my AI-powered searches better and more accurate?"
The short answer is... there is no short answer. That shouldn't be a surprise--the AI-powered world is even more complicated that the old regime.
I thought I'd try to improve the state of the world a bit by doing a bunch of short tips in the blog.
So.. my plan is to do a bunch (at least 5 or 6 over the next couple of months) and see if I have more to say when I get to the end of the short list. We'll see.
Let's start off with a pro tip:
Be careful with those article summaries! They sometimes leave out important content!
People use AI for summarization all the time--it's probably the most common use case. Google Scholar, Acrobat, YouTube... basically every publishing platform offers some kind of summarization shortcut. Don't have the time to read the whole article? Do this instead!
But (and you knew I'd say this), you have to recognize that what the AI thinks is obvious and can be skipped in a summary, YOU might not know about.
Example: you might say to to summarize this paper (Bastani, et al., 2025)about the use of AI in education) in 3 bullet points. Here's what you get:
• In a field Randomized Controlled Trial of ~1,000 math students, unrestricted ChatGPT (GPT Base) boosted practice scores by +48%, but reduced subsequent closed-book exam scores by −17% relative to a no-AI control group.
• A safeguarded version (GPT Tutor), prompted with teacher-designed hints instead of direct answers, boosted practice performance by +127% while protecting long-term student learning.
• Takeaway: Deploying pedagogically guarded AI tutors eliminates the crutch effect and maximizes math achievement.
Suppose I ask you this key question: Should your students use this computational tutor? It's clear that freshmen score higher on their closed-book final exam than traditional unassisted practice.
Does GPT Tutor beat unassisted practice on the exam?
“In stark contrast, in the subsequent unassisted exam, student performance in the GPT Base arm degraded by 0.054 (out of 1) relative to that of the control arm. In other words, GPT Base diminished the average control student's performance on the unassisted exam by 17%. Student performance in the GPT Tutor arm was statistically indistinguishable from that of the control arm, and the point estimate was smaller by an order of magnitude (−0.004). The fact that students in the GPT Tutor arm performed similarly to students in the control arm on the unassisted exam may be surprising since they performed so much better on the practice problems. This difference may be partly explained by the fact that students with access to GPT Tutor during the practice session could ask it to check their answers, whereas students in the control arm were only shown the solutions after they had already submitted their answers.”
See the problem?
Based on the AI summary, would you recommend that the students use GPT Tutor to improve their scores? Suppose it cost $100,000 for your school to subscribe. Would it be worth it?
There's an interesting nuance here. The summary makes it seem that the AI tutor leads to a huge gain. Of course it would be worth the $100K.
But read that summary carefully: Putting the '+127% practice' beside 'protected learning' caused readers to interpolate an exam boost. In reality, GPT Tutor merely had the same score as pen-and-paper (−0.004), as is clear from the full text.
Why does +127% improvement yield 0% transfer in the end?
A: Because of a nuance in the text. This difference may be partly explained by GPT Tutor letting students check & repair answers BEFORE submitting (and thereby inflating practice scores), whereas control students had to commit before seeing feedback. That's a big difference.
