In late February I started a second AI grading project, a few days after the first. The first was Riverside Sec Oral Conversation, the AI oral grader for the language teachers at my school. The second was a math-side analogue: a student's working goes in, the AI grades it against the rubric, a teacher reviews and publishes. Same shape, different medium. Riverside Sec Oral Conversation grades transcribed speech; this one would grade handwritten math.
I built it. It worked. I tested it on a sample of my own students' work. I closed the laptop and chose not to launch it.
The reasons it stayed in the drawer were not engineering reasons. By the time I was making the decision, the engineering was solved. The reasons were pedagogical. Three of them, in roughly that order of weight: the feedback arrived too late, the feedback arrived in the wrong shape, and the unit economics did not make a strong case either.
This piece is those three reasons, with a paragraph at the end about what I built instead.
What I built
The shape was even simpler in practice than the launch plan. In testing, my students uploaded photos of their handwritten work to Google Classroom, which our school already uses. I pulled the submissions down from there into the prototype. For launch, the plan was for students to upload directly into the tool. In testing I kept the upload on Google Classroom: it was already in their workflow, and asking them to shift between platforms was friction I did not want to introduce.
From there, each page ran through Mathpix to convert handwriting into something machine-readable, then through Claude with the rubric and a prompt: grade this page, identify any errors, mark each one. The output came back as a JSON list of decisions, each anchored to a region on the page. The system overlaid those decisions on the original image: green ticks for correct steps, red arrows pointing at the precise mistake, partial-credit notations in the margin. Alongside the marks, the system produced a brief written commentary per question, explaining what the student had done, where it had gone wrong, and what the correct working would have been.
The teacher saw all of this on a review canvas. Click any mark to edit its position, text or judgement. Drag a tick somewhere else. Delete a red arrow that was wrong. Adjust the qualitative commentary. When the teacher was satisfied, the annotated pages were published back to the student.
Most of the engineering time went into the geometry of the marks. Claude is good at reading a page and identifying which line has a computational slip. Claude is much less good at saying where on the page that slip is, in pixels. The model gives you something approximately right; the wrong y-coordinate puts a red arrow one line above the actual error, which is the kind of mistake that makes a tool look broken. So I spent a few weekends on this: a piecewise-linear correction pass, then collision detection so two marks would not overlap, then auto-rotation for the landscape scans nobody in a hurry bothered to orient correctly. A name-matching pass auto-linked uploaded pages to the right student in the class roster, because nobody wants to drag forty submissions into forty slots one at a time.
By the time it was working, I had a prototype that produced annotated, marked pages for a class of forty students in under a minute per assignment. The marks landed in approximately the right place most of the time. The teacher's job was to scan the canvas, fix the occasional misplaced arrow, adjust the qualitative commentary, override the AI's judgement where the AI had got the working wrong, and publish.
Then I tested it on a sample of my own students' work. And then I closed the laptop.
Three reasons I didn't ship it
The feedback arrived too late
The oral grader put a band and feedback in front of a student about twenty-four seconds after they pressed stop. The math grader could not match that. Because I was not going to let AI-generated marks reach a student without a teacher reviewing them first, the loop was as fast as the teacher reviewed: typically a few hours, sometimes a day.
Twenty-four seconds is fast enough to feed into a student's next attempt at the practice question. A day is not. By the time the marked work came back, the lesson was over, the question was no longer live in the student's head, and the moment when feedback could have changed something was gone.
That was the obvious problem, the one I could have predicted before writing any code. The asymmetry I had not predicted was deeper. Oral practice is preparation for an event that happens later: the feedback after a practice recording feeds into the next recording, which feeds into the actual exam. The post-hoc shape is the right shape for oral practice. Math working is different. The working is the learning. There is no separate later event for the feedback to feed into. The cognitive moment ends when the working ends, and a teacher-reviewed loop was always going to deliver feedback on the wrong side of that ending.
The feedback arrived in the wrong shape
The marker did more than place red arrows. It also produced a brief written commentary per question, explaining what the student had done, where it had gone wrong, and what the correct working would have been. A student opening their marked work would see: marks on the page, an explanation of each error, and the correct working alongside.
It was, structurally, a graded assignment returned to the student. The student's job in that loop is to read what they have been told. To absorb the explanation. To understand the error. To do better next time.
What I noticed, watching this from the teacher side, was that the student's role was passive. The student receives a judgement. The student reads an explanation. The student is told what was wrong and what the correct working would have been. None of these are the cognitive moves that make a student a better mathematician.
The cognitive moves that matter are the active ones: articulating what you were trying to do, reconstructing your own reasoning, noticing where it went sideways without being told. The best move a math teacher makes in front of a stuck student is rarely "this is wrong"; it is "what are you trying to do here?" The Socratic question forces the student to reconstruct their thinking, and that reconstruction is the work that transfers. A mark, even one with rich qualitative commentary alongside, does not do that work for them.
A marker hands the student a judgement to absorb. A tutor hands the student a question to answer. Different cognitive activity. Different product.
The unit economics did not make a strong case
The third reason was about money, and at school scale it pointed in the same direction as the first two.
The Mathpix OCR step costs around half a cent per page. Claude grading adds another cent or two, depending on the model and the length of the page. Round it up: two to three cents per page of student work, processed end to end. A single assignment for a class of forty students, with each student turning in perhaps half a dozen pages of working, runs to a few dollars in raw API spend.
Scale it: one teacher, one class, two or three assignments a week, forty weeks of the school year. That is a few hundred dollars per teacher per year. Across a math department of a dozen teachers, several thousand. Across the school of around eighty teachers, tens of thousands.
If the pedagogical case had been clear, those numbers would have been a normal conversation about what a school invests in. The pedagogical case was not clear, and at school scale the cost was not small. Combined with the first two reasons, it was not money I wanted to ask the school to absorb.
What I built instead
The product I built next is called RSS Feedback. Instead of marking the last line, it asks about the next one. A student writes their working on an iPad, in their own hand, and when they submit it the tool returns a question, not a verdict: what are you trying to do here? Why did you start with this approach? What happens if you check the substitution again? It nudges, in the Socratic register, toward the student doing the thinking that is the actual point of the exercise.
RSS Feedback is running at the school now. I assigned the first task on it to a class of my own students this term, and what happened in that classroom is the subject of the next piece.