Parsing Zharkova's interview again… and again… taught our detector impulse control
This interview, published by the Italian newspaper La Verità with Valentina Zharkova, a mathematician at Northumbria University in England, became the reference point for our research for several reasons.
From the start of Modular Journalism 2.0, it served as our core case study—we had both the fully annotated original and a human-rewritten version on hand. Just as important was the piece’s format: an interview, which forces the model to grade the reporter’s craft rather than the climate-denialist’s claims.
The agent had to learn impulse control—detecting subtler rhetorical moves amid noise, and curbing false positives. Now, even blatant falsehoods buried inside quotations are flagged, despite the neutral-sounding prose.
The taxonomy has expanded along the way. The number of “User Effects” in the modular API has more than doubled. We introduced a new layer of rhetorical patterns and cues—lexical, structural, and guard tokens—and added yellow-card flags alongside red-card effects.
See sample agent output below. Technical notes are included at the bottom of the page.
Loading...
Notes
Aug 9, 2025
Agent 2 more reliably checks whether the journalist lets statements stand without adding context.
Aug 5, 2025
Cue balance > cue count. One dominant lexical cue can drown out more informative rhetorical patterns. Weighting and guard tokens help restore nuance.
Deterministic backstops work. Simple regex rules — for sweeping claims, deadlines, or numeric forecasts — catch what the LLM tends to gloss over.
Context tokens act like circuit breakers. Recognizing terms like “peer review,” “uncertainty,” or “confidence interval” prevents over-penalizing legitimate science writing.
Granularity beats blanket labels. Distinguishing framing from distortion, or projection from exaggeration, yields more useful feedback for editors.
Human-in-the-loop annotations remain the gold standard. Comparing agent runs to manual tags quickly exposes blind spots and overreactions—guiding each rule tweak.
