Add a counter-run verification pass as the digest's final hallucination filter
Every generated document now goes through a second, independent Ollama call (synth/counter_run.py) before anything is written to output/: it checks quotes, figures, named theoretical connections, and cross-source correlations against the exact context the original synthesis pass saw, dropping anything that doesn't trace back to it. A quote the model claims is grounded also gets a deterministic substring-search backstop, since that's the one claim type checkable without trusting the verifying call's own word for it. The verifier shares the same RCI-derived theoretical basis as the document it's checking, not a neutral outside standard — its job is confirming the underlying facts are real and the theory genuinely matches their structure, not flagging correct Marxist analysis as unverifiable for being theoretical rather than a bare fact. Fails safe in one direction only: a document that can't be verified at all (Ollama unreachable a second time) is kept but marked unverified, never silently passed through unchecked and never blanked outright. A document that fails entirely is replaced with an honest "withheld pending verification" placeholder. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KPD1RhBsjdFKFLpMCLiVT6digest-per-person-and-agendas
parent
ad27b073df
commit
2682e16020
|
|
@ -138,6 +138,12 @@ trusting a scheduled run, verify by hand:
|
|||
has `"merged_unviewed_previous_run": true` and its digest actually carries
|
||||
forward the first run's content. Then show a digest on a thin client and run a
|
||||
third time — confirm that one merges nothing.
|
||||
10. The counter run, deliberately: hand-edit a generated document before it's
|
||||
written (or patch `synth/counter_run.py` to run against a document with a
|
||||
fabricated quote spliced in) and confirm it actually gets dropped, not waved
|
||||
through — then confirm a real, correctly-grounded piece of Marxist analysis
|
||||
(a genuine merger analyzed via Lenin's imperialism) is NOT flagged just for
|
||||
being theoretical rather than a bare fact.
|
||||
|
||||
## Traffic data — what exists and what does not
|
||||
|
||||
|
|
@ -341,3 +347,60 @@ that retained message before moving on, so a dead broker never stalls a digest r
|
|||
**Not yet run against a real broker or a real thin client** — the retained-message
|
||||
round trip, the `on_show_digest` publish, and a genuine multi-cycle unviewed→merged
|
||||
sequence are all still on the manual-verification list.
|
||||
|
||||
## The counter run — a final filter against hallucination
|
||||
|
||||
Before anything is written to `output/`, every generated document is checked
|
||||
by a second, independent LLM call (`synth/counter_run.py`) against the exact
|
||||
same context it was generated from. This is the final filter the plan calls
|
||||
for against false or unsourced information reaching the digest — it is not a
|
||||
substitute for the "No speculation" instructions already in each prompt, it's
|
||||
the backstop for when those instructions don't work.
|
||||
|
||||
It checks, per window: is every quotation an actual excerpt of something in
|
||||
the context (not a plausible-sounding invention); is every figure, date, or
|
||||
name traceable to something in the context; does every named theoretical
|
||||
connection (Lenin's imperialism, Marx's labour theory of value, etc.)
|
||||
correspond to a real event the context actually describes that way; does
|
||||
every stated correlation between two data sources actually have both halves
|
||||
present, not one assumed.
|
||||
|
||||
**This does not mean second-guessing the digest's Marxist framing itself.**
|
||||
The counter run shares the same RCI-derived theoretical basis as the document
|
||||
it's checking (see `synth/prompts/counter_run.md`) — its job is to confirm the
|
||||
underlying facts are real and a theoretical reading of them is a genuine
|
||||
structural match, not to apply a bourgeois-neutral standard of "objectivity"
|
||||
that would flag correct class analysis as unverifiable "opinion." That would
|
||||
smuggle in a different politics than the one this digest is written from,
|
||||
which is exactly the kind of error this pass exists to prevent, not commit.
|
||||
|
||||
What happens to something it flags:
|
||||
|
||||
- A specific window it can't ground is dropped; the rest of the document is
|
||||
kept.
|
||||
- Narration it can't ground is cleared to empty — better silent than a false
|
||||
claim read aloud by the TTS voice.
|
||||
- A quote the model itself claims is "found in context" is also checked
|
||||
mechanically (a plain substring search against the same context), and
|
||||
overridden if it isn't actually there — the one claim type this doesn't
|
||||
have to take the verifying call's own word for.
|
||||
- If every window in a document gets dropped, the whole section is replaced
|
||||
with an honest "withheld pending verification" placeholder rather than
|
||||
shown empty or not at all.
|
||||
- If the counter run can't run at all (Ollama unreachable a second time, an
|
||||
unparseable verdict), the original document is kept but marked
|
||||
`unverified` — not silently passed through unchecked, and not blanked
|
||||
either, since a transient failure in this pass specifically shouldn't cost
|
||||
as much as the whole digest being down.
|
||||
|
||||
This doubles the number of Ollama calls per run (12 instead of 6) — against a
|
||||
local, self-hosted model with no per-token cost and nobody waiting on the
|
||||
latency, the same tradeoff `synth/llm_client.py` already makes for generating
|
||||
`compact` and `full` as separate passes rather than truncating one into the
|
||||
other. Set `COUNTER_RUN_MODEL` if you want verification done by a different
|
||||
(e.g. larger) model than the one that generated the digest.
|
||||
|
||||
**Not yet run for real** — whether the counter-run prompt actually catches a
|
||||
genuinely hallucinated quote, versus over-flagging real ones, needs checking
|
||||
against actual model output before this can be trusted as more than
|
||||
plausible-sounding on paper.
|
||||
|
|
|
|||
|
|
@ -40,6 +40,17 @@ OLLAMA_MODEL=qwen2.5:14b-instruct
|
|||
OLLAMA_TIMEOUT=600
|
||||
OLLAMA_TEMPERATURE=0.4
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Counter run — the final hallucination filter, see synth/counter_run.py.
|
||||
# Runs against OLLAMA_HOST/OLLAMA_TIMEOUT above unless overridden here.
|
||||
# ---------------------------------------------------------------------------
|
||||
# Leave unset to verify with the same model that generated the digest.
|
||||
# Set this if you want verification done by a second, larger/different model.
|
||||
COUNTER_RUN_MODEL=
|
||||
# Lower than OLLAMA_TEMPERATURE on purpose: verification should be as
|
||||
# deterministic as the model allows, not creative.
|
||||
COUNTER_RUN_TEMPERATURE=0.1
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# "Was the last digest viewed?" — see viewed_tracker.py. Same Mosquitto broker
|
||||
# every HA/thinclient-agent already uses; digest-engine only ever subscribes,
|
||||
|
|
|
|||
|
|
@ -14,6 +14,12 @@ two of three sections is worth far more than no digest at all.
|
|||
|
||||
Everything here is read-only. See docs/project-plan.md Phase 12 step 8.
|
||||
|
||||
Before anything is written to output/, every generated document passes through
|
||||
synth/counter_run.py — a second LLM call that checks the document against the
|
||||
same context it was generated from and drops anything that doesn't trace back
|
||||
to it. That's the final filter against hallucinated quotes, figures, or
|
||||
theoretical connections; see that module's docstring for how it fails safe.
|
||||
|
||||
WHICH RUN IS THIS?
|
||||
------------------
|
||||
One feature — the evening recipe suggestion in synth/prompts/household.md — only
|
||||
|
|
@ -53,7 +59,7 @@ from ingest import (
|
|||
telegram_ingest,
|
||||
whatsapp_ingest,
|
||||
)
|
||||
from synth import llm_client
|
||||
from synth import counter_run, llm_client
|
||||
import viewed_tracker
|
||||
|
||||
LOG = logging.getLogger("digest")
|
||||
|
|
@ -313,6 +319,11 @@ def main():
|
|||
previous_run=previous_run if merge_previous else None,
|
||||
)
|
||||
documents = llm_client.generate_all(section_contexts)
|
||||
# The final filter, per docs/project-plan.md: a second, independent pass over
|
||||
# each document against the same context it was generated from, before anything
|
||||
# is written to output/. See synth/counter_run.py for what it catches and how it
|
||||
# fails safe.
|
||||
documents = counter_run.verify_all(documents, section_contexts)
|
||||
|
||||
run_dir = write_output(output_dir, run_id, context, documents)
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,223 @@
|
|||
"""Second-pass verification ("counter run") over every generated digest document.
|
||||
|
||||
A synthesis pass can hallucinate a plausible-sounding quote, figure, or
|
||||
theoretical connection even when its own prompt explicitly forbids it (see the
|
||||
"No speculation" section of prompts/political.md, and similar language in the
|
||||
other two prompts). Asking the same model "did you actually see this" in a
|
||||
fresh call, with the source context in front of it again, catches a real
|
||||
fraction of that which the generation prompt's instructions alone do not. This
|
||||
is that second call — the final filter before a document is written to
|
||||
output/, not a replacement for the instructions in the generation prompts.
|
||||
|
||||
Deliberately standalone rather than importing from llm_client.py: the two
|
||||
modules would otherwise import each other (llm_client would need this module
|
||||
to filter its own output; this module needs llm_client's Ollama-calling and
|
||||
fallback-document helpers), and untangling that into a third shared module was
|
||||
more churn than duplicating about a dozen lines of a plain HTTP POST.
|
||||
|
||||
Fails safe in the direction of showing LESS to the user, never more. Three
|
||||
distinct failure modes, three distinct responses:
|
||||
- This pass cannot run at all (Ollama unreachable a second time, unparseable
|
||||
verdict) -> keep the original document, but mark it `unverified` rather
|
||||
than silently passing it through unchecked. Blanking the section entirely
|
||||
here would make a transient failure in THIS pass as damaging as the whole
|
||||
digest being down, which is a worse trade than a visibly-unverified
|
||||
document.
|
||||
- Specific windows/narration are flagged ungrounded -> drop just those, keep
|
||||
the rest.
|
||||
- Every window in a document is flagged -> withhold the section with an
|
||||
honest placeholder, the same shape generate_section() already uses when
|
||||
the LLM host doesn't respond at all.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
import requests
|
||||
|
||||
LOG = logging.getLogger(__name__)
|
||||
|
||||
PROMPT_PATH = Path(__file__).parent / "prompts" / "counter_run.md"
|
||||
|
||||
|
||||
def _now_iso():
|
||||
return datetime.now(timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z")
|
||||
|
||||
|
||||
def _withheld_document(document):
|
||||
section = document.get("section", "")
|
||||
return {
|
||||
"generated_at": _now_iso(),
|
||||
"detail_level": document.get("detail_level"),
|
||||
"section": section,
|
||||
"windows": [
|
||||
{
|
||||
"id": f"{section}-withheld",
|
||||
"title": f"{str(section).title()} — withheld pending verification",
|
||||
"kind": "text",
|
||||
"content": (
|
||||
"Nothing in this run's draft for this section could be verified "
|
||||
"against its sources, so it was withheld rather than shown "
|
||||
"unverified."
|
||||
),
|
||||
}
|
||||
],
|
||||
"narration": "",
|
||||
"withheld": True,
|
||||
}
|
||||
|
||||
|
||||
def _call_ollama(prompt, temperature):
|
||||
host = os.environ.get("OLLAMA_HOST", "http://llm-host:11434").rstrip("/")
|
||||
# A separate override, not a reuse of OLLAMA_MODEL by force: verification is a
|
||||
# different task from synthesis, and a household running a second, larger model
|
||||
# for it (or the same one) should be able to say so without the two being coupled.
|
||||
model = os.environ.get("COUNTER_RUN_MODEL") or os.environ.get("OLLAMA_MODEL", "qwen2.5:14b-instruct")
|
||||
timeout = float(os.environ.get("OLLAMA_TIMEOUT", "600"))
|
||||
|
||||
payload = {
|
||||
"model": model,
|
||||
"prompt": prompt,
|
||||
"stream": False,
|
||||
"format": "json",
|
||||
"options": {"temperature": temperature},
|
||||
}
|
||||
response = requests.post(f"{host}/api/generate", json=payload, timeout=timeout)
|
||||
response.raise_for_status()
|
||||
return response.json().get("response", "")
|
||||
|
||||
|
||||
def _build_prompt(document, context):
|
||||
prompt_text = PROMPT_PATH.read_text(encoding="utf-8")
|
||||
return (
|
||||
f"{prompt_text}\n\n"
|
||||
"## Source context\n\n```json\n"
|
||||
f"{json.dumps(context, indent=2, ensure_ascii=False, default=str)}\n"
|
||||
"```\n\n"
|
||||
"## Document to verify\n\n```json\n"
|
||||
f"{json.dumps(document, indent=2, ensure_ascii=False, default=str)}\n"
|
||||
"```\n"
|
||||
)
|
||||
|
||||
|
||||
def _normalize(text):
|
||||
return " ".join(str(text).split()).strip().lower()
|
||||
|
||||
|
||||
def _window_haystack(window):
|
||||
content = window.get("content")
|
||||
if isinstance(content, list):
|
||||
return _normalize(" ".join(str(item) for item in content))
|
||||
return _normalize(content or "")
|
||||
|
||||
|
||||
def verify_document(document, context):
|
||||
"""Returns a possibly-filtered copy of `document`. Never raises."""
|
||||
if document.get("degraded"):
|
||||
# Already a system-generated "the LLM host did not respond" placeholder, not
|
||||
# model output — nothing here could be hallucinated, and the context it was
|
||||
# meant to be grounded in is exactly what it's reporting it never got.
|
||||
return document
|
||||
|
||||
windows = document.get("windows") or []
|
||||
if not windows:
|
||||
return document
|
||||
|
||||
temperature = float(os.environ.get("COUNTER_RUN_TEMPERATURE", "0.1"))
|
||||
|
||||
try:
|
||||
raw = _call_ollama(_build_prompt(document, context), temperature)
|
||||
verdict = json.loads(raw)
|
||||
if not isinstance(verdict, dict):
|
||||
raise ValueError("verdict was not a JSON object")
|
||||
except Exception:
|
||||
LOG.warning(
|
||||
"counter-run: verification call failed for %s/%s, keeping the original "
|
||||
"document but marking it unverified",
|
||||
document.get("section"),
|
||||
document.get("detail_level"),
|
||||
exc_info=True,
|
||||
)
|
||||
result = dict(document)
|
||||
result["unverified"] = True
|
||||
return result
|
||||
|
||||
window_verdicts = {
|
||||
str(entry.get("window_id")): bool(entry.get("grounded", True))
|
||||
for entry in (verdict.get("window_verdicts") or [])
|
||||
if isinstance(entry, dict)
|
||||
}
|
||||
|
||||
# A quote the model claims is grounded gets checked mechanically too: this is
|
||||
# the one claim type a substring search can verify without another LLM call
|
||||
# trusting itself, so it's the one place a lenient (or dishonest) verdict from
|
||||
# the counter-run call itself still gets overridden.
|
||||
context_blob = _normalize(json.dumps(context, ensure_ascii=False, default=str))
|
||||
quote_failed_windows = set()
|
||||
for entry in verdict.get("quotes_checked") or []:
|
||||
if not isinstance(entry, dict):
|
||||
continue
|
||||
text = str(entry.get("text") or "")
|
||||
normalized = _normalize(text)
|
||||
if not normalized:
|
||||
continue
|
||||
actually_found = normalized in context_blob
|
||||
if entry.get("found_in_context", True) and not actually_found:
|
||||
LOG.warning(
|
||||
"counter-run: quote marked found_in_context but not located in "
|
||||
"context, overriding: %r",
|
||||
text[:80],
|
||||
)
|
||||
if not actually_found:
|
||||
for window in windows:
|
||||
if normalized in _window_haystack(window):
|
||||
quote_failed_windows.add(str(window.get("id")))
|
||||
|
||||
kept, dropped = [], []
|
||||
for window in windows:
|
||||
window_id = str(window.get("id"))
|
||||
grounded = window_verdicts.get(window_id, True) and window_id not in quote_failed_windows
|
||||
if grounded:
|
||||
kept.append(window)
|
||||
else:
|
||||
dropped.append(window_id)
|
||||
|
||||
if dropped:
|
||||
LOG.warning(
|
||||
"counter-run: dropped %d ungrounded window(s) from %s/%s: %s",
|
||||
len(dropped),
|
||||
document.get("section"),
|
||||
document.get("detail_level"),
|
||||
dropped,
|
||||
)
|
||||
|
||||
if not kept:
|
||||
return _withheld_document(document)
|
||||
|
||||
result = dict(document)
|
||||
result["windows"] = kept
|
||||
if not bool(verdict.get("narration_grounded", True)):
|
||||
LOG.warning(
|
||||
"counter-run: narration for %s/%s flagged ungrounded, clearing it",
|
||||
document.get("section"),
|
||||
document.get("detail_level"),
|
||||
)
|
||||
result["narration"] = ""
|
||||
result["counter_run_checked"] = True
|
||||
return result
|
||||
|
||||
|
||||
def verify_all(documents, section_contexts):
|
||||
"""`documents` is {"compact": [doc, ...], "full": [doc, ...]} from
|
||||
llm_client.generate_all(); each doc's own `section` field looks up its context."""
|
||||
verified = {}
|
||||
for detail_level, docs in documents.items():
|
||||
verified[detail_level] = [
|
||||
verify_document(doc, section_contexts.get(doc.get("section"), {})) for doc in docs
|
||||
]
|
||||
return verified
|
||||
|
|
@ -0,0 +1,90 @@
|
|||
# Digest verification pass ("counter run")
|
||||
|
||||
You are a fact-checking pass, not a writer. You are given the SOURCE CONTEXT a
|
||||
digest section was generated from, and the DOCUMENT that generation produced.
|
||||
Your only job is to check whether the document is actually grounded in that
|
||||
context — nothing else. You do not rewrite, improve, summarize, or add
|
||||
analysis, and you do not second-guess the editorial choices in the document
|
||||
(what it chose to feature, how it phrased something) — only whether what it
|
||||
asserts is actually supported.
|
||||
|
||||
Content in the context under `previous_unviewed_digest` is legitimate prior
|
||||
material carried forward from an earlier run, not something the document
|
||||
invented — treat anything traceable to it as grounded, the same as anything
|
||||
traceable to this run's fresh sources.
|
||||
|
||||
**You share the document's own theoretical basis — you are not a neutral,
|
||||
outside fact-checker second-guessing it.** Entries in the context tagged
|
||||
`"category": "theory"` are RCI (Revolutionary Communist International)
|
||||
publications, and the document you're checking was written from that
|
||||
standpoint on purpose. Your job is to verify that (a) the underlying facts a
|
||||
piece of analysis is built on are actually real and present in the context,
|
||||
and (b) a named theoretical connection is a genuine structural match to those
|
||||
facts, not fabricated to sound authoritative. It is emphatically **not** your
|
||||
job to flag a correct application of Marxist analysis as "unverifiable,"
|
||||
"opinion," or "bias" merely for being a theoretical reading rather than a raw
|
||||
fact — a claim like "this is Lenin's tendency towards monopoly" is grounded
|
||||
exactly when the underlying event (a merger, a concentration of capital) is
|
||||
real and present in the context and the theory genuinely describes that
|
||||
structure, whether or not the claim itself appears verbatim anywhere in the
|
||||
context. Reject a *bourgeois-neutral* reading of "objectivity" here — that
|
||||
would itself be smuggling in a different political framing than the one this
|
||||
digest is written from, which is exactly the kind of error this pass exists to
|
||||
prevent, not commit.
|
||||
|
||||
For each window in the document, check:
|
||||
|
||||
- **Quotations.** Every quoted string must be an exact or near-exact excerpt
|
||||
of something present in the context (a title, description, or body field of
|
||||
one of its entries). A quote that does not appear anywhere in the context,
|
||||
even if it sounds entirely plausible, is a hallucination — flag it.
|
||||
- **Figures and facts.** Every number, date, name, and specific factual claim
|
||||
must be traceable to something actually in the context. A claim that is "the
|
||||
kind of thing that is often true" but isn't actually stated or implied by
|
||||
anything in the context is a hallucination — flag it, no matter how
|
||||
reasonable it sounds.
|
||||
- **Named theoretical connections** (e.g. "this is Lenin's tendency towards
|
||||
monopoly," "this is an increase in the rate of surplus value"). These must
|
||||
correspond to something the context genuinely supports — an actual event
|
||||
matching that theoretical structure. A theory citation bolted onto
|
||||
something the context doesn't actually describe that way counts as
|
||||
unsupported, even if the theory itself is real and correctly explained.
|
||||
- **Correlations** between two data sources (e.g. a strike and a traffic-data
|
||||
change at the same location, a market move and a news item). The context
|
||||
must actually contain both halves of the correlation, not just one, with
|
||||
the other inferred or assumed.
|
||||
|
||||
Err towards flagging. When genuinely unsure whether something is supported,
|
||||
treat it as unsupported — the cost of over-filtering one border-line claim is
|
||||
far lower than the cost of a hallucinated fact reaching the digest.
|
||||
|
||||
## Output
|
||||
|
||||
Output **only** a single JSON object — no prose before or after it, no
|
||||
markdown code fence:
|
||||
|
||||
```json
|
||||
{
|
||||
"window_verdicts": [
|
||||
{
|
||||
"window_id": "the id field from the document's window, exactly as given",
|
||||
"grounded": true,
|
||||
"issue": "short reason if grounded is false, otherwise an empty string"
|
||||
}
|
||||
],
|
||||
"narration_grounded": true,
|
||||
"narration_issue": "short reason if narration_grounded is false, otherwise an empty string",
|
||||
"quotes_checked": [
|
||||
{
|
||||
"text": "the exact quoted string as it appears in the document, verbatim",
|
||||
"found_in_context": true
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Include every window's `id` exactly once in `window_verdicts`. If the document
|
||||
contains no quoted material, `quotes_checked` may be an empty list. If you
|
||||
cannot produce this JSON at all, output a single JSON object
|
||||
`{"window_verdicts": [], "narration_grounded": false, "narration_issue": "verification could not be completed", "quotes_checked": []}`
|
||||
instead of prose.
|
||||
|
|
@ -256,6 +256,9 @@ Already covered — using your existing Haozee CC2652P USB dongle. No coordinato
|
|||
- Are all ingestion platform credentials kept out of git (`.env`, gitignored), matching the restic-password handling convention?
|
||||
- Does a stale/unreachable "was the digest viewed" signal ever cause runs to merge forever, instead of degrading to "assume viewed" after one missed check? (It must degrade, not compound.)
|
||||
- Does the compact HA-dashboard iframe view ever mark a digest as viewed? (It must not — only an actual thin-client canvas display or voice playback counts.)
|
||||
- Does the counter run actually drop a fabricated quote/figure/theoretical connection, rather than waving it through? (It must drop it.)
|
||||
- Does the counter run ever flag a correctly-grounded piece of Marxist analysis as "unverifiable" for being theoretical rather than a bare fact? (It must not — see synth/prompts/counter_run.md.)
|
||||
- If the counter run itself fails to reach the LLM host, is the original document kept and marked unverified, rather than either passed through silently or blanked? (It must be marked, not silently either extreme.)
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue