The_miro
/
SmartestHome
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
SmartestHome/docs/project-plan.md

31 KiB
Raw Blame History

AI-Managed Smart Home — Full Build Plan (v2)

Local-first, open-source stack: Home Assistant + RuView presence + Bermuda BLE identity + local LLM (Ollama) + Zigbee sensors/lighting + Frigate (peephole face recognition) + Grocy kitchen kiosk + Nextcloud calendar sync + Node-RED glue logic + a Sway thin-client media station + a quarter-daily LLM digest.

1. Hardware

1.1 Already in your possession

Item Use in this build
Framework 12 (primary) Dev machine, dotfiles/Claude Code work
ThinkPad T440p (secondary) Spare compute, webcam presence daemon testing
Nextcloud instance Calendar backend (CalDAV sync target)
"Scraps" x86 machine Candidate for LLM host (CPU-only tier) or Proxmox host
Haozee Zigbee USB dongle (CC2652P + CH340C) Zigbee coordinator — USB passthrough into container, adapter type zstack

1.2 Core AI / HA hosts

Item Est. Price (EUR) Notes
Mini PC / SFF for Home Assistant + attached services (Raspberry Pi 5, 8GB) €80 Runs HA Core (Container), Mosquitto, Zigbee2MQTT, Frigate, Grocy, Node-RED, monitoring
SSD for boot (USB3, 128GB+) €20 Avoid SD card for reliability
Official Pi 5 PSU + case €25
Used RTX 3060 12GB €200–250 LLM inference host (Qwen2.5-14B Q4)
Cheap SFF/tower host for GPU (spare PCIe x16 slot, 170W+ on 12V rail) €40–60 Only needs to feed the GPU
(Fallback: skip GPU) Use existing "scraps" x86 box, CPU-only Ollama €0 Slower (5–10 tok/s), fine to validate before buying GPU

Subtotal (with GPU tier): ~€365–435 | Subtotal (CPU-only fallback): ~€125

1.3 Zigbee backbone

Already covered — using your existing Haozee CC2652P USB dongle. No coordinator purchase needed. Contingency: if Zigbee2MQTT can't establish a network, the dongle may need CC2652P coordinator firmware reflashed (generic-branded gateways sometimes ship with non-coordinator firmware).

1.4 Zigbee sensors (price per unit — multiply by room count)

Item Est. Price/unit Suggested qty Notes
innr RB 285 C (Zigbee RGBWW bulb) €15–20 1 per AI-managed room Tunable 2200K–6500K, acts as Zigbee router
Sonoff SNZB-02P (temp/humidity) €10–12 1 per room Avoid older Aqara — mesh drop issues
Sonoff SNZB-06P (presence/illuminance) or Aqara Light Sensor €12–15 1 per room needing lux data Feeds LLM brightness decisions
Sonoff SNZB-04P (door/window contact) €13–15 1 per door/window of interest Gates automations
IKEA Vindstyrka (air quality, optional) ~€30 1–2 Requires Zigbee2MQTT

1.5 BLE identity layer (Bermuda)

Item Est. Price/unit Suggested qty Notes
ESP32 board (D1 Mini32 or similar — no CSI needed, just BT+WiFi) €5–10 1 per room for identity resolution Runs ESPHome bluetooth_proxy, separate hardware from RuView nodes
Fixed-MAC BLE tag per household member (optional but recommended) €5–8 1 per person More reliable anchor than phone MAC (which randomizes on iOS/Android); Bermuda + HA's Private BLE Device can also resolve phones via IRK if you skip this

1.6 RuView presence mesh

Item Est. Price/unit Suggested qty Notes
ESP32-S3 dev board €8–12 1 per room/zone needing CSI presence Runs RuView firmware — separate boards from the Bermuda BLE proxies above (one chip = one firmware)

1.7 Door-spy / peephole camera

Item Est. Price Notes
Peephole IP camera (ONVIF/RTSP, color, "-IR" variant only if hallway is dark) €50–65 Isolate on its own VLAN, block WAN egress
(You're handling the physical mounting yourself) €0

1.8 Frigate acceleration (optional)

Item Est. Price Notes
Hailo-8L (Raspberry Pi 5 AI kit) €70 Only if CPU/iGPU isn't keeping up with more cameras later
(Default: none needed for 1 camera — use CPU or Intel iGPU/OpenVINO) €0

1.9 Kitchen inventory kiosk (camera + touchscreen)

Item Est. Price Notes
Raspberry Pi 3B+ or 4 (2–4GB) €35–50 Chromium kiosk + camera decode
7" official/compatible touchscreen €30–40
USB webcam w/ macro focus €10–15 Barcode scanning via Grocy's browser PWA
Case/mount for kitchen wall or cupboard €10–15

Subtotal: ~€85–120

1.10 Deferred / out of scope for now

  • Projector + camera tabletop AI surface — genuinely interesting but high-effort (projector-camera calibration, fingertip touch detection is an active CV research problem, poor legibility with ambient light). If pursued, treat as an isolated experimental side project, not wired into critical automations. Not included in this plan's budget or phases.

1.11 Sway thin-client hardware

Item Est. Price (EUR) Notes
Mini PC / SFF (Intel N100/N305-class, Quick Sync, wired GbE) €150–220 Wired Ethernet strongly preferred over WiFi for Steam Link latency
Display (HDMI monitor/TV) €0–150 May already have one
Keyboard + mouse or remote €20–40 Local fallback input — primary control is HA/MQTT + wayvnc, not this
USB mic + speaker (mic-enabled rooms only) €25–50/room Only for the specific rooms chosen for voice interactivity — see §3 Phase 11 open decisions

(No new hardware for Phase 12 — digest-engine/digest-web run as containers on the existing container-host from Phase 1.)

2. Software (all open source / self-hosted)

Layer Software Purpose
Home automation core Home Assistant (Container install) Central hub, automations, dashboards
Zigbee Zigbee2MQTT + Mosquitto Broadest device compatibility, USB passthrough to your CC2652P dongle
BLE identity Bermuda (HACS) + ESPHome (bluetooth_proxy) + HA's Private BLE Device Room-level BLE presence, resolves randomized MACs via IRK, feeds person.* entities
Automation glue Node-RED Visual flows for cross-system logic — home for the face↔MAC identity-correlation logic, easy iteration/debugging
LLM runtime Ollama Serves Qwen2.5-14B-Instruct (GPU tier) or Qwen2.5-7B/3B (CPU tier)
Conversation agent HA Ollama conversation integration Ties LLM into Assist + AI Task
Voice STT/TTS Wyoming faster-whisper + Piper Local speech pipeline
Presence (CSI) RuView (ESP32-S3 firmware + server) Anonymous room-level presence, feeds gating automations
Presence (webcam) Your custom daemon (dotfiles repo) Desktop-specific, not tied to HA sensors directly
Camera NVR / face recognition Frigate (0.16+) Native face recognition, object detection, peephole cam ingest
Identity store Small dict/table (SQLite or JSON) — name, face label, associated MACs, confidence Built up conversationally via LLM tool calls (propose_person_link, confirm_person, rename_person); merges require confirmation, never silent
Container runtime Docker + Docker Compose Hosts all of the above
Inventory / shopping list Grocy Self-hosted stock + shopping list, native browser barcode scanning
Meal planning (optional) Mealie Recipe manager + shopping list + meal-plan calendar
Calendar backend Nextcloud Calendar (already running) Source of truth for household schedule
Calendar bridge HA CalDAV integration Read/write bridge, calendar.create_event supported; keep recurring events created in Nextcloud directly
Monitoring Netdata Consistent with your existing Ansible-fleet monitoring evaluation
Dashboard Homepage or Homarr Single landing page for HA / Grocy / Frigate / Zigbee2MQTT / Node-RED UIs
Container management Portainer (optional) GUI over the Docker host
Notifications ntfy Self-hosted push, replaces relying on a phone OS's proprietary push channel
Long-term stats (optional) InfluxDB + Grafana Only if you want history beyond HA's default recorder retention
Backup restic Scheduled encrypted backups of all stateful volumes
Kiosk browser Chromium (kiosk mode) Displays Grocy PWA on the kitchen touchscreen
Thin-client OS build live-build Builds the thin-client ISO from hosts/thin-client/live-build/
Thin-client compositor Sway Kiosk Wayland compositor
Thin-client autologin greetd Autologin straight into sway, no separate greeter UI
Thin-client remote view/control wayvnc VNC for wlroots compositors — the chosen remote-control channel (Sway/wlroots has no maintained RDP path; wayvnc replaces RDP for this project)
Thin-client scripted control thinclient-agent (custom) HA MQTT-discovery entity + swaymsg/app-process control; the only surface the LLM can reach, always mediated through HA
Thin-client media mpv + mpv-mpris Local playback with MPRIS2 D-Bus control, bridged into thinclient-agent's HA media_player entity
Thin-client music spotifyd / librespot Headless Spotify Connect receiver (Premium required, unofficial protocol)
Thin-client game streaming Steam Link (Flatpak/Flathub) + Xwayland Remote Steam play; Xwayland avoids native-Wayland black-screen/flicker bugs on wlroots
Thin-client browser Firefox (kiosk) General browsing + the rendering surface for the LLM-generated digest canvas
Thin-client voice wyoming-satellite + openWakeWord Local wake-word spotting, streams to the existing Phase 3 Wyoming faster-whisper/Piper Assist pipeline — no new STT/TTS infrastructure
Digest scheduling systemd timer 4x/day cadence, same OnCalendar pattern as the existing restic backup timer
Digest engine digest-engine (custom Python) Ingests mail/messages/news/financial data, calls the Phase 3 Ollama host, renders the personal/political/household digest sections
Digest static serving digest-web (nginx:alpine/Caddy) Serves the rendered digest artifact read-only to both the thin client and an HA iframe card
Digest ingestion — Signal signal-cli Linked-device (JSON-RPC) read access to Signal messages
Digest ingestion — Telegram Telethon MTProto client logged in as the real account — the Bot API can't read personal DMs
Digest ingestion — Discord discord.py Bot with Message Content intent, scoped to servers you own/admin only — no personal-DM access
Digest ingestion — WhatsApp whatsapp-bridge (custom, Node.js + whatsapp-web.js/Puppeteer + Xvfb, opt-in) Real WhatsApp Web session in a virtual display, headful Chromium to avoid headless-detection; still opt-in via ENABLE_WHATSAPP_INGEST, off by default
Digest ingestion — mail imapclient IMAP fetch (App Password or OAuth2/XOAUTH2)
Digest ingestion — news feedparser Curated OPML feed list, including https://www.marxist.com/feed/rss
Digest ingestion — financial FRED API + Stooq Macro/unemployment indicators + stock/oil/commodity data
Digest rendering digest-canvas SDK (custom, vendored) Offline globe (addMarker(lat, lon, {icon, color, glow})), window/panel chrome, and glow/holo CSS primitives the LLM composes against each run

3. Implementation Plan

Phase 0 — Infrastructure prep

  1. Decide: Proxmox host (if virtualizing) vs. bare-metal. Given your homelab habits, Proxmox with HAOS/container-host as VMs/LXCs is reasonable.
  2. Set up a dedicated camera VLAN (peephole cam + future cameras), no WAN egress, firewalled from the main LAN.
  3. Provision the container host (Debian 12 or Raspberry Pi OS Lite).

Phase 1 — Core containers + Zigbee

  1. Docker + Compose stack: Home Assistant, Mosquitto, Zigbee2MQTT (USB passthrough to the Haozee dongle), Node-RED.
  2. Pair Zigbee sensors room by room: bulb → temp/humidity → illuminance → door/window contact.
  3. Confirm entities populate correctly in HA.

Phase 2 — Presence: RuView + Bermuda, in parallel

  1. Deploy RuView ESP32-S3 nodes per room (dedicated CSI firmware).
  2. Separately, flash plain ESP32 boards with ESPHome (bluetooth_proxy), install Bermuda via HACS, configure Private BLE Device for phone IRK resolution, and/or distribute fixed-MAC BLE tags per person.
  3. Build plain HA automations: presence (RuView) on → light on at neutral default; off (with delay) → light off. Validate this works with the LLM host powered off — this is your safety-net baseline.

Phase 3 — LLM host + conversation agent

  1. Stand up the GPU host (or CPU fallback) with Ollama, pull Qwen2.5-14B-Instruct (or 7B/3B for CPU).
  2. Add the Ollama integration in HA, point it at the LLM host over LAN.
  3. Set up Wyoming faster-whisper + Piper, configure an Assist pipeline.
  4. Test open-ended conversation and basic tool-calling before wiring into presence logic.

Phase 4 — AI-managed brightness/color

  1. Script triggered after the presence-automation light-on event, calling ai_task.generate_data with room/time/occupancy context, requesting structured JSON (brightness, rgb_color).
  2. Apply via light.turn_on; fallback to neutral default on timeout/error.
  3. Feed illuminance sensor data into the LLM's exposed-entity list.

Phase 5 — Peephole camera + Frigate

  1. Mount your color camera (your own work), isolate on the camera VLAN.
  2. Deploy Frigate, add the RTSP stream, enable native face recognition.
  3. Enroll known faces; automation for known vs. unknown at the door.
  4. Set conservative confidence thresholds given the narrow FOV/low light.

Phase 6 — Identity correlation (face ↔ MAC ↔ name)

  1. Build the correlation logic as a Node-RED flow first: listen to Frigate face-recognition MQTT events + Bermuda/BLE MQTT topics, tally co-occurrence, apply a confidence threshold.
  2. Expose LLM tool calls (propose_person_link, confirm_person, rename_person) so the LLM can name people conversationally and propose merges — never auto-commit a merge silently, require confirmation.
  3. Once the logic stabilizes, consider porting to a small Python service if it outgrows Node-RED's comfort zone (git/test-ability).

Phase 7 — Kitchen inventory kiosk

  1. Deploy Grocy via Compose.
  2. Pi + touchscreen running Chromium kiosk mode pointed at Grocy's PWA.
  3. Attach USB webcam, test in-browser barcode scanning (Open Food Facts lookup).
  4. Use Grocy's own add/consume UI — no extra hardware/logic needed for the "in vs. out" problem.

Phase 8 — Nextcloud calendar sync

  1. Add HA's CalDAV integration, point at Nextcloud's CalDAV URL.
  2. Confirm read + write (calendar.create_event) both work.
  3. Expose the calendar entity to the LLM's tool list.
  4. Gate delete/move actions behind a confirmation step; create recurring events directly in Nextcloud, not via HA.

Phase 9 — Observability, dashboard, backup

  1. Deploy Netdata (or your team's eventual choice from the Netdata/Checkmk evaluation) for container/host monitoring.
  2. Deploy Homepage/Homarr as a single landing page across all service UIs.
  3. Deploy ntfy for local push notifications (door alerts, automation failures).
  4. Set up restic scheduled backups of all stateful volumes (HA config, Zigbee2MQTT device DB, Grocy data, Frigate face embeddings, Node-RED flows).

Phase 10 — Optional expansion

  1. Mealie for meal-plan-aware shopping lists.
  2. InfluxDB + Grafana if you want longer-term historical dashboards than HA's recorder retains.
  3. Additional RuView/Bermuda nodes as more rooms are covered.
  4. Dedicated Frigate accelerator (Hailo-8L) only if you expand beyond 1–2 cameras.

Phase 11 — Sway thin-client ISO

  1. Scaffold a live-build tree at hosts/thin-client/live-build/ (Debian 12, matching container-host's OS). config/package-lists/thin-client.list.chroot pulls sway, greetd, wayvnc, xwayland, firefox-esr, mpv, mpv-mpris, spotifyd (or librespot), flatpak (Steam Link), wyoming-satellite + openwakeword deps, plus pipewire/wireplumber. hosts/thin-client/scripts/build-thin-client-iso.sh drives lb config && lb build.
  2. Autologin straight into a kiosk Sway session via greetd (initial_session block runs sway directly, no greeter UI) — not the older getty+.bash_profile hack.
  3. Remote control: wayvnc for interactive screen view/control. Sway/wlroots has no maintained RDP path (wlroots dropped its RDP backend; xrdp is X11-only) — wayvnc is the deliberate, confirmed replacement for "RDP" in this project, not a stopgap.
  4. Build thinclient-agent (Python, hosts/thin-client/agent/) as a systemd service baked into the image:
    • Connects to Mosquitto, does HA MQTT-discovery: a media_player entity (driven by mpv's MPRIS2 D-Bus state via mpv-mpris, bridged in-process), plus button/select entities for launching apps, switching Sway workspaces, and opening/expanding the digest canvas.
    • On MQTT command, shells out to swaymsg ($SWAYSOCK) and manages app processes (Firefox, Steam Link, mpv).
    • Security principle: the LLM never gets a raw network path to the thin client. Every control path is LLM tool call → HA service call → MQTT → thinclient-agent, mirroring the Phase 6/8 "HA mediates, nothing auto-acts" precedent.
  5. Dedicate one Sway workspace to a kiosk Firefox window pointed at digest-engine's local HTTP endpoint (Phase 12) — this doubles as the LLM's rendering surface for "free windows/graphics."
  6. Spotify via spotifyd/librespot (Connect receiver, no GUI login, Premium required, unofficial protocol — minor ongoing-maintenance risk, not a blocker).
  7. Steam Link via Flathub Flatpak, run under Xwayland (documented workaround for native-Wayland black-screen/flicker bugs on wlroots).
  8. Voice interactivity:
    • Only the rooms with a chosen thin client and an attached mic run wyoming-satellite (openWakeWord), streaming to the existing Phase 3 Wyoming faster-whisper/Piper pipeline — no new STT/TTS infrastructure.
    • thinclient-agent accepts an Assist-resolved "play my digest" intent: switches the dedicated Firefox workspace into the "full/thorough" canvas view (vs. the HA dashboard's "compact" view) and narrates via the existing Piper TTS output.
    • Room/person routing: reuse Phase 2's presence system (person.*/area entities). If exactly one recognized person is in the room where the wake word fired, play that person's personal digest section. If more than one is present, Assist asks "whose digest?" and disambiguates by spoken name before playing the personal section — never guesses. Political/household sections always play regardless of presence ambiguity.
  9. Network placement: plain trusted LAN for now (no VLAN precedent exists for a general client device class yet — only the unbuilt camera-VLAN concept). Revisitable later as a Phase-10-style expansion item, not a blocker now.
  10. Validate the image boots to a working kiosk session (Sway, local mpv/Spotify playback) with Mosquitto/HA/container-host powered off — must not hang waiting on the network, same "reactive path never depends on a remote service" philosophy applied to the thin client's own boot path.

Phase 12 — Quarter-daily LLM digest

  1. Add digest-engine to hosts/container-host's compose stack via the existing ENABLE_X/X_BLOCK pattern (ENABLE_DIGEST_ENGINE="false", off by default until credentials are provisioned). Its build context points at the new top-level digest-engine/ directory — the first locally-built image in the stack (everything else pulls prebuilt registry images).
  2. Add a companion digest-web static-file service (nginx:alpine/Caddy) in the same block, serving the shared output volume read-only to both display surfaces: the thin client's Firefox workspace (compact-vs-full toggle) and an HA Lovelace HTML/iframe card.
  3. Schedule via a systemd timer, mirroring the existing restic-backup convention: smart-home-digest.service (oneshot, docker compose run --rm digest-engine) + smart-home-digest.timer (OnCalendar=*-*-* 00,06,12,18:00:00, adjust once Ollama contention is settled — see open decisions).
  4. Ingestion modules, each independently toggleable, each reading credentials from a not-committed .env:
    • Email — IMAP via imapclient; Gmail needs an App Password (2FA-gated) or OAuth2/XOAUTH2 — App Password recommended for this personal-use case.
    • Signal — signal-cli linked as a secondary device (JSON-RPC daemon mode); lowest risk of the four message platforms.
    • Telegram — Telethon (MTProto, logs in as the real account) since the Bot API can't read personal DMs; this is a userbot, ToS-grey but lower enforcement risk than WhatsApp/Discord-selfbot.
    • Discord — discord.py bot with Message Content intent, scoped to servers you own/admin only; cannot read personal DMs or others' servers without a selfbot (not built — real ban risk, explicit ToS violation).
    • WhatsApp — no officially-sanctioned API option exists. Rather than a protocol-reimplementation library (Baileys), run a small whatsapp-bridge sidecar (Node.js, digest-engine/whatsapp-bridge/): a real Chromium logged into the actual web.whatsapp.com client via whatsapp-web.js (Puppeteer), inside its own container running Xvfb so Chromium executes headful (not headless: true) — WhatsApp's automation detection specifically fingerprints headless Chrome, so a virtual-display "real browser" session is meaningfully lower-risk than either Baileys or true-headless whatsapp-web.js, though not zero-risk (it's still automated use of a personal account). One-time interactive QR-code login persists a session directory (mounted volume) so subsequent runs don't need re-scanning. The bridge exposes incoming messages over a local-only channel (e.g. a Unix socket or a small internal HTTP endpoint on the compose network, never published to the LAN) that digest-engine/ingest/whatsapp_ingest.py reads each run. Still gate behind ENABLE_WHATSAPP_INGEST="false", off by default, with a warning in script output + digest-engine/README.md; recommend a secondary/non-critical number if enabled. Build this one last.
    • News — feedparser over a curated OPML list (digest-engine/feeds/curated-feeds.opml), seeded with the confirmed https://www.marxist.com/feed/rss plus a mainstream-outlet list (exact outlets: see open decisions).
    • Financial — FRED API (macro/unemployment, e.g. UNRATE) + Stooq keyless CSV (stocks/oil/commodities, preferred over Alpha Vantage's tight free-tier cap).
  5. LLM synthesis: assemble the run's ingested content into context, call the existing Phase 3 Ollama host with three separate prompt templates (digest-engine/synth/prompts/{personal,political,household}.md):
    • Personal — from personal-flagged mail/messages.
    • Political — Marxist/working-class analytical framing (marxist.com feed as theoretical basis) synthesizing mainstream news + financial indicators + politically-flagged mail, laid out on the "holo globe" with colored/glowing markers (e.g. revolutionary-situation markers in red with a hammer-and-sickle/star motif).
    • Household/calendar — from the existing Nextcloud CalDAV integration (Phase 8) and Grocy state (Phase 7).
    • A detail-level parameter (compact for the HA iframe, full for the thin-client fullscreen view) makes the thin-client rendering genuinely more thorough without needing two independent generation passes.
  6. Rendering: vendor the offline digest-canvas SDK under digest-engine/render/digest-canvas-sdk/ (globe + addMarker(), window/panel chrome, glow/holo CSS utility, no CDN dependency). Each run's LLM job is to call into this SDK with structured content, not hand-roll projection math. Use a custom inline SVG or Unicode ☭ (U+262D, explicit font-fallback + CSS glow) for hammer-and-sickle iconography since Nerd Fonts has no such glyph.
  7. Live follow-up voice Q&A: persist each run's actually-used ingested-context bundle (not full raw content) as digest-engine/output/<run-timestamp>/context.json. Expose a small HA tool (digest_followup_query) so a spoken follow-up ("tell me more about the unemployment numbers") feeds the cached context + question back into Ollama for a grounded, low-latency answer — no fresh ingestion pass. The answer can push a new small window/card onto the already-open thin-client canvas via a websocket, keeping the "flexible windows" idea alive live, not just at generation time.
  8. Security/scope principle: digest-engine is the first component in this project with routine WAN egress (mail, message platforms, news, financial APIs). Run it as its own compose service, no inbound port exposure beyond digest-web's read-only LAN serving; credentials in a git-ignored .env. Everything here is read-only summarization — it must never perform a write action anywhere (no auto-reply, no mail archive/delete beyond what IMAP fetch requires, no CalDAV/Grocy writes, no message-platform writes), extending the Phase 6/8 "no silent mutation" precedent to its logical extreme: no mutation path exists at all.

Testing checklist before calling any phase "done"

  • Does the reactive path (presence → light on) work with the LLM host powered off? (It must.)
  • Does a bad/slow LLM response ever block a light switch? (It must not.)
  • Are cameras verified to have zero WAN egress?
  • Does the calendar integration survive a Nextcloud restart without orphaning entities?
  • Does an identity merge ever get auto-committed without confirmation? (It must not.)
  • Does the thin client boot to a usable kiosk session with Mosquitto/HA/container-host powered off? (It must not hang.)
  • Can the LLM reach the thin client through any path other than HA service call → MQTT → thinclient-agent? (It must not.)
  • Does digest-engine ever perform a write action anywhere? (It must not — read-only only.)
  • Is WhatsApp ingestion off by default, requiring the explicit opt-in + warning? (It must be.)
  • Does voice digest playback ever read the wrong person's personal section when multiple people are present? (It must not — must ask, never guess.)
  • Does a follow-up voice question ever trigger a write, or only grounded read/synthesis against cached context? (It must stay read-only.)
  • If a run's LLM output produces malformed canvas-SDK calls, does the digest fall back to plain text instead of a broken/blank page?
  • Are all ingestion platform credentials kept out of git (.env, gitignored), matching the restic-password handling convention?
  • Does a stale/unreachable "was the digest viewed" signal ever cause runs to merge forever, instead of degrading to "assume viewed" after one missed check? (It must degrade, not compound.)
  • Does the compact HA-dashboard iframe view ever mark a digest as viewed? (It must not — only an actual thin-client canvas display or voice playback counts.)
  • Does the counter run actually drop a fabricated quote/figure/theoretical connection, rather than waving it through? (It must drop it.)
  • Does the counter run ever flag a correctly-grounded piece of Marxist analysis as "unverifiable" for being theoretical rather than a bare fact? (It must not — see synth/prompts/counter_run.md.)
  • If the counter run itself fails to reach the LLM host, is the original document kept and marked unverified, rather than either passed through silently or blanked? (It must be marked, not silently either extreme.)

4. Open decisions (Phases 11–12)

These need a decision before their respective implementation steps can be built — everything above is written to accommodate any answer, but nothing should be built against an unresolved item.

  1. RDP vs. VNC vs. desktop-environment swap — resolved: wayvnc (VNC) replaces RDP for this project; Sway is kept.
  2. WhatsApp ingestion approach — resolved: whatsapp-bridge (headful Chromium + whatsapp-web.js in an Xvfb virtual display), not Baileys. Still opt-in (ENABLE_WHATSAPP_INGEST); still recommend a secondary/non-critical number, since automating a personal account carries some risk even via the real web client.
  3. Mainstream news source list — digest-engine/feeds/curated-feeds.opml ships with marxist.com/feed/rss (real) plus BBC World/Al Jazeera/Guardian World/DW as clearly-marked placeholders (Reuters/AP were skipped — both have restricted their public RSS and guessing a live URL seemed worse than an honest placeholder). Still needs the user's actual sign-off/edit.
  4. Ollama contention — not yet resolved; DIGEST_SCHEDULE defaults to 00,06,12,18 (every 6h) in setup-container-host.sh, unadjusted for Assist-traffic overlap. Revisit once real usage patterns are known.
  5. Credential storage — implemented as a git-ignored .env seeded from digest-engine.env.example (matches the restic-password precedent); age/sops was not built, considered adequate for now.
  6. Exact mic-enabled room list — still needed; ENABLE_VOICE_SATELLITE in build-thin-client-iso.sh defaults to false per-image until rooms are chosen.
  7. Exact thin-client hardware target — still needed; nothing in the built image assumes specific hardware, but Steam Link/Xwayland decode performance can't be validated without it.
  8. Personal-digest visibility on shared displays — not yet resolved; not blocking, since the thin-client rendering built so far doesn't yet distinguish "shared screen" from "private."
  9. Household/calendar ingestion has no real data source yet (new, found during Phase 12 implementation) — the plan named CalDAV (Phase 8) and Grocy (Phase 7) as the source but no ingest module was written for either; digest-engine/run.py currently passes empty calendar/Grocy context and the household prompt is told to say "nothing scheduled" rather than hallucinate. Needs either a new ingest module or a decision to pull this from HA directly.
  10. HA has no core MQTT media_player platform (new, found during Phase 11 implementation) — thinclient-agent publishes the media_player discovery payload as specified, but stock Home Assistant ignores it without the HACS "MQTT Media Player" custom integration installed. button/sensor/number entities are also published as a fallback that works on a plain HA install; decide whether to install the HACS integration or keep relying on the fallback entities.
  11. Several package-availability items still need verification on real hardware before first boot, all flagged in-code rather than guessed: the Steam Link Flatpak app ID (com.valvesoftware.SteamLink), spotifyd/librespot packaging on Debian bookworm (not in main — three fallback install routes documented, none wired to a hardcoded download URL), and mpv-mpris packaging.
  12. wayvnc ships with no password — start-wayvnc fails closed on a sentinel value (CHANGEME-SET-ON-FIRST-BOOT) rather than serving unauthenticated VNC; a real password must be generated on the booted machine before wayvnc will start (see hosts/thin-client/README.md).