SmartestHome/docs/project-plan.md

AI-Managed Smart Home — Full Build Plan (v2)

Local-first, open-source stack: Home Assistant + RuView presence + Bermuda BLE identity + local LLM (Ollama) + Zigbee sensors/lighting + Frigate (peephole face recognition) + Grocy kitchen kiosk + Nextcloud calendar sync + Node-RED glue logic + a Sway thin-client media station + a Sway touch panel + a camera-vision kitchen/fridge display + a quarter-daily LLM digest.

---

1. Hardware

1.1 Already in your possession

Item	Use in this build
Framework 12 (primary)	Dev machine, dotfiles/Claude Code work
ThinkPad T440p (secondary)	Spare compute, webcam presence daemon testing
Nextcloud instance	Calendar backend (CalDAV sync target)
"Scraps" x86 machine	Candidate for LLM host (CPU-only tier) or Proxmox host
Haozee Zigbee USB dongle (CC2652P + CH340C)	Zigbee coordinator — USB passthrough into container, adapter type zstack

1.2 Core AI / HA hosts

Item	Est. Price (EUR)	Notes
Mini PC / SFF for Home Assistant + attached services (Raspberry Pi 5, 8GB)	€80	Runs HA Core (Container), Mosquitto, Zigbee2MQTT, Frigate, Grocy, Node-RED, monitoring
SSD for boot (USB3, 128GB+)	€20	Avoid SD card for reliability
Official Pi 5 PSU + case	€25
Used RTX 3060 12GB	€200–250	LLM inference host (Qwen2.5-14B Q4)
Cheap SFF/tower host for GPU (spare PCIe x16 slot, 170W+ on 12V rail)	€40–60	Only needs to feed the GPU
(Fallback: skip GPU) Use existing "scraps" x86 box, CPU-only Ollama	€0	Slower (5–10 tok/s), fine to validate before buying GPU

Subtotal (with GPU tier): ~€365–435 | Subtotal (CPU-only fallback): ~€125

1.3 Zigbee backbone

Already covered — using your existing Haozee CC2652P USB dongle. No coordinator purchase needed. Contingency: if Zigbee2MQTT can't establish a network, the dongle may need CC2652P coordinator firmware reflashed (generic-branded gateways sometimes ship with non-coordinator firmware).

1.4 Zigbee sensors (price per unit — multiply by room count)

Item	Est. Price/unit	Suggested qty	Notes
innr RB 285 C (Zigbee RGBWW bulb)	€15–20	1 per AI-managed room	Tunable 2200K–6500K, acts as Zigbee router
Sonoff SNZB-02P (temp/humidity)	€10–12	1 per room	Avoid older Aqara — mesh drop issues
Sonoff SNZB-06P (presence/illuminance) or Aqara Light Sensor	€12–15	1 per room needing lux data	Feeds LLM brightness decisions
Sonoff SNZB-04P (door/window contact)	€13–15	1 per door/window of interest	Gates automations
IKEA Vindstyrka (air quality, optional)	~€30	1–2	Requires Zigbee2MQTT

1.5 BLE identity layer (Bermuda)

Item	Est. Price/unit	Suggested qty	Notes
ESP32 board (D1 Mini32 or similar — no CSI needed, just BT+WiFi)	€5–10	1 per room for identity resolution	Runs ESPHome bluetooth_proxy, separate hardware from RuView nodes
Fixed-MAC BLE tag per household member (optional but recommended)	€5–8	1 per person	More reliable anchor than phone MAC (which randomizes on iOS/Android); Bermuda + HA's Private BLE Device can also resolve phones via IRK if you skip this

1.6 RuView presence mesh

Item	Est. Price/unit	Suggested qty	Notes
ESP32-S3 dev board	€8–12	1 per room/zone needing CSI presence	Runs RuView firmware — separate boards from the Bermuda BLE proxies above (one chip = one firmware)

1.7 Door-spy / peephole camera

Item	Est. Price	Notes
Peephole IP camera (ONVIF/RTSP, color, "-IR" variant only if hallway is dark)	€50–65	Isolate on its own VLAN, block WAN egress
(You're handling the physical mounting yourself)	€0

1.8 Frigate acceleration (optional)

Item	Est. Price	Notes
Hailo-8L (Raspberry Pi 5 AI kit)	€70	Only if CPU/iGPU isn't keeping up with more cameras later
(Default: none needed for 1 camera — use CPU or Intel iGPU/OpenVINO)	€0

1.9 Kitchen inventory kiosk (camera + touchscreen)

Item	Est. Price	Notes
Raspberry Pi 3B+ or 4 (2–4GB)	€35–50	Chromium kiosk + camera decode
7" official/compatible touchscreen	€30–40
USB webcam w/ macro focus	€10–15	Barcode scanning via Grocy's browser PWA
Case/mount for kitchen wall or cupboard	€10–15

Subtotal: ~€85–120

1.10 Deferred / out of scope for now

• Projector + camera tabletop AI surface — genuinely interesting but high-effort (projector-camera calibration, fingertip touch detection is an active CV research problem, poor legibility with ambient light). If pursued, treat as an isolated experimental side project, not wired into critical automations. Not included in this plan's budget or phases.

1.11 Sway thin-client hardware

Item	Est. Price (EUR)	Notes
Mini PC / SFF (Intel N100/N305-class, Quick Sync, wired GbE)	€150–220	Wired Ethernet strongly preferred over WiFi for Steam Link latency
Display (HDMI monitor/TV)	€0–150	May already have one
Keyboard + mouse or remote	€20–40	Local fallback input — primary control is HA/MQTT + wayvnc, not this
USB mic + speaker (mic-enabled rooms only)	€25–50/room	Only for the specific rooms chosen for voice interactivity — see §3 Phase 11 open decisions

(No new hardware for Phase 12 or Phase 13 — digest-engine/digest-web and admin-canvas/admin-web all run as containers on the existing container-host from Phase 1.)

1.12 ESP32-S3 voice/display satellite hardware (Phase 14)

Item	Est. Price (EUR)	Notes
Waveshare ESP32-S3-Touch-LCD-1.85C, V2 revision	€35–45	Round 360×360 LCD + dual mic (ES7210, with echo-cancellation reference path) + speaker (ES8311) + 8Ω 2W speaker. Must be V2 — V1 has no AEC circuit and different audio pins, see firmware/esp32-s3-touch-lcd-1.85c/README.md. An alternative to Home Assistant Voice PE for rooms that also want a status display, not a wholesale replacement of §1.11's mic-enabled-room hardware

1.13 Headless audio endpoint hardware (Phase 15)

Item	Est. Price (EUR)	Notes
Raspberry Pi (Zero 2 W is enough for one Spotify Connect stream; any 40-pin-header Pi works)	€15–35	arm64 variant
HiFiBerry Amp2 HAT	~€45	60W Class D, drives 4–8Ω passive speakers directly — GPIO HAT, Raspberry-Pi-only
Passive bookshelf speakers (e.g. Micca MB42X G2)	~€100/pair	Same pairing already recommended for the thin-client rooms
(amd64 alternative) Spare x86 mini PC + USB DAC/amp (e.g. Fosi Audio V3, ~€130)	€0–130	No GPIO HAT involved — a USB Audio Class device instead, for rooms getting a repurposed mini PC rather than a Raspberry Pi

1.14 Touch panel hardware (Phase 16)

Item	Est. Price (EUR)	Notes
Mini PC or SBC with a capacitive touchscreen (e.g. 10–15" USB-touch monitor + small x86 mini PC, or an all-in-one touch panel PC)	€150–300	No specific unit has been chosen — the one hard requirement is that the touchscreen exposes itself as a native Wayland wl_touch device under libinput, not merely an emulated-mouse HID interface, since hosts/touch-panel/ has no gesture/pointer-translation fallback the way the thin client does
Speakers or a wired connection to existing room audio	€0–100	For local Spotify GUI-client playback — same speaker guidance as the thin-client/audio-endpoint rooms

(No container-host changes — touchpanel-agent talks to the existing Mosquitto/HA from Phase 1, same as every other MQTT-connected host in this plan.)

1.15 Kitchen/fridge display hardware (Phase 17)

Item	Est. Price (EUR)	Notes
Mini PC or SBC with a touchscreen, mounted near the fridge/pantry	€150–300	Same "must be a native Wayland wl_touch device" requirement as §1.14 — hosts/kitchen-display/ has no pointer-emulation fallback either
USB webcam, pointed at wherever items get held up for scanning	€15–40	Either built into the panel or on a short cable/gooseneck so its angle can be set independently of the screen. Any UVC webcam works — this is the same "no depth camera, no accelerator needed" bar as the thin client's gesture-control camera (§1.11), just used for a photo instead of continuous tracking

(No new container-host hardware — pantry-vision is a container on the existing Phase 1 host, calling the existing Phase 3/9 LLM host for vision inference and the already-running grocy container for storage.)

---

2. Software (all open source / self-hosted)

Layer	Software	Purpose
Home automation core	Home Assistant (Container install)	Central hub, automations, dashboards
Zigbee	Zigbee2MQTT + Mosquitto	Broadest device compatibility, USB passthrough to your CC2652P dongle
BLE identity	Bermuda (HACS) + ESPHome (bluetooth_proxy) + HA's Private BLE Device	Room-level BLE presence, resolves randomized MACs via IRK, feeds person.* entities
Automation glue	Node-RED	Visual flows for cross-system logic — home for the face↔MAC identity-correlation logic, easy iteration/debugging
LLM runtime	Ollama	Serves Qwen2.5-14B-Instruct (GPU tier) or Qwen2.5-7B/3B (CPU tier)
Conversation agent	HA Ollama conversation integration	Ties LLM into Assist + AI Task
Voice STT/TTS	Wyoming faster-whisper + Piper	Local speech pipeline
Presence (CSI)	RuView (ESP32-S3 firmware + server)	Anonymous room-level presence, feeds gating automations
Presence (webcam)	Your custom daemon (dotfiles repo)	Desktop-specific, not tied to HA sensors directly
Camera NVR / face recognition	Frigate (0.16+)	Native face recognition, object detection, peephole cam ingest
Identity store	Small dict/table (SQLite or JSON) — name, face label, associated MACs, confidence	Built up conversationally via LLM tool calls (propose_person_link, confirm_person, rename_person); merges require confirmation, never silent
Container runtime	Docker + Docker Compose	Hosts all of the above
Inventory / shopping list	Grocy	Self-hosted stock + shopping list, native browser barcode scanning
Meal planning (optional)	Mealie	Recipe manager + shopping list + meal-plan calendar
Calendar backend	Nextcloud Calendar (already running)	Source of truth for household schedule
Calendar bridge	HA CalDAV integration	Read/write bridge, calendar.create_event supported; keep recurring events created in Nextcloud directly
Monitoring	Netdata	Consistent with your existing Ansible-fleet monitoring evaluation
Dashboard	Homepage or Homarr	Single landing page for HA / Grocy / Frigate / Zigbee2MQTT / Node-RED UIs
Container management	Portainer (optional)	GUI over the Docker host
Notifications	ntfy	Self-hosted push, replaces relying on a phone OS's proprietary push channel
Long-term stats (optional)	InfluxDB + Grafana	Only if you want history beyond HA's default recorder retention
Backup	restic	Scheduled encrypted backups of all stateful volumes
Kiosk browser	Chromium (kiosk mode)	Displays Grocy PWA on the kitchen touchscreen
Thin-client OS build	live-build	Builds the thin-client ISO from hosts/thin-client/live-build/
Thin-client compositor	Sway	Kiosk Wayland compositor
Thin-client autologin	greetd	Autologin straight into sway, no separate greeter UI
Thin-client remote view/control	wayvnc	VNC for wlroots compositors — the chosen remote-control channel (Sway/wlroots has no maintained RDP path; wayvnc replaces RDP for this project)
Thin-client scripted control	thinclient-agent (custom)	HA MQTT-discovery entity + swaymsg/app-process control; the only surface the LLM can reach, always mediated through HA
Thin-client media	mpv + mpv-mpris	Local playback with MPRIS2 D-Bus control, bridged into thinclient-agent's HA media_player entity
Thin-client music	spotifyd / librespot	Headless Spotify Connect receiver (Premium required, unofficial protocol)
Thin-client game streaming	Steam Link (Flatpak/Flathub) + Xwayland	Remote Steam play; Xwayland avoids native-Wayland black-screen/flicker bugs on wlroots
Thin-client browser	Firefox (kiosk)	General browsing + the rendering surface for the LLM-generated digest canvas
Thin-client voice	wyoming-satellite + openWakeWord	Local wake-word spotting, streams to the existing Phase 3 Wyoming faster-whisper/Piper Assist pipeline — no new STT/TTS infrastructure
Digest scheduling	systemd timer	4x/day cadence, same OnCalendar pattern as the existing restic backup timer
Digest engine	digest-engine (custom Python)	Ingests mail/messages/news/financial data, calls the Phase 3 Ollama host, renders the personal/political/household digest sections
Digest static serving	digest-web (nginx:alpine/Caddy)	Serves the rendered digest artifact read-only to both the thin client and an HA iframe card
Digest ingestion — Signal	signal-cli	Linked-device (JSON-RPC) read access to Signal messages
Digest ingestion — Telegram	Telethon	MTProto client logged in as the real account — the Bot API can't read personal DMs
Digest ingestion — Discord	discord.py	Bot with Message Content intent, scoped to servers you own/admin only — no personal-DM access
Digest ingestion — WhatsApp	whatsapp-bridge (custom, Node.js + whatsapp-web.js/Puppeteer + Xvfb, opt-in)	Real WhatsApp Web session in a virtual display, headful Chromium to avoid headless-detection; still opt-in via ENABLE_WHATSAPP_INGEST, off by default
Digest ingestion — mail	imapclient	IMAP fetch (App Password or OAuth2/XOAUTH2)
Digest ingestion — news	feedparser	Curated OPML feed list, including https://www.marxist.com/feed/rss
Digest ingestion — financial	FRED API + Stooq	Macro/unemployment indicators + stock/oil/commodity data
Digest rendering	digest-canvas SDK (custom, vendored)	Offline globe (addMarker(lat, lon, {icon, color, glow})), window/panel chrome, and glow/holo CSS primitives the LLM composes against each run
Admin canvas write API	admin-canvas (custom Python, stdlib http.server)	Small always-on service, POST /show + POST /media/<f>, bearer-token gated, no published port — reachable only from Home Assistant on the compose network. The sys-admin-llm's on-demand counterpart to the scheduled digest-engine
Admin canvas static serving	admin-web (nginx:alpine)	Serves the admin canvas's rendered JSON + uploaded media read-only to the thin client — same role as digest-web, separate instance
Admin canvas rendering	canvas-sdk (custom, vendored, duplicated from digest-canvas SDK)	Same window/panel chrome and glow theme, minus the globe, plus stat/image/video/chart window kinds (the chart kind is a dependency-free inline-SVG bar/sparkline)
Voice/display satellite firmware	ESPHome (custom config, firmware/esp32-s3-touch-lcd-1.85c/)	Display (LVGL), on-device wake word (micro_wake_word, okay_nabu), voice_assistant streaming into the existing Phase 3 Assist pipeline, and a media/cover-art-over-idle-weather-time-date priority display
Headless audio endpoint (arm64)	rpi-image-gen (custom config, hosts/audio-endpoint/rpi-image-gen/)	Raspberry Pi Foundation's current custom-image tool; builds a flashable .img with a per-room Spotify Connect receiver + HiFiBerry Amp2 audio
Headless audio endpoint (amd64)	live-build (custom config, hosts/audio-endpoint/live-build-amd64/)	Reuses the thin client's own build tool, stripped of the whole graphical/kiosk stack — headless boot straight to the same per-room Spotify Connect receiver
Headless audio endpoint software	librespot or spotifyd (apt-first, documented fallback)	Independent, per-room Spotify Connect — no Snapcast, no shared/synced stream; each endpoint is its own device in Spotify's picker, same model as the thin client's own per-room spotifyd/librespot (Phase 11.6)
Touch-panel OS build	live-build (custom config, hosts/touch-panel/live-build/)	Reuses the thin client's own build tool and directory-split convention, not its live-build tree — a different, smaller image for a different device
Touch-panel compositor	Sway	Same choice as the thin client, configured for direct touch instead of remote control: an always-on dock instead of no bars, no gesture-camera/wayvnc machinery
Touch-panel scripted control	touchpanel-agent (custom)	HA MQTT-discovery entity + swaymsg/app-process control, identical security shape to thinclient-agent — LLM tool call → HA service call → MQTT → this agent, never a direct path
Touch-panel music	Spotify (official Linux client, Flathub com.spotify.Client)	Full GUI client, not a headless Connect receiver — the point of a touch panel is a screen you interact with directly (Premium required, same as the headless receivers)
Touch-panel home dashboard	Chromium (kiosk mode, --app=)	Dedicated, always-open window pointed at Home Assistant, auto-restarted if it crashes
Touch-panel browser	Firefox	General browsing, minimal (not kiosk-locked) chrome — back/forward/reload/address bar
Touch-panel on-screen keyboard	wvkbd (apt-first, documented fallback)	Manually toggled from the touch dock — no shell-level input-method integration on this image, so no automatic show-on-focus
Touch-panel navigation	eww (apt-first, documented fallback)	An always-on, layer-shell-reserved touch dock (Spotify / Home / Web / Keyboard) — the touch-first equivalent of the thin client's HA-only Screen select
Kitchen-display OS build	live-build (custom config, hosts/kitchen-display/live-build/)	Same toolchain/convention reuse as the touch panel, a third and simpler sibling image — one workspace, one app
Kitchen-display camera capture	Browser-native getUserMedia() (Chromium)	No native capture app on-device at all — the kiosk page itself talks to the webcam and hands a frame straight to pantry-vision
Kitchen-display scripted control	kitchen-display-agent (custom)	HA MQTT-discovery entity for Show scan/inventory/recipes only — the actual camera/Grocy read-write path is pantry-vision's own published API, called directly by the kiosk browser, not mediated through this agent
Grocery vision recognition	Ollama (a vision-capable model, e.g. llava/qwen2.5vl — TBD, not yet pulled or benchmarked)	Identifies a grocery item from one photo and estimates its shelf life; the proposal is always human-reviewed before anything is written (see pantry-vision/README.md)
Grocery inventory backend	pantry-vision (custom Python, stdlib http.server)	POST /identify (photo → proposal), POST /confirm (human-reviewed proposal → Grocy stock write), GET /inventory/GET /recipes (proxy Grocy, reshaped). Bearer-token gated, and — unlike admin-canvas — LAN-published, since the kitchen display's kiosk browser calls it directly rather than through Home Assistant
Grocery inventory storage	Grocy (already running, Phase 1/7)	The single source of truth for stock/best-before-dates/recipes; pantry-vision is a client of Grocy's own REST API, not a replacement for it
Kitchen-display static serving	pantry-web (nginx:alpine)	Serves pantry-vision/frontend/'s Scan/Inventory/Recipes single-page app read-only to the kitchen display — same role digest-web/admin-web play for their own hosts

---

3. Implementation Plan

Phase 0 — Infrastructure prep

1. Decide: Proxmox host (if virtualizing) vs. bare-metal. Given your homelab habits, Proxmox with HAOS/container-host as VMs/LXCs is reasonable.
2. Set up a dedicated camera VLAN (peephole cam + future cameras), no WAN egress, firewalled from the main LAN.
3. Provision the container host (Debian 12 or Raspberry Pi OS Lite).

Phase 1 — Core containers + Zigbee

1. Docker + Compose stack: Home Assistant, Mosquitto, Zigbee2MQTT (USB passthrough to the Haozee dongle), Node-RED.
2. Pair Zigbee sensors room by room: bulb → temp/humidity → illuminance → door/window contact.
3. Confirm entities populate correctly in HA.

Phase 2 — Presence: RuView + Bermuda, in parallel

1. Deploy RuView ESP32-S3 nodes per room (dedicated CSI firmware).
2. Separately, flash plain ESP32 boards with ESPHome (bluetooth_proxy), install Bermuda via HACS, configure Private BLE Device for phone IRK resolution, and/or distribute fixed-MAC BLE tags per person.
3. Build plain HA automations: presence (RuView) on → light on at neutral default; off (with delay) → light off. Validate this works with the LLM host powered off — this is your safety-net baseline.

Phase 3 — LLM host + conversation agent

1. Stand up the GPU host (or CPU fallback) with Ollama, pull Qwen2.5-14B-Instruct (or 7B/3B for CPU).
2. Add the Ollama integration in HA, point it at the LLM host over LAN.
3. Set up Wyoming faster-whisper + Piper, configure an Assist pipeline.
4. Test open-ended conversation and basic tool-calling before wiring into presence logic.

Phase 4 — AI-managed brightness/color

1. Script triggered after the presence-automation light-on event, calling ai_task.generate_data with room/time/occupancy context, requesting structured JSON (brightness, rgb_color).
2. Apply via light.turn_on; fallback to neutral default on timeout/error.
3. Feed illuminance sensor data into the LLM's exposed-entity list.

Phase 5 — Peephole camera + Frigate

1. Mount your color camera (your own work), isolate on the camera VLAN.
2. Deploy Frigate, add the RTSP stream, enable native face recognition.
3. Enroll known faces; automation for known vs. unknown at the door.
4. Set conservative confidence thresholds given the narrow FOV/low light.

Phase 6 — Identity correlation (face ↔ MAC ↔ name)

1. Build the correlation logic as a Node-RED flow first: listen to Frigate face-recognition MQTT events + Bermuda/BLE MQTT topics, tally co-occurrence, apply a confidence threshold.
2. Expose LLM tool calls (propose_person_link, confirm_person, rename_person) so the LLM can name people conversationally and propose merges — never auto-commit a merge silently, require confirmation.
3. Once the logic stabilizes, consider porting to a small Python service if it outgrows Node-RED's comfort zone (git/test-ability).

Phase 7 — Kitchen inventory kiosk

1. Deploy Grocy via Compose.
2. Pi + touchscreen running Chromium kiosk mode pointed at Grocy's PWA.
3. Attach USB webcam, test in-browser barcode scanning (Open Food Facts lookup).
4. Use Grocy's own add/consume UI — no extra hardware/logic needed for the "in vs. out" problem.

Phase 8 — Nextcloud calendar sync

1. Add HA's CalDAV integration, point at Nextcloud's CalDAV URL.
2. Confirm read + write (calendar.create_event) both work.
3. Expose the calendar entity to the LLM's tool list.
4. Gate delete/move actions behind a confirmation step; create recurring events directly in Nextcloud, not via HA.

Phase 9 — Observability, dashboard, backup

1. Deploy Netdata (or your team's eventual choice from the Netdata/Checkmk evaluation) for container/host monitoring.
2. Deploy Homepage/Homarr as a single landing page across all service UIs.
3. Deploy ntfy for local push notifications (door alerts, automation failures).
4. Set up restic scheduled backups of all stateful volumes (HA config, Zigbee2MQTT device DB, Grocy data, Frigate face embeddings, Node-RED flows).

Phase 10 — Optional expansion

1. Mealie for meal-plan-aware shopping lists.
2. InfluxDB + Grafana if you want longer-term historical dashboards than HA's recorder retains.
3. Additional RuView/Bermuda nodes as more rooms are covered.
4. Dedicated Frigate accelerator (Hailo-8L) only if you expand beyond 1–2 cameras.

Phase 11 — Sway thin-client ISO

1. Scaffold a live-build tree at hosts/thin-client/live-build/ (Debian 12, matching container-host's OS). config/package-lists/thin-client.list.chroot pulls sway, greetd, wayvnc, xwayland, firefox-esr, mpv, mpv-mpris, spotifyd (or librespot), flatpak (Steam Link), wyoming-satellite + openwakeword deps, plus pipewire/wireplumber. hosts/thin-client/scripts/build-thin-client-iso.sh drives lb config && lb build.
2. Autologin straight into a kiosk Sway session via greetd (initial_session block runs sway directly, no greeter UI) — not the older getty+.bash_profile hack.
3. Remote control: wayvnc for interactive screen view/control. Sway/wlroots has no maintained RDP path (wlroots dropped its RDP backend; xrdp is X11-only) — wayvnc is the deliberate, confirmed replacement for "RDP" in this project, not a stopgap.
4. Build thinclient-agent (Python, hosts/thin-client/agent/) as a systemd service baked into the image:
   • Connects to Mosquitto, does HA MQTT-discovery: a media_player entity (driven by mpv's MPRIS2 D-Bus state via mpv-mpris, bridged in-process), plus button/select entities for launching apps, switching Sway workspaces, and opening/expanding the digest canvas.
   • On MQTT command, shells out to swaymsg ($SWAYSOCK) and manages app processes (Firefox, Steam Link, mpv).
   • Security principle: the LLM never gets a raw network path to the thin client. Every control path is LLM tool call → HA service call → MQTT → thinclient-agent, mirroring the Phase 6/8 "HA mediates, nothing auto-acts" precedent.
5. Dedicate one Sway workspace to a kiosk Firefox window pointed at digest-engine's local HTTP endpoint (Phase 12) — this doubles as the LLM's rendering surface for "free windows/graphics."
6. Spotify via spotifyd/librespot (Connect receiver, no GUI login, Premium required, unofficial protocol — minor ongoing-maintenance risk, not a blocker).
7. Steam Link via Flathub Flatpak, run under Xwayland (documented workaround for native-Wayland black-screen/flicker bugs on wlroots).
8. Voice interactivity:
   • Only the rooms with a chosen thin client and an attached mic run wyoming-satellite (openWakeWord), streaming to the existing Phase 3 Wyoming faster-whisper/Piper pipeline — no new STT/TTS infrastructure.
   • thinclient-agent accepts an Assist-resolved "play my digest" intent: switches the dedicated Firefox workspace into the "full/thorough" canvas view (vs. the HA dashboard's "compact" view) and narrates via the existing Piper TTS output.
   • Room/person routing: reuse Phase 2's presence system (person.*/area entities). If exactly one recognized person is in the room where the wake word fired, play that person's personal digest section. If more than one is present, Assist asks "whose digest?" and disambiguates by spoken name before playing the personal section — never guesses. Political/household sections always play regardless of presence ambiguity.
9. Network placement: plain trusted LAN for now (no VLAN precedent exists for a general client device class yet — only the unbuilt camera-VLAN concept). Revisitable later as a Phase-10-style expansion item, not a blocker now.
10. Validate the image boots to a working kiosk session (Sway, local mpv/Spotify playback) with Mosquitto/HA/container-host powered off — must not hang waiting on the network, same "reactive path never depends on a remote service" philosophy applied to the thin client's own boot path.

Phase 12 — Quarter-daily LLM digest

1. Add digest-engine to hosts/container-host's compose stack via the existing ENABLE_X/X_BLOCK pattern (ENABLE_DIGEST_ENGINE="false", off by default until credentials are provisioned). Its build context points at the new top-level digest-engine/ directory — the first locally-built image in the stack (everything else pulls prebuilt registry images).
2. Add a companion digest-web static-file service (nginx:alpine/Caddy) in the same block, serving the shared output volume read-only to both display surfaces: the thin client's Firefox workspace (compact-vs-full toggle) and an HA Lovelace HTML/iframe card.
3. Schedule via a systemd timer, mirroring the existing restic-backup convention: smart-home-digest.service (oneshot, docker compose run --rm digest-engine) + smart-home-digest.timer (OnCalendar=*-*-* 00,06,12,18:00:00, adjust once Ollama contention is settled — see open decisions).
4. Ingestion modules, each independently toggleable, each reading credentials from a not-committed .env:
   • Email — IMAP via imapclient; Gmail needs an App Password (2FA-gated) or OAuth2/XOAUTH2 — App Password recommended for this personal-use case.
   • Signal — signal-cli linked as a secondary device (JSON-RPC daemon mode); lowest risk of the four message platforms.
   • Telegram — Telethon (MTProto, logs in as the real account) since the Bot API can't read personal DMs; this is a userbot, ToS-grey but lower enforcement risk than WhatsApp/Discord-selfbot.
   • Discord — discord.py bot with Message Content intent, scoped to servers you own/admin only; cannot read personal DMs or others' servers without a selfbot (not built — real ban risk, explicit ToS violation).
   • WhatsApp — no officially-sanctioned API option exists. Rather than a protocol-reimplementation library (Baileys), run a small whatsapp-bridge sidecar (Node.js, digest-engine/whatsapp-bridge/): a real Chromium logged into the actual web.whatsapp.com client via whatsapp-web.js (Puppeteer), inside its own container running Xvfb so Chromium executes headful (not headless: true) — WhatsApp's automation detection specifically fingerprints headless Chrome, so a virtual-display "real browser" session is meaningfully lower-risk than either Baileys or true-headless whatsapp-web.js, though not zero-risk (it's still automated use of a personal account). One-time interactive QR-code login persists a session directory (mounted volume) so subsequent runs don't need re-scanning. The bridge exposes incoming messages over a local-only channel (e.g. a Unix socket or a small internal HTTP endpoint on the compose network, never published to the LAN) that digest-engine/ingest/whatsapp_ingest.py reads each run. Still gate behind ENABLE_WHATSAPP_INGEST="false", off by default, with a warning in script output + digest-engine/README.md; recommend a secondary/non-critical number if enabled. Build this one last.
   • News — feedparser over a curated OPML list (digest-engine/feeds/curated-feeds.opml), seeded with the confirmed https://www.marxist.com/feed/rss plus a mainstream-outlet list (exact outlets: see open decisions).
   • Financial — FRED API (macro/unemployment, e.g. UNRATE) + Stooq keyless CSV (stocks/oil/commodities, preferred over Alpha Vantage's tight free-tier cap).
5. LLM synthesis: assemble the run's ingested content into context, call the existing Phase 3 Ollama host with three separate prompt templates (digest-engine/synth/prompts/{personal,political,household}.md):
   • Personal — from personal-flagged mail/messages.
   • Political — Marxist/working-class analytical framing (marxist.com feed as theoretical basis) synthesizing mainstream news + financial indicators + politically-flagged mail, laid out on the "holo globe" with colored/glowing markers (e.g. revolutionary-situation markers in red with a hammer-and-sickle/star motif).
   • Household/calendar — from the existing Nextcloud CalDAV integration (Phase 8) and Grocy state (Phase 7).
   • A detail-level parameter (compact for the HA iframe, full for the thin-client fullscreen view) makes the thin-client rendering genuinely more thorough without needing two independent generation passes.
6. Rendering: vendor the offline digest-canvas SDK under digest-engine/render/digest-canvas-sdk/ (globe + addMarker(), window/panel chrome, glow/holo CSS utility, no CDN dependency). Each run's LLM job is to call into this SDK with structured content, not hand-roll projection math. Use a custom inline SVG or Unicode ☭ (U+262D, explicit font-fallback + CSS glow) for hammer-and-sickle iconography since Nerd Fonts has no such glyph.
7. Live follow-up voice Q&A: persist each run's actually-used ingested-context bundle (not full raw content) as digest-engine/output/<run-timestamp>/context.json. Expose a small HA tool (digest_followup_query) so a spoken follow-up ("tell me more about the unemployment numbers") feeds the cached context + question back into Ollama for a grounded, low-latency answer — no fresh ingestion pass. The answer can push a new small window/card onto the already-open thin-client canvas via a websocket, keeping the "flexible windows" idea alive live, not just at generation time.
8. Security/scope principle: digest-engine is the first component in this project with routine WAN egress (mail, message platforms, news, financial APIs). Run it as its own compose service, no inbound port exposure beyond digest-web's read-only LAN serving; credentials in a git-ignored .env. Everything here is read-only summarization — it must never perform a write action anywhere (no auto-reply, no mail archive/delete beyond what IMAP fetch requires, no CalDAV/Grocy writes, no message-platform writes), extending the Phase 6/8 "no silent mutation" precedent to its logical extreme: no mutation path exists at all.

Phase 13 — On-demand admin canvas (sys-admin-llm display surface)

No new hardware — this reuses the Phase 11 thin client and Phase 12's container host.

1. New top-level admin-canvas/ directory (admin-canvas/README.md): a small stdlib-only Python HTTP service (server.py), the write-side counterpart to digest-engine — except long-running (restart: unless-stopped) rather than a oneshot, since content here arrives whenever Home Assistant's tool-calling LLM (the household's "sys-admin-llm", in its admin/ops-facing role — distinct from digest-engine's own synthesis LLM) decides to push something, not on a schedule. Two bearer-token-gated endpoints: POST /show (JSON {"windows": [...]}, overwrites output/latest.json, no run history — this is "what's on screen right now", not a scheduled artifact) and POST /media/<filename> (raw image/video bytes, filename allowlist-validated by both pattern and extension before being written under output/media/).
2. No published port on admin-canvas itself. It is reachable only from other containers on the container host's compose network — i.e. Home Assistant — the same trust boundary mosquitto/homeassistant already share. A companion admin-web (nginx:alpine, LAN-published, read-only, wired into setup-container-host.sh behind ENABLE_ADMIN_CANVAS) serves the shared output/ volume plus the vendored SDK, exactly mirroring digest-web.
3. Rendering: admin-canvas/render/canvas-sdk/ is a duplicated, not shared, copy of the digest-canvas SDK's window chrome and glow theme (renamed digest- → admin- throughout) — a deliberate choice to keep the two canvases fully decoupled rather than extracting a shared library out of a working, already-documented Phase 12 component. The globe kind is dropped (nothing here is a lat/lon marker); four kinds are added instead: stat (a big number/label/unit/trend), image and video (same "no scheme, no leading /, no .." src validation as the thin-client's own MQTT-payload invariant below, just relocated to this write API's trust boundary), and chart (dependency-free inline SVG — bar or sparkline, no charting library, consistent with this project's existing no-unnecessary-deps calls). Same degrade-instead-of-throw philosophy as the digest's renderer: a malformed window falls back to a <pre> dump, never a blank page.
4. Thin client: a fourth Sway workspace 4:admin (hosts/thin-client/configs/sway/config), a new thinclient_agent/admin_canvas.py module (sibling of digest_canvas.py, but simpler — it never inspects its own MQTT payload at all, since there is nothing content-specific for it to decide), and a single new HA button entity "Show admin canvas" (mqtt_discovery.py's register_admin_canvas). Unlike the digest workspace, 4:admin is not auto-launched at session start — this surface is on-demand by nature, so it starts empty until the first "Show admin canvas" command. Its own Firefox profile and launcher script (admin-browser), on the same kill-and-relaunch approach as digest-browser, with both scripts' pkill/pgrep patterns scoped to their own --profile path specifically so the two can never kill or race against each other.
5. Security principle, unchanged from Phase 11.4: this does not add a network path to the thin client. "Show admin canvas" only ever switches workspace and opens a fixed, locally-configured URL (ADMIN_WEB_URL/canvas.html) — identical in shape to "Show digest canvas". All actual content takes a completely separate path that never touches the thin client's MQTT surface: sys-admin-llm → HA tool call → HA rest_command → admin-canvas's write API → admin-web → the browser's own poll (every 15s, shorter than the digest's 5 minutes since this is meant to feel closer to live).
6. Nothing under this repo builds the HA side — same convention as every other HA integration point in this project (Node-RED flows, CalDAV/Grocy wiring, the Lovelace card mentioned in hosts/thin-client/README.md). admin-canvas/README.md documents the expected rest_command: shape and worked example JSON for each window kind, but the actual HA config, the specific tool/intent definition, and which entities/history it reads to answer something like "show me the kitchen outlet's power draw" are the household's own to build.

Phase 14 — ESP32-S3 voice/display satellite

New hardware: §1.12, V2 revision specifically — V1 has no echo-cancellation circuit and different audio pins, and this phase's firmware will not work on it.

1. New top-level firmware/esp32-s3-touch-lcd-1.85c/ directory — the first ESPHome firmware this repo actually ships (firmware/ruview/ and firmware/esphome-ble-proxy/ remain unbuilt placeholders from Phase 2). voice-display.yaml's display/touch/audio hardware bring-up (the ST77916 QSPI init sequence, CST816 touch, PCA9554 reset-pin wiring, I2S mic/speaker pins) is adapted from a community-verified config for this exact board rather than re-derived, since a wrong register sequence just shows static; see the file's own header comment and firmware/esp32-s3-touch-lcd-1.85c/README.md for the source and the cross-checked V2-specific quirks (EXIO2 reset, 80MHz data rate).
2. Wake word stays on-device, consistent with Phase 11.8's principle: ESPHome's micro_wake_word component (okay_nabu model — the same phrase as the thin client's VOICE_WAKE_WORD, so the household has one wake phrase regardless of which kind of satellite answers) runs TensorFlow Lite wake-word detection on the ESP32-S3 itself and explicitly starts a voice_assistant session on detection, rather than streaming continuously to Home Assistant for server-side spotting. Everything after the wake word streams into the existing Phase 3 Wyoming faster-whisper/Piper Assist pipeline — no new STT/TTS infrastructure, same as Phase 11.8's wyoming-satellite rooms.
3. Screen priority, in order: (1) a media/cover-art page, shown the instant the configured media_player entity's state becomes playing — cover art fetched via ESPHome's online_image platform (entity_picture resolved against a configured ha_base_url, since the device fetches it directly over HTTP rather than through the HA connection); (2) failing that, an idle page cycling every 8 seconds between a clock and the weather; (3) a voice-state visualizer (a colour-coded ring, listening/thinking/replying/error) as an LVGL top_layer overlay, shown regardless of which of the above is underneath — "a visualizer when speaking" is an overlay concern, not a fourth competing page.
4. Both HA entities this device mirrors are placeholders (media_player_entity_id, weather_entity_id in secrets.yaml, see secrets.yaml.example) — no real data source is picked yet, same "don't build against a guess" rule as Phase 12's calendar/Grocy sourcing and Phase 13's power-monitoring entity.
5. This is a per-room device — media status must be that specific room's, never any other room's. A room substitution (used in the device's hostname/AP name/friendly name) and a per-unit secrets.yaml (never shared across units) are both required, mirroring the thin client's per-image THINCLIENT_NAME convention. For rooms with more than one real audio source (thin client + a Spotify Connect speaker + a cast device, say), media_player_entity_id should point at a Home Assistant Universal Media Player that aggregates that room's real entities, rather than any single hardcoded device — see firmware/esp32-s3-touch-lcd-1.85c/README.md's "Multiple rooms" section. This firmware has no way to detect a misconfigured media_player_entity_id pointed at the wrong room; it is a configuration invariant, not something the code can verify at runtime.
6. Nothing under this repo builds the HA side here either — the device is a standard ESPHome device once flashed and adopted (Settings → Devices → Add Device → ESPHome), and plugs into whichever Assist pipeline Phase 3 already has configured. No new add-on, no new container.
7. Validated so far with esphome config voice-display.yaml (ESPHome's own schema validator — passes cleanly), not flashed to real hardware. See the itemized unverified list in firmware/esp32-s3-touch-lcd-1.85c/README.md.

Phase 15 — Headless audio endpoint (rooms without a thin client)

New hardware: §1.13. Two separate build pipelines, arm64 and amd64 — see below for why.

1. New top-level hosts/audio-endpoint/ directory. Per-room independent Spotify Connect, not a synced whole-house stream: each endpoint is its own Spotify Connect device with its own name, exactly mirroring the thin client's own per-room spotifyd/librespot receiver (Phase 11.6). No Snapcast, no shared source, no container-host changes — a self-contained appliance per room, by explicit choice over the alternative (a single synced multi-room source) considered and rejected during planning.
2. Two build pipelines, not one image with a flag: the HiFiBerry Amp2 (arm64's assumed audio hardware) is a Raspberry Pi GPIO HAT with no amd64 equivalent — an x86 mini PC instead uses a USB DAC/amp (e.g. Fosi Audio V3, USB Audio Class, no driver needed). arm64 uses rpi-image-gen (the Raspberry Pi Foundation's current officially-recommended tool for custom images, superseding pi-gen for this use case) to produce a real flashable .img. amd64 reuses the thin client's own live-build toolchain rather than a third, unfamiliar one — a new, much smaller live-build tree with the entire graphical/kiosk stack stripped out, headless boot straight to multi-user.target.
3. Spotify Connect install is apt-first with a documented fallback, reused identically on both variants from hosts/thin-client/live-build/config/hooks/normal/0500-spotify-connect.hook.chroot's own already-established logic (neither spotifyd nor librespot is in Debian bookworm main) — not two independent judgment calls about the same package.
4. Audio output is direct ALSA, not PipeWire, on both variants — deliberately different from the thin client, since this is a single-purpose headless appliance with no multi-app mixing need and no desktop session to run a per-user PipeWire daemon in the first place.
5. Per-room identity differs by variant, each using whichever mechanism its own platform actually supports well: arm64 builds one generic image and sets hostname/Wi-Fi per physical unit via Raspberry Pi Imager's OS Customisation dialog (works on any .img, no rebuild needed); amd64 has no equivalent tool for a generic x86 ISO, so it bakes IMAGE_HOSTNAME in at build time instead, one build per room, matching the thin client's own THINCLIENT_NAME convention. Both converge on the same mechanism at the software layer: spotify-connect-start reads $(hostname) fresh at every service start, regardless of which point in the pipeline set it.
6. rpi-image-gen's exact config/layer YAML schema is the single highest-risk unverified part of this phase — written from the tool's documented structure, not hands-on validated (no ARM build environment available). See hosts/audio-endpoint/README.md's prominent callout before a real build.
7. Neither image gives Home Assistant any control or visibility over these endpoints — no MQTT, no HA entities. Deliberate scope decision matching the "per-room independent" choice, not an oversight; revisit as a separate addition if remote control/monitoring from HA is wanted later.
8. Nothing built or flashed on real hardware — no Raspberry Pi, no HiFiBerry Amp2, no x86 test box available. See the itemized unverified list in hosts/audio-endpoint/README.md.

Phase 16 — Sway touch panel

New hardware: §1.14. No container-host changes.

1. New top-level hosts/touch-panel/ directory — a different device, not a variant of the Phase 11 thin client: that image is a couch-distance media station deliberately built around remote control (HA/MQTT, wayvnc) with no on-screen bars; this one is touched directly, so it inverts that — an always-on touch dock, native Wayland touch input, no gesture-camera/remote-view machinery. It reuses the thin client's live-build toolchain and configs/+agent/ split, not its live-build tree, exactly the relationship hosts/audio-endpoint/'s amd64 build already has to the thin client's.
2. Three fixed workspaces, no on-demand ones: 1:spotify (the real, official Spotify Linux GUI client via Flathub com.spotify.Client — not the thin client's headless spotifyd/librespot Connect receiver, since the whole point of a touch panel is a screen you interact with directly), 2:home (a persistent Chromium --kiosk --app=$HA_URL window, auto-restarted on crash by its own supervising loop rather than by sway or systemd), and 3:web (Firefox, minimal chrome — general browsing, not kiosk-locked, mirroring the thin client's own web_browser app in shape but on a separate, duplicated profile).
3. Touch navigation is a persistent on-screen dock (eww, layer-shell :exclusive true), not a keyboard/mouse-driven bar {}: reserving a strip at the bottom means the dock can never be covered by, or lose screen area behind, whatever app is on screen — the opposite trade-off from the thin client's deliberately bar-less, overlay-only widgets. Same session-scoped-not-agent ownership rule as the thin client's now-playing widget: this is compositor UI, not device control, and dies with sway rather than opening a second inbound channel into the security-sensitive agent.
4. touchpanel-agent mirrors thinclient-agent's exact security boundary: LLM tool call → HA service call → MQTT → this agent, never a direct network path. It publishes a Screen select (the touch dock's HA-reachable equivalent) and Show Spotify / Show Home / Show web browser buttons — the latter two focus the already-running app rather than relaunching it (both are meant to stay open and stateful), unlike the thin client's stateless kill-and-relaunch digest/ admin kiosk pages. Also publishes a media_player-shaped entity set bridged from Spotify's own MPRIS interface, identical in shape to the thin client's MPRIS bridge, pointed at the spotify player name instead of mpv/spotifyd.
5. An on-screen keyboard (wvkbd) with no auto-show, by explicit choice: proper show-on-text-field-focus needs the text-input-v3/virtual-keyboard-v1 protocols wired through a shell component (the way Phosh does it for squeekboard), which this image doesn't run. Rather than build that integration now, the dock's Keyboard button is a manual toggle — a documented, accepted limitation, not a gap hidden from the README.
6. No wayvnc on this image, unlike the thin client — a deliberate scope decision, not an oversight: the primary interaction model here is a finger at the panel, not a remote viewer, so SSH alone covers the "something's broken, fix it from a shell" case. Revisit only if a real need for remote view shows up once this runs on real hardware; adding wayvnc back would be a small, isolated change (see hosts/touch-panel/README.md).
7. Both apt-first/documented-fallback installs already established elsewhere in this repo are reused, not reinvented: eww (same as the thin client's now-playing widget) and wvkbd — neither is in Debian bookworm main, and both hooks stop at a documented placeholder rather than a hardcoded release URL if apt doesn't have them.
8. Nothing built or flashed on real hardware — no touch-panel unit has been chosen (§1.14), and in particular whether the touchscreen exposes itself as a native Wayland wl_touch device (this image's one hard assumption, since there is no pointer-emulation fallback the way the thin client's gesture control has one) is unverified. See the itemized list in hosts/touch-panel/README.md.

Phase 17 — Kitchen/fridge display (camera-vision grocery cataloguing)

New hardware: §1.15. New top-level pantry-vision/ directory (container-host service) plus hosts/kitchen-display/ (a third, simpler kiosk image).

1. The workflow this phase is built around, stated once because everything below serves it: come home, put down the shopping bag, hold one item up to the kitchen display's camera, the system proposes what it is and roughly how long it keeps, the person confirms (editing anything wrong) before anything is written anywhere, put the item away. The same display then shows inventory ordered by what expires soonest, and Grocy's recipes, on request.
2. pantry-vision never writes to Grocy from /identify alone — only /confirm, a separate human-reviewed call, ever does. This is the single guardrail the whole phase is designed around, the same "propose, never auto-commit" rule this project already applies to identity-merge confirmation (the Identity store row in §2). A wrong camera guess costs one tap to fix on the kiosk screen, not a wrong fact silently written into the household's inventory.
3. A real, LAN-published network listener — the one deliberate exception to this project's usual "the LLM only ever reaches a device through HA→MQTT" shape. admin-canvas (Phase 13) has no published port because only Home Assistant calls it; pantry-vision is different on purpose because the kitchen display is a separate physical device that has to get an identification back synchronously, with no HA round-trip in the loop for that specific call. The bearer token — not network placement — is the actual boundary here; see pantry-vision/README.md's "A real network listener, unlike admin-canvas" section. Controlling which screen is showing (Scan/Inventory/Recipes) is a separate, narrower path that does stay HA/MQTT-mediated: kitchen-display-agent, identical in shape to every other host's agent in this project.
4. Grocy is the system of record, not a new inventory store — pantry-vision is a client of Grocy's own REST API (already running unconditionally since Phase 1/7) for both reads (/inventory, /recipes) and the one write (/confirm → Grocy stock). Nothing here duplicates or replaces Grocy's own data model.
5. The vision-identification prompt asks for one photo, not a live video stream, and degrades to a low-confidence placeholder proposal (never an error page) if the model call fails or its response isn't parseable JSON — same "degrade, don't blank" rule as the digest/admin canvas renderers and llm_client.py's own _fallback_document.
6. The kitchen display has no native camera-capture app at all — Chromium's own getUserMedia(), called from pantry-vision/frontend/app.js, talks to the webcam directly inside the kiosk page and hands a captured frame straight to POST /identify as a blob. --use-fake-ui-for-media-stream on the kiosk launch auto-accepts the permission prompt that would otherwise sit unanswered on an unattended screen.
7. hosts/kitchen-display/ is one workspace, one app — no touch dock, no multi-app switching the way hosts/touch-panel has: Scan/Inventory/Recipes is in-page tab navigation inside pantry-vision/frontend/'s single-page app, since there's only one thing this device does. kitchen-display-agent's MQTT surface is correspondingly narrow: three "Show X" buttons and nothing else — no media_player, no capture/audio/remote-desktop entities.
8. The Grocy API integration is written from documentation, not verified against a live instance — the exact GET /api/stock response shape (whether product names arrive nested by default), the minimum required fields for POST /api/objects/products, and whether the Recipes/fulfillment endpoints need explicit setup before they return anything meaningful are all flagged in pantry-vision/README.md alongside the live OpenAPI spec URL every real Grocy instance exposes (/api/openapi/specification) as the way to actually check.
9. The vision model choice is unpicked and unbenchmarked (OLLAMA_VISION_MODEL defaults to llava, not confirmed pulled or even correct for a given LLM host's GPU/CPU tier) — same "don't build against a guess" flag as every other LLM-model/hardware pairing in this plan. Latency in particular is unmeasured and is the number that decides whether "hold it up, wait, confirm" feels usable at the counter or like standing there for 30 seconds.
10. Nothing built or run against real hardware, a real camera, a real vision model, or a real Grocy instance. See the itemized lists in pantry-vision/README.md and hosts/kitchen-display/README.md.

Testing checklist before calling any phase "done"

• Does the reactive path (presence → light on) work with the LLM host powered off? (It must.)
• Does a bad/slow LLM response ever block a light switch? (It must not.)
• Are cameras verified to have zero WAN egress?
• Does the calendar integration survive a Nextcloud restart without orphaning entities?
• Does an identity merge ever get auto-committed without confirmation? (It must not.)
• Does the thin client boot to a usable kiosk session with Mosquitto/HA/container-host powered off? (It must not hang.)
• Can the LLM reach the thin client through any path other than HA service call → MQTT → thinclient-agent? (It must not.)
• Does digest-engine ever perform a write action anywhere? (It must not — read-only only.)
• Is WhatsApp ingestion off by default, requiring the explicit opt-in + warning? (It must be.)
• Does voice digest playback ever read the wrong person's personal section when multiple people are present? (It must not — must ask, never guess.)
• Does a follow-up voice question ever trigger a write, or only grounded read/synthesis against cached context? (It must stay read-only.)
• If a run's LLM output produces malformed canvas-SDK calls, does the digest fall back to plain text instead of a broken/blank page?
• Are all ingestion platform credentials kept out of git (.env, gitignored), matching the restic-password handling convention?
• Does a stale/unreachable "was the digest viewed" signal ever cause runs to merge forever, instead of degrading to "assume viewed" after one missed check? (It must degrade, not compound.)
• Does the compact HA-dashboard iframe view ever mark a digest as viewed? (It must not — only an actual thin-client canvas display or voice playback counts.)
• Does the counter run actually drop a fabricated quote/figure/theoretical connection, rather than waving it through? (It must drop it.)
• Does the counter run ever flag a correctly-grounded piece of Marxist analysis as "unverifiable" for being theoretical rather than a bare fact? (It must not — see synth/prompts/counter_run.md.)
• If the counter run itself fails to reach the LLM host, is the original document kept and marked unverified, rather than either passed through silently or blanked? (It must be marked, not silently either extreme.)
• Does admin-canvas ever accept a request without a valid bearer token? (It must not — an unset ADMIN_CANVAS_TOKEN must fail closed, reject everything, not "auth optional".)
• Is admin-canvas's write port ever published to the LAN in the generated compose file? (It must not be — reachable only from other containers on the compose network.)
• Does admin-canvas ever accept an image/video window whose src isn't a bare media/<filename> path (no scheme, no leading /, no ..)? (It must not — same invariant as mqtt_discovery.py's "a payload never becomes a URL host", enforced server-side and again client-side.)
• Can the sys-admin-llm reach the thin client through any path other than "Show admin canvas" → MQTT → thinclient-agent, with all actual content arriving via the separate admin-canvas write API instead? (It must not — same boundary as the digest canvas.)
• Does opening the admin canvas ever kill the digest canvas's kiosk Firefox window, or vice versa? (It must not — both digest-browser and admin-browser's pkill/pgrep patterns are scoped to their own --profile path.)
• If admin-canvas's output/latest.json holds a malformed or unrecognized window, does the admin canvas fall back to plain text instead of a broken/blank page? (Same rule as the digest's canvas-SDK renderer.)
• Does the voice/display satellite's wake-word spotting ever stream continuously to Home Assistant instead of triggering locally? (It must not — on-device micro_wake_word, same Phase 11.8 principle as the thin client's wyoming-satellite rooms.)
• Does the idle weather/time/date cycle ever show over an active media page, or does media ever fail to take priority the instant playback starts? (It must not — media priority is the one hard behavioral requirement of Phase 14.)
• Does the voice-state visualizer ever replace or hide the underlying page's content instead of overlaying it? (It must not — it's a top_layer overlay by design, never a page swap.)
• Is the ESP32-S3-Touch-LCD-1.85C firmware ever flashed onto a V1 board? (It must not be — V1 has no AEC circuit and different audio pins; this phase's config assumes V2 throughout.)
• Does a headless audio endpoint's Spotify Connect device name ever fail to match the room it's actually in? (It must not — spotify-connect-start always reads $(hostname), which is set per-room by Raspberry Pi Imager on arm64 or IMAGE_HOSTNAME at build time on amd64; a copy-pasted/reused image for two rooms is a configuration mistake this software has no way to detect, same class of risk already called out for the ESP32 firmware's media_player_entity_id.)
• Does the headless audio endpoint ever gain a shared/synced audio path, an MQTT connection, or an HA entity without a deliberate new decision to add one? (It must not — Phase 15 is scoped to per-room independent Spotify Connect only, on purpose.)
• Can the LLM reach the touch panel through any path other than HA service call → MQTT → touchpanel-agent? (It must not — identical boundary to the thin client's.)
• Does the touch panel boot to a usable session (Spotify + the browser) with the container host powered off? (It must not hang — Home showing a connection error is the expected/acceptable degraded state, since unlike the thin client this device has no local media source to fall back to.)
• Does tapping the touch dock's Home/Spotify buttons ever relaunch (rather than just focus) an already-running window? (It must not — both are stateful apps; relaunching would throw away login/scroll/playback state for no reason.)
• Does pantry-vision's /identify endpoint ever write to Grocy by itself? (It must not — only a separate, human-reviewed /confirm call ever writes anything.)
• Does pantry-vision ever accept a request without a valid bearer token, on any of its four endpoints — including the two GETs? (It must not — unlike admin-canvas, this service is LAN-published, so the token is the actual boundary, not network placement.)
• Can the LLM reach the kitchen display through any path other than HA service call → MQTT → kitchen-display-agent, for which screen is showing? (It must not — reading/writing the actual inventory is a separate, intentionally-published path through pantry-vision itself, not a violation of this rule.)
• If the vision model's response is unparseable or the call fails outright, does /identify ever return a broken/blank result instead of a flagged, low-confidence placeholder proposal? (It must not.)

---

4. Open decisions (Phases 11–17)

These need a decision before their respective implementation steps can be built — everything above is written to accommodate any answer, but nothing should be built against an unresolved item.

1. RDP vs. VNC vs. desktop-environment swap — resolved: wayvnc (VNC) replaces RDP for this project; Sway is kept.
2. WhatsApp ingestion approach — resolved: whatsapp-bridge (headful Chromium + whatsapp-web.js in an Xvfb virtual display), not Baileys. Still opt-in (ENABLE_WHATSAPP_INGEST); still recommend a secondary/non-critical number, since automating a personal account carries some risk even via the real web client.
3. Mainstream news source list — digest-engine/feeds/curated-feeds.opml ships with marxist.com/feed/rss (real) plus BBC World/Al Jazeera/Guardian World/DW as clearly-marked placeholders (Reuters/AP were skipped — both have restricted their public RSS and guessing a live URL seemed worse than an honest placeholder). Still needs the user's actual sign-off/edit.
4. Ollama contention — not yet resolved; DIGEST_SCHEDULE defaults to 00,06,12,18 (every 6h) in setup-container-host.sh, unadjusted for Assist-traffic overlap. Revisit once real usage patterns are known.
5. Credential storage — implemented as a git-ignored .env seeded from digest-engine.env.example (matches the restic-password precedent); age/sops was not built, considered adequate for now.
6. Exact mic-enabled room list — still needed; ENABLE_VOICE_SATELLITE in build-thin-client-iso.sh defaults to false per-image until rooms are chosen.
7. Exact thin-client hardware target — still needed; nothing in the built image assumes specific hardware, but Steam Link/Xwayland decode performance can't be validated without it.
8. Personal-digest visibility on shared displays — not yet resolved; not blocking, since the thin-client rendering built so far doesn't yet distinguish "shared screen" from "private."
9. Household/calendar ingestion has no real data source yet (new, found during Phase 12 implementation) — the plan named CalDAV (Phase 8) and Grocy (Phase 7) as the source but no ingest module was written for either; digest-engine/run.py currently passes empty calendar/Grocy context and the household prompt is told to say "nothing scheduled" rather than hallucinate. Needs either a new ingest module or a decision to pull this from HA directly.
10. HA has no core MQTT media_player platform (new, found during Phase 11 implementation) — thinclient-agent publishes the media_player discovery payload as specified, but stock Home Assistant ignores it without the HACS "MQTT Media Player" custom integration installed. button/sensor/number entities are also published as a fallback that works on a plain HA install; decide whether to install the HACS integration or keep relying on the fallback entities.
11. Several package-availability items still need verification on real hardware before first boot, all flagged in-code rather than guessed: the Steam Link Flatpak app ID (com.valvesoftware.SteamLink), spotifyd/librespot packaging on Debian bookworm (not in main — three fallback install routes documented, none wired to a hardcoded download URL), and mpv-mpris packaging.
12. wayvnc ships with no password — start-wayvnc fails closed on a sentinel value (CHANGEME-SET-ON-FIRST-BOOT) rather than serving unauthenticated VNC; a real password must be generated on the booted machine before wayvnc will start (see hosts/thin-client/README.md).
13. The sys-admin-llm's HA-side wiring has no real data source or tool definition yet (new, Phase 13) — same shape as open decision #9 above: admin-canvas/README.md documents the expected rest_command:/tool-call contract and worked example JSON, but which HA entities/history back something like "the kitchen outlet's power draw" is unresolved, and no metering-capable Zigbee smart plug is in this plan's hardware list (§1.4) yet. Needs a hardware decision (a power-monitoring outlet) and an actual HA tool/script, neither of which exists in this repo by design — see the Phase 13 "nothing under this repo builds the HA side" note.
14. The voice/display satellite has no real hardware verification, and two of its data sources are unpicked (new, Phase 14) — firmware/esp32-s3-touch-lcd-1.85c/voice-display.yaml passes ESPHome's own config validator but has never been flashed to a physical unit; the display init sequence, the AEC audio path, and wake-word sensitivity are all adapted/assumed, not measured (see the itemized list in that directory's README). Separately, media_player_entity_id and weather_entity_id are placeholders — which media player this unit should mirror needs a decision if the household ends up with more than one active at a time.
15. The headless audio endpoint has no real hardware verification on either architecture (new, Phase 15) — neither the arm64 (rpi-image-gen) nor amd64 (live-build) pipeline has been run to completion, let alone flashed and booted. rpi-image-gen's exact config/layer YAML schema in particular was written from documentation only, with no ARM build environment available to validate it against the real tool — see hosts/audio-endpoint/README.md's prominent risk callout. Also still open: whether spotifyd or librespot actually installs cleanly from current bookworm apt sources at all (the fallback list is documented, not exercised), and how stable USB DAC/amp ALSA enumeration turns out to be on a real amd64 mini PC.
16. No touch-panel hardware has been chosen (new, Phase 16) — nothing in hosts/touch-panel/ has been booted on real metal or a real touchscreen. In particular, whether the panel's touch input actually surfaces as a native Wayland wl_touch device (this image's one hard assumption) is unverified — see hosts/touch-panel/README.md's verification list, item 3.
17. The touch panel's Spotify/Chromium integration details are assumed, not confirmed (new, Phase 16) — the Flathub app ID com.spotify.Client, the Spotify Flatpak's real MPRIS bus name (assumed spotify), and Chromium's Wayland app_id for a --app= kiosk window (assumed to start with chromium) are all flagged for on-hardware verification in hosts/touch-panel/README.md.
18. No vision-capable Ollama model has been picked, pulled, or benchmarked (new, Phase 17) — pantry-vision's OLLAMA_VISION_MODEL defaults to llava with no confirmation it's the right choice for any given LLM host's hardware tier, and grocery-item identification latency/accuracy is completely unmeasured. This is the single highest-risk unknown in Phase 17: if it's too slow or too inaccurate, "hold item up to camera" stops being a usable interaction and the phase needs a different approach (a smaller/faster model, a barcode-scan fallback via Grocy's existing scanning support, or both).
19. pantry-vision's Grocy API integration is written from documentation, not a live instance (new, Phase 17) — GET /api/stock's response shape, POST /api/objects/products's minimum required fields, and the Recipes/fulfillment endpoints' setup requirements are all assumed; pantry-vision/README.md points at each real Grocy instance's own live OpenAPI spec (/api/openapi/specification) as the way to check before trusting any of it. GROCY_DEFAULT_LOCATION_ID/GROCY_DEFAULT_QU_ID are fresh-install-default guesses that need confirming against Settings → Locations/Quantity units on the real instance too.