Phase 19: trash-calendar (Kennelbach ICS -> CalDAV) and transit (GTFS
departures + OpenTripPlanner-backed voice route planning, slow walking
speed assumed). Phase 20: Tapo camera integration via identity's new
Frigate face-recognition presence signal, and chores, a presence/
calendar-driven household task nudger ("I don't care who does it, as
long as it gets done") with a passive fairness tally, litter culprit
attribution, and per-person chore_exempt/chore_reminder_style settings
on identity.
Also: the HA integrations + hardware-monitoring catalogs, Music
Assistant wiring, docs/network-integration.md (OPNsense VLAN
segmentation, no WAN port-forwards), the ESPHome BLE-proxy firmware,
and RuView CSI-presence integration (github.com/ruvnet/ruview) with
household automations for sleep/distress/concurrent-vitals/bathroom-
occupancy — all flagged with their real unverified assumptions and
open decisions. Fixed two real port collisions surfaced while wiring
this in (OTP vs. zigbee2mqtt on 8080, Music Assistant vs. pantry-vision
on 8095).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K6CrKjW3yVptUnjG35HjC1
identity/ (Phase 6, rewritten from the original Node-RED sketch): a
person <-> BLE-identifier registry built directly as a Python service.
Solves multi-phone (multiple identifiers per person), anti-spoofing
(only allowlisted IRK-resolved/fixed-tag entities are ever accepted as
registration candidates, never a raw or randomized MAC), device-less
people (a grandmother without a smartphone gets a no_device registration
plus a hand-operated home/away toggle, reported as "unknown" rather than
a false "away"), and anonymous guests (POST /register/guest, no name
needed). Every person gets an automatic profile picture from their most
recent registration photo. /presence also reports a best-effort room per
person as groundwork for an eventual floor-plan view (not built).
Registration is single-utterance voice ("register me as <name>") or a
touchscreen form; ambiguous/conflicting candidates never auto-commit.
hosts/door-panel/ (Phase 18, new host): structurally kitchen-display's
twin - one Sway workspace, one Chromium kiosk window - defaulting to
identity's weather+clothing/who's-home/groceries-running-low dashboard,
with voice registration as its actual purpose (mic on by default, unlike
every other host's opt-in).
hosts/kitchen-display/: adds opt-in voice satellite and a "Show
registration" screen pointed at identity's register.html, reusing the
same camera-equipped-endpoint registration flow.
pantry-vision/: adds GET /shopping-list (Grocy's own volatile/missing
products, reshaped) for the door panel's "running low" section.
setup-container-host.sh: wires ENABLE_IDENTITY (identity + identity-web,
published like pantry-vision since kiosk browsers call it directly, plus
a persistent SQLite/photos volume unlike pantry-vision's stateless
design) and fixes the HA_URL example to the host's real LAN IP (HA runs
network_mode: host, unreachable by container name - the same situation
Node-RED's own config already documents).
docs/: Phase 6 rewritten, Phase 18 added, hardware/software/guardrail/
open-decision entries throughout project-plan.md and README.md.
components.md gains RuView presence nodes (one per room), fixed BLE tags,
and 2 spare webcams, with the price estimate updated to match.
Nothing here has been run against real hardware, a real HA instance, or
real Private BLE Device entities - TRUSTED_ENTITY_PREFIXES above all
needs checking against a live instance before registration finds
anything. See identity/README.md and hosts/door-panel/README.md for the
itemized verification lists.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K6CrKjW3yVptUnjG35HjC1
hosts/touch-panel/: a touch-driven Sway kiosk with a full Spotify GUI client
(Flathub), a dedicated Home Assistant Chromium kiosk window, a general web
browser, an always-on eww touch dock for app switching, an on-screen keyboard
(wvkbd), and touchpanel-agent giving HA/the LLM the same MQTT-mediated control
the thin client has.
hosts/kitchen-display/ + pantry-vision/: a camera-vision grocery cataloguing
endpoint for the fridge/pantry. The kiosk's browser captures a photo via
getUserMedia and sends it to the new pantry-vision service, which asks an
Ollama vision model to identify the item and estimate shelf life; the
proposal is always human-confirmed before being written into Grocy stock.
The same display shows inventory sorted by soonest-to-expire and Grocy's
recipes.
Wires ENABLE_PANTRY_VISION into setup-container-host.sh and documents both
phases in docs/project-plan.md and README.md, matching this project's
existing per-host hardware/software/guardrail/open-decision conventions.
Nothing here has been run against real hardware, a real camera, or a real
Grocy/vision-model instance — see each new README's verification list.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K6CrKjW3yVptUnjG35HjC1
Three additions to the thin-client/container-host stack:
- admin-canvas + admin-web (Phase 13): an on-demand display surface for the
sys-admin-llm, mirroring digest-engine/digest-web's write/read split but as
a small always-on token-gated write API instead of a scheduled job. New
"4:admin" thin-client workspace, MQTT button, and duplicated canvas-sdk with
stat/image/video/chart window kinds.
- ESPHome firmware for the Waveshare ESP32-S3-Touch-LCD-1.85C-V2 (Phase 14):
a voice satellite + round-display status screen, evaluated as a DIY
alternative to Home Assistant Voice PE. Media/cover-art always takes
priority over an idle weather/time/date cycle, with a voice-state
visualizer overlaid on top. On-device wake word (micro_wake_word) feeding
the existing Assist pipeline, per-room "media_player_entity_id" wiring
documented explicitly to avoid cross-room bleed. Validated with ESPHome's
own config validator; not yet flashed to real hardware.
- Capture-card ("receiver box") video source selection on the thin client: a
new "5:capture" workspace and HA select that dynamically enumerates
USB/PCIe capture cards (re-scanned periodically for hot-plug), launches
mpv against the chosen one, and unconditionally excludes the gesture-
control camera from the list to preserve its existing no-HA-entity
invariant.
Also fixes a pre-existing bug found along the way: digest-browser's pkill
pattern matched any --kiosk mpv/firefox process rather than its own, which
would have let the new admin-browser/capture-view scripts kill each other's
windows.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WgTwMqiho8DT6ue5S49WBP
Implements both new subsystems from the project plan end to end:
- hosts/thin-client/: Debian live-build Sway kiosk image — greetd autologin,
wayvnc (replaces RDP, per open decision #1), thinclient-agent (HA MQTT
discovery + swaymsg control, the sole LLM-reachable surface), mpv/mpv-mpris,
spotifyd/Steam Link, minimal-chrome Firefox with uBlock Origin/SponsorBlock
preinstalled, a fullscreen-aware now-playing widget, persistent audio-output
selection, outbound RDP/VNC via Remmina, HA mobile-app text/pointer remote
control via ydotool, an optional opt-in gesture-control camera (MediaPipe
hand tracking, off by default at two independent gates), a maintenance-shell
keybind, a German default keymap, and an idle-timeout photo slideshow from a
new gallery SMB share on the container host.
- digest-engine/: quarter-daily LLM digest — email/Signal/Telegram/Discord/
WhatsApp ingestion (WhatsApp via a headful-Chromium bridge, opt-in, highest
risk), news (including RCI/marxist.com theoretical sources and a broad
mainstream/state-affiliated outlet list), financial indicators, flight/naval
traffic and an OPNsense Suricata-alerts summary as political-analysis
evidence, Grocy/CalDAV household ingestion, an evening-only recipe and
shopping-list suggestion from soon-to-expire stock, and a vendored
digest-canvas SDK (globe/window/glow) rendering compact and full views. A
digest that goes unviewed (no thin-client canvas display or voice playback)
is merged into the next run instead of being discarded, tracked via a
retained MQTT message on the shared broker.
Both are wired into hosts/container-host/scripts/setup-container-host.sh via
the existing ENABLE_X/X_BLOCK pattern, off by default. docs/project-plan.md and
README.md are updated to match; remaining open decisions and manual-
verification items are tracked in-repo (project-plan.md §4, digest-engine/
README.md, hosts/thin-client/README.md).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPD1RhBsjdFKFLpMCLiVT6
Extends the container-host script to cover the rest of the Phase 1/9 stack
(README's outstanding checklist item) so a single Debian host setup gets
Home Assistant plus the full companion stack: automation glue, monitoring,
a dashboard, push notifications, container management, and optional
scheduled backups. Also fixes a stale comment referencing a network-attached
SLZB-06 coordinator when the plan and config target a USB CC2652P dongle.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Amir Alexander Abdelbaki