The_miro
/
SmartestHome
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Workshop assistant, fleet monitoring, infra health, and shared endpoint surfaces 

Adds the workshop/office assistant and the plumbing several other features
were waiting on. The through-line: every new capability that could act on
its own proposes instead, and says out loud when it does not know something.

New service — workshop/
  Project notebook (workshop.db) plus a never-pruned knowledge store
  (workshop-knowledge.db): standing workflow instructions by activity,
  keyword facts, durable project learnings, and the household's ONE hardware
  inventory. GET /context returns everything applying right now in one call,
  so the assistant is told the standing considerations rather than reminded
  of them. Two databases because they have different lifetimes: rebuilding
  the project store must not take the note about how you solder with it.

  Hardware statuses distinguish reserved (still on the shelf) from in_use
  (installed and working) — "can I use this right now" has different answers
  for the two, and naming a project on an in_use item never silently demotes
  it.

  Gitea repos with append-only history: commit/push/branch yes, unattended;
  force-push/rebase/amend/reset/filter-repo never, enforced server-side by
  branch protection rather than only by this code refusing. When history
  genuinely must be scrubbed, /scrub-request prints the commands for a human
  to run — the manual step is the safety mechanism.

  Fleet scripts: one monitoring-agent script per kind of machine, fetched by
  each endpoint's fleet-bootstrap timer. Remote code execution by design, so
  the constraints are the design — upload is a draft, publishing is separate,
  scripts live in SQLite rather than on the writable share, every version is
  kept, and the endpoint verifies the checksum and reports pass or fail.
  Slots exist for the ESP32s and network appliances that cannot run a script
  at all, holding the CheckMK-server-side config instead.

Infrastructure health
  opnsense becomes a LIST of firewalls, each named, keyed by name rather than
  index. CheckMK joins it. Both are polled by workshop (always-on) and read
  by digest-engine, so the digest can say "critical since Tuesday" instead of
  quoting a six-hour-old snapshot. Three states, because "I could not ask" is
  not "nothing is wrong".

pantry-vision
  All four stock movements are camera-driven; stock counts individual units
  and folds brand-free via Grocy product groups. Door-sensor-triggered
  appliance cameras record sightings as hints with timestamps, never as
  stock — a camera at a door cannot tell in from out.

identity
  Per-person colour and settable profile picture, assigned to avoid collisions
  between people sharing an initial, on the 2-bit-per-channel lattice a colour
  Pebble renders natively.

render/ — shared, vendored, dependency-free
  media-visualiser: two-tier by necessity, since most endpoints have no local
  audio; the synthetic tier says on screen that it is not an analysis.
  floorplan-3d: canvas 2D rather than three.js — the scene is prisms on a
  plane, which an isometric projection draws in ~200 lines, predictably on
  weak panels, with the frontend still at zero dependencies.

Config and fleet plumbing
  Rooms are one vocabulary (an HA area_id) from CoreSystemConfig through the
  builders to suggested_area. Keycloak and FreeIPA are coupled as one
  decision with USR_HA_ group naming, declaration-only for now and validated
  as such. Immich alongside the photo share, read-only. Thin clients get the
  full media-key set for a wireless remote.

Docs: fridge-item-location, workshop-assistant, rooms-and-endpoints,
endpoint-surfaces, pebble-presence-watchface.

Testing is stubbed suites and headless unit checks only — no real Grocy,
camera, vision model, CheckMK, Gitea, Samba or browser has been involved.
The CheckMK API shape and Gitea's branch-protection payload are written from
documentation and have version-sensitive field names.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FanS1vyE2gLhGkqKq6HtYj
main
Amir Alexander Abdelbaki 2026-08-10 14:54:37 +02:00
parent a4cae7d830
commit a948f4b375
72 changed files with 10399 additions and 385 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

74
CoreSystemConfig.json.template
View File

@ -66,6 +66,8 @@
"portainer": true,
"mealie": false,
"gallery_smb": false,
"photos_web": false,
"workshop": false,
"music_assistant": false,
"backups": false
}
@ -109,6 +111,9 @@
"transit": 8099,
"otp": 8100,
"music_assistant": 8101,
"workshop": 8102,
"workshop_web": 8103,
"photos_web": 2283,
"ollama": 11434,
"proxy_http": 80,
"proxy_https": 443
@ -122,15 +127,30 @@
"mqtt_username": "",
"mqtt_password": "",
"ha_token": "",
"opnsense_api_key": "",
"opnsense_api_secret": "",
"_opnsense_keys": "One entry per firewall in the opnsense list above, keyed by its `name`. A single-firewall household writes {\"main\": {...}} and is done. Keyed rather than positional because a list that has to line up with another list by index is a bug waiting for the day somebody reorders one of them.",
"opnsense_keys": {
"main": { "api_key": "", "api_secret": "" }
},
"_checkmk": "A CheckMK automation user with the read-only Guest role, and its AUTOMATION SECRET (Setup -> Users -> the user -> Automation secret) \u2014 not its login password.",
"checkmk_username": "",
"checkmk_secret": "",
"_freeipa_bind_password": "The password for freeipa.bind_dn. Prefer a dedicated service account with READ-ONLY access to the household subtree \u2014 the mirror never writes to the directory, so a bind that can is a grant with no matching use.",
"freeipa_bind_password": "",
"_workshop_token": "Required when container_host.enable.workshop is true. openssl rand -hex 32",
"workshop_token": "",
"_gitea": "Optional, and only for the workshop assistant's own repositories. Gitea -> Settings -> Applications -> Generate New Token, scope write:repository. STRONGLY PREFER a dedicated 'workshop-bot' user scoped to one organisation over a token on your own account: this token can create repositories, and the blast radius of a leaked env file should be one org of generated repos rather than everything you own. The assistant commits and pushes but never rewrites history, and repo deletion is not implemented at all \u2014 see workshop/README.md.",
"gitea_url": "",
"gitea_token": "",
"gitea_owner": "",
"ssh_authorized_key": "",
"kiosk_password": "",
"admin_password_hash": ""
},
"opnsense": {
"_comment": "The household's existing OPNsense firewall. Used by digest-engine's network digest, which pulls a Suricata intrusion-detection summary from it (GET /api/ids/service/status and POST /api/ids/service/query_alerts — two read endpoints, nothing else, ever). A build writes this block plus the secrets above into the container host's IDSconf.json; the long-form documentation for every field lives in digest-engine/IDSconf.json.example. Leave base_url empty to skip that file entirely. TWO THINGS THIS DOES NOT DO: it does not enable Suricata (do that at Services -> Intrusion Detection on the firewall, then download a ruleset), and it does not enable the ingest (set ENABLE_OPNSENSE_IDS_INGEST=true in digest-engine.env). SCOPE THE API KEY: give it its own OPNsense user with only the 'Services: Intrusion Detection' privilege — that privilege still covers api/ids/* including start/stop, because OPNsense ACLs are page-level, so the read-only guarantee comes from digest-engine calling exactly two endpoints and not from the firewall enforcing it.",
"opnsense": [
{
"_comment": "EVERY OPNsense firewall in the household, as a list. It was a single object until more than one firewall became plausible; a list is the shape that does not need changing again, and a one-firewall household just has one entry. `name` distinguishes them everywhere downstream — in the digest's network section, in the workshop health table, and in the alert text itself — so 'the IDS is quiet' can never silently mean 'one of the two is quiet'. Used by digest-engine's network digest (GET /api/ids/service/status and POST /api/ids/service/query_alerts \u2014 two read endpoints, nothing else, ever) and by workshop's health poller. A build writes this block plus the secrets into IDSconf.json; long-form field docs live in digest-engine/IDSconf.json.example. An empty list skips that file entirely. TWO THINGS THIS DOES NOT DO: it does not enable Suricata (Services -> Intrusion Detection on the firewall, then download a ruleset), and it does not enable the ingest (ENABLE_OPNSENSE_IDS_INGEST=true in digest-engine.env). SCOPE EACH API KEY: its own OPNsense user with only the 'Services: Intrusion Detection' privilege \u2014 which still covers api/ids/* including start/stop, because OPNsense ACLs are page-level, so the read-only guarantee comes from this project calling exactly two endpoints and not from the firewall enforcing it.",
"name": "main",
"base_url": "",
"verify_tls": true,
"interfaces": [],
@ -138,6 +158,46 @@
"top_signatures": 8,
"top_hosts": 5,
"packet_capture_reference": ""
}
],
"checkmk": {
"_comment": "An existing CheckMK server, polled read-only for host/service state. Feeds two places: digest-engine's network section (so a failing disk shows up in the quarter-daily digest) and workshop's own infra_status table (so the workshop display can overlay system health on the gallery). Nothing here ever acknowledges, downtimes or reschedules anything \u2014 the API user only needs to read. site is the CheckMK site name, which is the path segment in every URL: http://<host>/<site>/check_mk/api/1.0/...",
"_credentials": "CheckMK: Setup -> Users -> add a user, Roles = 'Guest' (read-only), then its 'Automation secret' \u2014 NOT its login password. Put the username and secret in secrets.checkmk_username / secrets.checkmk_secret. A Guest-role automation user cannot change anything, which is the actual boundary here rather than a promise about which endpoints get called.",
"base_url": "",
"site": "cmk",
"verify_tls": true,
"_only_problems": "true keeps the payload to hosts and services that are not OK, which is what both consumers want; false pulls everything and is mostly useful once, to see what the site knows about.",
"only_problems": true,
"max_rows": 200
},
"identity_provider": {
"_comment": "An EXTERNAL identity provider (Keycloak) in front of the household's web surfaces. NOTHING IN THIS REPO IMPLEMENTS SSO YET \u2014 this block exists so the decision and its values are recorded in one place before the work happens, and so the proxy config can be generated from it when it does. Leave issuer_url empty to skip it entirely, which is the current default and the tested path.",
"_pairs_with_freeipa": "KEYCLOAK AND FREEIPA ARE ONE DECISION, NEVER TWO. Keycloak federates FreeIPA as its user store; it is not a place people are created. Configuring Keycloak without FreeIPA would mean a second, parallel set of household accounts \u2014 which is the specific outcome having a directory exists to prevent \u2014 so the validator refuses either one alone. Group membership comes from FreeIPA and rides through Keycloak as a claim; roles are never assigned in Keycloak itself, or the directory stops being the answer to 'who is in this household'. WHAT IT WOULD PROTECT: the proxy-fronted web UIs (Home Assistant, the photo frontend, workshop, identity's admin panel). WHAT IT MUST NEVER PROTECT: the kiosk-to-service APIs \u2014 pantry-vision, identity's /presence, workshop's own API \u2014 which are bearer-token gated because a wall panel cannot complete an interactive login. Putting an OIDC redirect in front of those turns every kiosk into a dead screen.",
"issuer_url": "",
"_realm": "Keycloak realm name; part of the issuer URL too, kept separately because the proxy config and any client library both want it on its own.",
"realm": "smarthome",
"client_id": "smarthome-proxy",
"_protected_hosts": "Which proxied hostnames would sit behind SSO. Advisory until the work is done \u2014 see the _comment above.",
"protected_hosts": []
},
"freeipa": {
"_comment": "An existing FreeIPA domain, mirrored INTO identity as a source of people and group memberships. NOT IMPLEMENTED YET \u2014 this block records the decision and its values ahead of the work, like identity_provider above. Leave server empty to skip it, which is the current default and the only tested path.",
"_direction": "ONE WAY, FreeIPA -> identity, and it must stay that way. identity holds household facts FreeIPA has no opinion about (BLE identifiers, chore reminder style, digest preferences, a colour) and writing any of that back would make a directory serving real logins into a store of smart-home preferences. A mirrored person is matched on uid and their name/group memberships are refreshed; everything else identity knows about them is left alone.",
"_groups": "Group membership maps to roles here. `chore_exempt_group` members are dropped from the chore rotation exactly as the per-person flag does today \u2014 which means a FreeIPA group can grant an exemption but must never remove one somebody set by hand, or a directory sync would silently re-enrol a guest. `household_group` is who gets mirrored at all: without it every service account in the directory becomes a household member.",
"_naming": "EVERY group this project reads is named USR_HA_<parameter>, matching the field name after the _group suffix is dropped: household_group -> USR_HA_household, chore_exempt_group -> USR_HA_chore_exempt, admin_group -> USR_HA_admins. The prefix is what makes a directory shared with other systems auditable \u2014 'which groups does the smart home read?' is answerable with one filter instead of by reading this file. The validator warns on anything that does not follow it rather than erroring, since an existing directory may already have its own convention and renaming groups in FreeIPA is not a thing a config file should force.",
"server": "",
"domain": "",
"base_dn": "",
"bind_dn": "",
"verify_tls": true,
"household_group": "USR_HA_household",
"chore_exempt_group": "USR_HA_chore_exempt",
"admin_group": "USR_HA_admins",
"_sync_interval_minutes": "How often the mirror runs, once it exists. Directory changes are not urgent \u2014 somebody joining the household is a thing you also tell the door panel about.",
"sync_interval_minutes": 60
},
"proxy": {
@ -159,6 +219,7 @@
"_comment": "type must be one of: thin-client, touch-panel, door-panel, kitchen-display. hostname must be unique and a valid DNS label — it is what the HA device shows up as.",
"type": "door-panel",
"hostname": "door-panel",
"room": "hallway",
"friendly_name": "Door panel",
"kiosk_username": "kiosk",
"voice_satellite": true,
@ -167,6 +228,7 @@
{
"type": "kitchen-display",
"hostname": "kitchen-display",
"room": "kitchen",
"friendly_name": "Kitchen fridge display",
"kiosk_username": "kiosk",
"voice_satellite": false,
@ -175,6 +237,7 @@
{
"type": "thin-client",
"hostname": "thin-client-living",
"room": "living_room",
"friendly_name": "Living room thin client",
"kiosk_username": "kiosk",
"voice_satellite": false,
@ -185,6 +248,7 @@
{
"type": "touch-panel",
"hostname": "touch-panel-kitchen",
"room": "kitchen",
"friendly_name": "Kitchen touch panel",
"kiosk_username": "kiosk",
"voice_satellite": false,
@ -196,11 +260,13 @@
{
"_comment": "Headless Spotify Connect appliances for rooms with no thin client. arch picks the toolchain — and they are genuinely different toolchains producing different artifacts, not one image for both: 'amd64' is a mini PC + USB DAC built with live-build (an .iso), 'arm64' is a Raspberry Pi + HiFiBerry Amp2 built with rpi-image-gen (an .img). build-all.sh builds every entry here, so listing both architectures gets you both. hostname doubles as the Spotify Connect device name and must be unique across kiosks too — they're all devices on one network.",
"hostname": "audio-endpoint-livingroom",
"room": "living_room",
"friendly_name": "Living room",
"arch": "amd64"
},
{
"hostname": "audio-endpoint-kitchen",
"room": "kitchen",
"friendly_name": "Kitchen",
"arch": "arm64"
}

26
README.md
View File

@ -26,8 +26,9 @@ hosts/
browser, switched via an always-on touch dock or by HA/
the local LLM over MQTT
kitchen-display/ Single-purpose Sway kiosk for the fridge/pantry: one
Chromium window showing pantry-vision's Scan/Inventory/
Recipes frontend, camera capture via the browser itself
Chromium window showing pantry-vision's unload/consume/
expired/edit frontend, camera capture via the browser
itself
door-panel/ Single-purpose Sway kiosk by the door/wardrobe:
identity's weather+clothing/who's-home/groceries-low
dashboard by default, registration on demand — this
@ -60,7 +61,10 @@ admin-canvas/ On-demand sys-admin-llm display surface for the thin
static serving)
pantry-vision/ Kitchen-display backend: a photo held up to the camera
-> an Ollama vision-model proposal -> human-confirmed
write into Grocy stock; also proxies Grocy's inventory
write into Grocy stock. All four stock movements are
camera-driven (unload / consume / bin what expired /
correct by hand), counted in individual units and folded
brand-free; also proxies Grocy's inventory
(soonest-expiring first) and recipes to the kiosk
frontend (write API + frontend/ static serving)
trash-calendar/ Reads Kennelbach's personal trash-collection ICS feed,
@ -94,12 +98,24 @@ chores/ Presence/calendar-driven household chore nudging +
- [ ] ESP32-S3-Touch-LCD-1.85C-V2 voice satellite + status display (`firmware/esp32-s3-touch-lcd-1.85c/`) — ESPHome config written and passes `esphome config`, not yet flashed to real hardware; `media_player`/`weather` entity IDs still need to be chosen, see `docs/project-plan.md` §4
- [ ] Headless audio endpoint (`hosts/audio-endpoint/`) — per-room independent Spotify Connect appliance for rooms without a thin client, arm64 (Raspberry Pi + HiFiBerry Amp2, rpi-image-gen) and amd64 (mini PC + USB DAC/amp, live-build) build pipelines written, **neither built/flashed/booted on real hardware** — rpi-image-gen's exact config schema in particular is unverified, see `hosts/audio-endpoint/README.md`
- [ ] Sway touch panel (`hosts/touch-panel/`) — touch-driven Sway image: full Spotify GUI (Flathub), a dedicated Home Assistant Chromium kiosk window, a general web browser, an always-on touch dock for app switching, an on-screen keyboard (toggled manually, no auto-show), and `touchpanel-agent` (HA MQTT control, same LLM-mediated-through-HA security model as the thin client) — built, **no touch-panel hardware chosen and nothing booted on real metal**, see `hosts/touch-panel/README.md`
- [ ] Kitchen/fridge display + `pantry-vision` (`hosts/kitchen-display/`, `pantry-vision/`) — hold a grocery item up to the camera, an Ollama vision model proposes what it is and roughly how long it keeps, a human confirms (never auto-committed) before it's written into Grocy stock; the display then shows inventory sorted by soonest-to-expire, groceries running low, and Grocy's recipes — built and wired into `setup-container-host.sh` (`ENABLE_PANTRY_VISION`, off by default), **nothing run against a real camera, vision model, or Grocy instance** — the Grocy API call shapes in particular are written from documentation only, see `pantry-vision/README.md` and `hosts/kitchen-display/README.md`
- [ ] Kitchen/fridge display + `pantry-vision` (`hosts/kitchen-display/`, `pantry-vision/`) — hold a grocery item up to the camera, an Ollama vision model proposes what it is and roughly how long it keeps, a human confirms (never auto-committed) before it's written into Grocy stock; the display then shows inventory sorted by soonest-to-expire, groceries running low, and Grocy's recipes. All four stock movements now run off the camera — **unload** (a scan loop, one confirm per item, with pack size and where to put it away), **consume** (asks which brand and how many), **list expired** (cleared by scanning what you're binning, booked out as spoiled), and **edit inventory** (the deliberately camera-free correction screen) — with two invariants: stock counts *individual units* (a twelve-pack of eggs is twelve) and folds *brand-free* via Grocy product groups (12 of brand X + 10 of brand Y = 22 eggs, expandable per brand). Built and wired into `setup-container-host.sh` (`ENABLE_PANTRY_VISION`, off by default), **nothing run against a real camera, vision model, or Grocy instance** — the Grocy API call shapes in particular are written from documentation only, see `pantry-vision/README.md` and `hosts/kitchen-display/README.md`
- [x] Where-is-it-actually, for multiple fridges — `docs/fridge-item-location.md`: separates "which appliance" (a software-only change: Grocy locations + a transfer action) from "which shelf" (a *hint* at best) and "exact position" (occlusion makes it unbuildable), and rules out interior cameras on power/condensation/−18 °C grounds — which is exactly the compartment the question starts from. **Built**: appliances as Grocy locations, `POST /transfer` with a "Move to…" picker on the edit screen, and the door-sensor→camera→hint path (`pantry-vision/doorway.py`, `POST /doorway-event` — an HA automation on a Zigbee contact sensor, answered 202 with the camera burst running off-thread; what it recognises is stored in its own SQLite file with a timestamp and a confidence and is **never** written to stock). **Not bought, not tested**: no door sensor, no doorway camera, and the assumption the camera half rests on — that a local vision model can identify an item in a moving hand at ~1.5 m — has never been checked (open decision #40). An appliance can be configured with a sensor and no camera, which is still the recommended way to start
- [x] Per-person **colour** and a settable **profile picture** in `identity` — eight colours assigned automatically at registration, avoiding any colour already worn by somebody with the same initial (an Anna and an Amir are two identical "A"s on a wall panel, and the colour is what makes that readable), then least-used overall; editable in the admin panel, backfilled oldest-first for existing people so nobody's colour reshuffles on restart. `color` + `initial` now ride on `/people`, `/presence` and every `/floorplan/presence` occupant, so no consumer derives an initial or invents a palette. New `POST /people/<id>/photo` sets a picture without a walk to the door panel — the registration capture was the only source before, which left a device-less household member unable to have a face at all. **The palette values sit on the 2-bits-per-channel lattice a colour Pebble renders natively**, so the colour on a watch is the colour in the panel, see below
- [x] **Rooms are one vocabulary, and devices declare theirs** — `docs/rooms-and-endpoints.md`: the room id is an HA `area_id` everywhere (`CoreSystemConfig.json` → `config-export.py` → the ISO builder → the agent → `suggested_area` in MQTT discovery), which is the same string `identity`'s floorplan rooms join on. Every kiosk and audio endpoint now carries `room`, `tools/validate-config.py` rejects anything that isn't already an area_id (helpfully slugifying it is how you end up with two rooms) and warns rather than errors when it's missing. **The honest limit: `suggested_area` is only honoured at first discovery** — move a device and you move it in HA once, by hand
- [x] **`workshop/`** — built: project notebook (`workshop.db`), a **never-pruned knowledge store** (`workshop-knowledge.db`: workflow instructions by activity, keyword facts, project learnings, and the household's **one** hardware inventory), `GET /context` to hand an assistant everything that applies at once, a **web inventory editor**, **Gitea repos with append-only history** (commit/push/branch yes; force-push/rebase/reset/filter-repo never, enforced by branch protection server-side, with a print-only scrub-request for the token-in-history case), a **health poller** for CheckMK + every OPNsense firewall, a **cameras** tab over go2rtc, and a **fleet-scripts admin surface**: one monitoring-agent script per kind of machine (Debian x86, arm64 Pi, the Docker host, the GPU host, plus slots for the ESP32s and network appliances that *can't* run a script and whose CheckMK-server-side config goes there instead), fetched by each endpoint's `fleet-bootstrap` timer. It's remote code execution by design, so the constraints are the design: **upload is a draft, publishing is a separate click**, the service never executes anything, scripts live in SQLite rather than on the writable share (a share credential must not be a whole-fleet root-execution credential), every version is kept, and the endpoint verifies the checksum, runs a version once, and **reports back pass or fail** — a script that was served is not a script that succeeded. Off by default (`ENABLE_WORKSHOP`), **nothing run against a real deployment** — the CheckMK API shape and Gitea's branch-protection payload are both written from documentation and have version-sensitive field names
- [x] **Multi-firewall + CheckMK, and a config that says so** — `opnsense` is a **list** now (each with its own name and key pair, keyed by name rather than by index), `checkmk` is a first-class block, and both feed the digest (`ingest/infra_health.py`, which reads the *poller* rather than polling, so the digest can say "critical since Tuesday" instead of quoting a six-hour-old snapshot) **and** the workshop's `infra_status` table. Three states — `ok`/`problem`/**`unreachable`** — because "I could not ask" is not "nothing is wrong". Validator rejects duplicate firewall names and half-configured credentials for both
- [x] **Photo web frontend (Immich)** alongside the SMB share — `ENABLE_PHOTOS_WEB`, search/albums/faces over the **same** directory, mounted **read-only**: two writers to one photo tree with different ideas of the layout is how a collection gets quietly reorganised. The share stays right for bulk copy and for keeping photos openable with no software at all
- [x] **Keycloak + FreeIPA config blocks** — `identity_provider` and `freeipa`, validated and exported, **declaration-only**: nothing implements SSO or the directory mirror yet, and the validator says so out loud rather than letting a filled-in block imply it works. **They are one decision, not two** — Keycloak federates FreeIPA rather than being its own user store, so the validator errors on either configured alone; a Keycloak without the directory behind it is a second parallel set of household accounts, which is the thing a directory exists to prevent. Groups follow **`USR_HA_<parameter>`** (`USR_HA_household`, `USR_HA_chore_exempt`, `USR_HA_admins`), warned-not-errored so an existing directory's convention isn't overridden by a config file. Two rules recorded before the work: the mirror is **one-way** (a directory serving real logins must not become a store of chore-reminder preferences), and a group may **grant** a chore exemption but never **remove** one somebody set by hand
- [ ] Workshop/office assistant — `docs/workshop-assistant.md`: **read the label, don't recognise the object** (a closed T480 and T490 are the same black rectangle; the identity is in the service tag and the PCB silkscreen, so the pipeline is OCR/barcode-first with the VLM only locating the label). Specs must be **quoted from a fetched document with its URL, never generated** — a hallucinated pinout destroys hardware — which also makes this the first component here that deliberately reaches the open internet (outbound-only, allowlisted, cached). Per-room scoping = the room selects the toolset, riding the `room` plumbing above. An SMB **workspace share** for everything it produces (reusing the existing Samba container — a second one would collide on 445, which the gallery already holds — with its own volume, own account, and read-write where the gallery is read-only). Display widgets collapse to **one `svg` window kind plus server-side renderers** (Graphviz for code-flow/data-structure, netlistsvg for schematics, KiCad export for board plans), because a megabyte of JS on a kiosk buys a picture the server could render once — **IEC/EU notation is a symbol-library decision**, so "can I supply my own symbols?" disqualifies a tool before output quality does. Purple/magenta holo theme is one CSS variable override over the existing `glow.css`, with the drawings left untinted on purpose. Analysis only, nothing built
- [x] **Now-playing visualiser + 3D floorplan** — built, in `render/` (shared, vendored, dependency-free, config from `?query=params`). `media-visualiser/`: circular spectrum, album-art palette, LRC lyrics — **two-tier by necessity**, since most endpoints have no local audio (a kitchen panel showing what the living room plays cannot analyse anything), so it is real FFT where audio is local and a tempo-driven ring elsewhere, **which says on screen that it is not an analysis**. Palette rejects near-greys/near-blacks before ranking and lifts each colour until it clears the background — the step whose absence makes art-coloured visualisers invisible on dark covers. Plain lyrics are shown but never auto-scrolled at a guessed rate. `floorplan-3d/`: the same `/floorplan/presence` payload as the Pebble app, extruded — **canvas 2D, not three.js**, a change from the plan made while building it: the scene is prisms on a plane, which an isometric projection with painter's sorting draws in ~200 lines, predictably on weak panels, with the frontend still at zero dependencies. Lit/dark rooms plus the dashed third state for rooms HA never reports on, photos-or-initials in the person's colour ring, and a visible shelf for people who are home but unlocatable. Geometry unit-tested headlessly; **never opened in a real browser**
- [ ] ~~Now-playing visualiser + 3D floorplan (design)~~ — `docs/endpoint-surfaces.md`: a circular CAVA-style ring behind every playing screen, coloured from the album art, lyrics under the cover when they exist; and the Pebble app's presence view in 3D on any endpoint. Two findings shape both: **most endpoints have no local audio** (a kitchen panel showing what the living room plays cannot analyse anything), so the visualiser is explicitly two-tier — real FFT where audio is local, tempo-driven "mood light" elsewhere, and the synthetic tier must never claim to be the real one; and the 3D plan is an **extrusion of the existing 2D polygons**, not a hand-authored model, so it can't go stale when a room is redrawn. Three.js is ~1MB and a deliberate break with the dependency-free SDK rule — make it once, explicitly. Analysis only, nothing built
- [ ] Pebble presence **watchapp** feasibility — `docs/pebble-presence-watchface.md`: the floorplan drawn as quantised polygons (~200–250 bytes for a whole floor, against a ~2 KB AppMessage budget), occupants as initial-on-colour rather than photos (a face gets ~20×20px in 64 colours — four skin-toned blobs, and three of them exceed the message budget the entire floor fits in). Targets the **Pebble Round 2** that's actually owned: the plan is inscribed in the circle (`w = D·a/√(a²+1)`, so ~170×170 for a squarish plan at a 240px usable diameter — comparable to a Time 2 after chrome), the projection change is one line of phone-side JS, and the only real loss is corner furniture like a status footer. Occupancy reads as **dark room / lit room** in lightness rather than hue, since colour is already spoken for by *who* — with a third dashed state for rooms HA never reports on, because drawing "no data" as "empty" is a quiet lie. **Button-cycling through rooms forces a watchapp, not a watchface** — Pebble gives watchfaces no button events and restricts touch to apps — and the watchapp is the decision: Up/Down cycles rooms into a plain-text list of who is in each (plus a final "somewhere in the house" entry for people who are home but unlocatable, who are exactly who you picked the watch up to find). The cost is that it's a menu entry rather than your default screen, and its data only lives while it's open; a watchface variant later is a second `main()`, not a second project. Analysis only, nothing built. The finding that actually decides it is upstream: room-level presence has never been measured in this house, and the admin panel's floorplan tab with **Live** ticked tests it today for free
- [ ] "Put the groceries away" chore — `chores`' new `groceries_out_of_place` watch point: `litter`'s twin (same "whoever the camera just saw" attribution, same exemption-proof, same unassignable) with a **30-minute** neglect fuse instead of four hours, because the failure is spoiled food rather than untidiness. Its prompt names what to *ignore* (fruit bowls, bread bins, coffee, cookware) as well as what to look for — the general question gets a YES for any normal kitchen, and a false chore every two hours is how a household learns to mute the channel (open decision #41). Knows food is out, never what the food is; nothing here touches Grocy. Untested against a real camera or model, like every other watch point
- [ ] `identity` + door panel (`identity/`, `hosts/door-panel/`) — the person <-> BLE-identifier registry: "register me as `<name>`" by voice or touchscreen, multi-phone support (multiple identifiers per person), anti-spoofing (only allowlisted IRK-resolved/fixed-tag entities are ever accepted as candidates, never a raw MAC), device-less people (a "no device" flag plus a hand-operated Home/Away toggle — the concrete case: a grandmother without a smartphone), and an anonymous "Guest" path. Backs `hosts/door-panel/`'s weather+clothing/who's-home/groceries-running-low dashboard and `hosts/kitchen-display/`'s "Show registration" screen — built and wired into `setup-container-host.sh` (`ENABLE_IDENTITY`, off by default), **nothing run against a real HA instance, real Private BLE Device entities, or a real voice pipeline** — `TRUSTED_ENTITY_PREFIXES` above all needs checking against Developer Tools -> States, see `identity/README.md` and `hosts/door-panel/README.md`
- [ ] `identity` also corroborates presence from Frigate face recognition (Phase 20, Tapo pan/tilt cameras) — an OR-ed-in second signal only, **never** a registration signal; and owns the per-person chore-system settings (`chore_exempt`, `chore_reminder_style`, plus chore assignment) consumed by `chores/`, see `identity/README.md`
- [ ] `identity`'s admin panel (`identity/frontend/admin.html`, Phase 6b) — managing people/guests: edit every field, **nicknames** (an input alias only — `/resolve` accepts them, but the assistant always speaks the real `speak_name`), **visit history** sampled from `/presence` plus a derived "who was home with whom" view, **"select all that last visited before `<date>`"** pruning (the filter selects, a human confirms the exact list, the filter is never re-run at delete time), **per-device rights** for self-entry via a smart lock (`identity` only ever *answers* `GET /device-access` — HA asks and HA acts, deny is the default), chore assignment, a **floorplan editor** (draw levels and rooms as polygons, map each to an HA area, and watch occupied rooms light up — resolves the long-deferred open decision #22), and **opt-in arrival push notifications** ("tell me when someone gets home", via the self-hosted ntfy this stack already runs — `identity` itself never touches the WAN; ntfy stays LAN-only and remote delivery rides a WireGuard split tunnel — see `docs/network-integration.md` §2.2 for why a DMZ/port-forward was weighed and rejected). Deliberately **not** a kiosk page and not linked from any wall panel. Covered by API-level tests; **never opened in a real browser**, and `DEPARTURE_GRACE_SECONDS` is an untuned guess — see `identity/README.md`
- [ ] `trash-calendar` + `transit` (Phase 19, Kennelbach AT trash pickup + Vorarlberg public transit) — built and wired into `setup-container-host.sh` (`ENABLE_TRASH_CALENDAR`/`ENABLE_TRANSIT`/`ENABLE_TRIP_PLANNING`, all off by default), **nothing run against a live ICS feed, a live GTFS feed, or a real OpenTripPlanner instance** — trip planning also needs a manually-built OTP graph this repo does not build for you, see `trash-calendar/README.md` and `transit/README.md`'s "Route planning scope"
- [ ] `chores` (Phase 20) — presence/calendar-driven household chore nudging: "I don't care who does it, as long as it gets done" — prefers whoever's been assigned a chore in `identity`'s admin panel but falls through to whoever's actually home rather than waiting (`CHORE_ASSIGNMENT_STRICT` flips that), redirects to someone else if a chore goes neglected, keeps a passive fairness tally that never feeds back into who gets nudged, and camera-checks trash bins/dishes/litter via Frigate + an Ollama vision model. **Litter remains the exception to everything** — it ignores both chore-exemption and assignment, because cleaning up what you left out was never a task anyone could be assigned. Built and wired into `setup-container-host.sh` (`ENABLE_CHORES`, off by default, every-2-hours systemd timer), **no Tapo camera hardware chosen and nothing run against real hardware**, see `chores/README.md`
- [ ] `chores` (Phase 20) — presence/calendar-driven household chore nudging: "I don't care who does it, as long as it gets done" — prefers whoever's been assigned a chore in `identity`'s admin panel but falls through to whoever's actually home rather than waiting (`CHORE_ASSIGNMENT_STRICT` flips that), redirects to someone else if a chore goes neglected, keeps a passive fairness tally that never feeds back into who gets nudged, and camera-checks trash bins/dishes/litter/groceries-left-out via Frigate + an Ollama vision model. **Litter and groceries-left-out remain the exception to everything** — both ignore chore-exemption and assignment, because cleaning up what you left out was never a task anyone could be assigned; groceries additionally get a 30-minute fuse instead of four hours, since that failure spoils food rather than merely looking untidy. Built and wired into `setup-container-host.sh` (`ENABLE_CHORES`, off by default, every-2-hours systemd timer), **no Tapo camera hardware chosen and nothing run against real hardware**, see `chores/README.md`
- [ ] Music Assistant (optional, additive multi-room audio) — wired into `setup-container-host.sh` (`ENABLE_MUSIC_ASSISTANT`, off by default), **its default port is an unverified guess that collides with `PANTRY_VISION_PORT`** if both are enabled together, see `docs/project-plan.md` open decision #31
- [ ] `docs/network-integration.md` (OPNsense VLAN segmentation, the WireGuard split tunnel that carries arrival notifications, and why nothing here — ntfy included — gets port-forwarded to the WAN) — written, not run against a real OPNsense instance
- [ ] `tools/` + `CoreSystemConfig.json` — every build and setup script in one place, reading one config. The container host and LLM host build as a **twinned pair**: you set two last octets and the container host's `OLLAMA_HOST` is *derived* from the LLM host's, so the two ISOs cannot be built disagreeing about where the other one is; every kiosk's service URLs derive from the container host's address the same way. `build-all.sh` builds the set, `validate-config.py` refuses a build on duplicate ports (the `music_assistant`/`pantry_vision` 8095 clash, open decision #31), placeholder or padded tokens, duplicate hostnames, or a kiosk pointed at a disabled service. All secrets are burnt into the images so installs are unattended — **which makes every ISO a credential**; the filled-in config and `iso-out/` are gitignored. **No ISO has ever been built with this** (`lb build` needs live-build, root and a long fetch) — what is tested is config validation/derivation and every generated artifact, with `lb` stubbed. See `tools/README.md`

65
chores/README.md
View File

@ -21,18 +21,43 @@ module docstring for the full reasoning on each:
Frigate snapshot per configured watch point (optionally moving a PTZ camera to a
preset first), asks an Ollama vision model a one-word question ("is this bin
FULL/PARTIAL/EMPTY", "is this counter DIRTY/CLEAN", "is there litter left out
here, YES/NO"), opens a chore on "needs attention," auto-closes one on "clear."
here, YES/NO", "are there groceries standing out that belong in a fridge/freezer/
cupboard, YES/NO"), opens a chore on "needs attention," auto-closes one on "clear."
3. **Nudging** — ASAP, not on a fixed schedule: the first run after a chore opens
nudges whoever `identity` reports home right now — preferring anyone **assigned**
that chore type, minus anyone `chore_exempt`, both below. If the chore is still open `NEGLECT_THRESHOLD_HOURS` after the last
nudge (and the household calendar isn't showing a busy window), the nudge goes
to **someone different from who was last asked** — "the next person that walks
by" — rather than re-nagging the same person. `litter` chores are special-cased
to prefer whoever the camera most recently recognized nearby (a best-effort
"who left this" guess), since the point there is telling the actual person, not
just whoever's around — and `litter` also ignores `chore_exempt` entirely, see
below. Each nudge's wording is a plain template unless the target has a
`chore_reminder_style` set, see below.
by" — rather than re-nagging the same person. `litter` and
`groceries_out_of_place` chores are special-cased to prefer whoever the camera
most recently recognized nearby (a best-effort "who left this" guess), since the
point there is telling the actual person, not just whoever's around — and both
also ignore `chore_exempt` entirely, see below. Each nudge's wording is a plain
template unless the target has a `chore_reminder_style` set, see below.
## Putting the groceries away is litter with a shorter fuse
`groceries_out_of_place` is built as `litter`'s twin on purpose — same "whoever the
camera just saw" attribution, same exemption-proofing, same unassignability, because
it is the same situation: somebody carried something in and put it down. Its one
deliberate difference is **`GROCERIES_NEGLECT_THRESHOLD_HOURS`, 30 minutes instead of
the usual four**, and that is the whole reason it isn't simply another watch point on
the `litter` prompt: litter left for an afternoon is untidiness, and a tub of ice
cream left for an afternoon is a bin bag. It is the only chore type in `check.py`
treated as more urgent than the others, because it is the only one where being late
costs something other than tidiness.
Two things worth knowing before pointing a camera at this:
- **The prompt names what to ignore, not just what to look for** — fruit bowls, bread
bins, coffee, oil, spices, appliances, cookware. Ask a vision model the general
question ("is anything out of place?") and it will say YES to a kitchen that is
simply a kitchen, every two hours, until somebody mutes the topic.
- **It is a chore, not an inventory update.** It notices that food is standing out; it
does not know *what* the food is and never books anything into Grocy. That is
`pantry-vision`'s job, on a display someone is actually standing at. Two systems
looking at the same counter for two different reasons is the intended shape, not a
duplication to collapse.
## Assignment is a preference, not a lock
@ -52,20 +77,22 @@ legitimate and this file can't pick for you, so it's one env var rather than a
hard-coded opinion. Strict mode still never stalls a chore that was assigned to
*nobody* — that falls through regardless.
**Litter can't be assigned** (`_ASSIGNMENTS_DONT_APPLY` in `check.py`), for the same
reason it ignores exemptions: it goes to whoever left the mess, and cleaning up after
yourself was never a task anyone could be handed.
**Litter and groceries-left-out can't be assigned** (`_ASSIGNMENTS_DONT_APPLY` in
`check.py`), for the same reason they ignore exemptions: they go to whoever left the
mess, and cleaning up after yourself was never a task anyone could be handed.
## Chore-exempt people — everyone except litter
## Chore-exempt people — everyone except litter and groceries
Set in the same place (`identity`'s admin panel, or `POST /people/<id>/chore-settings`
directly). A `chore_exempt` person is
dropped from the nudge rotation entirely — the "cousin visits often but doesn't owe
me chores" case. **Litter is the deliberate exception** (`_EXEMPTIONS_DONT_APPLY`
in `check.py`): an exempt person still gets told to put trash they left out into
the bin, because that isn't "doing a chore," it's cleaning up after yourself. If
everyone currently home is chore_exempt for a non-litter chore, that run just logs
and skips — the chore stays open until someone eligible is around.
me chores" case. **Litter and groceries-left-out are the deliberate exceptions**
(`_EXEMPTIONS_DONT_APPLY` in `check.py`): an exempt person still gets told to put
trash they left out into the bin and to put the milk away, because neither is "doing
a chore," both are cleaning up after yourself — and a guest who helped unpack the
shopping is exactly as able to finish the job as anyone else. If everyone currently
home is chore_exempt for one of the other chore types, that run just logs and skips —
the chore stays open until someone eligible is around.
## Reminder tone is per-person and LLM-phrased, but never LLM-decided
@ -163,7 +190,11 @@ when left unconfigured.
2. Whether the vision model's one-word FULL/PARTIAL/EMPTY/DIRTY/CLEAN/YES/NO
answers are actually reliable for a real bin/sink/hallway from a real camera
angle — completely unmeasured, same caveat as pantry-vision's own vision-model
accuracy note.
accuracy note. **`groceries_out_of_place` is the one to check first**: its whole
design rests on a model reliably telling a carton of milk from a fruit bowl, and
its failure mode is not a missed chore but a false one every two hours, which is
how a household learns to ignore the notification channel. Point it at a *clean*
counter for a day before trusting a YES.
3. `_likely_culprit()`'s reliance on `identity`'s `face_seen_recently` field
assumes Frigate face-recognition presence corroboration is actually wired up
and working (`identity/README.md`'s own "Camera face recognition" section is

149
chores/check.py
View File

@ -18,8 +18,9 @@ in case something else is running" jitter) and does three things, in order:
camera hardware has been chosen yet, see docs/project-plan.md §1.18): for each
configured watch point (a Frigate camera + optional PTZ preset), grabs a
snapshot via Frigate's own API and asks an Ollama vision model whether it shows
a full bin / dirty dishes. A "needs attention" result opens a chore if one isn't
already open; a "clear" result auto-closes one if it was.
a full bin, dirty dishes, litter left out, or **groceries still standing out that
belong in a fridge/freezer/cupboard**. A "needs attention" result opens a chore if
one isn't already open; a "clear" result auto-closes one if it was.
3. **Nudging**: for every open chore, ASAP, not "wait for a schedule" — the first
run after a chore opens nudges whoever `identity` reports home right now (using
`identity`'s own room field to prefer someone actually near the relevant spot,
@ -35,10 +36,20 @@ in case something else is running" jitter) and does three things, in order:
`identity`'s `chore_exempt` flag (set via `POST /people/<id>/chore-settings`, see
identity/README.md) takes a person out of the nudge rotation entirely — a frequent
guest who isn't a household member doesn't owe chores. **`litter` is the one
exception** (`_EXEMPTIONS_DONT_APPLY` below): everyone, exempt or not, still gets
told to put trash they left out into the bin — that isn't "doing a chore," it's
cleaning up after yourself.
guest who isn't a household member doesn't owe chores. **`litter` and
`groceries_out_of_place` are the exceptions** (`_EXEMPTIONS_DONT_APPLY` below):
everyone, exempt or not, still gets told to put trash they left out into the bin and
to put the milk away — neither is "doing a chore," both are cleaning up after
yourself, and a guest who unpacked the shopping is exactly as able to finish the job
as anyone else.
`groceries_out_of_place` is deliberately built as litter's twin — same
culprit-attribution, same exemption-proofing, same unassignability — with **one
difference that matters: a much shorter fuse** (`_NEGLECT_HOURS_OVERRIDE`, 30 minutes
against the usual four hours). Litter left for an afternoon is untidiness; a tub of
ice cream left for an afternoon is a bin bag. It is the only place in this file where
one chore type is treated as more urgent than another, and the reason is that the cost
of being late differs in kind, not degree.
## Assignment is a preference, not a lock
@ -54,9 +65,10 @@ waiting. Set `CHORE_ASSIGNMENT_STRICT=true` if you'd rather it wait for the assi
that's the honest opposite reading of the same feature, and which one a household
wants isn't something this file can decide for it.
**`litter` ignores assignment entirely** (`_ASSIGNMENTS_DONT_APPLY`), for the same
reason it ignores exemptions: it goes to whoever left the mess, and "cleaning up after
yourself" was never a task anyone could be assigned in the first place.
**`litter` and `groceries_out_of_place` ignore assignment entirely**
(`_ASSIGNMENTS_DONT_APPLY`), for the same reason they ignore exemptions: they go to
whoever left the mess, and "cleaning up after yourself" was never a task anyone could
be assigned in the first place.
`identity`'s `chore_reminder_style` free-text field (same endpoint) is passed to an
LLM that **phrases** the ntfy message in that person's preferred tone ("be
@ -139,26 +151,65 @@ _CHORE_PROMPTS = {
"that does not belong there (not properly disposed of in a bin)? Answer with exactly "
"one word: YES or NO."
),
# Groceries left standing out — the same shape as litter (somebody put something
# down and walked away) with a much shorter fuse, because the failure here is not
# untidiness, it is food going off. The prompt names the perishable cases
# explicitly rather than asking the general "is anything out of place" question,
# which a vision model will happily answer YES to for a fruit bowl, a bread bin,
# or a kettle — and a chore that fires every two hours about the fruit bowl is one
# the household will mute within a week.
"groceries_out_of_place": (
"Look at this photo of a kitchen counter, table or worktop. Are there GROCERIES sitting "
"out that belong in a fridge, freezer or cupboard — for example milk, yoghurt, cheese, "
"meat, fish, eggs, opened jars, frozen food, or a shopping bag that has not been put "
"away? Ignore things that normally live on a worktop: fruit bowls, bread bins, coffee, "
"spices, oil, salt, appliances, cookware and dishes. Answer with exactly one word: "
"YES or NO."
),
}
# Which watch points get "who was just seen here" culprit-attribution treatment
# (see _likely_culprit()) instead of the general "whoever's around" nudge — litter
# is specifically about telling whoever left it, not just whoever's nearby now.
_ATTRIBUTE_TO_RECENT_VIEWER = {"litter"}
# Groceries left out is the same situation: somebody carried it there.
_ATTRIBUTE_TO_RECENT_VIEWER = {"litter", "groceries_out_of_place"}
# Chore types where identity's chore_exempt flag does NOT apply — everyone still
# gets told to clean up litter they left out, exempt household member or not (see
# module docstring). Currently the same set as _ATTRIBUTE_TO_RECENT_VIEWER, but
# they mean different things — one is about attribution, this is about eligibility
# — so they're kept as separate names rather than reusing one for both purposes.
_EXEMPTIONS_DONT_APPLY = {"litter"}
_EXEMPTIONS_DONT_APPLY = {"litter", "groceries_out_of_place"}
# Chore types that can't be assigned to anyone — see the module docstring's
# "Assignment is a preference, not a lock". Third set with the same one member as the
# "Assignment is a preference, not a lock". Third set with the same members as the
# two above, and kept separate for the third distinct reason: attribution, then
# eligibility, now assignability. If they ever diverge (a chore that's assignable but
# exempt-proof, say) collapsing them now would be the thing that made that painful.
_ASSIGNMENTS_DONT_APPLY = {"litter"}
_ASSIGNMENTS_DONT_APPLY = {"litter", "groceries_out_of_place"}
# How the one-word camera answer is read, per chore type. Extracted from an inline
# expression once there were four of them: adding a chore type should be a line in a
# table, not a new clause in a boolean nobody can read.
_ATTENTION_ANSWERS = {
# "PARTIAL" contains neither FULL nor a separate word, but a model answering
# "PARTIALLY FULL" must not be read as full — hence the exclusion, kept from the
# original inline check.
"bin_full": lambda a: "FULL" in a and "PARTIAL" not in a,
"dishes": lambda a: "DIRTY" in a,
"litter": lambda a: "YES" in a,
"groceries_out_of_place": lambda a: "YES" in a,
}
# Per-type override for how long a chore may sit before it is redirected to somebody
# else. A dropped crisp packet can wait the default four hours; a tub of ice cream on
# the counter cannot, and neither can the shopping nobody unpacked. This is the one
# place in this file where a chore type is treated as more urgent than another, and it
# is here because the cost of being late is different in kind — spoiled food, not an
# untidy room.
_NEGLECT_HOURS_OVERRIDE = {
"groceries_out_of_place": float(os.environ.get("GROCERIES_NEGLECT_THRESHOLD_HOURS", "0.5")),
}
# Whether an assigned person who ISN'T home blocks the chore from falling through to
# whoever is. Default false — "as long as it gets done" is the house rule; true makes
@ -261,7 +312,8 @@ def check_trash_day(conn) -> None:
# --- 2. Camera checks --------------------------------------------------------------
def _watchpoints() -> list[tuple[str, str, str | None]]:
"""CAMERA_WATCHPOINTS format: "type:camera[:preset],type:camera[:preset],...".
type must be a key in _CHORE_PROMPTS other than "trash" (bin_full, dishes).
type must be a key in _CHORE_PROMPTS other than "trash" — i.e. bin_full, dishes,
litter, or groceries_out_of_place.
"""
raw = os.environ.get("CAMERA_WATCHPOINTS", "").strip()
if not raw:
@ -269,7 +321,10 @@ def _watchpoints() -> list[tuple[str, str, str | None]]:
points = []
for entry in raw.split(","):
parts = [p.strip() for p in entry.split(":")]
if len(parts) < 2 or parts[0] not in _CHORE_PROMPTS or parts[0] == "trash":
# _ATTENTION_ANSWERS as well as _CHORE_PROMPTS: a chore type with a prompt but
# no way to read its answer is a KeyError in check_cameras(), i.e. a crash on
# the timer rather than here. Adding a type means adding both.
if len(parts) < 2 or parts[0] not in _CHORE_PROMPTS or parts[0] not in _ATTENTION_ANSWERS:
LOG.warning("chores: ignoring malformed CAMERA_WATCHPOINTS entry %r", entry)
continue
points.append((parts[0], parts[1], parts[2] if len(parts) > 2 else None))
@ -334,9 +389,7 @@ def check_cameras(conn) -> None:
if answer is None:
continue
needs_attention = ("FULL" in answer and "PARTIAL" not in answer) or "DIRTY" in answer or (
chore_type == "litter" and "YES" in answer
)
needs_attention = _ATTENTION_ANSWERS[chore_type](answer)
existing = _open_chore(conn, chore_type)
if needs_attention and existing is None:
@ -458,7 +511,8 @@ def nudge_open_chores(conn) -> None:
if last_nudged_at is not None:
elapsed_hours = (datetime.now(timezone.utc) - datetime.fromisoformat(last_nudged_at.replace("Z", "+00:00"))).total_seconds() / 3600
if elapsed_hours < NEGLECT_THRESHOLD_HOURS:
threshold = _NEGLECT_HOURS_OVERRIDE.get(chore["type"], NEGLECT_THRESHOLD_HOURS)
if elapsed_hours < threshold:
continue # not neglected yet — leave whoever was last nudged alone for now
# chore_exempt people are out of the rotation entirely, EXCEPT litter — see
@ -525,12 +579,54 @@ def nudge_open_chores(conn) -> None:
)
# The culprit-framed wording, per type. Both are "you probably left this, please deal
# with it" — but "put it in the bin" is exactly wrong for a tub of ice cream, and a
# reminder that tells you to do the wrong thing is one you stop reading.
_CULPRIT_MESSAGES = {
"litter": "{name}, looks like something was left out — could you put it in the bin?",
"groceries_out_of_place": (
"{name}, there are groceries still standing out — could you put them away before they spoil?"
),
}
# What the LLM phrasing pass is told the situation is, when there is a model
# configured. Same split, same reason.
_CULPRIT_SITUATIONS = {
"litter": (
"Something was left out and may belong to them specifically — ask them to put it away, "
"don't accuse them outright. "
),
"groceries_out_of_place": (
"Groceries have been left standing out instead of being put in the fridge, freezer or "
"cupboard, and they may be the person who left them — ask them to put the food away "
"before it spoils, don't accuse them outright. "
),
}
# Chore types are snake_case identifiers; people are not. Everything user-facing goes
# through here — a notification that says "could you take care of:
# groceries_out_of_place" is a notification that reads like a stack trace.
_CHORE_LABELS = {
"trash": "putting the bins out",
"bin_full": "emptying the bin",
"dishes": "the dishes",
"litter": "clearing up what was left out",
"groceries_out_of_place": "putting the groceries away",
}
def _label(chore_type: str) -> str:
return _CHORE_LABELS.get(chore_type, chore_type.replace("_", " "))
def _default_message(name: str, chore_type: str, redirected: bool, is_culprit: bool) -> str:
if is_culprit:
return f"{name}, looks like something was left out — could you put it in the bin?"
template = _CULPRIT_MESSAGES.get(chore_type, "{name}, looks like something was left out — could you deal with it?")
return template.format(name=name)
if redirected:
return f"{name}, this one's still open — could you take care of: {chore_type}?"
return f"{name}, could you take care of: {chore_type}?"
return f"{name}, this one's still open — could you take care of: {_label(chore_type)}?"
return f"{name}, could you take care of: {_label(chore_type)}?"
def _compose_message(name: str, chore_type: str, redirected: bool, is_culprit: bool, reminder_style: str | None) -> str:
@ -545,10 +641,13 @@ def _compose_message(name: str, chore_type: str, redirected: bool, is_culprit: b
prompt = (
f"Write ONE short household chore reminder (max 2 sentences) addressed to {name}. "
f"The chore is: {chore_type}. "
f"The chore is: {_label(chore_type)}. "
f"Follow {name}'s own stated preference for how they like to be reminded: \"{reminder_style}\". "
+ ("Something was left out and may belong to them specifically — ask them to put it away, don't accuse them outright. "
if is_culprit else "")
+ (_CULPRIT_SITUATIONS.get(
chore_type,
"Something was left out and may belong to them specifically — ask them to put it away, "
"don't accuse them outright. ",
) if is_culprit else "")
+ ("They were already asked about this once before and it's still not done. " if redirected else "")
+ "Reply with ONLY the message text itself — no preamble, no quotation marks."
)

23
chores/chores.env.example
View File

@ -48,15 +48,28 @@ CALDAV_QUIET_KEYWORDS=busy,meeting,call,movie,sleep
# sources.
#
# CAMERA_WATCHPOINTS format: "type:frigate_camera_name[:ptz_preset_name],...".
# type must be "bin_full", "dishes", or "litter" ("has someone left trash out
# somewhere it doesn't belong" — gets told to whoever the camera most recently
# recognized nearby, not just whoever's home in general, see README.md). preset
# is optional — omit it if the camera doesn't need to move. Example:
# CAMERA_WATCHPOINTS=bin_full:driveway_cam:trash_preset,dishes:kitchen_cam,litter:hallway_cam
# type must be one of:
# bin_full — is the bin full
# dishes — is the sink/counter dirty
# litter — has someone left trash out somewhere it doesn't belong
# groceries_out_of_place — is there food still standing out that belongs in a
# fridge/freezer/cupboard
# The last two are told to whoever the camera most recently recognized nearby, not
# just whoever's home in general, and neither can be assigned or exempted away —
# see README.md. groceries_out_of_place also has a much shorter neglect fuse than
# everything else (below), because the failure mode is spoiled food, not untidiness.
# preset is optional — omit it if the camera doesn't need to move. Example:
# CAMERA_WATCHPOINTS=bin_full:driveway_cam:trash_preset,dishes:kitchen_cam,litter:hallway_cam,groceries_out_of_place:kitchen_cam
# ---------------------------------------------------------------------------
FRIGATE_URL=
CAMERA_WATCHPOINTS=
# How long groceries may stand out before the nudge is redirected to somebody else,
# overriding NEGLECT_THRESHOLD_HOURS for that one chore type. 30 minutes by default:
# the default four hours is a sensible wait for a crisp packet and far too long for a
# tub of ice cream.
GROCERIES_NEGLECT_THRESHOLD_HOURS=0.5
# Same LLM host as digest-engine/pantry-vision. OLLAMA_VISION_MODEL must be a
# vision-capable model (see pantry-vision/README.md's identical caveat — plain text
# models cannot see images at all).

23
digest-engine/README.md
View File

@ -224,11 +224,15 @@ trusting a scheduled run, verify by hand:
half is verified — `ingest/rci_social.py` was run live against the committed
`feeds/rci-social.json` on 2026-08-06 and returned real YouTube and podcast
entries with durations — but no local model has yet been asked to produce the
four question windows, per-marker summaries and `sources` arrays in one JSON
five question windows, per-marker summaries and `sources` arrays in one JSON
document. Check on the first real run that a 14B model actually fills
`sources` rather than dropping the field, that it does not put episodes in
the analysis, and that the compact pass still fits the HA card now that the
section has more to say.
section has more to say. The roll-call (`political-struggles`) is the window
to read most sceptically on that first run: confirm every line traces to an
entry that is actually in the context rather than to the model's own memory
of a famous strike, and that a long-running dispute stays on the list across
consecutive runs instead of being dropped as stale.
14. **The Telegram channel entry in `feeds/rci-social.json`.** The channel name
comes from marxist.com's own footer but was never fetched — reading it needs
the Telethon session, which only exists on the real deployment. Also confirm
@ -264,9 +268,9 @@ trusting a scheduled run, verify by hand:
a Times of Israel claim about Palestinians never reaches the digest in the
section's own voice.
## The political section — sources, ownership, and the four questions
## The political section — sources, ownership, and the five questions
The political digest answers four questions, in this order, one window each
The political digest answers five questions, in this order, one window each
(`synth/prompts/political.md` is written around them):
1. What is relevant for the communist and class struggle **globally** right now.
@ -278,6 +282,17 @@ The political digest answers four questions, in this order, one window each
sources are the `theory` feeds, the organisation's own social output, and the
reports that arrive in the user's own **mail** — the one input here no feed
can supply.
5. What class struggles are **currently going on around the world** — the
`political-struggles` roll-call. This one is an inventory, not a curation:
one line per live strike, occupation or mass movement, kept on the list while
it runs even in a quarter that carried no fresh news of it, which is what the
archive's `times_seen`/`first_seen_at` annotations are for. It is deliberately
the one window where a story being old is not a reason to drop it, and it
survives the compact pass (shortened, and saying it is shortened) because a
roll-call folded into a summary line stops being a roll-call. Everything on it
still has to come from this run's context — the prompt says in as many words
that the model's own knowledge of the world is not a source, since an
inventory question is exactly the shape of prompt that invites one.
Plus a **watch-later** window: new videos and podcast episodes from the
organisation's channels, kept out of the analysis entirely because they are

96
digest-engine/ingest/infra_health.py Normal file
View File

@ -0,0 +1,96 @@
"""Infrastructure health for the network digest — CheckMK and every firewall.
READS THE POLLER, DOES NOT POLL. `workshop/health.py` already asks CheckMK and each
OPNsense box every few minutes and keeps a month of samples; this module fetches that
service's `GET /health` and reshapes it for the network section.
That indirection is the whole design and it is worth one paragraph. The digest runs
four times a day. "Is the NAS disk failing right now" is not a question with a
six-hour answer, so polling from here would have produced a snapshot taken at 06:00
and quoted at 12:00. Reading the poller instead gives the digest something a snapshot
cannot have: **how long the state has held**. "Critical since Tuesday" is a different
sentence from "critical", and it is the one that tells you whether to get up.
It also means one set of credentials in one place. digest-engine never learns the
CheckMK secret or any firewall's API key for this — those live in workshop's env file,
and this module needs only workshop's own bearer token.
WHY THIS IS NOT IN opnsense_ids.py
-----------------------------------
That module reads Suricata's *alert log*, with paging and a time window, from one
firewall. This one reads *service state* from many. They answer different questions
("what fired" vs "is it running and is anything broken"), and the alert query is
deliberately left where it already works rather than reimplemented here.
DEGRADES TO NOTHING, LOUDLY
----------------------------
An unreachable workshop service returns `None` and the run continues — same rule as
every other ingestion module here. But when the poller *is* reachable and reports a
target as `unreachable`, that is passed through as a finding rather than dropped: a
firewall nobody can reach is exactly the thing a network digest exists to mention.
"""
from __future__ import annotations
import json
import logging
import os
import urllib.error
import urllib.request
LOG = logging.getLogger(__name__)
WORKSHOP_URL = os.environ.get("WORKSHOP_URL", "").rstrip("/")
WORKSHOP_TOKEN = os.environ.get("WORKSHOP_TOKEN", "")
TIMEOUT = float(os.environ.get("WORKSHOP_TIMEOUT", "10"))
def fetch() -> list[dict]:
"""Health entries for the digest context, or [] when unavailable.
Every entry carries `target` — the firewall's name or the CheckMK site — because a
household with two firewalls must never be handed "the IDS is running" as if there
were one of them. The prompt is told to name the target in anything it says.
"""
if not (WORKSHOP_URL and WORKSHOP_TOKEN):
LOG.info("infra_health: WORKSHOP_URL/WORKSHOP_TOKEN unset, skipping")
return []
try:
req = urllib.request.Request(f"{WORKSHOP_URL}/health")
req.add_header("Authorization", f"Bearer {WORKSHOP_TOKEN}")
with urllib.request.urlopen(req, timeout=TIMEOUT) as resp:
payload = json.loads(resp.read() or b"{}")
except (urllib.error.URLError, urllib.error.HTTPError, TimeoutError, OSError, ValueError):
LOG.warning("infra_health: could not reach the workshop health poller", exc_info=True)
return []
if not payload.get("configured"):
# Nothing is being watched. Distinct from "everything is fine", and the prompt
# must never render the second when the truth is the first.
return []
entries = []
for row in payload.get("results", []) or []:
problems = row.get("detail") or []
entries.append({
"category": "infra_health",
"source": row.get("source"),
"target": row.get("target"),
"state": row.get("state"),
"title": f"{row.get('target')}: {row.get('state')}",
"summary": row.get("summary", ""),
"problem_count": row.get("problem_count", 0),
# When this state started. None means it has held for the whole retention
# window, which the prompt should read as "long-standing", not "just now".
"since": row.get("since"),
"checked_at": row.get("checked_at"),
# Capped: a site with 200 failing services is a real state, and putting all
# 200 in an LLM context is not how you say so.
"problems": problems[:12],
"problems_truncated": max(0, len(problems) - 12),
})
if entries:
LOG.info("infra_health: %d target(s), overall %s", len(entries), payload.get("overall"))
return entries

9
digest-engine/run.py
View File

@ -61,6 +61,7 @@ from ingest import (
financial,
flight_traffic,
grocy,
infra_health,
naval_traffic,
news_rss,
opnsense_ids,
@ -91,6 +92,9 @@ SOURCES = (
("ENABLE_FLIGHT_TRAFFIC_INGEST", "flight_traffic", flight_traffic),
("ENABLE_NAVAL_TRAFFIC_INGEST", "naval_traffic", naval_traffic),
("ENABLE_OPNSENSE_IDS_INGEST", "opnsense_ids", opnsense_ids),
# Reads workshop's health poller rather than polling CheckMK/the firewalls itself —
# see ingest/infra_health.py for why the digest must not be the thing that polls.
("ENABLE_INFRA_HEALTH_INGEST", "infra_health", infra_health),
("ENABLE_CALDAV_INGEST", "calendar", caldav_ingest),
("ENABLE_GROCY_INGEST", "grocy", grocy),
)
@ -109,7 +113,7 @@ SECTION_SOURCES = {
"political": ("news", "rci_social", "financial", "email", "flight_traffic",
"naval_traffic", "calendar"),
"household": ("calendar", "grocy"),
"network": ("opnsense_ids",),
"network": ("opnsense_ids", "infra_health"),
}
DEFAULT_SCHEDULE = "00,06,12,18"
@ -323,6 +327,9 @@ def build_section_contexts(collected, lookback_hours, is_evening_run, previous_r
contexts["network"] = {
"lookback_hours": lookback_hours,
"network_security": collected.get("opnsense_ids", []),
# Host/service state from CheckMK and every firewall's IDS status, each row
# naming its own target. See ingest/infra_health.py and synth/prompts/network.md.
"infra_health": collected.get("infra_health", []),
}
# A section nobody asked for is dropped here, before its context is ever built into

25
digest-engine/synth/prompts/network.md
View File

@ -31,6 +31,31 @@ section to pad.
no network data was collected this run. Do not infer that the network was
quiet, and do not invent an alert, a device, or a signature.
## Machine health: CheckMK, and every firewall by name
The context may carry entries tagged `"category": "infra_health"` — one per monitored
target, from the always-on poller rather than from this run (see
`ingest/infra_health.py`). Each has a `target`, a `state`, a `summary`, a `since`, and
up to twelve `problems`.
- **Always name the target.** There may be more than one firewall, and `main` being
healthy says nothing about `dmz`. "The IDS is running" is a sentence you must never
write when the context has two firewalls in it; write "Suricata is running on main;
dmz has not answered since 14:20."
- **`state: "unreachable"` is a finding, not an absence.** It means the poller asked
and got nothing — a machine that is off, a credential that expired, a cable. Report
it as prominently as a real failure, because it is one, and never as "no problems".
- **`since` is what makes this worth reading.** A critical service that went critical
four minutes ago and one that has been critical since Tuesday call for different
reactions. Say which, using the timestamp, whenever `since` is present. When it is
null the state has held for the whole retention window — say "long-standing", never
"just started".
- **`problems_truncated` is a count of what you were not shown.** If it is non-zero,
say so plainly ("12 of 47 shown"). Never summarise 47 failures from 12 of them.
- If there are no `infra_health` entries at all, say nothing about machine health.
Nothing being monitored and everything being fine look identical from here, and only
one of them is good news.
## History: one alert is noise, the same alert every night is a fact
The context may carry a `history` block from the digest's own archive of past

76
digest-engine/synth/prompts/political.md
View File

@ -15,9 +15,9 @@ Vorarlberg. Write for someone who already holds this politics and needs to be
oriented in the world this week — not for someone who needs to be convinced of
it.
## The four questions
## The five questions
Everything below serves four questions. They are the structure of the section,
Everything below serves five questions. They are the structure of the section,
not a checklist to append: decide what goes in the digest by asking which
question an item answers, and drop it if it answers none.
@ -46,10 +46,59 @@ question an item answers, and drop it if it answers none.
sections arrive there, and they are the one source in this context that no
feed can supply. Treat a comrade's report as a report, not as an anonymous
claim: say which section or comrade it came from where the mail says so.
5. **What class struggles are currently going on around the world?** Strikes,
occupations, factory takeovers, mass movements and general strikes that are
*live right now* — an inventory, not a curation. This is the one question that
does not ask what is most relevant or what moved this run: a dispute that has
been running for six weeks with nothing new to report still belongs on the
list, because the reader needs to know what is going on, not only what
happened since breakfast. See "The roll-call" below for how to build it.
Each question gets its own window (see "Output"). A question with nothing worth
saying this run gets one honest line, never filler.
Questions 1 and 5 look at the same material and are not the same job. Question 1
asks what a communist needs to understand — the featured, quoted, analysed items,
few of them. Question 5 asks what is happening — the full standing list, one line
each, no analysis. An item can be in both: featured in `political-global` with
its quote and its theory, and named again in the roll-call as one of the
struggles currently running. That is not duplication, it is the difference
between the briefing and the map.
## The roll-call
Question 5's window is a list of the class struggles this run's context shows to
be under way. It has its own rules, because a standing inventory fails in
different ways from a curated section.
- **Everything on it still comes from this run's context.** "No speculation"
below is not relaxed here: a struggle goes on the list because entries in this
run's material describe it, never because you know of it from elsewhere or
remember it from a previous digest. Your own knowledge of the world is not a
source.
- **`times_seen` and `first_seen_at` are what keep it standing.** An entry that
has appeared in six consecutive runs is exactly the long strike this question
exists to hold on to — it stays on the roll-call at full standing, and the
"a story featured for three runs needs a reason to be featured again" rule in
"History" governs questions 1–4, not this one.
- **One line each**: what, where, who is out or occupying, and since when if the
context says. "Rail workers, France — national strike over pensions, running
since 4 Aug (first seen 5 Aug, in every run since)." No analysis, no impact
paragraph; that belongs to question 1 if the item earns it.
- **Say when a struggle's state is unknown rather than assuming it continues.**
If the last thing the context said about a strike was a fortnight ago and
nothing since, the line says that: "no entry in this run's material since 27
Jul — outcome unknown here." A roll-call that quietly implies everything on it
is still live is worse than a short one.
- **Ended is a finding.** A strike settled, defeated, sold out or won this run
stays on the list once, marked with how it ended and by whose account, then
drops off. Defeats are not omitted to keep the list encouraging.
- Sources attach the same way as everywhere else — see "Curating and quoting".
A line with no entry behind it does not go on the list.
- Cap it at roughly a dozen entries. If more clear the bar, keep the largest and
those a reader could act on or be asked about, and say plainly that the list
is cut ("12 of ~20 disputes in this run's material").
## What this section is, and is not
This is a curation-and-correlation exercise, not a running commentary on
@ -149,7 +198,7 @@ the reader might choose to watch or listen to later.
says** — never guess at the content of a video from its title alone.
- Order newest first, cap it at about six entries, and omit the window entirely
when there are no new episodes this run. Never carry an episode over into one
of the four question windows or onto the globe.
of the five question windows or onto the globe.
- A `"theory"` article whose title is marked as a podcast (marxist.com prefixes
these with `[Podcast]`) belongs in this window too, not in the analysis.
@ -328,7 +377,9 @@ context — a specific entry, figure, or quote. This is not a style preference:
## Curating and quoting
For each entry that passes the relevance filter above — i.e. that answers one of
the four questions — and inside the window belonging to that question:
the five questions — and inside the window belonging to that question (question
5's window is the exception: it is one line per struggle with its sources, and
the quoting and analysis rules below do not apply to it, see "The roll-call"):
- Feature it explicitly with a short excerpt or quotation taken verbatim from
the entry's own text (its title/description/body field, not your
@ -523,6 +574,14 @@ Rules:
Omit the window entirely if the run genuinely has none.
- `political-rci` — question 4, the International and comrades' reports. Omit
if there is nothing; never manufacture organisational news.
- `political-struggles`, titled something like **"Struggles under way"** —
question 5, the roll-call. `kind: "list"`, one line per live struggle, per
"The roll-call" above. Most of the amber `star` and red `hammer-sickle`
markers on the globe should correspond to a line here; keep the two saying
the same thing. Omit the window only when this run's material describes no
live struggle anywhere — say so in one line in `political-global` when that
happens, because it is a surprising claim about the world rather than a quiet
run.
- `political-agenda` — the upcoming meetings and their agenda points, per "The
agendas" above. Omit when no agenda arrived.
- `political-todo`, titled **"Political todos"** — the tasks those agendas
@ -533,11 +592,14 @@ Rules:
You may add further windows beyond these when an item needs its own space (a
long piece of analysis, the financial indicators as their own `kind: "list"`
with each line stating the move *and* what it means for working people). Do not
drop or rename the four question windows to make room.
- At `detail_level: "compact"` keep the globe and fold the questions into a
drop or rename the five question windows to make room.
- At `detail_level: "compact"` keep the globe and fold questions 1–4 into a
single `kind: "list"` window — one or two lines each for questions 1 and 2, one
line each for 3 and 4 if they have anything. Sources still attach; they cost no
space when folded. **Keep `political-todo` as its own window even here**: it is
space when folded. **Keep `political-struggles` as its own window even here**,
shortened to about six lines and to the largest and nearest struggles, saying
it is cut: a roll-call folded into a summary line stops being a roll-call.
**Keep `political-todo` as its own window even here** too: it is
the one part of this section a person acts on rather than reads, and it is the
first thing they will look for on a phone.
- `narration` is spoken aloud by a TTS voice, so no markdown, no URLs, no emoji.

159
docs/components.md
View File

@ -44,6 +44,45 @@ Components
1xSound System
1xRuView Presence Node (ESP32-S3 CSI board)
#Workshop / Office
The room the workshop assistant runs in (`workshop/`, docs/workshop-assistant.md).
Structurally the kitchen display again — a Sway kiosk, a screen, a camera — with two
differences that matter for buying: the screen wants to be BIG (schematics, board
plans and a camera grid at the same time), and the camera is a MACRO problem, not a
wide-angle one.
1xTiny PC (have — "near infinite Tiny Pcs")
1xLarge monitor or TV, wall-mounted above the bench
1xUSB inspection/macro camera -> reading a service tag or PCB silkscreen at 10-20cm
1xVoice Reciever - Round screen thingy
1xRuView Presence Node (ESP32-S3 CSI board)
#Network cameras
Distinct from every USB camera in this project, which are all fixed-purpose at angles
useless for anything else (an item held to the kitchen lens; an appliance door). These
are the ones on the network, ingested by the Frigate/go2rtc this stack already runs,
and they are what the workshop display can actually show.
2-4xPoE or WiFi camera (RTSP/ONVIF) -> count depends on which rooms; see #Open items
#Appliance monitoring (fridges + freezers)
The household runs more than one cold appliance — the main fridge's freezer
compartment is too small, so there is at least a second freezer elsewhere. That is
what makes "which one is it in?" a real question; see
[`fridge-item-location.md`](fridge-item-location.md) for why the answer is *door
sensors plus an outward-facing camera*, and why nothing goes inside the appliance.
Per appliance (assume 3 to start — kitchen fridge, kitchen freezer compartment,
second freezer):
1xZigbee door contact sensor (mounted on the OUTSIDE face of the door, never inside)
1xDoorway camera (looking at the door opening from outside, ~1.8-2m up)
The appliances themselves are already owned and are not bought by this project. The
room the second freezer lives in is not decided yet — see #Open items.
##Lighting
~16x RGB smart bulb (exact count/room split TBD — see #Open items). Zigbee, not
@ -62,13 +101,17 @@ ecosystem. See #Need for the specific pick and why.
2x ThinClient (tiny)
1x Miniscreen System (Lenovo all-in-one, built-in touchscreen — already owned)
1x Beamer
4x Voice Reciever - round screen thingy (Living Room, Loggia, Linus Room, Kitchen)
5x Voice Reciever - round screen thingy (Living Room, Loggia, Linus Room, Kitchen, Workshop)
1x HA Voice PE
5x Sound System (all confirmed: Living Room, Loggia, Linus Room, Amirs Room, Kitchen)
1x Tiny PC with Mini Touchscreen (Kitchen)
1x Camera (Kitchen)
2x Spare webcam (destination TBD)
5x RuView Presence Node (ESP32-S3 CSI board — one per room: Living Room, Loggia, Linus Room, Amirs Room, Kitchen)
6x RuView Presence Node (ESP32-S3 CSI board — one per room: Living Room, Loggia, Linus Room, Amirs Room, Kitchen, Workshop)
3x Zigbee door contact sensor (one per cold appliance door — count depends on the final appliance list)
2x Doorway camera (network/RTSP, one per appliance that isn't already covered by a camera pointing the right way)
1x Tiny PC + 1x large monitor/TV + 1x USB inspection/macro camera (Workshop/Office)
2-4x Network camera (RTSP/ONVIF — rooms TBD, see #Open items)
#Lighting
~16x RGB smart bulb (Zigbee) — count/room split not finalized
@ -78,12 +121,30 @@ Living Room: 1x ThinClient, 1x Voice Reciever, 1x Sound System (have), 1x RuVie
Loggia: 1x Miniscreen System (have), 1x Sound System, 1x Voice Reciever, 1x RuView node
Linus Room: 1x Voice Reciever, 1x Sound System, 1x RuView node, +1 unresolved (???)
Amirs Room: 1x Beamer, 1x ThinClient, 1x HA Voice PE, 1x Sound System, 1x RuView node
Kitchen: 1x Tiny PC w/ Touchscreen, 1x Camera, 1x Voice Reciever, 1x Sound System, 1x RuView node
Kitchen: 1x Tiny PC w/ Touchscreen, 1x Camera, 1x Voice Reciever, 1x Sound System, 1x RuView node,
2x door contact sensor (fridge + freezer compartment), 1x doorway camera
Workshop: 1x Tiny PC (have), 1x large monitor/TV, 1x inspection camera, 1x Voice Reciever, 1x RuView node
Wherever the second freezer is: 1x door contact sensor, 1x doorway camera
#Open items
- Linus Room's "???" line is still undecided.
- Bulb count/room split is a guess ("like 16") — not yet mapped to specific rooms
or fixture counts per room.
- **Which room the second freezer is in**, and therefore whether its doorway camera
needs its own PoE/power run or can share the kitchen's. Also whether the kitchen
fridge and its freezer compartment are one door or two — that decides whether the
contact-sensor count is 2 or 3.
- **How many network cameras, and in which rooms.** Priced at 3 as a placeholder. The
workshop display can show any of them; which ones are worth watching is a decision
nobody has made, and unlike the bulbs this one has a privacy dimension — a camera in
a room is a camera in a room, the same caveat `hosts/thin-client`'s README already
spends a section on for gesture control.
- **Whether the workshop monitor needs to be new at all.** It is the only display here
with no touch requirement; almost any HDMI panel or old TV does the job.
- **Whether the kitchen's existing C920 can double as the kitchen fridge's doorway
camera.** It is already there and already on a host, so it is free if the display
happens to face the fridge — and useless if it doesn't. Nobody has measured the
angle; do that before buying a second camera for that room.
#Have
1xSound System (earmarked for Living Room)
@ -180,6 +241,74 @@ the identity registration flow, not just kitchen-display/door-panel).
pantry-vision's food-photo use case that specifically wants the C920's autofocus,
so there's no reason to pay for that here.
#Appliance door sensors — need ~3
One per cold-appliance door, feeding the Zigbee2MQTT mesh this project already runs
(§1.3) — the same reasoning as the bulbs: one local mesh, not a manufacturer cloud.
These are what trigger the doorway cameras below (`pantry-vision`'s
`POST /doorway-event`, see `pantry-vision/README.md`), and they are worth buying
**even if no camera is ever installed**: on their own they already answer "which
appliance was opened, when" and "has the freezer been standing open for four
minutes", which prevents more spoilage per euro than any amount of item-locating.
- [Aqara Door and Window Sensor (Zigbee)](https://www.amazon.com/Aqara-Window-Sensor-Wireless-Security/dp/B07D37FKGX)
(~€10–15 each) — plain reed-switch contact sensor, long-standing Zigbee2MQTT
support, coin-cell powered.
- **Mount the sensor body on the OUTSIDE face of the door and the magnet on the
outside of the frame** (or vice versa). Not inside the compartment: a coin cell at
−18 °C loses a large part of its usable capacity, and every door opening condenses
humid room air onto whatever is in there. The reed switch does not care which side
of the door it is on, so there is no reason to pay the cold penalty.
- Battery life in this application is **unverified** — a fridge door that opens forty
times a day is a much higher duty cycle than the window these are sold for. Budget
for spare CR1632/CR2032s and check one after a month.
#Doorway cameras (appliances) — need ~2
Outward-facing, above the door on the hinge side, ~1.8–2 m, angled so the field of
view is the door aperture and the ~40 cm in front of it. **Never inside the
appliance** — `docs/fridge-item-location.md` has the full argument (no interior
power, condensation on every opening, and −18 °C being outside every consumer camera
module's rating, which rules out exactly the compartment the question came from).
- [TP-Link Tapo C120 (2K, RTSP/ONVIF)](https://www.amazon.com/TP-Link-Tapo-Security-Detection-C120/dp/B0CL5RRZ2M)
(~€30–40 each) — needs to expose a plain **RTSP** stream so Frigate can ingest it,
the same way `chores`' watch points already work. **Unverified**: whether this
specific model serves RTSP without go2rtc as a bridge — the same open question
Phase 20 already carries for its own camera pick (project-plan open decision #23),
and worth resolving once for both rather than twice.
- One of the two **spare webcams** already on this list is a legitimate substitute
for whichever appliance sits next to an existing host — a USB webcam needs a
machine, and the kitchen has one; the second freezer's room probably doesn't.
- Buy **one** first, not both. `docs/fridge-item-location.md` recommends running door
sensors alone for a month before deciding whether item-level localisation is a
thing the household actually wanted, and the test that decides whether the camera
works at all — can a local vision model identify an item in a moving hand at
doorway distance? — can be run today with the kitchen's existing webcam.
#Workshop display — need 1 screen + 1 macro camera
The Tiny PC is covered by "Have". These two are not:
- Large monitor or TV (32-43", 1080p is enough; 4K only if you want two schematics
side by side). **Any HDMI display works** — this is the one screen in the project
with no touch requirement, because the workshop interaction is voice and keyboard
with dirty hands, not fingers on glass. Reuse anything you have before buying.
- [USB digital microscope / inspection camera, 1080p with adjustable stand](https://www.amazon.com/Microscope-Compatible-Magnification-Adjustable-Compatible/dp/B08BC7GDVL)
(~€30-60) — **not a webcam.** The job is reading 2mm text on a service tag or a PCB
silkscreen, so autofocus at 10-20cm and enough resolution to resolve small print
matter far more than field of view. The C920's autofocus does not go near enough.
This is the single most important pick for whether the OCR-first identification in
`docs/workshop-assistant.md` works at all, and it is cheap enough to test the
premise before committing to the rest.
#Network cameras — need 2-4
For the workshop display's camera view, and to give Frigate something to watch beyond
the peephole cam. Must expose **RTSP** — that is the ingest path Frigate and go2rtc
use, and it is the one thing worth checking on the listing before buying.
- [TP-Link Tapo C120 (2K, RTSP/ONVIF)](https://www.amazon.com/TP-Link-Tapo-Security-Detection-C120/dp/B0CL5RRZ2M)
(~€30-40 each) — same model already listed for the appliance doorways, deliberately:
one camera model across the house means one set of quirks to learn, one stream
configuration to get right, and spares that fit anywhere. **Unverified**: whether
this model serves RTSP without go2rtc as a bridge (project-plan open decision #23).
- Buy **one first** and get it into Frigate before ordering the rest. Every camera
after the first is a repeat of a solved problem; the first one is where you find out
whether the model was the right pick.
#Fixed BLE tags — need ~4 to start
For household members who don't (or shouldn't have to) carry a phone for presence to
work — a grandmother without a smartphone is the concrete case, but this is also just
@ -255,8 +384,8 @@ local pricing/VAT/shipping.
| TMY 1080P mini projector | 1 | €40–65 | €40–65 |
| HDMI cable | 1 | €6–10 | €6–10 |
| Ceiling/shelf mount *(optional)* | 1 | €15–25 | €15–25 |
| Waveshare ESP32-S3-Touch-LCD-1.85C board | 4 | €35–45 | €140–180 |
| USB-C 5V/3A wall adapter | 4 | €7–12 | €28–48 |
| Waveshare ESP32-S3-Touch-LCD-1.85C board | 5 | €35–45 | €175–225 |
| USB-C 5V/3A wall adapter | 5 | €7–12 | €35–60 |
| Fosi Audio V3 amp | 4 | €130 | €520 |
| Micca MB42X G2 speakers, pair | 4 | €100 | €400 |
| 16AWG speaker wire, 100ft spool *(one-time)* | 1 | €15–22 | €15–22 |
@ -265,10 +394,24 @@ local pricing/VAT/shipping.
| Logitech C270 spare webcam | 2 | €25–35 | €50–70 |
| innr RB 285 C Zigbee RGB bulb | ~16 (TBD) | €15–20 | €240–320 |
| iTag-Tiny fixed BLE tag | 4 | €3–8 | €12–32 |
| Espressif ESP32-S3-DevKitC-1-N8R2 (RuView) | 5 | €8–12 | €40–60 |
| Espressif ESP32-S3-DevKitC-1-N8R2 (RuView) | 6 | €8–12 | €48–72 |
| Aqara Zigbee door contact sensor (appliance doors) | ~3 | €10–15 | €30–45 |
| Tapo C120 doorway camera *(buy 1 first — see that section)* | ~2 | €30–40 | €60–80 |
| Tapo C120 network camera (rooms/workshop view) | ~3 | €30–40 | €90–120 |
| Workshop monitor/TV *(reuse one first if you have it)* | 1 | €120–250 | €120–250 |
| USB inspection/macro camera (workshop) | 1 | €30–60 | €30–60 |
**Subtotal (excludes the optional beamer mount): ~€1,831–2,182**
**Subtotal, including the optional beamer mount: ~€1,846–2,207**
**Subtotal (excludes the optional beamer mount): ~€2,244–2,846**
**Subtotal, including the optional beamer mount: ~€2,259–2,871**
The workshop line items are the ones to sequence rather than buy at once: the macro
camera is cheap and decides whether the OCR-first identification works at all, the
monitor can be anything you already own, and the network cameras are one-then-more.
The two appliance-monitoring lines are the only ones on this list bought in a
deliberate order rather than all at once: the door sensors are worth having on their
own merits, the cameras are not worth having until the sensors have proved the
question is real. Buying one camera instead of two costs €30–40 to find out.
All 5 Sound Systems (Living Room, Loggia, Linus Room, Amirs Room, Kitchen) are
confirmed — see `#Open items` above — so all 4 needed amp+speaker kits are in the

173
docs/endpoint-surfaces.md Normal file
View File

@ -0,0 +1,173 @@
# Two shared endpoint surfaces: the now-playing visualiser and the 3D floorplan
Design notes for two things that must look the same on every screen in the house — a
thin client on a TV, the Loggia all-in-one, the kitchen panel, the door panel — and
which therefore belong in a **shared, vendored SDK** rather than being written four
times.
> **Status: both are built** — `render/media-visualiser/` and `render/floorplan-3d/`.
> One thing changed in the building, and it is called out in section 2: the 3D view is
> **canvas 2D, not three.js**. The scene turned out to be prisms on a plane, which an
> isometric projection draws in ~200 lines without the megabyte, without a build step,
> and with predictable performance on the small panels. Everything else below was
> implemented as specified.
---
## 1. The now-playing visualiser
Wanted: a circular CAVA-style spectrum behind every playing-status screen, coloured
from the album art, with lyrics scrolling under the cover when they exist.
### The audio problem, which is the whole problem
CAVA reads an audio stream. The endpoints do not all have one:
| Endpoint | Is the audio local? | Can it get a spectrum? |
|---|---|---|
| Thin client running mpv/spotifyd | **yes** | Yes — real FFT of the actual output |
| Audio endpoint (Spotify Connect box) | yes, but it is headless | Yes, but there is no screen to draw on |
| Kitchen / door panel showing *what the living room is playing* | **no** | **No. There is no audio here at all** |
So a design that requires real audio analysis works on one endpoint and silently
degrades to a dead circle on the others — the worst outcome, because the screen looks
broken rather than looking deliberate.
**Two-tier, declared up front:**
- **Reactive tier** — where audio is local, CAVA (or a WebAudio `AnalyserNode` when the
player is in the browser) drives the bars. `cava` has a `raw` output mode writing
plain numbers to a FIFO, which a small agent can publish over MQTT; that is the least
fragile path on a machine already running an agent.
- **Synthetic tier** — everywhere else, the ring is animated from **track position and
tempo**, not from silence. It breathes with the beat rather than pretending to
analyse it. Nobody watching a kitchen panel from across the room can tell, and it
never looks broken.
Do not let the synthetic tier claim to be the reactive one anywhere in the UI. It is a
mood light, and the moment somebody believes it is a spectrum they will report it as a
bug every time it does not match a bass drop.
### Colours from the album art
Pull 3–5 dominant colours from the cover, client-side, at load:
1. Draw the cover into a small offscreen canvas (64×64 is plenty — this is a palette,
not a photograph).
2. Bucket pixels in RGB space, take the top buckets by population.
3. **Reject near-greys and near-blacks** before ranking. Album art is full of them, and
a palette extracted naively from a dark cover gives you four indistinguishable dark
greys and a visualiser that looks switched off.
4. **Force a minimum contrast against the background.** Lift the chosen colours in
lightness until they clear the panel behind them; a visualiser you cannot see is
the same as no visualiser, and this is the step that gets skipped.
No library needed — that is about forty lines of canvas work, and it keeps the
"vendored, dependency-free" property the canvas SDKs already have.
### Lyrics
- **Source**: whatever the player exposes. Music Assistant and MPD both surface
synced-lyrics fields when the provider has them; `.lrc` sidecar files are the other
common case for local libraries.
- **Synced (LRC) vs plain**: with timestamps, highlight the current line and scroll it
to centre. Without them, do not fake it — scroll slowly, or just show the text. A
plain-lyrics block auto-scrolled at a guessed rate is wrong within ten seconds and
stays wrong.
- **Absent is the normal case.** Most tracks in most libraries have no lyrics. The
layout must be designed for "no lyrics" as the default state, with lyrics as the
addition — not a gap where they would go.
- Never fetch lyrics from the internet at render time. If lyrics are worth having,
they are worth caching where the track is.
### Where it lives
A new `render/media-visualiser/` in the shared SDK style: one JS file, one CSS file,
vendored into each host that needs it, exactly as `digest-canvas-sdk` and
`canvas-sdk` already are. Inputs are a normalised now-playing object
(`{title, artist, album, art_url, position_ms, duration_ms, tempo?, lyrics?}`) and an
optional spectrum feed. Every endpoint already receives now-playing over MQTT; nothing
new has to be plumbed for the synthetic tier.
---
## 2. The 3D floorplan
Wanted: the Pebble app's presence view, in 3D, on any endpoint — rooms, and who is in
them, with profile pictures.
### It is the same data, and that is the point
`GET /floorplan/presence` already returns everything: room polygons normalised 0–1,
each room's occupants, each occupant's `color`, `initial` and `has_photo`, plus
`unplaced` for people who are home but not locatable. The Pebble app and this render
the same JSON at different fidelities — which is the reason to build the second one at
all. If they diverge, one of them is lying.
### Extrusion, not modelling
Do not author a 3D model. Take the existing 2D polygons and **extrude them to a wall
height**, which is one `THREE.ExtrudeGeometry` per room and needs no new data beyond a
single `wall_height_m` per level. A hand-authored model would be prettier, immediately
stale the first time a room is redrawn, and unmaintainable by the person who drew the
plan in a 2D editor.
Camera: fixed isometric by default, drag to orbit, no free-fly. An orbit camera on a
wall panel is something people knock out of alignment and cannot get back.
### The occupant markers
This is where 3D earns itself, because the Pebble's constraint — 18×18 px, no room for
a face — is gone:
- A **billboarded disc** above each occupied room, always facing the camera (never a
flat sprite lying on the floor, which is unreadable at a glance).
- **Profile picture inside the disc**, ringed in the person's colour, falling back to
the initial on the colour when `has_photo` is false. The ring matters even with a
photo: it is what ties this marker to the same person's marker on the watch and in
the admin panel.
- **Occupied rooms lit, empty rooms dark**, the same rule the watchapp uses, and for
the same reason: state should read before detail. In 3D that is an emissive floor
material rather than a fill colour, and the third state — *drawn but never reported
by HA* — stays visually distinct, because rendering "no data" as "empty" is a quiet
lie on any screen.
- **`unplaced` people get a shelf**, not a hidden list: a strip along the bottom
showing everyone who is home but unlocatable. They are the people you are most often
looking for.
### The honest constraint — and how it was resolved
The plan was to vendor three.js and call it a deliberate break with the
dependency-free rule. Building it made the cheaper answer obvious and it was taken
instead: **canvas 2D with an isometric projection and painter's-algorithm sorting.**
The scene is prisms standing on a plane — no lighting model worth the name, no
textures, no physics, no camera motion beyond an orbit. That is ~200 lines of canvas
2D, it redraws in well under a millisecond on the small panels (where a WebGL context
on integrated graphics is a much less predictable proposition), and the frontend stays
at zero dependencies with nothing to keep patched.
This is not a compromise version of the three.js plan; it is the smaller correct tool
for this specific scene. If the view ever grows real lighting or an imported model,
three.js becomes right and `floorplan3d.js` becomes the fallback for the weak panels.
And the same caveat as the watchapp, which no amount of rendering fixes: **this is only
as right as room-level presence is**, which has never been measured in this house. A
beautiful 3D house with everybody sitting in the `unplaced` shelf is a beautiful 3D
house that tells you nothing. Test that first — the admin panel's floorplan tab with
**Live** ticked answers it for free.
---
## Build order
1. **The visualiser's synthetic tier**, on one endpoint. It needs no new data and no
audio plumbing, and it is what tells you whether the effect is worth the reactive
tier's complexity.
2. **Album-art palette extraction**, which is where the look actually comes from.
3. **Lyrics**, synced-only at first.
4. **The reactive tier**, on the thin clients that have local audio.
5. **The 3D floorplan** — built. It turned out not to be a dependency addition at all
(see above), so the sequencing argument that put it last no longer applies. The
*other* reason still does, in full: **it is only as right as room-level presence,
which has never been measured here.** Test that before believing the picture.

223
docs/fridge-item-location.md Normal file
View File

@ -0,0 +1,223 @@
# Knowing *where* a thing is: multiple fridges, shelves, and what it would cost
A feasibility note that has since grown a build. `pantry-vision` (Phase 17) knows
**what** the household has and **when** it expires. This asks what it would take to
also know **where** — which of several fridges, and how far inside one.
The recommendation at the bottom is deliberately smaller than the question, and the
reason is in the middle.
> **Status, 2026-08-10.** Steps 1 and 4 of the recommendation are now written:
> appliances as Grocy locations with a `POST /transfer` action and a "Move to…"
> control on the edit screen, and the door-sensor→camera→hint path
> (`pantry-vision/doorway.py`, `POST /doorway-event`). **Step 2 is hardware nobody has
> bought and step 3 is a month nobody has spent** — which means the camera half is
> written against an assumption that has never been tested: that a local vision model
> can identify an item in a moving hand at doorway distance. An appliance can be
> configured with a door sensor and no camera, and that remains the recommended way to
> start. Nothing about writing the code changes the order in which it is worth
> switching on.
## First, decide which question is actually being asked
These are three different problems with three different price tags, and they get
conflated constantly:
| Level | The question | Honest difficulty |
| --- | --- | --- |
| **L1 — which appliance** | "Is the mustard in the kitchen fridge or the loggia one?" | Solved today, no new hardware, one extra tap |
| **L2 — which zone** | "Door shelf, middle shelf, or the crisper drawer?" | Achievable at *hint* quality (60–80%), real hardware, real work |
| **L3 — exact position** | "Second from the left, behind the milk" | Not worth it. See "Occlusion" below |
The value is overwhelmingly at L1, and it is the level this household actually has a
problem at — the whole premise of the question is *multiple fridges*, because the
freezer compartment of the main one is too small. Somebody standing in the kitchen
wondering whether to walk to the loggia is asking an L1 question. L2 saves ten seconds
of looking; L3 saves none, because by the time you have the door open you can see.
**Do not build downward from L3.** Every camera-in-the-fridge design starts there,
because that is the impressive demo, and every one of them founders on the same rock.
## L1: available now, and mostly a data-model decision
`pantry-vision` already resolves the confirm screen's fridge/freezer/cupboard answer
to a **Grocy location**, created by name (`PLACEMENT_LOCATION_NAMES` in `server.py`,
`PANTRY_LOCATION_*` in the env file), and passes `location_id` on the stock add. Grocy
has modelled multiple locations natively since long before this project existed.
So L1 is not a vision problem at all. It is three small changes, and two and a half of
them are now written:
1. **The placement answer becomes the real appliances** — "Fridge (kitchen)", "Freezer
(loggia)", "Cupboard" — named in `PANTRY_LOCATION_*` and resolved to Grocy
locations. *Done, with a caveat*: the confirm screen still offers the four
categories (fridge / freezer / cupboard / counter) and each maps to one location
name, so a household with two fridges has to pick which one that word means and
move the other by hand. Reading the appliance list straight off Grocy's locations
is the obvious next step and has not been taken.
2. **`/inventory` carries the location through**, so every list says where. *Done* —
`location_id`/`location` on every row, `locations` on the response.
3. **Moving something between fridges needs a "moved" action**, or the data rots
within a fortnight. *Done* — `POST /transfer` onto Grocy's own transfer endpoint,
with a picker on the edit screen.
Point 3 is the whole ballgame, and it is the thing every camera proposal below is
really trying to buy its way out of: **the expensive part is not learning where things
are, it is noticing when they move.**
## L2/L3: what the camera approaches actually run into
### Occlusion is the rock
A single camera sees the front row. Groceries are stored in rows. The mustard behind
the milk is invisible, and no model — local, cloud, or otherwise — recovers it from a
picture that does not contain it. Multi-camera stereo does not fix this either; it is
not a depth problem, it is a line-of-sight problem.
This is why L3 is off the table and why L2 is a *hint*, not an assertion. Any design
that needs the camera to enumerate a shelf's contents is buying an answer that is
silently incomplete, which for an inventory is the worst failure available: "we have
no mustard" from a system that simply could not see it produces a second jar of
mustard.
### The interior-camera problems, in the order they will bite
- **Power.** Fridges have no interior outlet. The three options are a battery (poor:
cold cuts usable lithium capacity substantially, and this is a duty-cycled camera,
not a doorbell), a flat ribbon cable through the door gasket (works — this is how
retrofit fridge cams do it — but it is a modification to the seal of an appliance
that may be rented), or drilling a grommet (irreversible).
- **The freezer is out of scope for interior cameras, full stop.** Off-the-shelf
camera modules — ESP32-CAM, Pi camera, USB webcams — are specified to roughly 0 °C
at best. −18 °C is outside every consumer part's rating: lubricants stiffen,
electrolytics lose capacitance, and condensation cycling on every door open
eventually gets inside the enclosure. Industrial cold-chain cameras exist and cost
more than the freezer. **Given that the second appliance exists precisely because
the built-in freezer is too small, this rules interior cameras out of the exact
compartment the question started from.**
- **Condensation.** Every door opening puts warm humid air on a cold lens. It fogs,
and it fogs for minutes — longer than the interaction. Mitigations (sealed housing
with desiccant, conformal coating, a lens heater) all cost power, which brings back
problem one.
- **Light.** The interior lamp is on only while the door is open, so the only usable
frames are the ones during an opening — at which point the camera might as well be
outside.
Every one of those bullets is a *class* of problem, not a part-selection detail. Taken
together they say: **do not put the camera inside.**
### The approach that survives all four: watch the doorway, not the shelf
Point a camera at the **plane of the door opening**, from outside, and trigger on a
door sensor. Then you are not parsing a packed shelf — you are watching one item, held
in one hand, unoccluded, at a known moment, against a known background, with the
kitchen's own lighting. That is the *same* recognition problem `/identify` already
solves, which means it reuses the prompt, the model, and the "propose, never
auto-commit" guardrail instead of needing a new perception stack.
What it buys: **transactions, not state.** "A jar of X went into the loggia fridge at
18:42." Transactions are what keep L1 from rotting (the problem above), and a
sequence of them gives L2 for free if the camera can also see *which shelf the hand
went to* — a much weaker inference than reading the shelf, and one that degrades
gracefully into "somewhere in this fridge".
What it costs in honesty: it misses. Two items at once, an arm across the lens, a foot
closing the door. So its output is a **hint with a timestamp** — "last seen going into
the loggia fridge, Tuesday 18:42" — never an assertion the UI treats as fact. That
framing is what makes a 70%-accurate system useful instead of infuriating, and it is
the same distinction `digest-engine` already draws between what a source *says* and
what is *so*.
This is what `pantry-vision/doorway.py` implements. An HA automation on the contact
sensor POSTs `/doorway-event`, the service answers 202 and takes a short burst off the
request thread (three frames, stopping at the first that recognises anything), and the
result lands in **its own SQLite file** — not in Grocy, because Grocy owns stock and
has nowhere to put "a camera thinks it saw something like this go past that door, and
might be wrong". Hints expire after 30 days, because "last seen a month ago" tells
nobody anything they didn't know. The edit screen shows the sighting next to the
location Grocy records, and acting on it is a tap on *Move*.
### Where the cameras go, concretely
- **One camera per appliance**, mounted above the door on the hinge side, ~1.8–2 m,
angled down so its field of view is the door aperture and the ~40 cm in front of it.
Not inside the door swing. `docs/components.md` already lists **2× spare webcam
(destination TBD)** — this is a plausible destination for one of them, which makes
the experiment nearly free.
- **A contact sensor on every fridge and freezer door.** Zigbee, on the coordinator
this project already has. This is the single highest value-per-euro item in this
whole note and it is worth doing *even if no camera is ever installed*: it is what
triggers the capture, it is what tells you which appliance a transaction belongs to,
and on its own it already supports "the loggia freezer has been open for four
minutes" — which prevents more spoilage per euro than any amount of localisation.
- **No new inter-VLAN paths.** Both devices sit where the existing camera/IoT devices
sit; the capture is pulled by the container host, same direction as everything else.
### Approaches ruled out, and why
- **RFID/NFC tags per item.** The only technology that genuinely answers L3. It also
requires tagging every single item by hand, which is a *worse* manual step than the
one tap L1 costs — and a fridge is a metal box full of water, which detunes UHF tags
badly. Rules itself out twice.
- **Load cells under each shelf.** Excellent at "how much is left", blind to identity,
and a per-shelf retrofit on appliances that may be rented.
- **BLE/UWB.** Nothing to attach a transmitter to. Not applicable to passive goods.
## The compute question nobody asks until it's too late
Reading a shelf is not the same task as reading one held-up jar: small objects,
partial labels, angles, occlusion. A 7–13B local VLM will be markedly worse at it than
at the current task, and the honest architecture is detect-then-crop — a small object
detector proposing boxes, the VLM reading each crop — which is a Frigate-class,
always-on GPU workload.
There is one RTX 3060 in this design, and contention on it is already an open
question (project plan open decision #4, `MAX_LOADED_MODELS=1`). Adding a
continuous detector to a GPU that also has to answer Assist within voice latency and
run four digests a day is a scheduling problem, not a spare-capacity problem. **The
doorway-camera approach avoids this too**: it is event-driven, a handful of inferences
per day, on the model that is already loaded.
## Recommendation
In order, stopping wherever the household stops caring:
1. **Make L1 real in software** — appliances as Grocy locations, location shown in
every list, and a transfer action so moves get recorded. No hardware. This is the
only step that is unambiguously worth doing, and it must come first regardless:
without it there is nowhere to *put* a camera's answer. **Written.**
2. **Contact sensors on every fridge/freezer door.** Cheap, Zigbee, immediately useful
on their own, and the prerequisite for everything below. See
`docs/components.md`'s "Appliance door sensors" — ~€10–15 each, mounted on the
*outside* face of the door, because a coin cell at −18 °C is a false economy.
**Not bought.**
3. **Live with 1+2 for a month.** The real finding will be whether anyone ever wanted
L2, or whether "which fridge" plus "the door's been open" was the whole need. This
step is not padding — it is the only way to avoid building L2 for a question nobody
turned out to be asking. **Not done, and writing step 4 early does not skip it.**
4. **Only then**, a doorway camera on one appliance, producing timestamped *hints*,
reusing `/identify` unchanged. One appliance, not all of them, until it earns the
second. **Written** (`pantry-vision/doorway.py`), **untested against a real camera,
and configurable per appliance so it can stay off.**
5. **Never** an interior camera in the freezer, and probably not in the fridge either.
A related chore was added alongside this: `chores`' **`groceries_out_of_place`** watch
point, which notices food standing out on a counter and nudges whoever the camera last
saw — litter's twin, with a 30-minute fuse instead of four hours, because the failure
there is spoilage rather than untidiness. It is a *different* system looking at a
*different* surface for a *different* reason, and deliberately so: it knows food is
out, not what the food is, and it never touches Grocy. See `chores/README.md`.
## What in here is unverified
Everything with a number in it. Specifically: consumer camera modules' actual
low-temperature behaviour (the 0 °C figure is a datasheet-class generalisation, not a
part this project has picked or tested); how badly a lens really fogs on a door
opening in this kitchen's humidity; whether a local VLM can identify an item held in a
moving hand at doorway distance at all — that last one is the assumption step 4 lives
or dies on, and it is testable today with the kitchen display's existing camera and
about twenty minutes, long before anything is bought.
Related open decisions in `docs/project-plan.md`: #18 (no vision model picked,
benchmarked, or measured for latency) and #4 (GPU contention). Both apply here
unchanged, and neither gets easier with a second camera pointed at anything.

280
docs/pebble-presence-watchface.md Normal file
View File

@ -0,0 +1,280 @@
# A Pebble watchapp that shows who is in which room
Feasibility note. The question: take the floorplan drawn in `identity`'s admin panel,
put it on a Pebble, and mark each room with who is standing in it.
**Verdict: the watch is the easy part.** Drawing the plan is a few hundred bytes and
some `gpath` calls, and the data already exists at `GET /floorplan/presence`. It is a
**watchapp**, not a watchface — see "Zooming into a room" for why the buttons decide
that.
The hard part is upstream and unchanged by any of this: **room-level presence
itself.** Build it if you want it, but do not expect it to be more right than the
presence data feeding it, which today is "which HA area a trusted BLE entity reports" and has
never been measured for room-level accuracy in this house.
## The hardware: a Pebble Round 2
**That is the target — it is the watch that exists.** 260×260, 64-colour e-paper,
touch. For reference, the rest of the line:
| Model | Screen | Colour | Notes |
|---|---|---|---|
| **Pebble Round 2** | 260×260 | 64-colour e-paper, touch | **The target.** The most pixels in the line, and circular — see the next section, which is entirely about that |
| Pebble Time 2 | 200×228 | 64-colour e-paper, touch | Rectangular, so a plan fits without the geometry below. Fewer pixels, more usable ones |
| Pebble 2 Duo | 144×168 | Black and white | The per-person colour collapses to a grey; two people sharing an initial stop being distinguishable |
Classic Pebbles (144×168) run the same code, cramped.
## Fitting a rectangular plan on a round screen
A floorplan is a rectangle and the screen is a circle, so the plan gets inscribed in
the circle and the corners are simply not available. The arithmetic, for a plan of
aspect ratio `a = w/h` on a usable diameter `D`:
```
w = D·a / √(a²+1)
h = D / √(a²+1)
```
With `D = 240` (260 less a ~10 px margin, because a circular screen's outermost pixels
are where you least want a room boundary):
| Plan aspect | Usable box |
|---|---|
| 1:1 | 170 × 170 |
| 4:3 | 192 × 144 |
| 16:9 | 209 × 118 |
So a squarish plan gets **170×170** — comparable to what a Time 2 gives after chrome,
and more than the 144×168 classics ever had. The Round is not a downgrade for this;
it just has to be told the truth about its shape.
Three consequences to design to:
- **Compute the box in PebbleKit JS, not on the watch.** The projection step already
planned for (normalised polygons → pixels) simply takes the inscribed box instead of
the full screen. No watch-side change at all.
- **No corner furniture.** The "3 of 4 home placed" footer has nowhere to live on a
circle — a line of text at the bottom gets clipped by the bezel radius. Put it at the
vertical centre-bottom, short, or move it into the room-detail screen where there is
a full-width line to use.
- **Round screens have less usable area than their diameter suggests**, and a ten-room
plan at 170×170 gives each room roughly 40×35 px. That is still enough for a 16 px
occupant dot with an initial in it — which is the number that decided against
profile pictures above, and it does not change here.
## The data path
```
identity ──HTTP──> PebbleKit JS ──AppMessage/BT──> watchapp (C)
(container host) (runs on the Android ~2 KB budget gpath + text
/floorplan/presence phone, inside the rendering
Pebble app)
```
Three consequences, all of which shape the design:
- **The watch has no network of its own.** Everything goes through PebbleKit JS, which
runs on the phone, inside the Pebble app, and — this is the important part —
**only while the app is open.** It cannot poll in the background — which bounds the
battery cost, and also means every launch begins with a fetch.
- **The phone has to be able to reach `identity`.** On the home network that is direct.
Off it, that is the WireGuard split tunnel this project already uses for exactly this
class of problem — nothing here justifies putting `identity` on the WAN. If the
tunnel is down it shows its last state with an age on it, not a blank plan.
- **Android only**, which this household is anyway, so no iOS-side caveats apply.
## The payload budget, and why it is not a problem
Pebble's guaranteed AppMessage buffers are small — 124 bytes in / 636 out at the
documented minimum, ~2 KB each way in practice for a JS-backed app. That sounds
alarming until you count what a floorplan actually is:
- Room polygons are already stored **normalised 0.0–1.0** (`floorplan_rooms.points`).
Quantise each coordinate to one byte and a vertex costs 2 bytes.
- A ten-room plan at eight vertices a room: `10 × (1 + 8×2) = 170 bytes`.
- An occupant marker is 3 bytes: room index, colour index, initial.
So a whole floor with everybody on it lands around **200–250 bytes** — comfortably
inside one message, with room to spare for a level name and a timestamp. Send it as a
single byte-array tuple rather than one tuple per room; the dictionary overhead is
what would actually cost you.
**Do the projection on the phone, not the watch.** PebbleKit JS picks the level, scales
the normalised polygons to the watch's pixel box — the inscribed box, on the Round —
drops rooms too small to draw, and sends integers. The watch does `gpath_create` → `gpath_draw_filled` → outline → text.
No floating point, no layout logic, no second copy of the floorplan model on a device
with 64 KB to its name.
## Empty rooms dark, occupied rooms lit
The plan should read as a *state* at a glance, not as a drawing you have to search. So
occupancy is carried by the room fill itself, before you look at any marker:
| | Fill | Outline | Contents |
|---|---|---|---|
| **Empty** | near-black, barely above the background | dim | nothing, or the room's initial letter in the dim outline colour |
| **Occupied** | light — a pale warm grey/white | bright | the occupant dots, in their own colours |
Three details that decide whether this works:
- **Carry it in lightness, not hue.** The occupant dots are already using colour to
mean *who*; if the room fill also used colour to mean *occupied*, the two would
compete on the one channel that matters most on a 64-colour panel. Dark-vs-light is
the strongest signal e-paper has and it costs nothing.
- **A lit room must be lighter than any occupant dot is dark**, or the dot vanishes into
its own room. With `PERSON_COLORS` being mid-to-bright, a near-white fill and a dark
dot outline keeps every one of the eight readable — this is the same reason the admin
panel draws initials in near-black on the person's colour.
- **"Unknown" is not "empty".** A room the plan has drawn but whose area HA never
reports is neither occupied nor confirmed-empty, and rendering it as empty is a
quiet lie. Give it the empty fill with a **dashed or dotted outline** — a third state
that costs one drawing call and is the difference between "nobody is in the study"
and "nothing can see the study".
E-paper is reflective, so "lit" here means a lighter fill, not a backlight. The effect
is exactly the one you want in a dark hallway at 2am: the rooms with people in them are
the bright shapes.
## Profile pictures: no, and the arithmetic says so
On the Round 2, a ten-room plan inscribed at 170×170 gives each room about 40×35 px. A
face inside one, with the room outline still visible, gets about **18×18 px, in 64
colours, on e-paper**. That is not a picture of a person; it is four skin-toned blobs.
And each one costs ~330 bytes to ship, so three of them exceed the entire message
budget that currently carries the whole floor. The rectangular Time 2 is no better —
45×35 px rooms, a 20×20 face — so this is a conclusion about the class of device, not
about the shape of this one.
**Initial plus colour is the right answer, and it is why the colour exists.** A filled
16 px circle in the person's own colour with their initial in `GOTHIC_14_BOLD` reads
at arm's length, costs 3 bytes, and degrades honestly: on a black-and-white Pebble the
colour becomes a grey and the letter still works.
`identity` now serves both fields ready-made — `color` and `initial` on `/people`,
`/presence`, and each occupant in `/floorplan/presence` — so no consumer has to derive
an initial from a name or invent a palette. **The eight palette colours are chosen on
the 2-bits-per-channel lattice (`00/55/AA/FF`) that a colour Pebble renders natively**,
precisely so the colour on the watch is the colour in the admin panel and not a
dithered approximation of it.
## Zooming into a room: why this is an app and not a face
The wanted behaviour — press a button, cycle to the next room, see a plain list of who
is in it — is easy to draw and cheap in bytes. **The catch is that it forces a
watchapp instead of a watchface**, and that is a real product decision, not a detail:
- **Watchfaces do not receive button events.** The buttons belong to the system there
(Select opens the app menu, Up/Down are the system shortcuts), and Pebble's own docs
say **touch is deliberately restricted to watchapps too** — "easier to allow it later
than to take it away once apps depend on it."
- A **watchapp** gets buttons and touch, and can do exactly the requested cycling. What
it does not get is being your default screen: you launch it from the menu, look, and
leave.
So pick which the thing actually is:
| | Watchface | Watchapp |
|---|---|---|
| Shows without launching | **yes** — this is the whole glance-at-wrist value | no, it's a menu entry |
| Button cycling through rooms | no | **yes** |
| Shake to cycle | yes (`accel_tap_service_subscribe`, the long-standing shake-to-reveal trick — worth confirming on your firmware) | yes |
| Practical shape | the plan, occupant dots, nothing else | the plan **plus** the per-room drill-down |
**Decided: build the watchapp.** Button cycling is the point, so the drill-down wins
over being the default screen. What that costs, stated plainly so it isn't a surprise
later: you launch it from the app menu rather than seeing it by raising your wrist, and
its JS — and therefore its data — only lives while it is open, so every launch starts
with one fetch and a moment of "loading". Design for that: draw the last-known state
immediately with its age on it, then repaint when the fetch lands, rather than showing
a spinner on a screen that already has something true to say.
One package is either a face or an app — no binary is both — but the rendering, the
AppMessage handler and the JS are all shared, so **a watchface variant later is a
second `main()` and a build target, not a second project.** Worth keeping that seam
clean while writing it, in case the glance turns out to be what you actually reach for.
### What the drill-down shows
Rooms in the order they are drawn, wrapping at both ends, with Back leaving. (On the
Round 2, confirm which of buttons and touch you actually want to drive this — it has
both, and a circular screen makes a swipe more natural than a button press for
"next". The click config provider and a touch handler are the same twenty lines either
way.)
```
Kitchen <- room name, Up/Down cycles
─────────────
● Amir <- the person's colour, then their full name
● Anna
<- "Nobody here" when empty, never a blank screen
3 of 4 home placed <- the honest footer, see below
```
Full names cost bytes the overview does not need — ten people at ~12 bytes is ~120,
still nothing against the ~2 KB budget, so send them with the plan rather than making a
second request per room. The colour dot stays even though there is room for the name:
it is what ties this screen back to the marker on the plan.
**Give the empty and the unknown cases real text.** "Nobody here" is a finding.
"3 of 4 home placed" is the truth that the overview can only gesture at — and the
`unplaced` list from `/floorplan/presence` deserves its own entry at the end of the
cycle ("Somewhere in the house: Bibi"), because a person the system cannot locate is
exactly who you were looking for when you picked up the watch.
Where a photo *does* belong: nowhere here either. Once you have room to print
"Amir" you have already solved the problem the picture was for.
## Update cadence
The JS only lives while the face is displayed, so "polling" means "while you are
looking at it":
- Fetch once on load.
- Refresh on a `tick_timer` every 2–5 minutes while visible.
- Refresh on tap/shake, for the "who's home *right now*" glance that is the actual use
case.
Do not refresh every second, and do not attempt a background service to keep it warm:
Bluetooth wakeups are the battery cost on both devices, and a presence display that is
four minutes stale is not wrong in any way that matters.
## What is already in place, and what is missing
Already there:
- `GET /floorplan/presence` — the drawn plan joined to who is in each room, with
`unplaced` for people who are home but not locatable and `unmapped_areas` for areas
HA reports that nothing on the plan claims. Both matter on a small screen: "3 home,
1 not locatable" is honest, and quietly dropping two people is not.
- `color` and `initial` on every person, in every presence payload.
- Bearer-token auth on the whole API — the token would have to live in the app's
Clay settings, which is a real consideration: it is stored on the phone in the Pebble
app's config, and it is a token that reads the household's presence history.
Missing, in the order it would need doing:
1. **A compact serialisation.** Doing the quantisation in PebbleKit JS keeps the server
general and is the right first move. If polygon vertex counts turn out to be large
(a hand-drawn plan can easily have 20-vertex rooms), a `?format=compact` on
`/floorplan/presence` that simplifies and quantises server-side is the fallback.
2. **The watchapp itself** — `gpath` rendering for the overview, a click config
provider, the room-detail window, an AppMessage handler, and a Clay settings page
for the API URL and token. This is a weekend, not a project.
3. **A level picker**, because the plan is per-level and a watch shows one at a time.
Another button, or fold the levels into the same Up/Down cycle after the last room.
## The thing that decides whether this is worth building
**Room-level presence.** `identity` resolves a person's room from whatever
`AREA_ATTRIBUTE` holds on their trusted BLE entity — which is as good as the BLE
proxy layout and HA's area assignment, and this project has never measured it. If in
practice everyone resolves to "home, room unknown", the app is a picture of a
floorplan with everybody sitting in the `unplaced` list at the bottom, and no amount
of watch-side work fixes that.
That is testable today, without buying anything: open the admin panel's floorplan tab
with **Live** ticked, walk between two rooms, and see whether the marker moves. If it
does, the app is a weekend. If it doesn't, the work is in the BLE proxies, and
the watch is a distraction from it.

116
docs/project-plan.md
View File

@ -113,10 +113,13 @@ from Phase 1, same as every other MQTT-connected host in this plan.)*
|---|---|---|
| Mini PC or SBC with a touchscreen, mounted near the fridge/pantry | €150–300 | Prefers a native Wayland `wl_touch` device, same as §1.14 — unlike that host, `hosts/kitchen-display/` does not (yet) ship the `type:pointer`/udev-override fallback described there; a misclassified touchscreen here is still an open problem |
| USB webcam, pointed at wherever items get held up for scanning | €15–40 | Either built into the panel or on a short cable/gooseneck so its angle can be set independently of the screen. Any UVC webcam works — this is the same "no depth camera, no accelerator needed" bar as the thin client's gesture-control camera (§1.11), just used for a photo instead of continuous tracking |
| Zigbee door contact sensor, one per cold appliance door | €10–15 | ~3 to start (fridge, freezer compartment, second freezer). Mounted on the **outside** face of the door — a coin cell at −18 °C is a false economy and every opening condenses room air onto whatever is inside. Worth buying on their own merits ("has the freezer been open four minutes") before any camera exists; see `docs/fridge-item-location.md` |
| Doorway camera, one per appliance (optional, later) | €30–40 | Outward-facing, above the door on the hinge side, ~1.8–2 m, ingested by Frigate like every other camera in this project. **Never inside the appliance.** Buy one, not two — and only after the door sensors have been lived with, per `docs/fridge-item-location.md`'s recommended order |
*(No new container-host hardware — `pantry-vision` is a container on the existing
Phase 1 host, calling the existing Phase 3/9 LLM host for vision inference and the
already-running `grocy` container for storage.)*
already-running `grocy` container for storage. It does gain one small bind mount,
`/data`, for the doorway-hint database — see Phase 17 item 4a.)*
### 1.16 Door/wardrobe panel hardware (Phase 18)
| Item | Est. Price (EUR) | Notes |
@ -215,7 +218,7 @@ real hardware" callouts for everything downstream of this.)*
| Grocery vision recognition | **Ollama** (a vision-capable model, e.g. `llava`/`qwen2.5vl` — TBD, not yet pulled or benchmarked) | Identifies a grocery item from one photo and estimates its shelf life; the proposal is always human-reviewed before anything is written (see `pantry-vision/README.md`) |
| Grocery inventory backend | **pantry-vision** (custom Python, stdlib `http.server`) | `POST /identify` (photo → proposal), `POST /confirm` (human-reviewed proposal → Grocy stock write), `GET /inventory`/`GET /recipes` (proxy Grocy, reshaped). Bearer-token gated, **and — unlike admin-canvas — LAN-published**, since the kitchen display's kiosk browser calls it directly rather than through Home Assistant |
| Grocery inventory storage | **Grocy** (already running, Phase 1/7) | The single source of truth for stock/best-before-dates/recipes; `pantry-vision` is a client of Grocy's own REST API, not a replacement for it |
| Kitchen-display static serving | **pantry-web** (nginx:alpine) | Serves `pantry-vision/frontend/`'s Scan/Inventory/Recipes single-page app read-only to the kitchen display — same role `digest-web`/`admin-web` play for their own hosts |
| Kitchen-display static serving | **pantry-web** (nginx:alpine) | Serves `pantry-vision/frontend/`'s single-page app (unload / consume / expired / edit, plus Inventory and Recipes) read-only to the kitchen display — same role `digest-web`/`admin-web` play for their own hosts |
| Identity registry backend | **identity** (custom Python, stdlib `http.server` + `sqlite3`) | Person <-> BLE-identifier registry: registration (voice or touchscreen), presence resolution, visit history, per-device rights, a weather proxy. **Published**, unlike admin-canvas — `hosts/kitchen-display/`'s and `hosts/door-panel/`'s kiosk browsers call it directly, bearer-token gated |
| Identity static serving | **identity-web** (nginx:alpine) | Serves `identity/frontend/`'s `register.html`/`dashboard.html`/`admin.html` read-only — same role as `pantry-web`/`digest-web`/`admin-web`. The two kiosk pages are what the wall panels load; `admin.html` is deliberately not linked from either (see Phase 6b) |
| Door-panel OS build | **live-build** (custom config, `hosts/door-panel/live-build/`) | Reuses the thin client's build tool/convention, structurally `hosts/kitchen-display/`'s twin — see Phase 18 |
@ -627,7 +630,34 @@ service) plus `hosts/kitchen-display/` (a third, simpler kiosk image).
whole phase is designed around, the same "propose, never auto-commit" rule this
project already applies to identity-merge confirmation (the *Identity store* row
in §2). A wrong camera guess costs one tap to fix on the kiosk screen, not a wrong
fact silently written into the household's inventory.
fact silently written into the household's inventory. The book-out direction obeys
it too, and it costs a tap there as well: recognition never consumes anything by
itself, an ambiguous brand is asked about instead of picked, and a throw-away
states its amount first.
2a. **Stock only moves in four ways, so the display has four screens** — *Unload
groceries* (the camera runs in a loop, one confirm per item, straight on to the
next), *Consume article* (hold up what you are eating; multi-unit rows ask how
many, multi-brand kinds ask which), *List expired foods* (cleared by scanning what
you are binning, booked out as **spoiled** so Grocy keeps waste apart from eating),
and *Edit inventory* (`−`/`+` and a freeform amount — the one screen that
deliberately does not use the camera, because it is where you go when the camera
got something wrong). Corrections use Grocy's inventory-correction endpoint, not
consume/add, so the stock journal doesn't fill up with corrections dressed as
meals.
2b. **Two invariants the whole feature rests on.** (i) *Stock is counted in individual
units, never packages* — a twelve-pack of eggs is twelve, because that is the
question people ask and it makes consuming three of them arithmetic instead of
fractions of a pack; the model's `units_per_package` is a multiplier applied once
on the confirm screen, where it is editable and where the result is stated before
anything is written. (ii) *The fold key is the brand-free product kind* — twelve
eggs of brand X plus ten of brand Y are twenty-two eggs, recorded as a Grocy
**product group** so the grouping lives in Grocy's own data model rather than in a
second classification scheme here, with the per-brand rows kept underneath because
"take ten off brand Y" has to stay possible on the edit screen. The folding is
deliberately dumb (casefold, strip the brand suffix); the clever version is a
synonym problem nobody wants adjudicated by a kitchen display at 19:00, and its
failure mode — two lines that should be one — is at least visible, which a wrongly
merged line is not.
3. **A real, LAN-published network listener — the one deliberate exception to this
project's usual "the LLM only ever reaches a device through HA→MQTT" shape.**
`admin-canvas` (Phase 13) has no published port because only Home Assistant calls
@ -636,18 +666,51 @@ service) plus `hosts/kitchen-display/` (a third, simpler kiosk image).
with no HA round-trip in the loop for that specific call. The bearer token — not
network placement — is the actual boundary here; see
`pantry-vision/README.md`'s "A real network listener, unlike admin-canvas"
section. Controlling *which screen is showing* (Scan/Inventory/Recipes) is a
separate, narrower path that does stay HA/MQTT-mediated: `kitchen-display-agent`,
identical in shape to every other host's agent in this project.
section. Controlling *which screen is showing* is a separate, narrower path that
does stay HA/MQTT-mediated: `kitchen-display-agent`, identical in shape to every
other host's agent in this project. Its "Show scan" button still publishes the
fragment `scan`, which `frontend/app.js` treats as an alias for the unload screen —
an old name living on costs nothing next to an HA button that silently stops
working.
4. **Grocy is the system of record, not a new inventory store** — `pantry-vision` is
a client of Grocy's own REST API (already running unconditionally since Phase 1/7)
for both reads (`/inventory`, `/recipes`) and the one write (`/confirm` → Grocy
stock). Nothing here duplicates or replaces Grocy's own data model.
for the reads and for every stock write (`/confirm`, `/consume`, `/adjust`,
`/transfer`). Nothing here duplicates or replaces Grocy's own data model. The one
thing `pantry-vision` keeps in a file of its own is the doorway hints below, and
that is precisely because they are *not* inventory.
4a. **Multiple cold appliances, answered twice over — and only one of the answers is
authoritative.** Grocy **locations** are the record: the confirm screen's placement
resolves to one, `/inventory` reports it, and `POST /transfer` moves an amount
between appliances. That transfer action is the whole point — without it a location
decays into "where it was when it was bought", which is worse than no answer.
Alongside it, **doorway hints** (`pantry-vision/doorway.py`): a Zigbee contact
sensor on each appliance door drives an HA automation that POSTs `/doorway-event`,
which answers 202 immediately and pulls a short camera burst off the request
thread. Whatever the vision model recognises is written to its own SQLite file with
a timestamp and a confidence, **never to stock** — a camera at a door cannot tell
in from out, misses two-items-at-once, and sees nothing behind an arm, so it
produces "last seen at Freezer (loggia), 20 min ago" for a person to evaluate.
`docs/fridge-item-location.md` is the full argument, including why nothing goes
*inside* an appliance (no interior power, condensation on every opening, −18 °C
outside every consumer camera module's rating) and why the door sensors are worth
buying even if no camera ever is. **An appliance configured with a sensor and no
camera is a supported, recommended starting point.**
5. **The vision-identification prompt asks for one photo, not a live video stream**,
and degrades to a low-confidence placeholder proposal (never an error page) if the
model call fails or its response isn't parseable JSON — same "degrade, don't
blank" rule as the digest/admin canvas renderers and `llm_client.py`'s own
`_fallback_document`.
`_fallback_document`. The unload screen's "camera searching for products" is still
built out of single photos: `frontend/app.js` samples a 32×24 greyscale thumbnail
every 700 ms and spends an `/identify` call only when the frame has **settled** and
**changed** since the last identification. Both gates are latency economics — see
open decision #18 — and a degraded answer stops the loop rather than
re-photographing the counter at a model that is down.
5a. **The model is asked to read a printed best-before date off the packaging**, and
that is the one place the "propose, never auto-commit" rule is deliberately looser
than a pure estimate would be. The screen always labels which of the two it is
showing ("read off the packaging — check it" vs "estimated from the category"), and
the server discards any date more than a year past or ten years out, because a
misread label is this feature's most likely failure and that is its shape.
6. **The kitchen display has no native camera-capture app at all** — Chromium's own
`getUserMedia()`, called from `pantry-vision/frontend/app.js`, talks to the webcam
directly inside the kiosk page and hands a captured frame straight to
@ -655,11 +718,12 @@ service) plus `hosts/kitchen-display/` (a third, simpler kiosk image).
auto-accepts the permission prompt that would otherwise sit unanswered on an
unattended screen.
7. **`hosts/kitchen-display/` is one workspace, one app — no touch dock, no
multi-app switching** the way `hosts/touch-panel` has: Scan/Inventory/Recipes is
in-page tab navigation inside `pantry-vision/frontend/`'s single-page app, since
there's only one thing this device does. `kitchen-display-agent`'s MQTT surface
is correspondingly narrow: three "Show X" buttons and nothing else — no
media_player, no capture/audio/remote-desktop entities.
multi-app switching** the way `hosts/touch-panel` has: Home/Inventory/Recipes is
in-page tab navigation inside `pantry-vision/frontend/`'s single-page app, with the
four stock-movement screens opening from Home, since there's only one thing this
device does. `kitchen-display-agent`'s MQTT surface is correspondingly narrow:
three "Show X" buttons and nothing else — no media_player, no
capture/audio/remote-desktop entities.
8. **The Grocy API integration is written from documentation, not verified against a
live instance** — the exact `GET /api/stock` response shape (whether product
names arrive nested by default), the minimum required fields for
@ -758,13 +822,28 @@ entirely container-host services plus additional Frigate camera sources.
"Camera face recognition" section) specifically so that Tapo cameras plug into
the exact same anti-spoofing-respecting presence pipeline as everything else,
rather than this phase inventing its own.
3. **Trash-bin fullness / dishes / litter checks are `chores/`'s job**: every ~2
hours (systemd timer, `RandomizedDelaySec=1800` — the "+/-30 min in case
something else is running" jitter, systemd's own built-in feature, not custom
3. **Trash-bin fullness / dishes / litter / groceries-left-out checks are `chores/`'s
job**: every ~2 hours (systemd timer, `RandomizedDelaySec=1800` — the "+/-30 min in
case something else is running" jitter, systemd's own built-in feature, not custom
code), `chores/check.py` grabs a Frigate snapshot per configured watch point
(optionally moving a PTZ camera to a preset first) and asks an Ollama vision
model a one-word question. A "needs attention" result opens a chore; a "clear"
result auto-closes one.
3a. **`groceries_out_of_place` is `litter`'s twin, with a shorter fuse.** Same
culprit-attribution (whoever the camera last recognised nearby), same
exemption-proofing, same unassignability — because it is the same situation,
somebody carried something in and put it down. The one deliberate difference is
`GROCERIES_NEGLECT_THRESHOLD_HOURS` (30 minutes against the usual four), and it is
the only place in `chores/` where one chore type is treated as more urgent than
another: litter left for an afternoon is untidiness, a tub of ice cream left for an
afternoon is a bin bag. Its prompt names what to **ignore** (fruit bowls, bread
bins, coffee, spices, cookware) as well as what to look for, because the general
question — "is anything out of place?" — is one a vision model answers YES to for a
kitchen that is simply a kitchen, every two hours, until the household mutes the
topic. It knows food is out; it never knows *what* the food is and never writes to
Grocy. That is `pantry-vision`'s job, at a display someone is standing at — two
systems looking at the same counter for two different reasons, which is the
intended shape rather than a duplication to collapse.
4. **Household chore distribution — presence/calendar-driven nudging, not LLM
assignment.** The governing principle, stated in `chores/check.py`'s own module
docstring: *"I don't care who does it, as long as it gets done."* This is a
@ -901,3 +980,6 @@ These need a decision before their respective implementation steps can be built
37. ~~iOS cannot receive `identity`'s arrival notifications without WAN egress through a third party~~ — **household decision made** (Phase 6b): **no Apple devices are used here**, which removes the only forcing function for exposing ntfy at all. ntfy therefore **stays LAN-only** — no DMZ, no port forward, no NAT-reflection hairpin; `identity → ntfy` is a container-to-container call that never reaches OPNsense, and remote delivery rides a **WireGuard split tunnel** routing only the smart-home VLAN. The full comparison (including why the hairpin variant, which avoids an inter-VLAN rule, was rejected for making an internal path depend on the ISP) is recorded in `docs/network-integration.md` §2.2 so it doesn't get re-litigated. **Still genuinely open**: nothing has been delivered to a real phone yet, and the split tunnel's `AllowedIPs` is the fiddly part — routing all of `192.168.0.0/16` would collide with typical café/hotel LANs and break the phone's connectivity there. Revisit §2.2 only if an Apple device ever joins the household.
38. **Both of `chores`' shipped container URLs pointed at `127.0.0.1` and could never have worked** (found while wiring Phase 6b's arrival notifications) — inside the `chores` container that address is the chores container itself, not the host. `NTFY_URL=http://127.0.0.1:8090` meant every nudge failed to send, and `IDENTITY_URL=http://127.0.0.1:8097` meant `_presence()` never reached identity at all, so **`chores` would have done nothing whatsoever** — and silently, since both failure paths log and continue by design (a deliberate never-block-on-a-dependency choice that here hid a total outage). Corrected in the template to `http://ntfy` and `http://identity:8097` (compose-bridge DNS; ntfy on its internal port 80, not the 8090 published to the LAN). **An already-deployed `/opt/smart-home/chores/chores.env` still has the old values** — templates are copied once at setup and never re-synced, so existing installs need this edited by hand. The broader lesson worth acting on: several services fail soft on an unreachable dependency, which means a misconfigured address produces silence rather than an error — nothing in this repo currently distinguishes "nothing to do" from "never reached the thing that would have told me".
39. **Nothing enforces that voice/TTS consumers actually read `speak_name` rather than `nickname`** (new, Phase 6b) — the field exists, is documented, and `chores/` uses it, but a future HA intent script that reaches for the friendlier-looking `nickname` would break the "the assistant uses real names" rule silently. Worth checking whenever a new consumer of `/presence` or `/resolve` is written.
40. **Whether a vision model can identify a grocery item in a moving hand at doorway distance** (new, Phase 17) — the assumption the entire `/doorway-event` path rests on, and a materially harder task than the kitchen display's held-still-against-a-plain-background one. It is also the cheapest open decision on this list to close: point the kitchen's existing webcam at somebody walking past with a carton and run `/identify` on the frames. Until then the door sensors are the part worth deploying, and `PANTRY_DOOR_APPLIANCES` entries can be configured without a camera — see `docs/fridge-item-location.md`, whose recommended order (sensors, then a month of living with them, *then* one camera) the code deliberately does not shortcut.
41. **Whether `groceries_out_of_place` can tell a carton of milk from a fruit bowl** (new, Phase 20) — its prompt lists what to ignore precisely because the general question gets a YES for any normal kitchen, but that is a mitigation written blind. The failure mode is not a missed chore, it is a false one every two hours, which is how a household learns to ignore the notification channel entirely. Point it at a *clean* counter for a day before trusting a YES, and note this is the first chore type whose false positives cost more than its false negatives.
42. **Contact-sensor battery life on an appliance door is unverified** (new, Phase 17) — a fridge door opens far more often than the window these sensors are sold for, and the sensor sits in a cold, humid draught even when mounted outside the compartment (which `docs/components.md` says to do, for the separate reason that a coin cell at −18 °C is a false economy). Check one after a month before buying more.

78
docs/rooms-and-endpoints.md Normal file
View File

@ -0,0 +1,78 @@
# Rooms, and what is in them
One vocabulary for rooms across the whole project, and one answer to "what hardware is
in this room". Written because three separate features now need it — the floorplan
presence view, the Pebble app, and anything room-scoped like the workshop assistant —
and each of them was about to invent its own.
## The room id is an HA `area_id`, everywhere
`living_room`, `kitchen`, `workshop`. Lowercase, digits, underscores; Home Assistant
slugifies area names into exactly that shape.
That one string is the join key for every part of this project that cares where
something is:
| Where it appears | As what |
|---|---|
| Home Assistant | the device's **area** |
| `CoreSystemConfig.json` | `room` on each kiosk and audio endpoint |
| Each host's agent config | `<PREFIX>_ROOM`, baked in at build time |
| MQTT discovery | `suggested_area` on the device object |
| `identity` | `floorplan_rooms.ha_area_id` — the drawn polygon's tie to reality |
| `identity`'s `/presence` | the `room` field, read off `AREA_ATTRIBUTE` of a trusted entity |
Nothing translates between these. A room named `Living Room` in one file and
`living_room` in another is two rooms as far as every join above is concerned, which is
why `tools/validate-config.py` rejects anything that is not already an area_id rather
than helpfully slugifying it — helpfully slugifying it is how you end up with two.
## How a device learns which room it is in
```
CoreSystemConfig.json tools/config-export.py the ISO builder
kiosks[].room ────────────> CORE_KIOSK_ROOM ───────────> <PREFIX>_ROOM in the
agent's config file
│
HA files the device in that area <───────────┘
(suggested_area, on first discovery)
```
Declared once, in the same file that already knows every other fact about that device.
Nobody drags devices into areas in the HA UI, and nobody types a room name twice.
**The one real limitation: `suggested_area` is a suggestion, and it is only honoured
when HA first discovers the device.** Move a panel to another room, rebuild its image,
and HA keeps it in the old area — the suggestion is not reapplied. Moving a device
means moving it in HA too, once. This is a property of HA's discovery, not something
this project can paper over, and it is why the field is named *suggested*.
An empty `room` is allowed and is a **warning, not an error**: a household that hasn't
settled its room names must still be able to build an image. What it loses is automatic
area assignment, which is a nuisance to fix by hand — not a broken device.
## What is not covered by this
- **Zigbee devices, cameras, and anything not built by `tools/`.** Their room lives in
HA only, set when they are paired or added. That is fine — HA is the registry; this
file is about the devices this repo builds images for, which otherwise had no way to
say where they were.
- **Where a *person* is.** That is `identity`'s `/presence`, and it is a different
problem with a different reliability story — see
`docs/pebble-presence-watchface.md`'s closing section.
- **Where an *item* is.** `pantry-vision` locations for food
(`docs/fridge-item-location.md`); nothing tracks tools or parts yet
(`docs/workshop-assistant.md`).
## Adding a room
1. Create the area in Home Assistant (or let a device's `suggested_area` create it).
2. Use its area_id as `room` on every kiosk/audio endpoint that lives there.
3. Draw it on the floorplan in `identity`'s admin panel and set its `ha_area_id` to
the same string. The room editor offers a pick-list from
`GET /floorplan/areas` — every area HA is currently reporting — so this step is
choosing from a list rather than retyping an id.
Step 3 is optional and independent: presence works without a drawn plan, and a drawn
plan is useful before presence is wired up. They only need each other for the views
that show people *on* the plan.

341
docs/workshop-assistant.md Normal file
View File

@ -0,0 +1,341 @@
# A workshop/office assistant: what's worth building
Feasibility note. The ask: identify laptops and computers by camera, identify
components (mainboards especially), pull up disassembly guides, research part specs
automatically, and hold a conversation about planning a project — all scoped per room.
**The one finding that reorganises the whole thing: stop trying to recognise the
object, and read the label on it.** Everything below follows from that.
## Why "identify it by shape" is the wrong problem
Ask a local 7–13B vision model what laptop it is looking at and you will get a
confident, wrong answer, because:
- **Laptops are visually identical by design.** A closed ThinkPad T480 and a T490 are
the same black rectangle. So are half of Dell's Latitude line. There is no visual
feature to learn, and a model that claims otherwise is pattern-matching on the
wallpaper.
- **Mainboards are worse, not better.** Board layout varies more than a laptop shell
does, but the discriminating detail — where the fourth M.2 slot is, which VRM
configuration — is exactly what a photo at workbench distance and a general-purpose
VLM cannot resolve.
- **Getting it wrong is expensive here**, unlike in the pantry. A wrong yoghurt costs a
tap. A disassembly guide for the wrong laptop revision costs a broken clip or a
ribbon cable, and a wrong pinout costs the board.
Meanwhile, every one of these objects **carries its own identity in printed text**:
| Object | Where its identity actually is |
|---|---|
| Laptop | Service tag / serial sticker on the base — and on Dell/Lenovo/HP that tag resolves to the *exact factory configuration* |
| Mainboard | Model silkscreened on the PCB (`PRIME B450M-A`), usually near the RAM slots or the PCIe bracket |
| GPU / PSU / drives | Model and part number on the label |
| ICs | Top-marking, when it hasn't been sanded |
So the pipeline is **OCR-first**: photo → detect text regions → read them → match
against a parts database. The VLM's job is the small, tractable one it is good at —
"what kind of thing is this, and where on it is the label" — and the identification
comes from characters, not from vibes. This also degrades honestly: an unreadable
label produces "I can't read it, hold it closer" rather than a plausible wrong model.
**What stays hard, permanently:** an unmarked part. A capacitor, a sanded IC, an
anonymous barrel connector. Nothing here will identify those, and the assistant should
say so rather than guess.
## The four capabilities, ranked by whether they're worth it
### 1. Conversational project planning — build this first
The cheapest and the most useful, and the only one with no perception problem in it.
It is a text LLM with a notebook: what am I building, what have I got, what's the next
step, what did I decide last Tuesday and why. The value is entirely in the *notebook*
being persistent and per-project, not in the model being clever.
This is also the piece that makes the others worth having: an identified mainboard is
only useful if there is somewhere to put it ("this is for the NAS build").
Shape: a small service in the pattern this repo already uses four times over
(`identity`, `chores`, `pantry-vision`) — SQLite, an HTTP API, bearer token, a static
frontend on a display. Projects, notes, parts, photos, decisions.
### 2. Part-spec research — build it, with one hard rule
**Every spec must be quoted from a fetched document, with its URL, and never generated
from the model's memory.** This is the same rule `digest-engine`'s political prompt
enforces on claims about the world and `pantry-vision` enforces on stock writes, and
here it is a safety property rather than a quality one: a hallucinated TDP wastes an
afternoon, a hallucinated pinout destroys hardware.
Practically: fetch the vendor page or datasheet PDF, cache it locally, extract, and
show the extract next to its source. If nothing was fetched, the answer is "I couldn't
find a datasheet", not a paragraph of plausible numbers.
**This is the first component in this project that deliberately reaches the open
internet for content**, which is worth stating plainly given the local-first doctrine
everything else follows. It is outbound-only, it does not require anything to be
exposed, and it should run against an explicit allowlist of sources (vendor domains,
iFixit, datasheet archives) with everything it fetches cached on disk — so the same
board is looked up once, not once per question.
### 3. Disassembly guides — mostly a link, and that's fine
iFixit is the corpus, and it has a public API. The honest scope is: resolve the
identified model to a guide, show the steps and images on the workshop display, cache
what you fetch. Do not rewrite the guide through an LLM — the steps are the thing, and
an LLM paraphrase of "disconnect the battery before removing the board" is strictly
worse than the sentence itself.
Vendor service manuals (Lenovo's HMM PDFs, Dell's service manuals) cover what iFixit
doesn't, and are the reason the cache is worth having: they are large PDFs you want
locally, once.
### 4. Camera identification — build it last, and OCR-first
By the time the three above exist, this is a convenience: it saves typing a model
number. Useful convenience — hands are usually dirty or full — but it is the piece
with the perception risk, and it should not be the first thing attempted.
The interaction that actually works: hold the label to the camera, get the text back
with a confidence, **confirm before anything acts on it** — the identical
propose-never-auto-commit rule `pantry-vision` runs on. Scanning a barcode or QR code
where one exists (most service tags have one) is strictly better than OCR and should
be tried first.
## Per-room scoping, which the plumbing now supports
"Make this a per-room system" splits into two questions, and the boring one is now
answered:
**Which devices are in which room** — done. Every kiosk and audio endpoint declares a
`room` (an HA area_id) in `CoreSystemConfig.json`; it is baked into the agent and
published as `suggested_area`, so HA files each device in the right area by itself. See
`docs/rooms-and-endpoints.md`. The workshop is a room like any other: give it a display
and it declares `room: workshop`.
**Which assistant is active in which room** — the design question, and the answer that
fits this project is: *the room selects the toolset, not the personality.* A voice
request in the workshop can reach the parts database and the project notebook; the same
words in the kitchen reach `pantry-vision` and the recipes. That is a routing table
from area_id to capability set, sitting in front of HA Assist, and it is small.
Two reasons to do it that way rather than one assistant with everything:
- **Ambiguity collapses.** "What have I got?" means stock in the kitchen and parts in
the workshop, and no amount of prompt engineering makes one agent reliably guess
which room the words came from — but `identity`'s `/speaker` already answers "who
spoke in this area", and the area is right there in the request.
- **The dangerous tools stay where they belong.** Nothing in the kitchen should be able
to open a disassembly guide, and nothing in the workshop needs to book out groceries.
## The workspace, and why it is an SMB share
The assistant needs somewhere to put things: fetched datasheets, extracted specs,
generated diagrams, photos of the board, project notes, a scratch directory per
project. Making that a **plain directory on the container host, exported over SMB**, is
right for a reason worth stating: it means every artefact is a file you can open from a
laptop with no API, no export step, and no dependence on this project still existing in
two years. The assistant's output should outlive the assistant.
```
/opt/smart-home/workshop/
projects/<slug>/
notes/ markdown the assistant and you both write
datasheets/ everything fetched, cached, named by part
diagrams/ generated SVGs (see the next section)
photos/ what the camera captured
scratch/ the assistant's working directory
```
**This project already runs Samba** — `gallery-smb`, from `setup-container-host.sh`.
Two things follow, and the second is a real trap:
- **Reuse the container, add a share.** Samba serves many shares on one port; a second
Samba container would fight the first for **445, which is already claimed**
(`ports.gallery_smb`). That is the same class of collision already recorded as open
decision #31 for Music Assistant, and it is avoidable here by construction: one
`smb` container, a `gallery` share and a `workshop` share, separate accounts.
- **The gallery share is `read only = yes`. This one cannot be**, and that is the
entire security difference between them. A writable share is fine; a writable share
that an LLM writes into deserves three limits, none of which are exotic:
- **Its own volume**, not a subdirectory of anything else. The blast radius of a bad
path is then "the workshop workspace", which is recoverable.
- **Its own account**, not the gallery's. Guest access off, same as the gallery.
- **No execution.** Nothing in the workspace is ever run by anything on the host —
it is a place for documents, and the moment it becomes a place for scripts it is a
different security question than this note answers.
Keep the assistant's *state* — projects, parts, decisions — in SQLite as with every
other service here, and the *artefacts* on the share. The database is the index; the
share is the filing cabinet. Mixing them (blobs in SQLite, or state in files) gets you
the worst of both: a database you cannot browse and files nothing can query.
## Technical display widgets, and the one decision that makes them cheap
The thin-client canvas SDK today has `stat`, `chart`, `image`, `video`, and plain
markdown-ish text. The wanted additions — schematics, board plans, code-flow and
data-structure diagrams, breadboard layouts, 3D — look like six new widgets. They are
mostly one:
> **Render to SVG on the container host; show it in the canvas.** One new `svg` window
> kind, plus renderers server-side.
That keeps the SDK's dependency-free, no-build-step property (the thing that makes it
maintainable), puts heavy tooling on the machine that already has heavy tooling, and
means a diagram is a *file in the workspace share* as well as something on a screen.
Client-side rendering libraries would put a megabyte of JavaScript on a kiosk to
produce a picture the server could have produced once and cached.
| Widget | How | Honest difficulty |
|---|---|---|
| **Circuit diagrams** | `netlistsvg` (or KiCad's own SVG export) from a netlist the assistant produces | Medium — see the notation note below, which is the whole job |
| **Board plans** | KiCad `.kicad_pcb`/Gerber → SVG, server-side, pre-rendered | Medium, and only for boards you have files for. There is no path from a *photo* of a board to a layout drawing |
| **Code-flow diagrams** | Graphviz `dot` → SVG | **Easy.** Graphviz is small, deterministic, and an LLM writes `dot` reliably |
| **Data-structure diagrams** | Graphviz (records/HTML-like labels) → SVG | Easy, same pipeline |
| **Assembly guides** | The guide's own step images in sequence — see the iFixit section | Easy in 2D |
| **Breadboard diagrams** | Fritzing-style, and Fritzing's value *is* its part library, which is not callable | **Hard, and least worth it** — a photo of the actual breadboard is usually better and always more honest |
| **3D models** | glTF + a vendored three.js on the kiosk | The only one that genuinely needs client-side code — see below |
### EU/IEC notation is a symbol-library decision, not a rendering one
Rectangular resistors, not zigzags. This matters more than it sounds: **most schematic
tooling defaults to ANSI/US symbols**, and a renderer that cannot be given a symbol set
will quietly produce American schematics forever.
So the selection criterion for the schematic renderer is *"can I supply my own symbol
library?"* before anything about output quality. `netlistsvg` takes custom skins, which
is the concrete lever — an IEC skin, written once, vendored in the repo like every
other asset here. KiCad can be configured the same way. Anything that hardcodes its
symbols is disqualified regardless of how good its output looks, because the first
zigzag resistor is the last time anyone trusts the diagram.
The same applies downstream: state the convention in the prompt that generates the
netlist, and check it on the way out. A diagram in the wrong notation is not a style
complaint — it is a diagram that reads wrong to the person holding the soldering iron.
### The holographic look is a variable override, not a new stylesheet
The canvas SDK is already a holo aesthetic — glow text, glowing panel edges, a pulse
animation, a dark blue base — and, importantly, it is already parameterised. Every
colour in `glow.css` comes from custom properties on `:root`
(`--admin-accent`, `--admin-glow-color`, `--admin-edge`, `--admin-panel`, `--admin-bg`).
So "make the workshop look holographic in purple/magenta" is **one small override file
loaded after the SDK**, not a fork of it:
```css
/* workshop-theme.css — loaded after glow.css. Overrides only. */
:root {
--admin-bg: #0a0510; /* near-black, violet-biased */
--admin-panel: rgba(28, 12, 44, 0.82);
--admin-edge: rgba(200, 120, 255, 0.32);
--admin-accent: #c084fc; /* the purple everything glows in */
--admin-text: #eadcff;
--admin-muted: #a98fc4;
--admin-glow-color: #ff3ec8; /* magenta glow against a purple accent */
}
```
Two magenta-on-purple choices worth making deliberately: the **accent** (borders,
labels, chrome) reads better as the lighter purple, and the **glow** as the magenta —
glow bleeds and saturates, so the more aggressive colour belongs to the effect rather
than to the thing being read. Keep `--admin-good`/`--admin-warn` as they are; green
and amber mean something, and recolouring them into the theme would cost that meaning.
**Do not tint the technical drawings.** This is the one rule that keeps the look from
becoming a liability: a schematic, a board plan or a pinout table needs contrast and
neutral colour far more than it needs to match the frame — trace colours mean things,
and a magenta glow over a resistor value is how you misread it. So:
- Holo treatment on the **chrome**: window frames, titles, edges, the pulse, the
background.
- The **drawing surface stays neutral** — a light panel for schematics (they are drawn
for paper and read best that way), or high-contrast line art on near-black. The
frame around it can glow all it likes.
- Anything conveying state by colour — a red LED in a diagram, an error, a
live measurement — keeps its own colour. The theme owns the furniture, not the data.
The same override trick works if a future room wants its own palette; the SDK never
needed to know about themes for this to be possible, which is why it costs one file.
### 3D: worth doing, but not first, and not with a live viewer at first
3D is the one item that cannot be an SVG. Two paths:
- **Pre-rendered orbit frames** — render N views server-side, ship them as an image
sequence, scrub with a drag. Cheap, works on every surface including the ones with
no GPU, and covers "let me see what this connector looks like from behind", which is
most of the actual need.
- **A real viewer** — vendored three.js plus glTF. It is roughly a megabyte of
JavaScript on a kiosk that currently ships none, which is a real break with the
SDK's "vendored, dependency-free" rule and should be a deliberate decision rather
than a drift into one.
Start with the frames. Move to a viewer if and only if manipulating the model turns
out to be what people do, rather than glancing at it.
**3D assembly guides** — exploded views with per-step visibility — are the most
speculative item in this entire note. They need a model *with assembly structure*,
which almost nothing you download has, and authoring one per project is more work than
the repair. Park it.
## Hardware
The workshop needs what the kitchen already has: a screen you can touch with one
knuckle, a camera, a microphone. `hosts/kitchen-display/` is exactly this shape
already — a Sway kiosk, a webcam, an agent, one workspace — so a workshop display is
that image with a different frontend, not a new class of device.
The camera wants to be better than the kitchen's, though, and differently: reading a
service tag or a PCB silkscreen is a **macro** problem. Autofocus at 10–20 cm and
enough resolution to resolve 2 mm text matter far more than field of view. A cheap
USB microscope/inspection camera is a better fit than a webcam, and is worth pricing
before assuming the C920 answer carries over.
## The compute problem, again
One RTX 3060, already carrying Assist, four digests a day, `pantry-vision` and
`chores`. Adding OCR is fine (it is small, and CPU OCR is viable). Adding a second
large model that has to be resident is not — and "the workshop assistant felt slow
because someone asked for a digest" is exactly the contention already flagged as open
decision #4. Design for one model swapped predictably, not two resident.
## Recommended order
1. **The project notebook + conversational planning, and the workspace share.** No
camera, no internet, no new hardware. Useful on day one, and it is where everything
else lands — including every artefact the later steps produce.
2. **Spec research with mandatory citation**, against an allowlist, cached on disk.
3. **The `svg` window kind + a Graphviz renderer.** One kind and one small tool buys
code-flow and data-structure diagrams immediately, and is the pipeline the schematic
and board-plan renderers then plug into.
4. **Guide lookup**, by typed model number.
5. **Schematics with an IEC symbol skin** — the notation work is the work.
6. **Camera identification**, barcode first and OCR second, proposing rather than
acting.
7. **3D**, as pre-rendered orbit frames. A live viewer only if manipulation turns out
to be what people actually do.
The purple/magenta holo theme is not in that order because it is not a step: it is one
override file and can land whenever.
Steps 1–3 need no new hardware at all — a browser on any existing thin client reaches
them. That matters: it means the whole idea can be proved before buying a workshop
display, and if the conversation-and-notebook half turns out to be the only part
anyone uses, that is a complete and useful outcome rather than a failure.
## What is unverified here
Everything past step 1. Specifically: whether iFixit's API terms permit caching what
this would cache; whether a local VLM can read a PCB silkscreen at all under workshop
lighting (testable today with any webcam and a spare board — do this before buying an
inspection camera); whether service-tag lookups work without a vendor account; and
whether the OCR-first pipeline holds up on the labels that are scratched, curved, or
under a warranty sticker, which in a workshop is most of them.
On the display side specifically: whether `netlistsvg`'s skin mechanism really can
express a full IEC symbol set (the notation requirement rests entirely on that, and it
is worth a two-hour spike before committing to the tool); whether an LLM produces
netlists that are *correct* rather than merely well-formed, which is a different and
much harder property; and how big a vendored three.js actually is against the kiosk's
budget before deciding 3D is affordable.

5
hosts/door-panel/agent/door_panel_agent/main.py
View File

@ -71,6 +71,9 @@ def main() -> int:
hostname = socket.gethostname()
node_id = "".join(c if c.isalnum() else "_" for c in hostname).strip("_") or "doorpanel"
friendly_name = config.get("DOOR_PANEL_NAME") or f"Door panel ({hostname})"
# The HA area this device sits in, published as suggested_area — see
# docs/rooms-and-endpoints.md. Blank is fine and means "no suggestion".
room = config.get("DOOR_PANEL_ROOM", "")
broker_host = config.get("MQTT_BROKER_HOST", "")
broker_port = int(config.get("MQTT_BROKER_PORT") or 1883)
@ -79,7 +82,7 @@ def main() -> int:
if config.get("MQTT_USERNAME"):
client.username_pw_set(config["MQTT_USERNAME"], config.get("MQTT_PASSWORD") or None)
discovery = Discovery(client, node_id, friendly_name)
discovery = Discovery(client, node_id, friendly_name, room)
def on_show(screen: str) -> None:
if screen == "register":

10
hosts/door-panel/agent/door_panel_agent/mqtt_discovery.py
View File

@ -25,10 +25,11 @@ DISCOVERY_PREFIX = "homeassistant"
class Discovery:
def __init__(self, client, node_id: str, friendly_name: str):
def __init__(self, client, node_id: str, friendly_name: str, room: str = ""):
self.client = client
self.node_id = node_id
self.friendly_name = friendly_name
self.room = (room or "").strip()
self.base = f"doorpanel/{node_id}"
self.availability_topic = f"{self.base}/availability"
self._handlers: dict[str, Callable[[str], None]] = {}
@ -40,6 +41,13 @@ class Discovery:
"model": "Sway door panel",
"sw_version": __version__,
}
# Which room this physically sits in, as an HA area_id. `suggested_area`
# is honoured by HA only when the device is FIRST discovered — moving a
# device later means moving it in HA too, this cannot un-file it. Omitted
# entirely when unset, because an empty suggested_area is not the same
# request as no suggestion. See docs/rooms-and-endpoints.md.
if self.room:
self.device["suggested_area"] = self.room
def _publish_config(self, component: str, object_id: str, payload: dict) -> None:
payload = {

11
hosts/kitchen-display/README.md
View File

@ -9,8 +9,9 @@ inventory (soonest-to-expire first) and Grocy's recipes, all touch-driven.
that's [`../../pantry-vision/`](../../pantry-vision/README.md), a container-host
service this device's kiosk browser calls directly. This host is deliberately thin:
a Sway session, one Chromium kiosk window, and `kitchen-display-agent`, which only
ever controls *which screen is showing* (Scan / Inventory / Recipes, plus
Registration — see below) over MQTT — the same "the agent controls the surface, a
ever controls *which screen is showing* (the unload-groceries scanner, Inventory or
Recipes, plus Registration — see below) over MQTT — the same "the agent controls the
surface, a
separate write API owns the content" split `hosts/thin-client`'s admin canvas already
established.
@ -88,7 +89,11 @@ would otherwise sit unanswered on a screen nobody is there to click "Allow" on.
`kitchen-display-agent` publishes four buttons on connect: **Show scan**, **Show
inventory**, **Show recipes**, **Show registration** — the first three kill and
relaunch the kiosk Chromium window at `PANTRY_WEB_URL/index.html?...#<tab>`; the
relaunch the kiosk Chromium window at `PANTRY_WEB_URL/index.html?...#<tab>`. ("Show
scan" still publishes the fragment `scan`, which the frontend treats as an alias for
its unload-groceries screen; the four stock-movement screens are reached by tapping,
not over MQTT, since they are all things a person is standing at the display doing.)
The
fourth relaunches it at `IDENTITY_WEB_URL/register.html?...` instead (a different
backend, `identity`, behind the same "Show X" shape — see
`kitchen_display_agent/main.py`'s `on_show()`). That's the entire MQTT surface; there

5
hosts/kitchen-display/agent/kitchen_display_agent/main.py
View File

@ -72,6 +72,9 @@ def main() -> int:
hostname = socket.gethostname()
node_id = "".join(c if c.isalnum() else "_" for c in hostname).strip("_") or "kitchendisplay"
friendly_name = config.get("KITCHEN_DISPLAY_NAME") or f"Kitchen display ({hostname})"
# The HA area this device sits in, published as suggested_area — see
# docs/rooms-and-endpoints.md. Blank is fine and means "no suggestion".
room = config.get("KITCHEN_DISPLAY_ROOM", "")
broker_host = config.get("MQTT_BROKER_HOST", "")
broker_port = int(config.get("MQTT_BROKER_PORT") or 1883)
@ -80,7 +83,7 @@ def main() -> int:
if config.get("MQTT_USERNAME"):
client.username_pw_set(config["MQTT_USERNAME"], config.get("MQTT_PASSWORD") or None)
discovery = Discovery(client, node_id, friendly_name)
discovery = Discovery(client, node_id, friendly_name, room)
def on_show(fragment: str) -> None:
# "register" is a different backend (identity's register.html) behind the

10
hosts/kitchen-display/agent/kitchen_display_agent/mqtt_discovery.py
View File

@ -28,10 +28,11 @@ DISCOVERY_PREFIX = "homeassistant"
class Discovery:
def __init__(self, client, node_id: str, friendly_name: str):
def __init__(self, client, node_id: str, friendly_name: str, room: str = ""):
self.client = client
self.node_id = node_id
self.friendly_name = friendly_name
self.room = (room or "").strip()
self.base = f"kitchendisplay/{node_id}"
self.availability_topic = f"{self.base}/availability"
self._handlers: dict[str, Callable[[str], None]] = {}
@ -43,6 +44,13 @@ class Discovery:
"model": "Sway kitchen display",
"sw_version": __version__,
}
# Which room this physically sits in, as an HA area_id. `suggested_area`
# is honoured by HA only when the device is FIRST discovered — moving a
# device later means moving it in HA too, this cannot un-file it. Omitted
# entirely when unset, because an empty suggested_area is not the same
# request as no suggestion. See docs/rooms-and-endpoints.md.
if self.room:
self.device["suggested_area"] = self.room
def _publish_config(self, component: str, object_id: str, payload: dict) -> None:
payload = {

46
hosts/thin-client/README.md
View File

@ -294,6 +294,44 @@ vendor/product IDs) and passes it to mpv as `--external-file=alsa://hw:X,0` if
found. Video-only playback (not a crash) if nothing matches. **Unverified against
real hardware** — see the checklist below.
## A wireless USB remote works out of the box
The kind with TV buttons on the front and a small keyboard on the back. To Linux that
dongle is two ordinary HID keyboards — a normal one and a "consumer control" one — so
there is nothing to configure per device: the front buttons arrive as `XF86*` keysyms
and the back keyboard arrives as keys. `configs/sway/config`'s remote-control section
binds the full standardised set:
| Buttons | What they do |
|---|---|
| ▶ ⏸ ⏹ ⏭ ⏮ | `playerctl -p mpv,spotifyd` — play/pause, pause, stop, next, previous |
| ⏪ ⏩ | seek 10s back / 30s forward. Skip, not scan: remotes get *pressed*, not held, and the podcast convention is already in people's fingers |
| Volume, mute, mic mute | the **sink**, not the player — the rocker should move the room's volume whatever is making the noise |
| Channel ± | next/previous workspace. A media station's "channels" are its workspaces, which is the closest honest analogy |
| Home / Back | the media workspace / `back_and_forth` |
| Power, Sleep | **the display, not the machine** — see below |
**Two deliberate choices worth knowing before you remap anything.**
*Plain arrows and Return are not bound, on purpose.* A remote's D-pad and OK send
exactly those, unmodified, and Chromium, mpv and every kiosk page need them — binding
them at the compositor would break scrolling a web page with the remote, which is most
of what the remote is for. Window focus stays on `$mod`+arrows.
*The power button turns the screen off, not the computer.* On a TV that button turns
the picture off; on a thin client that autologins into a kiosk, `poweroff` takes the
room's screen away until somebody walks over to press a physical button. So it runs
`display-toggle`, which is a **toggle** rather than two bindings because with the
outputs dark there is no other way back — Sway is still running and still receiving
keys, so the next press wakes it. A stuck button therefore cannot leave the screen
dark either.
**Remotes vary more than their marketing does.** Run `wev` (or
`sudo libinput debug-events`) from the maintenance shell (`$mod+Shift+Ctrl+m`), press
every button, and add any that comes back with an unbound keysym. A button that reports
*no keysym at all* is one the kernel has no mapping for — that is a udev hwdb entry,
not a Sway binding, and is worth knowing before blaming the config.
## Home Assistant entities
`thinclient-agent` publishes MQTT-discovery configs on connect. Under the MQTT
@ -452,6 +490,14 @@ default on most sets, and the reason a TV that sleeps fine refuses to wake.
## Manual verification still outstanding
0. **The remote.** No remote has been plugged into anything — the keysym list above is
the standardised set, not one read off a specific device. Expect one or two buttons
on any given remote to report something unbound (or nothing at all); `wev` from the
maintenance shell is the two-minute check, and `display-toggle`'s grep for
`'"dpms": true'` in `swaymsg -t get_outputs` is worth confirming against the Sway
version this image actually ships, since that field's spelling is the one thing
that would make the power button silently do nothing.
None of this has been run on hardware. In rough order:
1. The ISO builds at all (`lb build` is network-heavy and can fail on mirror hiccups).

5
hosts/thin-client/agent/thinclient_agent/main.py
View File

@ -152,6 +152,9 @@ def main() -> int:
hostname = socket.gethostname()
node_id = "".join(c if c.isalnum() else "_" for c in hostname).strip("_") or "thinclient"
friendly_name = config.get("THINCLIENT_NAME") or f"Thin client ({hostname})"
# The HA area this device sits in, published as suggested_area — see
# docs/rooms-and-endpoints.md. Blank is fine and means "no suggestion".
room = config.get("THINCLIENT_ROOM", "")
broker_host = config.get("MQTT_BROKER_HOST", "")
broker_port = int(config.get("MQTT_BROKER_PORT") or 1883)
@ -183,7 +186,7 @@ def main() -> int:
if config.get("MQTT_USERNAME"):
client.username_pw_set(config["MQTT_USERNAME"], config.get("MQTT_PASSWORD") or None)
discovery = Discovery(client, node_id, friendly_name)
discovery = Discovery(client, node_id, friendly_name, room)
mpris = MprisBridge(discovery.publish_media_state, sway.session_env)
def on_detail_level(payload: str) -> None:

10
hosts/thin-client/agent/thinclient_agent/mqtt_discovery.py
View File

@ -29,10 +29,11 @@ DISCOVERY_PREFIX = "homeassistant"
class Discovery:
def __init__(self, client, node_id: str, friendly_name: str):
def __init__(self, client, node_id: str, friendly_name: str, room: str = ""):
self.client = client
self.node_id = node_id
self.friendly_name = friendly_name
self.room = (room or "").strip()
self.base = f"thinclient/{node_id}"
self.availability_topic = f"{self.base}/availability"
self.media_state_topic = f"{self.base}/media/state"
@ -51,6 +52,13 @@ class Discovery:
"model": "Sway thin client",
"sw_version": __version__,
}
# Which room this physically sits in, as an HA area_id. `suggested_area`
# is honoured by HA only when the device is FIRST discovered — moving a
# device later means moving it in HA too, this cannot un-file it. Omitted
# entirely when unset, because an empty suggested_area is not the same
# request as no suggestion. See docs/rooms-and-endpoints.md.
if self.room:
self.device["suggested_area"] = self.room
# --- plumbing -----------------------------------------------------------
def _publish_config(self, component: str, object_id: str, payload: dict) -> None:

52
hosts/thin-client/configs/sway/config
View File

@ -133,12 +133,64 @@ bindsym $mod+Right focus right
bindsym $mod+Up focus up
bindsym $mod+Down focus down
# ---------------------------------------------------------------------------
# Remote control — the full standardised media-key set.
#
# THE DEVICE THIS IS FOR: a wireless USB remote with TV controls on the front and a
# small keyboard on the back. To Linux that is just two HID keyboards on one dongle
# (a normal keyboard plus a "consumer control" device), so there is nothing to
# configure per-device — the front buttons arrive as the XF86* keysyms below and the
# back keyboard arrives as ordinary keys. Any remote in that class works; the bindings
# are what make it useful.
#
# WHAT IS DELIBERATELY *NOT* BOUND: plain Up/Down/Left/Right and Return. A remote's
# D-pad and OK button send exactly those, unmodified — and Chromium, mpv and every
# kiosk page need them. Stealing them at the compositor would break scrolling a web
# page with the remote, which is most of what the remote is for. Window focus stays on
# $mod+arrows above; the bare arrows belong to whatever is on screen.
#
# CHECKING WHAT YOUR REMOTE ACTUALLY SENDS: remotes vary more than their marketing
# does. `wev` (or `sudo libinput debug-events`) prints the keysym for each button —
# run it once from the maintenance shell and add any button that comes back unbound.
# A button that reports no keysym at all is one the kernel has no mapping for, which
# is a udev hwdb entry, not a Sway binding.
# ---------------------------------------------------------------------------
# Transport. `playerctl -p mpv,spotifyd` matches the two players this image runs;
# the order is the priority when both are alive.
bindsym XF86AudioPlay exec playerctl -p mpv,spotifyd play-pause
bindsym XF86AudioPause exec playerctl -p mpv,spotifyd pause
bindsym XF86AudioStop exec playerctl -p mpv,spotifyd stop
bindsym XF86AudioNext exec playerctl -p mpv,spotifyd next
bindsym XF86AudioPrev exec playerctl -p mpv,spotifyd previous
# Skip rather than scan: a remote's ⏪/⏩ are pressed repeatedly, not held, and
# playerctl has no scan mode. 30s forward / 10s back is the podcast convention and is
# the one people already have in their fingers.
bindsym XF86AudioForward exec playerctl -p mpv,spotifyd position 30+
bindsym XF86AudioRewind exec playerctl -p mpv,spotifyd position 10-
# Volume. Sinks, not players — the remote's volume rocker should move the room's
# volume regardless of what is making the noise.
bindsym XF86AudioRaiseVolume exec wpctl set-volume @DEFAULT_AUDIO_SINK@ 5%+
bindsym XF86AudioLowerVolume exec wpctl set-volume @DEFAULT_AUDIO_SINK@ 5%-
bindsym XF86AudioMute exec wpctl set-mute @DEFAULT_AUDIO_SINK@ toggle
bindsym XF86AudioMicMute exec wpctl set-mute @DEFAULT_AUDIO_SOURCE@ toggle
# Navigation. A media station's "channels" are its workspaces, which is the closest
# honest analogy and means the channel rocker does something useful instead of nothing.
bindsym XF86ChannelUp workspace next
bindsym XF86ChannelDown workspace prev
bindsym XF86HomePage workspace $ws_media
bindsym XF86Back workspace back_and_forth
bindsym XF86Forward workspace back_and_forth
# Power. **The display, not the machine.** A remote's power button on a TV turns the
# picture off, and that is what people expect it to do here — while `poweroff` on a
# thin client that autologins is a button that takes the room's screen away until
# somebody walks over to it. See the display-toggle script for the wake side.
bindsym XF86PowerOff exec display-toggle
bindsym XF86Sleep exec display-toggle
bindsym XF86ScreenSaver exec display-toggle
# Deliberately no exit binding: `swaymsg exit` would drop to a black VT, and greetd
# would just autologin straight back in. Use SSH or the local terminal to administer.

26
hosts/thin-client/configs/sway/display-toggle Normal file
View File

@ -0,0 +1,26 @@
#!/bin/sh
# Turn every output off, or back on. Installed to /usr/local/bin/display-toggle and
# bound to the remote's power/sleep buttons in the Sway config.
#
# WHY THE POWER BUTTON DOES NOT POWER ANYTHING OFF: on a TV, that button turns the
# picture off. On a thin client that autologins into a kiosk, `poweroff` takes the
# room's screen away until somebody walks over and presses a physical button — which
# is a worse outcome than any it prevents. So the remote's power button does the thing
# the person pressing it actually meant.
#
# THE WAKE SIDE IS THE HARD HALF. With outputs powered off, Sway is still running and
# still receiving keys, so pressing power again lands here and turns them back on.
# That is why this is a toggle rather than two bindings: there is no other way back.
# It also means a *stuck* remote button cannot leave the screen dark — the next press
# fixes it.
set -eu
# `swaymsg -t get_outputs` reports each output's dpms state. If ANY output is still on,
# the intent of a press is "turn it off"; only when everything is already dark does a
# press mean "wake up". That ordering matters on a multi-output machine, where asking
# per-output would leave the remote toggling one screen at a time.
if swaymsg -t get_outputs | grep -q '"dpms": true'; then
swaymsg 'output * dpms off'
else
swaymsg 'output * dpms on'
fi

3
hosts/thin-client/live-build/config/package-lists/thin-client.list.chroot
View File

@ -22,6 +22,9 @@ mpv
mpv-mpris
libmpv2
playerctl
# wev prints the keysym for each key press — the two-minute way to find out what a
# given wireless remote's buttons actually send. See README's remote-control section.
wev
# --- Audio ---
pipewire

5
hosts/touch-panel/agent/touchpanel_agent/main.py
View File

@ -117,6 +117,9 @@ def main() -> int:
hostname = socket.gethostname()
node_id = "".join(c if c.isalnum() else "_" for c in hostname).strip("_") or "touchpanel"
friendly_name = config.get("TOUCHPANEL_NAME") or f"Touch panel ({hostname})"
# The HA area this device sits in, published as suggested_area — see
# docs/rooms-and-endpoints.md. Blank is fine and means "no suggestion".
room = config.get("TOUCHPANEL_ROOM", "")
broker_host = config.get("MQTT_BROKER_HOST", "")
broker_port = int(config.get("MQTT_BROKER_PORT") or 1883)
@ -128,7 +131,7 @@ def main() -> int:
if config.get("MQTT_USERNAME"):
client.username_pw_set(config["MQTT_USERNAME"], config.get("MQTT_PASSWORD") or None)
discovery = Discovery(client, node_id, friendly_name)
discovery = Discovery(client, node_id, friendly_name, room)
mpris = MprisBridge(discovery.publish_media_state, sway.session_env)
def on_launch(key: str) -> None:

10
hosts/touch-panel/agent/touchpanel_agent/mqtt_discovery.py
View File

@ -30,10 +30,11 @@ DISCOVERY_PREFIX = "homeassistant"
class Discovery:
def __init__(self, client, node_id: str, friendly_name: str):
def __init__(self, client, node_id: str, friendly_name: str, room: str = ""):
self.client = client
self.node_id = node_id
self.friendly_name = friendly_name
self.room = (room or "").strip()
self.base = f"touchpanel/{node_id}"
self.availability_topic = f"{self.base}/availability"
self.media_state_topic = f"{self.base}/media/state"
@ -46,6 +47,13 @@ class Discovery:
"model": "Sway touch panel",
"sw_version": __version__,
}
# Which room this physically sits in, as an HA area_id. `suggested_area`
# is honoured by HA only when the device is FIRST discovered — moving a
# device later means moving it in HA too, this cannot un-file it. Omitted
# entirely when unset, because an empty suggested_area is not the same
# request as no suggestion. See docs/rooms-and-endpoints.md.
if self.room:
self.device["suggested_area"] = self.room
# --- plumbing -----------------------------------------------------------
def _publish_config(self, component: str, object_id: str, payload: dict) -> None:

59
identity/README.md
View File

@ -72,6 +72,57 @@ actually decides who's registering. **If you want camera-based identity later,
Frigate's own face-recognition + enrollment (0.16+) is the piece to wire in, not a
new pipeline here.**
## How a person looks on a screen too small for their name
Every person carries two derived fields, served on `/people`, `/presence` and each
occupant in `/floorplan/presence`:
- **`initial`** — the first letter of the **real name**, never the nickname. Same rule
as `speak_name`, same reason: what a machine shows for somebody should follow who
they are, not what the household happens to call them this year.
- **`color`** — one of eight, assigned automatically at registration and editable in
the admin panel.
**The colour exists because initials collide.** A household with an Anna and an Amir
gets two identical "A"s on a wall panel or a watch face, and the colour is what makes
that readable. So the assignment rule is: first avoid any colour already worn by
somebody with the same initial — that is worth spending the whole palette on — then
take the least-used colour overall. Past eight people sharing one letter it repeats,
and you pick by hand.
**Why those eight values.** Every channel is `00`/`55`/`AA`/`FF`, which is exactly the
2-bits-per-channel space a colour Pebble renders natively. Anything else is snapped or
dithered by the watch, and a colour that shifts between the admin panel and the watch
defeats the point of having one. They also vary in lightness rather than only in hue,
so they stay distinguishable when a black-and-white screen reduces them to greys, and
they survive the common colour-vision deficiencies better than a rainbow would. A hex
outside the palette is accepted if you set one deliberately; it just won't be exact on
a watch. See `docs/pebble-presence-watchface.md`.
Existing people are backfilled with a colour at startup, oldest first, so the
assignment is stable across restarts and never reshuffles a colour somebody has
already learned.
## The photo, and where it comes from
Two ways a person gets a profile picture:
- **Registration** — the door panel's capture becomes the picture, "most recent
registration wins". Good for the picture registration took; useless for a device-less
household member registered by hand, who could never have one at all.
- **`POST /people/<id>/photo`** — upload one from the admin panel. Written into the
same photo directory with the same filename shape, so anything that serves or backs
up a registration capture handles this identically. Deliberately **not** recorded as
a registration event: nobody registered.
Removing a picture (`clear_photo`) blanks the reference but leaves the file on disk —
it may also be a `registration_events` audit artifact, and "stop showing this photo" is
a different request from "destroy the record that it was taken". The same reasoning
applies to replacing one.
A person with no picture is not a gap in a UI: their initial on their colour is the
fallback everywhere, and on the small screens it is the *preferred* rendering anyway.
## People without a device
Two paths, distinct on purpose because they solve different problems:
@ -559,9 +610,11 @@ not network placement.
| `POST /register` | `{"name", "device_id", "photo_id"?, "entity_id"?, "no_device"?}` -> registers, or returns a reason it couldn't (see above) |
| `POST /register/guest` | `{"device_id", "photo_id"?}` -> registers "Guest N", no name needed |
| `GET /people` | admin/audit list of every person: identifiers, device grants, chore assignments, `nickname`/`speak_name`, `last_visit_at`, `visit_count`, `currently_home_since` |
| `POST /people/<id>` | edit any editable field — `{"name"?, "nickname"?, "note"?, "chore_exempt"?, "chore_reminder_style"?, "notify_on_arrival"?, "announce_arrivals"?, "notify_topic"?, "digest_sections"?, "clear_photo"?}`. Omitted keys are left alone |
| `POST /people/<id>` | edit any editable field — `{"name"?, "nickname"?, "note"?, "color"?, "chore_exempt"?, "chore_reminder_style"?, "notify_on_arrival"?, "announce_arrivals"?, "notify_topic"?, "digest_sections"?, "clear_photo"?}`. Omitted keys are left alone; an empty `color` re-derives one rather than blanking it |
| `POST /people/<id>/test-notification` | push a test message to this person's ntfy topic, to prove it works |
| `GET /people/<id>/photo` | the person's profile picture (raw JPEG) — their most recent registration photo |
| `GET /people/<id>/photo` | the person's profile picture (raw JPEG) |
| `POST /people/<id>/photo` | raw image bytes -> set the profile picture directly, without a walk to the door panel |
| `GET /person-colors` | the eight-colour palette the admin panel offers (see below) |
| `POST /people/<id>/identifiers` | `{"entity_id"}` — attach an identifier by hand (a fixed BLE tag not in range yet). Still enforces `TRUSTED_ENTITY_PREFIXES` |
| `DELETE /people/<id>/identifiers/<id>` | revoke a mistaken or compromised identifier |
| `DELETE /people/<id>` | remove a person entirely (their identifiers, grants and visits go with them) |
@ -589,7 +642,7 @@ not network placement.
| `GET /floorplan/areas` | area values HA is currently reporting — the editor's pick list |
| `POST /presence/manual` | `{"person_id", "home"}` — hand-operated Home/Away for anyone with no identifiers |
| `POST /people/<id>/chore-settings` | `{"chore_exempt"?, "chore_reminder_style"?}` — see below; either field omitted/`null` leaves it unchanged |
| `GET /presence` | `{"people": [{"id", "name", "nickname", "speak_name", "home", "room", "has_device", "has_photo", "chore_exempt", "chore_reminder_style", "chore_assignments"}], "generated_at"}` — `home` is `true`/`false`/`null` (unknown), `room` is best-effort floor-plan groundwork (see below) |
| `GET /presence` | `{"people": [{"id", "name", "nickname", "speak_name", "home", "room", "has_device", "has_photo", "color", "initial", "chore_exempt", "chore_reminder_style", "chore_assignments"}], "generated_at"}` — `home` is `true`/`false`/`null` (unknown), `room` is best-effort floor-plan groundwork (see below) |
| `GET /weather` | proxies `smarthome/weather/current`, same JSON shape (`temperature`/`condition`/`location`) `hosts/thin-client`'s weather overlay already uses |
**Every person gets a profile picture, automatically** — whichever registration photo

22
identity/frontend/admin.html
View File

@ -203,6 +203,28 @@
<input id="edit-note" type="text" autocomplete="off" placeholder="e.g. cousin, visits at Christmas">
</label>
<fieldset class="field-group">
<legend>How they look on small screens</legend>
<p class="hint">
A wall panel or a watch face has no room for a name — it shows this colour and
the first letter of the real name. Two people whose names start the same letter
get different colours automatically; change one here if you'd rather pick.
</p>
<div class="field">Colour
<div id="edit-colors" class="swatch-row"></div>
</div>
<div class="field">Profile picture
<div class="photo-actions">
<input id="edit-photo-file" type="file" accept="image/*" hidden>
<button type="button" class="btn ghost" id="edit-photo-upload">Upload a picture</button>
<button type="button" class="btn ghost" id="edit-photo-clear">Remove</button>
</div>
<span class="hint">
Otherwise this is whatever the door panel captured at their last registration.
</span>
</div>
</fieldset>
<fieldset class="field-group">
<legend>Chores</legend>
<label class="check">

92
identity/frontend/admin.js
View File

@ -90,6 +90,82 @@ function duration(fromIso, toIso) {
// Same blob-fetch approach as register.js/dashboard.js — every identity endpoint
// requires an Authorization header, and a plain <img src="..."> has no way to send
// one. See identity/README.md.
// --- Colour + profile picture --------------------------------------------------------
// The palette comes from the server (GET /person-colors) rather than being repeated
// here: those eight values are chosen to render exactly on a colour Pebble's 2-bit-per-
// channel screen, and a second copy in this file would drift the first time somebody
// "improved" one of them. Falls back to whatever the person already has if the fetch
// fails, so the editor still opens.
let personColors = [];
function loadPersonColors() {
return api("/person-colors")
.then((data) => {
personColors = data.colors || [];
})
.catch(() => {
personColors = [];
});
}
function renderColorSwatches() {
const row = document.getElementById("edit-colors");
const current = (editing.color || "").toUpperCase();
const palette = personColors.length ? personColors : [current].filter(Boolean);
// A colour set by hand outside the palette still gets a swatch, so it is visible and
// reselectable rather than silently absent from its own editor.
const colors = palette.includes(current) || !current ? palette : palette.concat([current]);
row.innerHTML = colors
.map(
(color) =>
`<button type="button" class="swatch${color.toUpperCase() === current ? " selected" : ""}"
data-color="${escapeHtml(color)}" style="background:${escapeHtml(color)}"
title="${escapeHtml(color)}"><span>${escapeHtml(editing.initial || "")}</span></button>`
)
.join("");
row.querySelectorAll(".swatch").forEach((btn) =>
btn.addEventListener("click", () => {
editing.color = btn.dataset.color;
renderColorSwatches();
const avatar = document.getElementById("edit-avatar");
if (!editing.has_photo) avatar.style.background = editing.color;
})
);
}
document.getElementById("edit-photo-upload").addEventListener("click", () => {
document.getElementById("edit-photo-file").click();
});
document.getElementById("edit-photo-file").addEventListener("change", (event) => {
const file = event.target.files && event.target.files[0];
if (!file || !editing) return;
setStatus("Uploading…");
// Raw bytes, not multipart — same shape as the level-image upload and
// /register/photo. The picture is written immediately rather than waiting for Save,
// because it is a file on the server, not a field in this form.
api(`/people/${editing.id}/photo`, { method: "POST", body: file, headers: { "Content-Type": file.type || "image/jpeg" } })
.then((result) => {
if (!result.ok) throw new Error(result.message || "Could not upload.");
return refreshEditing("Picture updated.");
})
.catch((err) => setStatus(err.message, true))
.finally(() => {
event.target.value = "";
});
});
document.getElementById("edit-photo-clear").addEventListener("click", () => {
if (!editing) return;
setStatus("Removing…");
postJson(`/people/${editing.id}`, { clear_photo: true })
.then((result) => {
if (!result.ok) throw new Error(result.message || "Could not remove.");
return refreshEditing("Picture removed — showing their initial instead.");
})
.catch((err) => setStatus(err.message, true));
});
function loadAvatar(container, personId) {
fetch(`${API}/people/${personId}/photo`, { headers: { Authorization: `Bearer ${TOKEN}` } })
.then((res) => (res.ok ? res.blob() : Promise.reject()))
@ -154,7 +230,9 @@ function loadPeople() {
.map(
(p) =>
`<button type="button" class="card as-button" data-person="${p.id}">
<span class="avatar" data-avatar="${p.id}">👤</span>
<span class="avatar" data-avatar="${p.id}" style="background:${escapeHtml(
p.color || ""
)}">${escapeHtml(p.initial || "")}</span>
<span class="card-body">
<span class="card-name">${escapeHtml(p.name)}${p.currently_home_since ? ' <span class="dot-home" title="home now"></span>' : ""}</span>
<span class="card-meta">${escapeHtml(personSubtitle(p))}</span>
@ -208,9 +286,12 @@ function openEditor(personId) {
: '<span class="error">No ntfy topic reachable — set NTFY_URL and NTFY_DEFAULT_TOPIC on the server, or a topic here.</span>';
const avatar = document.getElementById("edit-avatar");
avatar.replaceChildren(document.createTextNode("👤"));
avatar.replaceChildren(document.createTextNode(editing.initial || "👤"));
avatar.style.background = editing.color || "";
if (editing.has_photo) loadAvatar(avatar, editing.id);
renderColorSwatches();
document.getElementById("edit-digest-sections").innerHTML = DIGEST_SECTIONS.map(
([key, label]) =>
`<label class="chip"><input type="checkbox" data-digest="${key}"${
@ -323,6 +404,9 @@ document.getElementById("edit-save").addEventListener("click", () => {
notify_on_arrival: document.getElementById("edit-notify-on-arrival").checked,
announce_arrivals: document.getElementById("edit-announce-arrivals").checked,
notify_topic: document.getElementById("edit-notify-topic").value.trim(),
// Whatever swatch is currently selected — the picker mutates `editing.color` and
// saves with the rest of the dialog, since it is one more column on `people`.
color: editing.color || "",
// Sent with the rest of the person's own fields rather than as a third call: unlike
// chore assignments (their own table), this is one column on `people`, so it saves
// or is refused together with everything else in the dialog.
@ -992,4 +1076,6 @@ document.getElementById("fp-image").addEventListener("change", (e) => {
document.getElementById("fp-live").addEventListener("change", loadFloorplanPresence);
loadPeople();
// The palette first, so the first editor opened already has its swatches — it is one
// small request and it never has to be repeated.
loadPersonColors().then(loadPeople);

41
identity/frontend/style.css
View File

@ -130,6 +130,47 @@ main.no-tabs {
object-fit: cover;
}
/* The initial is drawn on the person's own colour, so a household with an Anna and an
* Amir can be read at a glance on a surface with no room for names — the same pairing
* a floorplan marker and a watch face use. Dark text because every colour in
* PERSON_COLORS is mid-to-bright; a white glyph would vanish on the amber. */
.avatar {
color: #101014;
font-weight: 700;
}
.swatch-row {
display: flex;
flex-wrap: wrap;
gap: 8px;
margin-top: 6px;
}
.swatch {
width: 44px;
height: 44px;
border-radius: 50%;
border: 2px solid transparent;
cursor: pointer;
display: flex;
align-items: center;
justify-content: center;
color: #101014;
font-weight: 700;
font-size: 16px;
}
.swatch.selected {
border-color: #e8e8ec;
box-shadow: 0 0 0 2px rgba(0, 0, 0, 0.6) inset;
}
.photo-actions {
display: flex;
gap: 8px;
margin: 6px 0;
}
.card-meta {
font-size: 14px;
color: #9a9aa6;

197
identity/server.py
View File

@ -61,6 +61,9 @@ Endpoints:
- POST /register the main call, described above
- GET /people admin/audit list of registered people + identifiers
- POST /people/<id> edit any editable field on a person (the admin panel)
- GET/POST /people/<id>/photo the profile picture — GET serves it, POST replaces
it from the admin panel without a walk to the door panel
- GET /person-colors the per-person colour palette (see PERSON_COLORS)
- DELETE /people/<id>/identifiers/<id> revoke a mistaken/compromised identifier
- GET /presence who's currently home, resolved from registered identifiers
- GET /resolve spoken name OR nickname -> the canonical person
@ -436,6 +439,98 @@ def init_db() -> None:
_ensure_column(conn, "people", "announce_arrivals", "INTEGER NOT NULL DEFAULT 1")
_ensure_column(conn, "people", "notify_topic", "TEXT")
_ensure_column(conn, "people", "digest_sections", "TEXT")
_ensure_column(conn, "people", "color", "TEXT")
_backfill_colors(conn)
# --- Per-person colour -------------------------------------------------------------
# WHAT THIS IS FOR: telling two people apart at a glance on a surface too small for
# their name. The floorplan view marks an occupied room with each occupant's initial,
# and a household with an Anna and an Amir gets two identical "A"s — the colour is
# what makes that readable. Everything downstream (the admin panel, the dashboard's
# floorplan, and any watch/panel face built on GET /floorplan/presence) uses the same
# assignment rather than each picking its own, so a person is the same colour
# everywhere they appear.
#
# WHY THESE EIGHT: every channel is 0x00/0x55/0xAA/0xFF, which is exactly the 2-bit-
# per-channel colour space a colour Pebble renders natively (64 colours). Anything
# else gets dithered or snapped by the watch, and a colour that shifts between the
# admin panel and the watch defeats the entire point of having one. They are also
# spread across lightness, not just hue, so they stay distinguishable when a
# black-and-white watch reduces them to a grey — and for the same reason they survive
# the most common colour-vision deficiencies better than a rainbow would.
PERSON_COLORS = [
"#FF0000", # red
"#0055FF", # blue
"#FFAA00", # amber
"#00AA00", # green
"#AA00FF", # purple
"#00AAAA", # teal
"#FF55AA", # pink
"#AA5500", # brown
]
COLOR_RE = re.compile(r"^#[0-9a-fA-F]{6}$")
def _person_initial(name: str) -> str:
"""The single character a small display shows for this person.
First letter of the real name, never the nickname — the same rule speak_name
follows, and for the same reason: what a machine shows for somebody should be
derived from who they are, not from what the household happens to call them this
year. Non-alphanumeric leading characters are skipped so a name that starts with a
quote or an accent-mark artifact still yields a letter.
"""
for char in (name or "").strip():
if char.isalnum():
return char.upper()
return "?"
def _pick_color(conn: sqlite3.Connection, name: str, person_id: int | None = None) -> str:
"""The least-contended colour for this person.
Two rules, in order. First, avoid any colour already worn by somebody whose name
starts with the same letter — that collision is the entire reason this field
exists, and it is worth spending the whole palette on. Second, among what is left,
take the least-used colour overall, so a small household ends up with eight
distinct colours rather than three people sharing red.
Falls back to the least-used colour when the palette is exhausted (more than eight
people sharing an initial), because a repeat is better than an empty field — the
admin panel can always override it by hand.
"""
rows = conn.execute("SELECT id, name, color FROM people WHERE color IS NOT NULL").fetchall()
taken: dict[str, int] = {color: 0 for color in PERSON_COLORS}
same_initial: set[str] = set()
initial = _person_initial(name)
for row in rows:
if person_id is not None and row["id"] == person_id:
continue
color = str(row["color"]).upper()
taken[color] = taken.get(color, 0) + 1
if _person_initial(row["name"]) == initial:
same_initial.add(color)
preferred = [c for c in PERSON_COLORS if c not in same_initial] or PERSON_COLORS
return min(preferred, key=lambda c: (taken.get(c, 0), PERSON_COLORS.index(c)))
def _backfill_colors(conn: sqlite3.Connection) -> None:
"""Give everyone who predates this column a colour, oldest first.
Oldest first so the assignment is stable across restarts and matches the order the
household actually acquired people — re-running this must never reshuffle colours
somebody has already learned.
"""
rows = conn.execute(
"SELECT id, name FROM people WHERE color IS NULL OR color = '' ORDER BY id"
).fetchall()
for row in rows:
conn.execute("UPDATE people SET color = ? WHERE id = ?", (_pick_color(conn, row["name"], row["id"]), row["id"]))
if rows:
LOG.info("identity: assigned a colour to %d person/people that had none", len(rows))
def _ha_get(path: str):
@ -519,7 +614,10 @@ def _find_or_create_person(conn: sqlite3.Connection, name: str) -> tuple[int, bo
raise AmbiguousName([m["name"] for m in matches])
if matches:
return matches[0]["id"], False
cur = conn.execute("INSERT INTO people (name, created_at) VALUES (?, ?)", (name, _now()))
cur = conn.execute(
"INSERT INTO people (name, created_at, color) VALUES (?, ?, ?)",
(name, _now(), _pick_color(conn, name)),
)
assert cur.lastrowid is not None
return cur.lastrowid, True
@ -749,6 +847,10 @@ def _person_payload(conn: sqlite3.Connection, person: sqlite3.Row) -> dict:
# filesystem detail — just whether GET /people/<id>/photo has
# anything to serve.
"has_photo": person["photo_path"] is not None,
# How a display that has no room for a name shows this person: their colour and
# their initial, decided here so every surface agrees. See PERSON_COLORS.
"color": person["color"] or PERSON_COLORS[0],
"initial": _person_initial(person["name"]),
"chore_exempt": bool(person["chore_exempt"]),
"chore_reminder_style": person["chore_reminder_style"],
"chore_assignments": [a["chore_type"] for a in assignments],
@ -1056,6 +1158,26 @@ def update_person(person_id: int, fields: dict) -> dict:
("digest_sections", ",".join(s for s in DIGEST_SECTIONS if s in cleaned))
)
if "color" in fields:
color = str(fields["color"] or "").strip()
if not color:
# Emptying the field re-derives one rather than leaving a person with
# no colour: every surface that draws people needs *a* colour, and a
# blank would only push that decision into four different renderers.
updates.append(("color", _pick_color(conn, person["name"], person_id)))
elif not COLOR_RE.match(color):
return {
"ok": False,
"reason": "bad_color",
"message": "A colour has to look like #RRGGBB.",
}
else:
# Any valid hex is accepted, but only PERSON_COLORS render exactly on a
# colour Pebble (2 bits per channel) — anything else is snapped by the
# watch and will not match what the admin panel shows. The panel offers
# the palette; this permits going outside it knowingly.
updates.append(("color", color.upper()))
for text_field in ("chore_reminder_style", "note"):
if text_field in fields:
value = fields[text_field]
@ -1192,6 +1314,38 @@ def get_person_photo(person_id: int) -> bytes | None:
return path.read_bytes()
def set_person_photo(person_id: int, image: bytes) -> dict:
"""Set someone's profile picture directly, from the admin panel.
Until this existed the only way to get a profile picture was to walk to the door
panel and re-register, because `_set_profile_photo()` only ever runs on the
registration path ("most recent registration photo wins"). That is a fine rule for
the picture the *registration* captured and a poor one for "this is what this
person looks like" — a device-less household member registered by hand had no way
to have a face at all.
Written into the same PHOTO_DIR as a registration capture and with the same
filename shape, so anything that serves or backs up one serves and backs up the
other. It is deliberately NOT recorded as a registration_event: nobody registered.
"""
with _db_lock, _db() as conn:
person = conn.execute("SELECT id FROM people WHERE id = ?", (person_id,)).fetchone()
if person is None:
return {"ok": False, "reason": "no_such_person", "message": "No person with that id."}
PHOTO_DIR.mkdir(parents=True, exist_ok=True)
photo_path = PHOTO_DIR / f"{int(time.time())}{os.getpid() % 10000:04d}.jpg"
photo_path.write_bytes(image)
# The previous file is left on disk on purpose, the same reasoning as
# clear_photo's: it may also be a registration_events audit artifact, and
# replacing a profile picture is not a request to destroy the record that an
# earlier one was taken.
conn.execute("UPDATE people SET photo_path = ? WHERE id = ?", (str(photo_path), person_id))
updated = conn.execute("SELECT * FROM people WHERE id = ?", (person_id,)).fetchone()
LOG.info("identity: set profile photo for person %d (%d bytes)", person_id, len(image))
return {"ok": True, "person": _person_payload(conn, updated)}
def delete_identifier(person_id: int, identifier_id: int) -> bool:
with _db_lock, _db() as conn:
cur = conn.execute(
@ -1334,6 +1488,11 @@ def presence() -> dict:
"room": room,
"has_device": bool(person["identifiers"]),
"has_photo": person["has_photo"],
# Carried through from /people so a consumer that only ever calls
# /presence (the floorplan, a watch face) can draw somebody without a
# second request per person — see PERSON_COLORS.
"color": person["color"],
"initial": person["initial"],
"face_seen_recently": face_seen,
# Chore-system settings, straight passthrough — see chores/README.md
# for how these are used (never anything presence-related itself).
@ -1997,7 +2156,16 @@ def floorplan_presence() -> dict:
for person in people:
if person.get("home") is not True:
continue
entry = {"id": person["id"], "name": person["name"], "has_photo": person["has_photo"]}
# Everything a floorplan occupant marker needs, and nothing else: a name for
# the surfaces with room for one, an initial and a colour for the ones without
# (a wall panel at a glance, a watch face — see docs/pebble-presence-watchface.md).
entry = {
"id": person["id"],
"name": person["name"],
"initial": person["initial"],
"color": person["color"],
"has_photo": person["has_photo"],
}
area = person.get("room")
if area:
by_area.setdefault(str(area), []).append(entry)
@ -2199,6 +2367,11 @@ class Handler(BaseHTTPRequestHandler):
self._respond(HTTPStatus.OK, floorplan_presence())
elif path == "/floorplan/areas":
self._respond(HTTPStatus.OK, area_suggestions())
elif path == "/person-colors":
# Served rather than duplicated in the admin panel's JS, so the palette
# has one definition — see PERSON_COLORS for what makes these eight
# specific values the palette.
self._respond(HTTPStatus.OK, {"colors": PERSON_COLORS})
elif level_image_match:
self._handle_level_image(int(level_image_match.group(1)))
elif assignments_match:
@ -2261,6 +2434,23 @@ class Handler(BaseHTTPRequestHandler):
self.end_headers()
self.wfile.write(data)
def _handle_set_person_photo(self, person_id: int) -> None:
"""POST /people/<id>/photo — raw image bytes, same shape as the level-image
upload and as /register/photo. No multipart: every client of this API is
either this project's own JS or a curl, and multipart parsing in the stdlib is
more failure surface than a Blob body is inconvenience.
"""
try:
data = self._read_body(MAX_IMAGE_BYTES)
except ValueError as exc:
self._respond(HTTPStatus.REQUEST_ENTITY_TOO_LARGE, {"error": str(exc)})
return
if not data:
self._respond(HTTPStatus.BAD_REQUEST, {"error": "empty image body"})
return
result = set_person_photo(person_id, data)
self._respond(HTTPStatus.OK if result.get("ok") else HTTPStatus.NOT_FOUND, result)
def _handle_level_image(self, level_id: int) -> None:
data = get_level_image(level_id)
if data is None:
@ -2289,6 +2479,7 @@ class Handler(BaseHTTPRequestHandler):
identifiers_match = re.match(r"^/people/(\d+)/identifiers$", path)
test_notify_match = re.match(r"^/people/(\d+)/test-notification$", path)
person_match = re.match(r"^/people/(\d+)$", path)
person_photo_match = re.match(r"^/people/(\d+)/photo$", path)
post_level_image_match = re.match(r"^/floorplan/levels/(\d+)/image$", path)
# /people/prune is checked before the bare /people/<id> edit route so it is
# never parsed as a person id (it can't be — it's not digits — but the ordering
@ -2326,6 +2517,8 @@ class Handler(BaseHTTPRequestHandler):
self._handle_save_room()
elif post_level_image_match:
self._handle_upload_level_image(int(post_level_image_match.group(1)))
elif person_photo_match:
self._handle_set_person_photo(int(person_photo_match.group(1)))
elif person_match:
self._handle_update_person(int(person_match.group(1)))
else:

6
pantry-vision/Dockerfile
View File

@ -8,8 +8,8 @@ ENV PYTHONUNBUFFERED=1 \
WORKDIR /app
# stdlib only (urllib for the Ollama/Grocy calls) — no requirements.txt, same call as
# admin-canvas/server.py.
COPY server.py ./
# stdlib only (urllib for the Ollama/Grocy calls, sqlite3 for doorway.py's hint
# store) — no requirements.txt, same call as admin-canvas/server.py.
COPY server.py doorway.py ./
CMD ["python", "server.py"]

206
pantry-vision/README.md
View File

@ -9,21 +9,126 @@ display then shows the resulting inventory ordered by what expires soonest, and
Grocy's recipes, on request.
- **`pantry-vision`** (this directory) — a small always-on Python HTTP service.
`POST /identify` (a photo → a proposal via an Ollama vision model), `POST /confirm`
(a human-reviewed proposal → written into Grocy stock), `GET /inventory`,
`GET /recipes`, and `GET /shopping-list` (all three proxy Grocy, reshaped for a
frontend). All five endpoints are bearer-token gated. `/shopping-list` is also
consumed by `hosts/door-panel/`'s dashboard (Phase 18, "groceries running low") —
it's a thin reshape of Grocy's own `/api/stock/volatile` `missing_products`, not
new inventory logic.
`POST /identify` (a photo → a proposal via an Ollama vision model), the writes
`POST /confirm` / `POST /consume` / `POST /adjust` / `POST /transfer`, the
door-sensor hook `POST /doorway-event`, and the reads `GET /inventory`,
`GET /expired`, `GET /doorway-events`, `GET /recipes`, `GET /shopping-list` (which
proxy Grocy, reshaped for a frontend). Every endpoint is bearer-token gated.
`/shopping-list` is also consumed by `hosts/door-panel/`'s dashboard (Phase 18,
"groceries running low") — it's a thin reshape of Grocy's own
`/api/stock/volatile` `missing_products`, not new inventory logic.
- **`doorway.py`** — the appliance-door half: a contact sensor fires, a camera burst
is pulled from Frigate, and what it recognised is recorded as a **hint**. See "Which
fridge is it in" below, and `docs/fridge-item-location.md` for the argument.
- **`frontend/`** — the static single-page app the kitchen display's kiosk browser
loads: Scan / Inventory / Recipes, vanilla JS, no build step, no framework — same
"vendored, dependency-free" choice as the digest/admin canvas SDKs. Served
read-only by a `pantry-web` nginx container (`setup-container-host.sh`), the same
role `digest-web`/`admin-web` already play for their own hosts.
loads: the four stock-movement screens below, plus Inventory and Recipes. Vanilla
JS, no build step, no framework — same "vendored, dependency-free" choice as the
digest/admin canvas SDKs. Served read-only by a `pantry-web` nginx container
(`setup-container-host.sh`), the same role `digest-web`/`admin-web` already play for
their own hosts.
- **`hosts/kitchen-display/`** — the touch kiosk image that runs the frontend. See
that directory's own README for the device side of this.
## The four ways stock moves
Stock only ever changes in four ways, so the display has four buttons, and three of
them are camera-first.
| Screen | What it does | Grocy call underneath |
| --- | --- | --- |
| **Unload groceries** | The camera runs in a loop. Hold up an item, get a stats card — description, best-before, where to put it, and how many individual things are in the pack — confirm, and it moves straight on to the next item without you touching the screen again. | `stock/products/{id}/add` |
| **Consume article** | Hold up what you are about to eat. If the same kind is in stock under more than one brand, it asks which; if the row holds more than one unit, it asks how many. | `stock/products/{id}/consume` |
| **List expired foods** | Everything already past its date. Clear a line by scanning the item you are about to bin (or with the row's own button when the label is unreadable). Booked out as **spoiled**, not eaten — Grocy keeps those apart, and that is the only way you will ever find out what the household keeps buying and throwing away. | `stock/products/{id}/consume` with `spoiled` |
| **Edit inventory** | The keyboard-and-buttons fallback: every article, `−`/`+`, and a freeform amount. This is the screen that does *not* use the camera, on purpose — it is where you go when the camera got something wrong. | `stock/products/{id}/inventory` |
Corrections go through Grocy's *inventory-correction* endpoint rather than
consume/add, so the stock journal says "somebody fixed the number" instead of
quietly filling the household's consumption history with corrections dressed as
meals.
### Two invariants everything else rests on
**Stock is counted in individual units, never in packages.** A twelve-pack of eggs is
booked in as twelve. "How many eggs do we have" is the question people actually ask,
and it makes booking out three of them arithmetic instead of a fractions-of-a-pack
problem. The vision model's `units_per_package` is a multiplier applied once on the
confirm screen — where it is editable, and where the screen states the result ("Books
in 12 eggs") before anything is written. Nothing downstream stores it. Set
`GROCY_DEFAULT_QU_ID` to a *Piece*-like unit accordingly.
**The fold key is the brand-free product kind.** Twelve eggs of brand X and ten of
brand Y are twenty-two eggs. The vision model is asked for a brand-free `kind`
alongside the brand, `/confirm` files the Grocy product under a **product group**
named after that kind, and the folding is a read of that group — so the grouping is
visible and editable in Grocy's own UI instead of living in a second classification
scheme here. The per-brand rows are never summed away: they are what the edit screen
expands to, because "take ten off brand Y specifically" has to stay possible.
The folding itself is deliberately dumb (casefold, strip a trailing `(Brand)`,
collapse whitespace). The clever version is a synonym problem — *eggs* vs *egg* vs
*free-range eggs* — that nobody wants adjudicated by a kitchen display at 19:00. When
the model answers inconsistently you get two lines instead of one, which is visible
and fixable; a wrongly merged line is neither.
## Which fridge is it in
The household has more than one cold appliance, so "in the fridge" is not an answer.
Two mechanisms address that, and **only the first is authoritative**:
**1. Grocy locations — the record.** The confirm screen's placement choice resolves to
a real Grocy location (`PANTRY_LOCATION_*`, created by name) and rides along on the
stock add. `/inventory` reports it. `POST /transfer` moves an amount between
appliances, and the edit screen has a "Move to…" control per row. That transfer is the
part that makes locations worth recording at all: without it, a location decays into
"where it was when it was bought", which is worse than no answer because it is
confidently wrong.
**2. Doorway hints — an observation, never a fact.** A Zigbee contact sensor on each
appliance door drives a Home Assistant automation that POSTs to `/doorway-event`:
```yaml
# Home Assistant automation — one per appliance door.
trigger:
- platform: state
entity_id: binary_sensor.fridge_kitchen_door
to: "on"
action:
- service: rest_command.pantry_doorway_event
data: { appliance: "fridge-kitchen", state: "opened" }
```
`pantry-vision` answers **202 immediately** and does the work on its own thread — the
burst is up to three snapshots and a vision call each, and an appliance door is not
something to keep a home automation waiting on. It pulls frames from that appliance's
Frigate camera, stops at the first one that identifies something, and writes what it
saw to its own SQLite file with a timestamp and a confidence.
**Nothing in that path writes stock.** A camera at a door cannot tell in from out,
misses when two things are carried at once, and sees nothing behind an arm. So it
produces "camera last saw something like this at Freezer (loggia), 20 min ago — a
sighting, not a fact", which is a sentence a person can evaluate, shown next to the
location Grocy actually records. Acting on it is a tap on Move.
Configure it with `PANTRY_DOOR_APPLIANCES` (`id:Grocy location name:frigate_camera`,
camera optional) and `FRIGATE_URL` — the same Frigate `chores` already pulls snapshots
from. **An appliance with a door sensor and no camera is a legitimate configuration**
and is the recommended starting point: it still records that the door opened, which is
the half of this feature that pays for itself.
### What the camera loop actually does
`frontend/app.js` samples a 32×24 greyscale thumbnail of the video every 700 ms and
only spends an `/identify` call when the picture has **settled** (it stopped moving)
*and* **changed** since the last thing it identified. Both gates exist because vision
latency is this phase's known open risk: the frames worth spending it on are the ones
where somebody is holding something still, and the same tin must never be identified
twice because nobody moved. "Identify now" overrides both, for the shiny jar under a
downlight that never settles.
A model that is down or answering nonsense **stops the loop** rather than
re-photographing the counter at it; the unload screen drops you into the manual form
instead.
## A real network listener, unlike admin-canvas
`admin-canvas` deliberately has **no published port** — only Home Assistant, on the
@ -49,6 +154,21 @@ already applies to identity-merge confirmation (see the *Identity store* row in
`docs/project-plan.md` §2) — a wrong camera guess costs one tap to fix, not a wrong
fact silently written into the household's inventory.
The book-out screens follow the same rule in the other direction, and it costs a tap
there too: recognition never consumes anything by itself, an ambiguous brand is asked
about rather than picked, and "throw away" states the amount before it goes. A camera
that quietly books out the wrong yoghurt produces an inventory nobody trusts, and an
inventory nobody trusts is worth exactly as much as no inventory.
One place where that guardrail is deliberately looser: the vision model is asked to
read a **printed best-before date** off the packaging when one is legible, and that
date lands pre-filled in the confirm screen's date field. The screen always says which
it is showing — *"Date read off the packaging — check it"* versus *"Estimated from
the category"* — and the server throws out anything more than a year in the past or
ten years out, because a misread label (small print, dot-matrix ink, curved packaging)
is the single most likely failure of this feature and that is what it looks like when
it happens.
## Configure
```sh
@ -84,17 +204,40 @@ responsive — this needs to be measured on real hardware, not assumed.
## Grocy API assumptions — unverified against a real instance
`server.py`'s Grocy calls (`_find_or_create_product`, `_add_to_stock`, the `/inventory`
and `/recipes` proxies) are written against Grocy's *documented* API shape, not
checked against a running instance. In particular:
`server.py`'s Grocy calls (`_find_or_create_product`, `_add_to_stock`,
`_consume_stock`, `_set_stock_amount`, `_stock_items` and the `/recipes` proxy) are
written against Grocy's *documented* API shape, not checked against a running
instance. In particular:
- Whether `GET /api/stock` rows carry a nested `product` object with a `name` field
by default, or need an explicit embed/expand parameter — `_handle_inventory`
degrades to `Product #<id>` if not, rather than dropping the row, but that's a
fallback, not a fix.
by default, or need an explicit embed/expand parameter. `_stock_items` sidesteps
this by fetching `/api/objects/products` separately and joining on `product_id` —
it needs the group and location anyway, not just the name — and degrades to
`Product #<id>` if both are missing, rather than dropping the row.
- Whether `POST /api/objects/products` with just
`name`/`location_id`/`qu_id_purchase`/`qu_id_stock` is actually enough to create a
minimal product on your Grocy version, or whether it requires more fields.
- Whether `POST /api/objects/product_groups` and `POST /api/objects/locations` accept
a bare `{"name": ...}`. Both are best-effort: a failure means the product lands
without a group (so it folds on its name instead) or in `GROCY_DEFAULT_LOCATION_ID`
(so it is in the wrong place but still in stock). Neither costs you the book-in.
- Whether `POST /api/stock/products/{id}/consume` takes `spoiled` as a boolean, and
what it returns when you try to consume more than is in stock — that refusal is
passed through to the screen verbatim, on the assumption that it is a real answer
("we have fewer than you think") rather than a transport failure.
- Whether `POST /api/stock/products/{id}/inventory` takes `new_amount` as the absolute
new figure and whether it *requires* `best_before_date` when the figure goes up. The
edit screen currently sends the amount alone.
- Whether `POST /api/stock/products/{id}/transfer` takes `location_id_from` /
`location_id_to`, and — the subtle one — **which location Grocy considers a stock
entry to be in.** `/transfer` uses the *product's* location as the source, which is
what `/confirm` set. Grocy can also hold one product's stock across several
locations at once, per entry; if your instance does that, the source this sends will
sometimes be wrong and the transfer will fail rather than move the wrong thing.
Reading `/api/stock/products/{id}/entries` is the fix if it comes up.
- Whether Frigate's `/api/<camera>/latest.jpg` returns a usable still for a camera that
is idle — `doorway.py` and `chores/check.py` both assume it does, neither has called
it.
- Whether the Recipes feature (`GET /api/objects/recipes`,
`GET /api/recipes/{id}/fulfillment`) needs to be explicitly enabled/populated
before it returns anything meaningful — `_handle_recipes` degrades to
@ -115,7 +258,34 @@ and its own README. It builds two containers: `pantry-vision` (this API) and
1. All of the Grocy API assumptions above.
2. Real-world vision-model accuracy and latency for grocery items — untested with
any actual model or camera.
any actual model or camera. The unload loop is where latency bites hardest: it is
built to feel like a queue of items rather than a queue of round trips, and if an
`/identify` takes 20 seconds it will not feel like either.
3. **Whether the model returns a stable `kind` for the same product across scans.**
This is the assumption the whole brand-folding rests on, and the one most likely
to disappoint quietly: scan the same carton of eggs five times and see whether it
says "eggs" five times. If it drifts, the fix is a fixed `kind` vocabulary in the
prompt, not more clever folding on this side.
4. **Whether printed best-before dates are read correctly or confidently invented.**
The prompt says not to guess and the server bounds the range, but neither can catch
a plausible wrong date. Check a handful against the actual packets before trusting
the pre-filled field.
5. **That a twelve-pack really does arrive as twelve.** `units_per_package` is the
one model answer that gets multiplied rather than displayed, and the failure is
silent in both directions — twelve eggs booked in as one, or one jar booked in as
twelve.
6. **Whether a doorway camera can identify anything at all.** This is the assumption
the whole `/doorway-event` path rests on, and it is not the same task as the
kitchen display's: an item in a moving hand, at ~1.5 m, in whatever light the room
has, versus one held still against a plain background 30 cm from a webcam.
`docs/fridge-item-location.md` says to test it with the kitchen's existing camera
before buying a second one, and that advice applies to this code too — if the
answer is no, the door sensors are still worth having and this half simply stays
switched off (`PANTRY_DOOR_APPLIANCES` entries without a camera).
7. **Whether the hints are read as hints.** The wording on screen ("a sighting, not a
fact") is doing real work; if in practice people treat a sighting as the answer and
stop checking, that is a design failure this code cannot detect and the feature
should be turned off rather than tuned.
3. Whether Ollama's `/api/generate` `images` field is still the right call shape for
whichever vision model you pick — some multimodal models are only exposed through
Ollama's newer `/api/chat` with a `images` field per-message instead; this was

286
pantry-vision/doorway.py Normal file
View File

@ -0,0 +1,286 @@
"""Door-sensor-triggered appliance cameras — the "which fridge is it in" half of
pantry-vision, from docs/fridge-item-location.md.
A Zigbee contact sensor on each cold appliance fires an HA automation, which POSTs
`/doorway-event` here. This module then pulls a snapshot (or a short burst) from that
appliance's camera via Frigate, runs it through the same vision identification the
kitchen display uses, and records what it saw as a **hint**.
WHY THIS IS A HINT AND NOT A FACT
---------------------------------
Read `docs/fridge-item-location.md` before changing anything here; the short version:
- **A camera at the door can only ever see the doorway.** It does not know whether the
item was going in or coming out, it misses when two things are carried at once, and
it sees nothing at all when an arm is in the way. Anything that presented its output
as the truth about where food is would be wrong several times a week, silently.
- So a hint is written with a timestamp, an appliance, and a confidence, and it is
shown as "last seen going past the loggia fridge, Tue 18:42" — a sentence a person
can evaluate. **Nothing here writes stock.** Moving an item between appliances is
`/transfer`, which a person taps, exactly like every other write in this service.
- Which means a wrong hint costs a glance. That is the whole design budget.
WHY A SEPARATE SQLITE FILE, WHEN "GROCY IS THE SYSTEM OF RECORD"
----------------------------------------------------------------
Grocy owns *stock*: what exists, how much, until when, and — via locations — where it
is meant to be. It has nowhere to put "a camera thinks it saw something like this go
past that door 40 seconds ago, and might be wrong". That is not inventory, it is
observation, and it has a retention life measured in days. Keeping it out of Grocy is
what stops a guess from ever being mistaken for stock — the same separation
digest-engine draws between its archive and its output.
The file is best-effort in the same way `archive.py` is over in digest-engine: a
corrupt or unwritable database logs a warning and the identification still happens.
No hint is worth failing a request over.
"""
from __future__ import annotations
import logging
import os
import sqlite3
import time
import urllib.error
import urllib.request
from datetime import datetime, timedelta, timezone
from pathlib import Path
LOG = logging.getLogger("pantry-vision.doorway")
DB_PATH = Path(os.environ.get("PANTRY_HINTS_DB_PATH", "/data/pantry-hints.db"))
# Same Frigate this project's `chores` already pulls snapshots from, same env name.
# Unset means the door-event endpoint accepts the event and records the opening
# without a picture — which is still worth having, see the module docstring.
FRIGATE_URL = os.environ.get("FRIGATE_URL", "").rstrip("/")
FRIGATE_TIMEOUT = float(os.environ.get("FRIGATE_TIMEOUT", "15"))
# How many frames to take per door event, and how far apart. An item crosses the
# doorway in about a second, so one frame is a coin toss and ten is a queue at the
# vision model. Three is a compromise, and the burst stops early on the first frame
# that actually identifies something.
BURST_FRAMES = int(os.environ.get("PANTRY_DOORWAY_BURST", "3"))
BURST_INTERVAL = float(os.environ.get("PANTRY_DOORWAY_BURST_INTERVAL", "0.7"))
# Hints are worthless once they are old — "last seen a month ago" tells you nothing a
# person didn't already know. Pruned on write, so the file cannot grow without bound.
HINT_RETENTION_DAYS = int(os.environ.get("PANTRY_HINT_RETENTION_DAYS", "30"))
def _now() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def appliances() -> dict[str, dict]:
"""PANTRY_DOOR_APPLIANCES: "id:Grocy location name:frigate_camera, ..."
The Grocy location name is the same string `PANTRY_LOCATION_*` uses, because the
point of the whole feature is to answer *which appliance*, and "which appliance"
has to mean the same thing here as it does on the confirm screen. The camera is
optional: an appliance with a door sensor and no camera still records openings,
which is the half of this feature that pays for itself.
"""
raw = os.environ.get("PANTRY_DOOR_APPLIANCES", "").strip()
result: dict[str, dict] = {}
for entry in raw.split(","):
entry = entry.strip()
if not entry:
continue
parts = [p.strip() for p in entry.split(":")]
if len(parts) < 2 or not parts[0] or not parts[1]:
LOG.warning("pantry-vision: ignoring malformed PANTRY_DOOR_APPLIANCES entry %r", entry)
continue
result[parts[0]] = {
"id": parts[0],
"location_name": parts[1],
"camera": parts[2] if len(parts) > 2 and parts[2] else "",
}
return result
# --- the hint store ---------------------------------------------------------------
def _db() -> sqlite3.Connection | None:
try:
DB_PATH.parent.mkdir(parents=True, exist_ok=True)
conn = sqlite3.connect(DB_PATH, timeout=5)
conn.row_factory = sqlite3.Row
conn.execute(
"""
CREATE TABLE IF NOT EXISTS hints (
id INTEGER PRIMARY KEY,
appliance TEXT NOT NULL,
location_name TEXT NOT NULL,
door_state TEXT NOT NULL,
product_id INTEGER,
kind TEXT,
name TEXT,
confidence TEXT,
identified INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL
)
"""
)
conn.execute("CREATE INDEX IF NOT EXISTS hints_by_product ON hints (product_id, created_at)")
return conn
except (sqlite3.Error, OSError):
# OSError as well as sqlite3.Error, and this is not belt-and-braces: mkdir on
# an unmounted or read-only /data raises PermissionError, which is an OSError
# and not a database error at all. Without it, a deployment that forgot the
# bind mount would take down /inventory — the one screen that has nothing to
# do with hints — instead of quietly having no sightings to show.
LOG.warning("pantry-vision: hint database unusable at %s", DB_PATH, exc_info=True)
return None
def record_hint(appliance: dict, door_state: str, proposal: dict | None, product_id: int | None) -> None:
conn = _db()
if conn is None:
return
try:
with conn:
conn.execute(
"INSERT INTO hints (appliance, location_name, door_state, product_id, kind, name, "
"confidence, identified, created_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)",
(
appliance["id"],
appliance["location_name"],
door_state,
product_id,
(proposal or {}).get("kind") or "",
(proposal or {}).get("name") or "",
(proposal or {}).get("confidence") or "",
1 if proposal else 0,
_now(),
),
)
cutoff = (datetime.now(timezone.utc) - timedelta(days=HINT_RETENTION_DAYS)).isoformat()
conn.execute("DELETE FROM hints WHERE created_at < ?", (cutoff.replace("+00:00", "Z"),))
except sqlite3.Error:
LOG.warning("pantry-vision: could not write a doorway hint", exc_info=True)
finally:
conn.close()
def recent_hints(limit: int = 50) -> list[dict]:
conn = _db()
if conn is None:
return []
try:
rows = conn.execute(
"SELECT appliance, location_name, door_state, product_id, kind, name, confidence, "
"identified, created_at FROM hints ORDER BY created_at DESC LIMIT ?",
(max(1, min(500, limit)),),
).fetchall()
return [dict(row) for row in rows]
except sqlite3.Error:
LOG.warning("pantry-vision: could not read doorway hints", exc_info=True)
return []
finally:
conn.close()
def last_seen_by_product() -> dict[int, dict]:
"""{product_id: the most recent identified hint for it}.
Only identified hints: a door opening with nothing recognised in it says something
about the door, not about any particular jar, and attaching it to an item would be
inventing the very link this module refuses to invent.
"""
conn = _db()
if conn is None:
return {}
try:
# SQLite's documented bare-column rule: with MAX() in the select list, the
# other columns come from the row that matched it. That is what makes this one
# query instead of one per product.
rows = conn.execute(
"SELECT product_id, appliance, location_name, confidence, MAX(created_at) AS created_at "
"FROM hints WHERE product_id IS NOT NULL AND identified = 1 GROUP BY product_id"
).fetchall()
return {int(row["product_id"]): dict(row) for row in rows}
except sqlite3.Error:
LOG.warning("pantry-vision: could not read last-seen hints", exc_info=True)
return {}
finally:
conn.close()
# --- the camera -------------------------------------------------------------------
def _snapshot(camera: str) -> bytes | None:
"""One frame from Frigate, or None. Same endpoint shape `chores/check.py` uses.
VERIFY against a real Frigate: `/api/<camera>/latest.jpg` is its documented
always-available snapshot path, but nothing in this project has called it yet.
"""
if not (FRIGATE_URL and camera):
return None
try:
with urllib.request.urlopen(f"{FRIGATE_URL}/api/{camera}/latest.jpg", timeout=FRIGATE_TIMEOUT) as resp:
return resp.read()
except (urllib.error.URLError, urllib.error.HTTPError, TimeoutError, OSError):
LOG.warning("pantry-vision: could not fetch a snapshot from camera %r", camera, exc_info=True)
return None
def handle_event(appliance: dict, door_state: str, identify, match) -> dict:
"""Take a burst, identify the first frame that shows something, record the hint.
Runs off the request thread (see server.py) because a burst is up to three vision
calls and Home Assistant's `rest_command` should not be sitting on a socket for
however long the LLM host takes. `identify` and `match` are passed in rather than
imported so this module never has to know about Grocy — it deals in doors,
cameras and hints.
"""
camera = appliance.get("camera") or ""
if not camera:
record_hint(appliance, door_state, None, None)
LOG.info("pantry-vision: %s %s (no camera configured)", appliance["id"], door_state)
return {"appliance": appliance["id"], "identified": False, "reason": "no camera configured"}
proposal = None
for attempt in range(max(1, BURST_FRAMES)):
if attempt:
time.sleep(BURST_INTERVAL)
image = _snapshot(camera)
if image is None:
break
candidate = identify(image)
# `present: false` is the model saying the frame holds no grocery item — an
# arm, a closed door, an empty kitchen. That is the expected answer for most
# frames of most openings, and it is not a failure.
if candidate.get("present") and not candidate.get("degraded"):
proposal = candidate
break
if proposal is None:
record_hint(appliance, door_state, None, None)
LOG.info("pantry-vision: %s %s — nothing recognised in %d frame(s)", appliance["id"], door_state, BURST_FRAMES)
return {"appliance": appliance["id"], "identified": False, "reason": "nothing recognised"}
# An exact stock match makes the hint attachable to a row the household can act
# on. Without one it is still worth recording — "something like eggs went past the
# loggia freezer" is a useful thing to have seen, and it is exactly what a person
# would search for after failing to find eggs.
product_id = None
try:
matches = match(proposal.get("kind", ""), proposal.get("name", ""))
exact = [m for m in matches if m.get("exact")]
if exact:
product_id = int(exact[0]["product_id"])
elif len(matches) == 1:
product_id = int(matches[0]["product_id"])
except Exception:
LOG.warning("pantry-vision: could not match a doorway identification to stock", exc_info=True)
record_hint(appliance, door_state, proposal, product_id)
LOG.info(
"pantry-vision: %s %s — saw %r (confidence %s, stock row %s)",
appliance["id"], door_state, proposal.get("name"), proposal.get("confidence"), product_id,
)
return {
"appliance": appliance["id"],
"identified": True,
"name": proposal.get("name"),
"confidence": proposal.get("confidence"),
"product_id": product_id,
}

826
pantry-vision/frontend/app.js
View File

@ -6,6 +6,17 @@
// kitchen-display kiosk's own launch command, NOT hardcoded here — this file is a
// generic static asset with no secret in it, served read-only by pantry-web to
// whatever device points a browser at it.
//
// FOUR FLOWS, ONE CAMERA, ONE RULE
// --------------------------------
// Unload (book in), Consume (book out), Expired (throw away) and Edit (correct) are
// the four ways stock moves, and the first three are camera-first: hold the thing up,
// the model says what it is, a person confirms, it is written. The rule that shapes
// every one of them is server.py's — the camera proposes, the person disposes.
// Nothing here calls a write endpoint without a tap in between, including the flows
// where that costs an extra tap, because a camera that quietly books out the wrong
// brand of yoghurt produces an inventory nobody trusts, and an inventory nobody
// trusts is the same as no inventory.
"use strict";
const params = new URLSearchParams(location.search);
@ -32,45 +43,101 @@ function api(path, options) {
});
}
// --- Tabs --------------------------------------------------------------------
const tabs = document.querySelectorAll(".tab");
const panels = document.querySelectorAll(".panel");
function postJson(path, body) {
return api(path, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body),
});
}
const $ = (id) => document.getElementById(id);
// --- Navigation ----------------------------------------------------------------
// Three tabs (Home / Inventory / Recipes) plus four flow screens reached from Home.
// "scan" is kept as an alias for the unload flow: it is the fragment
// kitchen-display-agent's "Show scan" MQTT button has been publishing since Phase 17,
// and a button in Home Assistant that stops working is a worse outcome than an
// old name living on here.
const SCREENS = ["home", "unload", "consume", "expired", "edit", "inventory", "recipes"];
const TABS = ["home", "inventory", "recipes"];
let currentScreen = "home";
function show(name) {
if (name === "scan") name = "unload";
if (!SCREENS.includes(name)) name = "home";
// Leaving a flow always releases the camera and cancels its loop: the kiosk has one
// webcam and a scan loop left running behind another screen would keep firing
// /identify at the LLM host with nobody watching the answers.
if (currentScreen !== name) stopScanner();
currentScreen = name;
document.querySelectorAll(".panel").forEach((p) => p.classList.toggle("active", p.id === name));
document.querySelectorAll(".tab").forEach((t) => t.classList.toggle("active", t.dataset.tab === name));
// A flow screen is not a tab; keep Home lit while one is open so the tab bar never
// shows nothing selected.
if (!TABS.includes(name)) document.querySelector('.tab[data-tab="home"]').classList.add("active");
function activateTab(name) {
tabs.forEach((t) => t.classList.toggle("active", t.dataset.tab === name));
panels.forEach((p) => p.classList.toggle("active", p.id === name));
if (name === "inventory") loadInventory();
if (name === "recipes") loadRecipes();
if (name === "scan") resetScan();
if (name === "unload") startUnload();
if (name === "consume") startConsume();
if (name === "expired") startExpired();
if (name === "edit") loadEditInventory();
}
tabs.forEach((t) => t.addEventListener("click", () => activateTab(t.dataset.tab)));
document.querySelectorAll(".tab").forEach((t) => t.addEventListener("click", () => show(t.dataset.tab)));
document.querySelectorAll("[data-goto]").forEach((b) => b.addEventListener("click", () => show(b.dataset.goto)));
// A "Show <tab>" MQTT command from touchpanel-style HA control arrives as a URL
// fragment reload (hosts/kitchen-display/agent kills and relaunches Chromium at
// index.html#recipes, same pattern as the thin client's digest-browser) — honour it
// on load, same as any manual tap.
if (location.hash) {
const initial = location.hash.slice(1);
if (["scan", "inventory", "recipes"].includes(initial)) activateTab(initial);
function toast(message, kind) {
const el = $("toast");
el.textContent = message;
el.className = kind || "";
el.hidden = false;
clearTimeout(toast._timer);
toast._timer = setTimeout(() => {
el.hidden = true;
}, 2600);
}
// --- Scan ----------------------------------------------------------------------
const video = document.getElementById("camera-preview");
const captureBtn = document.getElementById("capture-btn");
const cameraError = document.getElementById("camera-error");
const scanCamera = document.getElementById("scan-camera");
const scanResult = document.getElementById("scan-result");
const capturedFrame = document.getElementById("captured-frame");
const confirmForm = document.getElementById("confirm-form");
const confirmStatus = document.getElementById("confirm-status");
const retakeBtn = document.getElementById("retake-btn");
// --- The camera ----------------------------------------------------------------
// One <video>, one MediaStream, moved between flows. The auto-scan loop samples a
// tiny greyscale thumbnail of each frame and only spends an /identify call when the
// picture has (a) settled and (b) actually changed since the last thing it
// identified. Both gates exist for the same reason: the vision model's latency is the
// known open risk in this phase (project-plan.md open decision #18), so the frames
// worth spending it on are the ones where somebody is holding something still, and
// the same tin should never be identified twice because nobody moved.
const SAMPLE_INTERVAL_MS = 700;
const STILL_THRESHOLD = 6; // mean per-pixel difference below which the frame is "settled"
const CHANGED_THRESHOLD = 12; // ...and above which it is a different item from the last one
const video = $("camera-preview");
const scanner = $("scanner");
const scanStatus = $("scan-status");
const cameraError = $("camera-error");
const captureNowBtn = $("capture-now");
const frameSample = $("frame-sample");
const capturedFrame = $("captured-frame");
let stream = null;
let scanTimer = null;
let scanBusy = false;
let previousSample = null;
let acceptedSample = null;
let onIdentified = null;
function mountScanner(screenId) {
const slot = document.querySelector(`#${screenId} .scanner-slot`);
if (slot && scanner.parentElement !== slot) slot.appendChild(scanner);
scanner.hidden = false;
}
function startCamera() {
if (stream) return;
navigator.mediaDevices
if (stream) return Promise.resolve();
return navigator.mediaDevices
.getUserMedia({ video: { facingMode: "environment" }, audio: false })
.then((s) => {
stream = s;
@ -80,79 +147,148 @@ function startCamera() {
.catch((err) => {
cameraError.textContent = `Camera unavailable: ${err.message}. See hosts/kitchen-display/README.md.`;
cameraError.hidden = false;
throw err;
});
}
function resetScan() {
scanResult.hidden = true;
scanCamera.hidden = false;
confirmStatus.textContent = "";
startCamera();
function stopScanner() {
clearInterval(scanTimer);
scanTimer = null;
scanBusy = false;
onIdentified = null;
previousSample = null;
acceptedSample = null;
scanner.hidden = true;
if (stream) {
stream.getTracks().forEach((t) => t.stop());
stream = null;
video.srcObject = null;
}
}
captureBtn.addEventListener("click", () => {
if (!stream) return;
/** A 32x24 greyscale thumbnail of the current frame, as a plain array. */
function sampleFrame() {
if (!video.videoWidth) return null;
frameSample.width = 32;
frameSample.height = 24;
const ctx = frameSample.getContext("2d", { willReadFrequently: true });
ctx.drawImage(video, 0, 0, 32, 24);
const data = ctx.getImageData(0, 0, 32, 24).data;
const grey = new Array(32 * 24);
for (let i = 0; i < grey.length; i++) {
const p = i * 4;
grey[i] = (data[p] * 299 + data[p + 1] * 587 + data[p + 2] * 114) / 1000;
}
return grey;
}
function frameDelta(a, b) {
if (!a || !b) return Infinity;
let total = 0;
for (let i = 0; i < a.length; i++) total += Math.abs(a[i] - b[i]);
return total / a.length;
}
/**
* Run the camera on `screenId` and call `handler(proposal)` the first time it sees
* something. The loop stops itself on a hit; the flow calls resumeScan() when the
* person is done with that item and ready for the next one.
*/
function startScan(screenId, handler, statusText) {
onIdentified = handler;
mountScanner(screenId);
scanStatus.textContent = "Starting camera…";
startCamera()
.then(() => {
scanStatus.textContent = statusText || "Hold an item up to the camera…";
previousSample = null;
clearInterval(scanTimer);
scanTimer = setInterval(tick, SAMPLE_INTERVAL_MS);
})
.catch(() => {
scanStatus.textContent = "";
});
}
function resumeScan(statusText) {
if (!stream || !onIdentified) return;
scanner.hidden = false;
scanStatus.textContent = statusText || "Ready for the next one…";
previousSample = null;
clearInterval(scanTimer);
scanTimer = setInterval(tick, SAMPLE_INTERVAL_MS);
}
function pauseScan() {
clearInterval(scanTimer);
scanTimer = null;
}
function tick() {
if (scanBusy) return;
const sample = sampleFrame();
if (!sample) return;
const settled = frameDelta(sample, previousSample) < STILL_THRESHOLD;
const changed = frameDelta(sample, acceptedSample) > CHANGED_THRESHOLD;
previousSample = sample;
if (!settled) return;
if (!changed) {
scanStatus.textContent = "Waiting for the next item…";
return;
}
identifyNow(sample);
}
/** Capture at full resolution and ask the server what it is. */
function identifyNow(sample) {
if (scanBusy) return;
scanBusy = true;
scanStatus.textContent = "Identifying…";
capturedFrame.width = video.videoWidth;
capturedFrame.height = video.videoHeight;
capturedFrame.getContext("2d").drawImage(video, 0, 0);
scanCamera.hidden = true;
scanResult.hidden = false;
confirmStatus.textContent = "Identifying…";
document.getElementById("f-confidence").textContent = "";
capturedFrame.toBlob(
(blob) => {
api("/identify", { method: "POST", body: blob, headers: { "Content-Type": "image/jpeg" } })
.then((proposal) => {
document.getElementById("f-name").value = proposal.name || "";
document.getElementById("f-category").value = proposal.category || "other";
const days = Number.isFinite(proposal.estimated_shelf_life_days)
? proposal.estimated_shelf_life_days
: 7;
const due = new Date();
due.setDate(due.getDate() + days);
document.getElementById("f-date").value = due.toISOString().slice(0, 10);
document.getElementById("f-confidence").textContent =
`Model confidence: ${proposal.confidence || "unknown"}` +
(proposal.note ? ` — ${proposal.note}` : "") +
". Review before confirming.";
confirmStatus.textContent = "";
scanBusy = false;
if (!proposal.present) {
scanStatus.textContent = "Nothing recognised — hold the item closer.";
return;
}
acceptedSample = sample || sampleFrame();
pauseScan();
if (onIdentified) onIdentified(proposal);
})
.catch((err) => {
confirmStatus.textContent = `Could not identify: ${err.message}. Fill in manually.`;
scanBusy = false;
pauseScan();
scanStatus.textContent = `Could not identify: ${err.message}`;
// A dead vision model must not turn into a loop that keeps photographing
// the counter at it. The flow decides what to offer instead — for unload,
// the manual form; for the others, a message and a way out.
if (onIdentified) onIdentified({ present: true, degraded: true, error: err.message });
});
},
"image/jpeg",
0.85
);
}
// "Identify now" is the escape hatch for the stillness gate: a shiny jar under a
// kitchen downlight can flicker enough to never settle, and standing there waving it
// is not an acceptable answer.
captureNowBtn.addEventListener("click", () => {
if (stream && !scanBusy) identifyNow(null);
});
retakeBtn.addEventListener("click", resetScan);
// --- Shared item helpers -------------------------------------------------------
function escapeHtml(s) {
return String(s).replace(/[&<>"']/g, (c) => ({ "&": "&amp;", "<": "&lt;", ">": "&gt;", '"': "&quot;", "'": "&#39;" }[c]));
}
confirmForm.addEventListener("submit", (event) => {
event.preventDefault();
confirmStatus.textContent = "Adding…";
api("/confirm", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
name: document.getElementById("f-name").value.trim(),
category: document.getElementById("f-category").value,
best_before_date: document.getElementById("f-date").value,
quantity: Number(document.getElementById("f-quantity").value) || 1,
}),
})
.then(() => {
confirmStatus.textContent = "Added. Put it away!";
setTimeout(resetScan, 1500);
})
.catch((err) => {
confirmStatus.textContent = `Could not add: ${err.message}`;
});
});
// --- Inventory -------------------------------------------------------------------
function urgencyClass(daysLeft) {
if (daysLeft === null || daysLeft === undefined) return "";
if (daysLeft < 0) return "urgent-expired";
@ -161,30 +297,508 @@ function urgencyClass(daysLeft) {
return "";
}
function loadInventory() {
const el = document.getElementById("inventory-list");
api("/inventory")
.then((data) => {
const items = data.items || [];
if (!items.length) {
el.innerHTML = '<p class="hint">Nothing in stock yet — scan something!</p>';
function dueLabel(days) {
if (days === null || days === undefined) return "no date";
if (days < 0) return `expired ${-days}d ago`;
if (days === 0) return "expires today";
return `${days}d left`;
}
function amountOf(row) {
const n = Number(row.amount);
return Number.isFinite(n) ? n : 0;
}
/** "12 × Eggs (Brand X)" the way every list in here writes it. */
function rowLabel(row) {
return `${amountOf(row)} × ${row.name}`;
}
// --- Unload groceries (book in) ------------------------------------------------
const confirmForm = $("confirm-form");
let unloadAdded = 0;
function startUnload() {
unloadAdded = 0;
$("unload-count").textContent = "";
confirmForm.hidden = true;
startScan("unload", onUnloadProposal, "Hold the first item up to the camera…");
}
function onUnloadProposal(proposal) {
scanner.hidden = false;
confirmForm.hidden = false;
$("confirm-status").textContent = "";
const degraded = !!proposal.degraded;
$("f-description").textContent = degraded
? "Could not identify this one — fill it in by hand."
: proposal.description || proposal.name || "";
$("f-name").value = proposal.name || "";
$("f-category").value = proposal.category || "other";
$("f-placement").value = proposal.recommended_placement || "cupboard";
$("f-date").value = proposal.best_before_date || new Date().toISOString().slice(0, 10);
$("f-date-source").textContent =
proposal.best_before_source === "label"
? "Date read off the packaging — check it."
: "Estimated from the category, not read off the packaging.";
$("f-packages").value = 1;
$("f-units").value = proposal.units_per_package || 1;
confirmForm.dataset.unitName = proposal.unit_name || "piece";
confirmForm.dataset.kind = proposal.kind || "";
confirmForm.dataset.brand = proposal.brand || "";
updateTotalLine();
// "You already have ten of these" at the only moment it can still change what
// somebody does — while the shopping is still on the counter.
const matches = proposal.stock_matches || [];
const held = matches.reduce((sum, m) => sum + amountOf(m), 0);
$("f-have").textContent = held
? `Already in stock: ${held} (${matches.length} ${matches.length === 1 ? "entry" : "entries"}).`
: proposal.stock_matches_unavailable
? "Could not check what is already in stock — Grocy did not answer."
: "";
$("f-confidence").textContent = degraded
? proposal.error || proposal.note || ""
: `Model confidence: ${proposal.confidence || "unknown"}. Review before confirming.`;
}
function updateTotalLine() {
const packs = Number($("f-packages").value) || 1;
const per = Number($("f-units").value) || 1;
const unit = confirmForm.dataset.unitName || "piece";
const total = packs * per;
$("f-total").textContent =
per > 1
? `Books in ${total} ${unit}${total === 1 ? "" : "s"} (${packs} × ${per}).`
: `Books in ${total} ${unit}${total === 1 ? "" : "s"}.`;
}
$("f-packages").addEventListener("input", updateTotalLine);
$("f-units").addEventListener("input", updateTotalLine);
$("skip-btn").addEventListener("click", () => {
confirmForm.hidden = true;
resumeScan("Skipped. Hold up the next item…");
});
confirmForm.addEventListener("submit", (event) => {
event.preventDefault();
$("confirm-status").textContent = "Adding…";
postJson("/confirm", {
name: $("f-name").value.trim(),
kind: confirmForm.dataset.kind || "",
brand: confirmForm.dataset.brand || "",
category: $("f-category").value,
placement: $("f-placement").value,
best_before_date: $("f-date").value,
quantity: Number($("f-packages").value) || 1,
units_per_package: Number($("f-units").value) || 1,
})
.then((result) => {
unloadAdded += 1;
$("unload-count").textContent = `${unloadAdded} booked in`;
$("confirm-status").textContent = "";
confirmForm.hidden = true;
toast(`Added ${result.amount} × ${$("f-name").value.trim()} — put it away!`);
resumeScan("Hold up the next item…");
})
.catch((err) => {
$("confirm-status").textContent = `Could not add: ${err.message}`;
});
});
// --- Consume article (book out) ------------------------------------------------
let consumeChoice = null;
let consumeUnitsPerPackage = 1;
function startConsume() {
$("consume-result").hidden = true;
$("consume-status").textContent = "";
startScan("consume", onConsumeProposal, "Hold up what you are about to use…");
}
function onConsumeProposal(proposal) {
$("consume-result").hidden = false;
$("consume-amount").hidden = true;
$("consume-status").textContent = "";
consumeUnitsPerPackage = proposal.units_per_package || 1;
if (proposal.degraded) {
$("consume-identified").textContent = "Could not identify that.";
$("consume-candidates").innerHTML =
'<p class="hint">Try again, or correct the amount by hand on the Edit inventory screen.</p>';
addRescanButton();
return;
}
el.innerHTML = items
.map((item) => {
const days = item.days_left;
const label =
days === null || days === undefined
? "no date"
: days < 0
? `expired ${-days}d ago`
: days === 0
? "expires today"
: `${days}d left`;
return `<div class="card ${urgencyClass(days)}">
<span class="card-name">${escapeHtml(item.name)}</span>
<span class="card-amount">×${escapeHtml(String(item.amount ?? ""))}</span>
<span class="card-due">${escapeHtml(label)}</span>
const matches = proposal.stock_matches || [];
$("consume-identified").textContent = `Looks like: ${proposal.name}`;
if (!matches.length) {
$("consume-candidates").innerHTML =
'<p class="hint">Nothing matching that is booked in, so there is nothing to book out. ' +
"If it should be in stock, add it on the Unload screen first.</p>";
addRescanButton();
return;
}
// One unambiguous match goes straight to the amount step; several mean the camera
// knows the kind but not the brand, and picking one at random here would be exactly
// the silent wrong write this whole service is arranged to avoid.
const exact = matches.filter((m) => m.exact);
if (exact.length === 1) {
chooseConsumeRow(exact[0]);
return;
}
if (matches.length === 1) {
chooseConsumeRow(matches[0]);
return;
}
$("consume-candidates").innerHTML =
'<p class="hint">Which one? (Same kind, different brands or dates.)</p>' +
matches
.map(
(m, i) => `<button class="card choice" data-index="${i}">
<span class="card-name">${escapeHtml(rowLabel(m))}</span>
<span class="card-due">${escapeHtml(dueLabel(m.days_left))}</span>
</button>`
)
.join("");
$("consume-candidates")
.querySelectorAll("button.choice")
.forEach((btn) => btn.addEventListener("click", () => chooseConsumeRow(matches[Number(btn.dataset.index)])));
}
function addRescanButton() {
const btn = document.createElement("button");
btn.className = "big-btn secondary";
btn.textContent = "Scan again";
btn.addEventListener("click", () => {
$("consume-result").hidden = true;
resumeScan("Hold up what you are about to use…");
});
$("consume-candidates").appendChild(btn);
}
function chooseConsumeRow(row) {
consumeChoice = row;
$("consume-candidates").innerHTML = "";
$("consume-amount").hidden = false;
$("consume-chosen").textContent = `${row.name} — ${amountOf(row)} in stock, ${dueLabel(row.days_left)}`;
// "If multipack, ask how many": in stock terms a multipack is simply a row holding
// more than one unit, since everything was booked in as individual units. The quick
// buttons are the answers people actually give — one, the whole pack, or all of it.
const held = amountOf(row);
const quick = [1];
if (consumeUnitsPerPackage > 1 && consumeUnitsPerPackage <= held) quick.push(consumeUnitsPerPackage);
if (held > 1 && !quick.includes(held)) quick.push(held);
$("consume-qty").value = 1;
$("consume-qty").max = held || 1;
$("consume-quick").innerHTML = quick
.map(
(n) =>
`<button type="button" class="quick-btn" data-amount="${n}">${
n === held ? `all ${n}` : n === consumeUnitsPerPackage && n > 1 ? `whole pack (${n})` : n
}</button>`
)
.join("");
$("consume-quick")
.querySelectorAll(".quick-btn")
.forEach((btn) =>
btn.addEventListener("click", () => {
$("consume-qty").value = btn.dataset.amount;
})
);
}
$("consume-cancel").addEventListener("click", () => {
consumeChoice = null;
$("consume-result").hidden = true;
resumeScan("Hold up what you are about to use…");
});
$("consume-confirm").addEventListener("click", () => {
if (!consumeChoice) return;
const amount = Number($("consume-qty").value);
if (!(amount > 0)) {
$("consume-status").textContent = "Enter how many.";
return;
}
$("consume-status").textContent = "Booking out…";
postJson("/consume", { product_id: consumeChoice.product_id, amount, spoiled: false })
.then(() => {
toast(`Booked out ${amount} × ${consumeChoice.name}`);
consumeChoice = null;
$("consume-result").hidden = true;
$("consume-status").textContent = "";
resumeScan("Hold up the next thing…");
})
.catch((err) => {
$("consume-status").textContent = `Could not book out: ${err.message}`;
});
});
// --- Expired foods -------------------------------------------------------------
let expiredRows = [];
function startExpired() {
scanner.hidden = true;
$("expired-status").textContent = "";
loadExpired();
}
function loadExpired() {
const el = $("expired-list");
api("/expired")
.then((data) => {
expiredRows = data.items || [];
if (!expiredRows.length) {
el.innerHTML = '<p class="hint">Nothing has expired. </p>';
return;
}
el.innerHTML = expiredRows
.map(
(row, i) => `<div class="card urgent-expired">
<span class="card-name">${escapeHtml(rowLabel(row))}</span>
<span class="card-due">${escapeHtml(dueLabel(row.days_left))}</span>
<button class="row-btn" data-index="${i}">Throw away</button>
</div>`
)
.join("");
el.querySelectorAll(".row-btn").forEach((btn) =>
btn.addEventListener("click", () => throwAway(expiredRows[Number(btn.dataset.index)]))
);
})
.catch((err) => {
el.innerHTML = `<p class="error">Could not load expired foods: ${escapeHtml(err.message)}</p>`;
});
}
// Scanning is the intended way to clear this list — you are standing at the bin with
// the thing in your hand — and the per-row button is the fallback for when the label
// is unreadable or the food no longer looks like itself.
$("expired-scan-btn").addEventListener("click", () => {
$("expired-status").textContent = "";
startScan("expired", onExpiredProposal, "Hold up what you are throwing away…");
});
function onExpiredProposal(proposal) {
if (proposal.degraded) {
$("expired-status").textContent = "Could not identify that — use the Throw away button on the row instead.";
resumeScan("Try another item…");
return;
}
const matches = (proposal.stock_matches || []).filter((m) =>
expiredRows.some((row) => row.product_id === m.product_id)
);
if (!matches.length) {
$("expired-status").textContent = `${proposal.name} is not on the expired list — nothing removed.`;
resumeScan("Hold up the next one…");
return;
}
throwAway(matches[0]);
}
function throwAway(row) {
if (!row) return;
const amount = amountOf(row) || 1;
// One tap between the camera and a write, same as everywhere else here — with the
// amount stated, because "throw away" on a row holding twelve means all twelve.
if (!window.confirm(`Throw away ${amount} × ${row.name}?`)) {
resumeScan("Hold up the next one…");
return;
}
$("expired-status").textContent = "Removing…";
postJson("/consume", { product_id: row.product_id, amount, spoiled: true })
.then(() => {
toast(`Binned ${amount} × ${row.name}`);
$("expired-status").textContent = "";
loadExpired();
resumeScan("Hold up the next one…");
})
.catch((err) => {
$("expired-status").textContent = `Could not remove: ${err.message}`;
});
}
// --- Edit inventory ------------------------------------------------------------
// The only screen that shows the folded view and its per-brand rows together: "22
// eggs" is the number the household thinks in, and "10 of brand Y" is the number you
// need to be able to correct exactly. Hence a <details> per kind rather than two
// separate screens.
let knownLocations = [];
function loadEditInventory() {
const el = $("edit-list");
$("edit-status").textContent = "";
api("/inventory")
.then((data) => {
knownLocations = data.locations || [];
const groups = data.groups || [];
if (!groups.length) {
el.innerHTML = '<p class="hint">Nothing booked in yet.</p>';
return;
}
el.innerHTML = groups.map(renderEditGroup).join("");
wireEditRows(el);
})
.catch((err) => {
el.innerHTML = `<p class="error">Could not load inventory: ${escapeHtml(err.message)}</p>`;
});
}
function renderEditGroup(group) {
const brands = group.brand_count > 1 ? ` · ${group.brand_count} brands` : "";
return `<details class="group ${urgencyClass(group.soonest_days_left)}">
<summary>
<span class="card-name">${escapeHtml(group.display_name || group.kind)}</span>
<span class="card-amount">${escapeHtml(String(group.total_amount))}${escapeHtml(brands)}</span>
<span class="card-due">${escapeHtml(dueLabel(group.soonest_days_left))}</span>
</summary>
${(group.entries || []).map(renderEditRow).join("")}
</details>`;
}
function renderEditRow(row) {
return `<div class="edit-row" data-product="${row.product_id}">
<span class="card-name">${escapeHtml(row.brand || row.name)}</span>
<span class="card-due">${escapeHtml(dueLabel(row.days_left))}</span>
<button class="step-btn" data-step="-1">−</button>
<input class="amount-input" type="number" min="0" step="1" value="${escapeHtml(String(amountOf(row)))}">
<button class="step-btn" data-step="1">+</button>
<button class="row-btn save-btn">Save</button>
</div>
${renderWhereRow(row)}`;
}
// Where a thing is, in two registers that are deliberately not merged: the location
// Grocy records (which somebody chose, and which "Move" changes) and the doorway
// camera's last sighting (which nobody chose, and which is only ever a hint — see
// docs/fridge-item-location.md). Showing the sighting as a sentence with a time in it
// is what keeps it readable as a clue rather than as a fact.
function renderWhereRow(row) {
if (!knownLocations.length && !row.last_seen) return "";
const seen = row.last_seen
? `Camera last saw something like this at ${escapeHtml(row.last_seen.location_name)}, ${escapeHtml(
relativeTime(row.last_seen.created_at)
)} — a sighting, not a fact.`
: "";
const options = knownLocations
.map(
(loc) =>
`<option value="${loc.id}"${loc.id === row.location_id ? " selected" : ""}>${escapeHtml(loc.name)}</option>`
)
.join("");
return `<div class="where-row" data-product="${row.product_id}">
<span class="hint">${seen}</span>
${
knownLocations.length
? `<label class="where-move">Move to
<select class="where-select">${options}</select>
</label>
<button class="row-btn move-btn">Move</button>`
: ""
}
</div>`;
}
function relativeTime(iso) {
const then = Date.parse(iso || "");
if (!Number.isFinite(then)) return "at an unknown time";
const minutes = Math.round((Date.now() - then) / 60000);
if (minutes < 1) return "just now";
if (minutes < 60) return `${minutes} min ago`;
const hours = Math.round(minutes / 60);
if (hours < 24) return `${hours}h ago`;
return `${Math.round(hours / 24)}d ago`;
}
function wireEditRows(root) {
root.querySelectorAll(".edit-row").forEach((rowEl) => {
const input = rowEl.querySelector(".amount-input");
rowEl.querySelectorAll(".step-btn").forEach((btn) =>
btn.addEventListener("click", () => {
const next = (Number(input.value) || 0) + Number(btn.dataset.step);
input.value = Math.max(0, next);
// +/- write immediately: a step button that needs a second tap on Save is a
// step button people will forget to save. The freeform field waits for Save,
// because it is mid-typing until then.
saveEditRow(rowEl);
})
);
rowEl.querySelector(".save-btn").addEventListener("click", () => saveEditRow(rowEl));
});
root.querySelectorAll(".where-row").forEach((whereEl) => {
const moveBtn = whereEl.querySelector(".move-btn");
if (!moveBtn) return;
moveBtn.addEventListener("click", () => {
const productId = Number(whereEl.dataset.product);
const rowEl = root.querySelector(`.edit-row[data-product="${productId}"]`);
const amount = Number(rowEl && rowEl.querySelector(".amount-input").value);
if (!(amount > 0)) {
$("edit-status").textContent = "Nothing to move — the amount is zero.";
return;
}
$("edit-status").textContent = "Moving…";
postJson("/transfer", {
product_id: productId,
amount,
to_location_id: Number(whereEl.querySelector(".where-select").value),
})
.then((result) => {
$("edit-status").textContent = "";
toast(result.unchanged ? "Already there" : "Moved");
loadEditInventory();
})
.catch((err) => {
$("edit-status").textContent = `Could not move: ${err.message}`;
});
});
});
}
function saveEditRow(rowEl) {
const productId = Number(rowEl.dataset.product);
const amount = Number(rowEl.querySelector(".amount-input").value);
if (!(amount >= 0)) {
$("edit-status").textContent = "Amount has to be zero or more.";
return;
}
$("edit-status").textContent = "Saving…";
postJson("/adjust", { product_id: productId, amount })
.then(() => {
$("edit-status").textContent = "";
toast("Corrected");
loadEditInventory();
})
.catch((err) => {
$("edit-status").textContent = `Could not save: ${err.message}`;
});
}
// --- Inventory (read-only) -----------------------------------------------------
function loadInventory() {
const el = $("inventory-list");
api("/inventory")
.then((data) => {
const groups = data.groups || [];
if (!groups.length) {
el.innerHTML = '<p class="hint">Nothing in stock yet — unload some groceries!</p>';
return;
}
el.innerHTML = groups
.map((group) => {
const brands = group.brand_count > 1 ? `${group.brand_count} brands` : "";
return `<div class="card ${urgencyClass(group.soonest_days_left)}">
<span class="card-name">${escapeHtml(group.display_name || group.kind)}</span>
<span class="card-amount">×${escapeHtml(String(group.total_amount))} ${escapeHtml(brands)}</span>
<span class="card-due">${escapeHtml(dueLabel(group.soonest_days_left))}</span>
</div>`;
})
.join("");
@ -196,7 +810,7 @@ function loadInventory() {
// --- Recipes -----------------------------------------------------------------
function loadRecipes() {
const el = document.getElementById("recipes-list");
const el = $("recipes-list");
api("/recipes")
.then((data) => {
const recipes = data.recipes || [];
@ -219,9 +833,7 @@ function loadRecipes() {
});
}
function escapeHtml(s) {
return String(s).replace(/[&<>"']/g, (c) => ({ "&": "&amp;", "<": "&lt;", ">": "&gt;", '"': "&quot;", "'": "&#39;" }[c]));
}
// Start on whatever tab is active (default: scan).
resetScan();
// A "Show <screen>" MQTT command from HA arrives as a URL fragment reload
// (kitchen-display-agent kills and relaunches Chromium at index.html#inventory, same
// pattern as the thin client's digest-browser) — honour it on load, same as any tap.
show(location.hash ? location.hash.slice(1) : "home");

117
pantry-vision/frontend/index.html
View File

@ -14,25 +14,53 @@
as URL query params from the kitchen-display kiosk's own launch command
(hosts/kitchen-display/configs/sway/pantry-kiosk), never baked into this file, so
this stays a plain static asset with no secret in git — see app.js's top comment.
Four stock movements, four screens, all reached from Home and all camera-first
except the last: Unload groceries (book in), Consume article (book out), List
expired foods (throw away), Edit inventory (the keyboard fallback for when the
camera got it wrong). #scanner below is ONE camera element, moved into whichever
flow is running — see app.js's mountScanner(); a second getUserMedia stream on a
single-webcam kiosk is a black rectangle, not a second camera.
-->
<nav id="tabs">
<button class="tab active" data-tab="scan">📷<span>Scan</span></button>
<button class="tab active" data-tab="home">🏠<span>Home</span></button>
<button class="tab" data-tab="inventory">🧊<span>Inventory</span></button>
<button class="tab" data-tab="recipes">🍳<span>Recipes</span></button>
</nav>
<main>
<section id="scan" class="panel active">
<div id="scan-camera">
<section id="home" class="panel active">
<div class="action-grid">
<button class="action-btn" data-goto="unload">🛍️<span>Unload groceries</span></button>
<button class="action-btn" data-goto="consume">🍽️<span>Consume article</span></button>
<button class="action-btn" data-goto="expired">🗑️<span>List expired foods</span></button>
<button class="action-btn" data-goto="edit">✏️<span>Edit inventory</span></button>
</div>
</section>
<!-- The camera, mounted into whichever flow is active. -->
<div id="scanner" hidden>
<video id="camera-preview" autoplay playsinline muted></video>
<button id="capture-btn" class="big-btn">Capture</button>
<p id="scan-status" class="hint">Starting camera…</p>
<p id="camera-error" class="error" hidden></p>
<button id="capture-now" class="big-btn secondary">Identify now</button>
<canvas id="captured-frame" hidden></canvas>
<canvas id="frame-sample" hidden></canvas>
</div>
<div id="scan-result" hidden>
<canvas id="captured-frame"></canvas>
<form id="confirm-form">
<label>Name <input id="f-name" type="text" required></label>
<!-- ---------------------------------------------------------------- Unload -->
<section id="unload" class="panel flow">
<header class="flow-head">
<button class="back-btn" data-goto="home">‹ Home</button>
<h1>Unload groceries</h1>
<span id="unload-count" class="flow-badge"></span>
</header>
<div class="scanner-slot"></div>
<form id="confirm-form" hidden>
<p id="f-description" class="lead"></p>
<label>Article <input id="f-name" type="text" required></label>
<div class="field-row">
<label>Category
<select id="f-category">
<option value="produce">Produce</option>
@ -45,18 +73,83 @@
<option value="other">Other</option>
</select>
</label>
<label>Put it in
<select id="f-placement">
<option value="fridge">🧊 Fridge</option>
<option value="freezer">❄️ Freezer</option>
<option value="cupboard">🚪 Cupboard</option>
<option value="counter">🧺 Counter</option>
</select>
</label>
</div>
<label>Best before <input id="f-date" type="date" required></label>
<label>Quantity <input id="f-quantity" type="number" min="1" step="1" value="1"></label>
<p id="f-date-source" class="hint"></p>
<div class="field-row">
<label>Packs <input id="f-packages" type="number" min="1" step="1" value="1"></label>
<label>Per pack <input id="f-units" type="number" min="1" step="1" value="1"></label>
</div>
<p id="f-total" class="total-line"></p>
<p id="f-have" class="hint"></p>
<p id="f-confidence" class="hint"></p>
<div class="form-actions">
<button type="button" id="retake-btn" class="big-btn secondary">Retake</button>
<button type="submit" id="confirm-btn" class="big-btn primary">Confirm &amp; add</button>
<button type="button" id="skip-btn" class="big-btn secondary">Skip</button>
<button type="submit" id="confirm-btn" class="big-btn primary">Confirm &amp; next</button>
</div>
<p id="confirm-status" class="hint"></p>
</form>
</section>
<!-- --------------------------------------------------------------- Consume -->
<section id="consume" class="panel flow">
<header class="flow-head">
<button class="back-btn" data-goto="home">‹ Home</button>
<h1>Consume article</h1>
</header>
<div class="scanner-slot"></div>
<div id="consume-result" hidden>
<p id="consume-identified" class="lead"></p>
<div id="consume-candidates" class="card-list"></div>
<div id="consume-amount" hidden>
<p id="consume-chosen" class="lead"></p>
<div id="consume-quick" class="quick-row"></div>
<label>How many? <input id="consume-qty" type="number" min="1" step="1" value="1"></label>
<div class="form-actions">
<button type="button" id="consume-cancel" class="big-btn secondary">Cancel</button>
<button type="button" id="consume-confirm" class="big-btn primary">Book out</button>
</div>
</div>
<p id="consume-status" class="hint"></p>
</div>
</section>
<!-- --------------------------------------------------------------- Expired -->
<section id="expired" class="panel flow">
<header class="flow-head">
<button class="back-btn" data-goto="home">‹ Home</button>
<h1>Expired foods</h1>
<button id="expired-scan-btn" class="flow-action">📷 Scan to remove</button>
</header>
<div class="scanner-slot"></div>
<p id="expired-status" class="hint"></p>
<div id="expired-list" class="card-list"><p class="hint">Loading…</p></div>
</section>
<!-- ------------------------------------------------------------------ Edit -->
<section id="edit" class="panel flow">
<header class="flow-head">
<button class="back-btn" data-goto="home">‹ Home</button>
<h1>Edit inventory</h1>
</header>
<p class="hint">
Everything booked in, same kind added up across brands. Open a line to correct one
brand exactly.
</p>
<p id="edit-status" class="hint"></p>
<div id="edit-list" class="card-list"><p class="hint">Loading…</p></div>
</section>
<!-- ------------------------------------------------------------- Reference -->
<section id="inventory" class="panel">
<div id="inventory-list" class="card-list">
<p class="hint">Loading…</p>
@ -70,6 +163,8 @@
</section>
</main>
<p id="toast" hidden></p>
<script src="app.js"></script>
</body>
</html>

258
pantry-vision/frontend/style.css
View File

@ -5,6 +5,14 @@
box-sizing: border-box;
}
/* Several elements below are flex/grid containers that the JS shows and hides with
* the `hidden` attribute. A `display:` rule in a stylesheet outranks the user agent's
* own `[hidden] { display: none }`, so without this every one of them would be
* permanently visible — the camera included. */
[hidden] {
display: none !important;
}
html, body {
margin: 0;
height: 100%;
@ -58,21 +66,262 @@ main {
display: block;
}
/* --- Scan ------------------------------------------------------------------- */
#scan-camera, #scan-result {
/* --- Home: the four stock movements ------------------------------------------ */
.action-grid {
display: grid;
grid-template-columns: repeat(2, 1fr);
gap: 12px;
}
.action-btn {
display: flex;
flex-direction: column;
align-items: center;
gap: 16px;
justify-content: center;
gap: 8px;
min-height: 140px;
padding: 12px;
border: none;
border-radius: 16px;
background: rgba(110, 168, 254, 0.16);
color: #e8e8ec;
font-size: 34px;
}
#camera-preview, #captured-frame {
.action-btn span {
font-size: 16px;
font-weight: 600;
text-align: center;
}
.action-btn:active {
background: rgba(110, 168, 254, 0.3);
}
/* --- Flow screens ------------------------------------------------------------ */
.flow-head {
display: flex;
align-items: center;
gap: 12px;
margin-bottom: 12px;
}
.flow-head h1 {
flex: 1;
margin: 0;
font-size: 20px;
}
.back-btn, .flow-action {
min-height: 44px;
padding: 0 14px;
border: none;
border-radius: 10px;
background: rgba(255, 255, 255, 0.14);
color: #e8e8ec;
font-size: 15px;
}
.flow-badge {
font-size: 14px;
color: #9a9aa6;
}
/* --- Scan ------------------------------------------------------------------- */
#scanner {
display: flex;
flex-direction: column;
align-items: center;
gap: 10px;
margin-bottom: 16px;
}
#camera-preview {
width: 100%;
max-width: 640px;
max-height: 38vh;
object-fit: cover;
border-radius: 12px;
background: #000;
}
.lead {
width: 100%;
max-width: 640px;
margin: 0;
font-size: 17px;
font-weight: 600;
}
.total-line {
margin: 0;
font-size: 16px;
font-weight: 600;
color: #6ea8fe;
}
.field-row {
display: flex;
gap: 12px;
}
.field-row label {
flex: 1;
}
.quick-row {
display: flex;
flex-wrap: wrap;
gap: 8px;
margin: 12px 0;
}
.quick-btn {
min-height: 48px;
padding: 0 18px;
border: 1px solid rgba(255, 255, 255, 0.16);
border-radius: 10px;
background: #1a1a20;
color: #e8e8ec;
font-size: 16px;
}
#consume-result, #consume-amount {
width: 100%;
max-width: 640px;
margin: 0 auto;
display: flex;
flex-direction: column;
gap: 12px;
}
#consume-amount label {
display: flex;
flex-direction: column;
gap: 4px;
font-size: 14px;
color: #9a9aa6;
}
#consume-amount input {
min-height: 48px;
font-size: 16px;
padding: 8px 12px;
border-radius: 8px;
border: 1px solid rgba(255, 255, 255, 0.16);
background: #1a1a20;
color: #e8e8ec;
}
/* --- Edit inventory ---------------------------------------------------------- */
.group {
border-radius: 10px;
background: rgba(255, 255, 255, 0.06);
overflow: hidden;
}
.group > summary {
display: flex;
align-items: center;
gap: 12px;
min-height: 56px;
padding: 8px 16px;
cursor: pointer;
}
.edit-row {
display: flex;
align-items: center;
gap: 8px;
padding: 8px 16px;
border-top: 1px solid rgba(255, 255, 255, 0.08);
}
.step-btn {
min-width: 52px;
min-height: 48px;
border: none;
border-radius: 10px;
background: rgba(255, 255, 255, 0.14);
color: #e8e8ec;
font-size: 22px;
}
.amount-input {
width: 84px;
min-height: 48px;
text-align: center;
font-size: 16px;
border-radius: 8px;
border: 1px solid rgba(255, 255, 255, 0.16);
background: #1a1a20;
color: #e8e8ec;
}
.where-row {
display: flex;
align-items: center;
flex-wrap: wrap;
gap: 8px;
padding: 0 16px 10px;
}
.where-row .hint {
flex: 1;
min-width: 180px;
}
.where-move {
display: flex;
align-items: center;
gap: 6px;
font-size: 14px;
color: #9a9aa6;
}
.where-select {
min-height: 44px;
font-size: 15px;
padding: 4px 10px;
border-radius: 8px;
border: 1px solid rgba(255, 255, 255, 0.16);
background: #1a1a20;
color: #e8e8ec;
}
.row-btn {
min-height: 48px;
padding: 0 14px;
border: none;
border-radius: 10px;
background: #6ea8fe;
color: #101014;
font-size: 15px;
font-weight: 600;
}
.card.choice {
width: 100%;
border: none;
text-align: left;
color: #e8e8ec;
}
/* --- Toast ------------------------------------------------------------------- */
#toast {
position: fixed;
left: 50%;
bottom: 24px;
transform: translateX(-50%);
margin: 0;
padding: 14px 22px;
border-radius: 12px;
background: rgba(20, 20, 26, 0.96);
border: 1px solid rgba(255, 255, 255, 0.16);
font-size: 16px;
max-width: 90vw;
}
.big-btn {
min-height: 64px;
min-width: 200px;
@ -96,6 +345,7 @@ main {
#confirm-form {
width: 100%;
max-width: 640px;
margin: 0 auto;
display: flex;
flex-direction: column;
gap: 12px;

62
pantry-vision/pantry-vision.env.example
View File

@ -31,18 +31,74 @@ OLLAMA_TIMEOUT=120
# network at this internal name/port regardless of ENABLE_PANTRY_VISION.
# ---------------------------------------------------------------------------
GROCY_URL=http://grocy:80
# Settings -> Manage API keys, inside Grocy's own UI. Required for /confirm (writes);
# /inventory and /recipes (reads) use the same key.
# Settings -> Manage API keys, inside Grocy's own UI. Required for every write
# (/confirm, /consume, /adjust); the reads use the same key.
GROCY_API_KEY=
# VERIFY against your own Grocy instance (Settings -> Locations / Quantity units) —
# these are fresh-install defaults ("Default" location, "Piece" unit), not guaranteed
# to match a Grocy that's already been customised.
# to match a Grocy that's already been customised. GROCY_DEFAULT_QU_ID is the unit
# stock is counted in, and pantry-vision counts INDIVIDUAL ITEMS, never packages: a
# twelve-pack of eggs is booked in as twelve. Point this at a "Piece"-like unit, not
# at "Pack".
GROCY_DEFAULT_LOCATION_ID=1
GROCY_DEFAULT_QU_ID=2
# ---------------------------------------------------------------------------
# Where things get put away. The confirm screen's fridge/freezer/cupboard/counter
# choice is resolved to a Grocy location BY NAME, creating it if it doesn't exist —
# so if your Grocy already calls them something else ("Kühlschrank", "Vorratskammer"),
# put those names here. A mismatch doesn't error, it quietly creates a second,
# duplicate location, which is the kind of thing you only notice a month later.
#
# With more than one cold appliance, name them apart here ("Fridge (kitchen)") and
# list every one of them in PANTRY_DOOR_APPLIANCES below — see
# docs/fridge-item-location.md.
# ---------------------------------------------------------------------------
PANTRY_LOCATION_FRIDGE=Fridge
PANTRY_LOCATION_FREEZER=Freezer
PANTRY_LOCATION_CUPBOARD=Cupboard
PANTRY_LOCATION_COUNTER=Counter
# ---------------------------------------------------------------------------
# Door-sensor-triggered appliance cameras (optional; off while unset).
#
# Format: id:Grocy location name:frigate_camera , ...
# The camera is optional — an appliance with only a door sensor still records that
# it was opened, which is the half of this feature that pays for itself.
#
# A Zigbee contact sensor on each door drives an HA automation that POSTs
# {"appliance": "<id>", "state": "opened"} to this service's /doorway-event. The
# service then pulls a short burst from that camera and records what it recognised
# as a HINT — with a timestamp and a confidence, in its own SQLite file, NEVER into
# Grocy stock. A camera at a door cannot tell in from out, misses two-items-at-once,
# and sees nothing behind an arm; read docs/fridge-item-location.md before treating
# any of it as authoritative. Moving stock between appliances is /transfer, which a
# person taps.
# ---------------------------------------------------------------------------
PANTRY_DOOR_APPLIANCES=
# Example:
# PANTRY_DOOR_APPLIANCES=fridge-kitchen:Fridge (kitchen):cam_fridge_kitchen,freezer-loggia:Freezer (loggia):cam_freezer_loggia
# The same Frigate this project's `chores` already pulls snapshots from. Unset means
# door events are still recorded, just without a picture.
FRIGATE_URL=
FRIGATE_TIMEOUT=15
# Frames per door event, and the gap between them. An item crosses a doorway in about
# a second: one frame is a coin toss, ten is a queue at the vision model. The burst
# stops early on the first frame that identifies something.
PANTRY_DOORWAY_BURST=3
PANTRY_DOORWAY_BURST_INTERVAL=0.7
# Hints are worthless once stale, and are pruned on write.
PANTRY_HINT_RETENTION_DAYS=30
PANTRY_HINTS_DB_PATH=/data/pantry-hints.db
# ---------------------------------------------------------------------------
# Run behaviour
# ---------------------------------------------------------------------------
PANTRY_VISION_PORT=8095
PANTRY_VISION_MAX_IMAGE_MB=15
# Upper bound on the vision model's claimed pack size, since that number gets
# multiplied into stock. It is editable on screen before anything is written; this is
# only the guard against a model that answers "units_per_package": 100000.
PANTRY_MAX_UNITS_PER_PACKAGE=240
LOG_LEVEL=INFO

930
pantry-vision/server.py
View File

File diff suppressed because it is too large Load Diff

76
render/README.md Normal file
View File

@ -0,0 +1,76 @@
# render/ — shared frontend surfaces
Two pages that every endpoint can show, kept here rather than inside one component
because they belong to no single service: the now-playing screen reads Home Assistant,
the floorplan reads `identity`, and both are shown on thin clients, touch panels, the
kitchen and door panels alike.
Same rules as `digest-canvas-sdk` and `canvas-sdk`: **vendored, dependency-free, no
build step**, config from `?query=params` so nothing here holds a secret. Copy the
directory into a host's image; do not serve it from one place, or a kiosk that cannot
reach the container host shows nothing.
| | What | Reads |
|---|---|---|
| `media-visualiser/` | now-playing: circular spectrum, album-art colours, lyrics | HA `media_player` state |
| `floorplan-3d/` | who is in which room, extruded | `identity` `GET /floorplan/presence` |
## media-visualiser
```
nowplaying.html?ha=http://ha:8123&token=<HA token>&entity=media_player.living_room
```
**Two tiers, and the distinction is load-bearing.** CAVA reads an audio stream; most
endpoints do not have one — a kitchen panel showing what the *living room* is playing
has no audio to analyse and never will. So:
- **reactive** — audio is local. Real FFT via WebAudio (`attachAudio(el)`), or levels
pushed from `cava -r` by the host's agent (`pushLevels([...])`).
- **synthetic** — everywhere else. The ring breathes from track position and tempo.
The synthetic tier **says so on screen** ("visual rhythm — not an audio analysis") and
sets `data-tier` on the canvas. That matters: the moment somebody believes it is a
spectrum, every bass drop it misses becomes a bug report.
Palette extraction rejects near-greys and near-blacks *before* ranking (album art is
full of both, and a naive palette off a dark cover is four indistinguishable greys),
then lifts each colour until it clears the background — the step that gets skipped, and
why so many art-coloured visualisers are invisible on dark covers.
Lyrics: synced LRC highlights and scrolls; **plain text is shown but never
auto-scrolled at a guessed rate**, which is wrong within ten seconds and stays wrong.
Most tracks have none, so the layout treats absent as the normal case.
## floorplan-3d
```
floorplan.html?api=http://identity:8097&token=<identity token>[&level=<id>]
```
**Canvas 2D, not three.js** — a change from what `docs/endpoint-surfaces.md` planned,
made while building it. The scene is prisms on a plane: no lighting model, no textures,
no model import. An isometric projection with painter's-algorithm sorting draws exactly
that in ~200 lines, runs predictably on the small panels (where a WebGL context is much
less predictable), and keeps the frontend at zero dependencies. If this ever needs real
lighting or an imported model, three.js becomes right and this becomes the fallback.
Inherited from the Pebble watchapp deliberately, because they render the same payload
and divergence would mean one of them is lying:
- Occupied rooms **lit**, empty rooms **dark** — carried in lightness, not hue, because
colour already means *who*.
- A **third state**, dashed, for rooms HA never reports on. Drawing "no data" as
"empty" is a quiet lie.
- Occupants as **colour + initial**, upgraded to their photo where there is room, with
the colour ring kept either way — it is what ties this marker to the same person on
the watch and in the admin panel.
- `unplaced` people get a **visible shelf**. They are who you are most often looking for.
Drag to orbit; pitch is clamped and there is no free-fly, because an unconstrained
camera on a wall panel is something people knock askew and cannot get back.
**The caveat no rendering fixes:** this is only as right as room-level presence, which
has never been measured in this house. A beautiful 3D house with everybody on the
unplaced shelf is a beautiful picture of nothing — test that first with the admin
panel's floorplan tab, **Live** ticked.

94
render/floorplan-3d/floorplan-app.js Normal file
View File

@ -0,0 +1,94 @@
/*
* Wires floorplan3d.js to identity's GET /floorplan/presence.
*
* One request gives the plan AND who is in it, so there is no second call and no window
* where the rooms are drawn but the people are missing.
*/
"use strict";
const params = new URLSearchParams(location.search);
const API = (params.get("api") || "").replace(/\/$/, "");
const TOKEN = params.get("token") || "";
const POLL_MS = Number(params.get("poll") || 15000);
const $ = (id) => document.getElementById(id);
if (!API || !TOKEN) {
$("error").hidden = false;
$("error").textContent = "Not configured — needs ?api=&token= in the URL.";
throw new Error("floorplan: missing ?api=/&token=");
}
const plan = new Floorplan3D($("plan"), {
// identity serves the picture; this builds the URL rather than the SDK knowing about
// any particular service.
photoUrl: (personId) => `${API}/people/${personId}/photo?token=${encodeURIComponent(TOKEN)}`,
});
let levels = [];
let activeLevelId = params.get("level") ? Number(params.get("level")) : null;
function load() {
fetch(`${API}/floorplan/presence`, { headers: { Authorization: `Bearer ${TOKEN}` } })
.then((res) => {
if (!res.ok) throw new Error(`${res.status} ${res.statusText}`);
return res.json();
})
.then((data) => {
$("error").hidden = true;
levels = data.levels || [];
if (activeLevelId === null && levels.length) activeLevelId = levels[0].id;
renderLevelButtons();
const level = levels.find((l) => l.id === activeLevelId) || levels[0] || null;
$("level-name").textContent = level ? level.name : "No levels drawn yet";
plan.setLevel(level, data.unplaced || []);
// The honest count, in the same sentence: placed people and the ones the system
// cannot locate. A view that silently omits the second number implies it knows
// where everybody is.
const placed = (level ? level.rooms || [] : []).reduce(
(n, room) => n + (room.occupants || []).length, 0
);
const unplaced = (data.unplaced || []).length;
$("summary").textContent = unplaced
? `${placed} placed · ${unplaced} home but not locatable`
: `${placed} placed`;
})
.catch((err) => {
$("error").hidden = false;
$("error").textContent = `identity: ${err.message}`;
})
.finally(() => setTimeout(load, POLL_MS));
}
function renderLevelButtons() {
if (levels.length < 2) {
$("levels").innerHTML = "";
return;
}
$("levels").innerHTML = levels
.map(
(l) =>
`<button data-id="${l.id}" class="${l.id === activeLevelId ? "active" : ""}">${escapeHtml(
l.name
)}</button>`
)
.join("");
$("levels")
.querySelectorAll("button")
.forEach((btn) =>
btn.addEventListener("click", () => {
activeLevelId = Number(btn.dataset.id);
load();
})
);
}
function escapeHtml(s) {
return String(s ?? "").replace(/[&<>"']/g, (c) =>
({ "&": "&amp;", "<": "&lt;", ">": "&gt;", '"': "&quot;", "'": "&#39;" }[c])
);
}
load();

32
render/floorplan-3d/floorplan.html Normal file
View File

@ -0,0 +1,32 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no">
<title>Where everyone is</title>
<link rel="stylesheet" href="floorplan3d.css">
</head>
<body>
<!--
The floorplan, extruded, with who is in each room. Same GET /floorplan/presence the
Pebble watchapp renders — at the fidelity a real screen allows.
Config from URL query params, never baked in:
?api=http://identity:8097&token=<identity token>[&level=<id>][&poll=15000]
Drag to orbit. Yaw-free, pitch clamped, no free-fly: an unconstrained camera on a
wall panel is something people knock askew and cannot get back.
-->
<header id="bar">
<h1 id="level-name">Loading…</h1>
<nav id="levels"></nav>
<span id="summary" class="muted"></span>
</header>
<canvas id="plan"></canvas>
<p id="error" class="error" hidden></p>
<script src="floorplan3d.js"></script>
<script src="floorplan-app.js"></script>
</body>
</html>

56
render/floorplan-3d/floorplan3d.css Normal file
View File

@ -0,0 +1,56 @@
/* floorplan-3d styling. Dark, because an occupied room is drawn LIGHT — the contrast
* between a lit room and its surroundings is the entire signal, and it only works
* against a dark stage. */
* { box-sizing: border-box; }
html, body {
margin: 0;
height: 100%;
background: #0a0510;
color: #eadcff;
font-family: system-ui, sans-serif;
overflow: hidden;
}
#bar {
display: flex;
align-items: baseline;
gap: 14px;
padding: 12px 18px;
border-bottom: 1px solid rgba(200, 120, 255, 0.2);
}
#bar h1 { margin: 0; font-size: 18px; }
.muted { color: #a98fc4; font-size: 14px; margin-left: auto; }
#levels { display: flex; gap: 6px; }
#levels button {
min-height: 32px;
padding: 2px 12px;
border-radius: 8px;
border: 1px solid rgba(200, 120, 255, 0.24);
background: transparent;
color: #a98fc4;
font-size: 13px;
cursor: pointer;
}
#levels button.active { background: rgba(200, 120, 255, 0.18); color: #eadcff; }
#plan {
display: block;
width: 100%;
height: calc(100% - 54px);
touch-action: none; /* the canvas owns drag-to-orbit */
cursor: grab;
}
#plan:active { cursor: grabbing; }
.error {
position: absolute;
bottom: 10px;
left: 0;
right: 0;
text-align: center;
color: #ff8080;
font-size: 14px;
}

377
render/floorplan-3d/floorplan3d.js Normal file
View File

@ -0,0 +1,377 @@
/*
* floorplan-3d — the drawn floorplan, extruded, with who is in each room.
*
* The same `GET /floorplan/presence` payload the Pebble watchapp renders, at the
* fidelity a real screen allows. If the two ever disagree, one of them is lying.
*
* WHY THIS IS CANVAS 2D AND NOT THREE.JS
* ---------------------------------------
* docs/endpoint-surfaces.md planned to vendor three.js and called it a deliberate break
* with the dependency-free rule. Building it made the cheaper answer obvious, so this
* takes it instead:
*
* - The scene is prisms standing on a plane. There is no camera motion beyond an
* orbit, no lighting model worth the name, no textures, no physics. An isometric
* projection with painter's-algorithm sorting produces exactly that picture in
* ~200 lines of canvas 2D.
* - The door panel and kitchen panel are small machines. Canvas 2D redraws this in
* under a millisecond; a WebGL context on integrated graphics is a much less
* predictable proposition, and "the floorplan is smooth on the TV and juddery on
* the panel" is a bad outcome for a view whose whole job is a glance.
* - A megabyte of vendored library is a megabyte to keep patched, and it would be the
* only dependency in the entire frontend.
*
* If this ever grows real lighting or a model import, three.js becomes the right answer
* and this file becomes the fallback. Until then it is not a compromise, it is the
* smaller correct tool.
*
* WHAT IT INHERITS FROM THE WATCHAPP, DELIBERATELY
* -------------------------------------------------
* - Occupied rooms lit, empty rooms dark, and a THIRD state for rooms HA never
* reports on. Rendering "no data" as "empty" is a quiet lie on any screen.
* - Occupants as their colour + initial, upgraded to their photo where there is room.
* The colour ring stays even with a photo: it is what ties this marker to the same
* person on the watch and in the admin panel.
* - `unplaced` people get a visible shelf, not a hidden list. They are who you are
* most often looking for.
*/
"use strict";
(function (global) {
const DEG = Math.PI / 180;
class Floorplan3D {
/**
* @param {HTMLCanvasElement} canvas
* @param {object} options { wallHeight, pitch, yaw, photoUrl }
* photoUrl(personId) -> a URL for that person's picture, or null.
*/
constructor(canvas, options) {
this.canvas = canvas;
this.ctx = canvas.getContext("2d");
this.options = Object.assign(
{ wallHeight: 0.16, pitch: 34, yaw: 30, photoUrl: null },
options || {}
);
this.level = null;
this.unplaced = [];
this.photos = new Map();
this.hover = null;
this._resize();
window.addEventListener("resize", () => this._resize());
this._bindOrbit();
}
_resize() {
const rect = this.canvas.getBoundingClientRect();
const dpr = Math.min(window.devicePixelRatio || 1, 2);
this.canvas.width = Math.max(1, Math.round(rect.width * dpr));
this.canvas.height = Math.max(1, Math.round(rect.height * dpr));
this.ctx.setTransform(dpr, 0, 0, dpr, 0, 0);
this.width = rect.width;
this.height = rect.height;
this.draw();
}
/**
* Drag to orbit. Deliberately yaw-only with a clamped pitch, and NO free-fly: an
* unconstrained camera on a wall panel is something people knock askew and cannot
* get back, and there is no keyboard next to it to reset with.
*/
_bindOrbit() {
let dragging = false;
let lastX = 0;
let lastY = 0;
const start = (x, y) => {
dragging = true;
lastX = x;
lastY = y;
};
const move = (x, y) => {
if (!dragging) return;
this.options.yaw += (x - lastX) * 0.4;
this.options.pitch = Math.max(12, Math.min(70, this.options.pitch - (y - lastY) * 0.25));
lastX = x;
lastY = y;
this.draw();
};
const end = () => {
dragging = false;
};
this.canvas.addEventListener("pointerdown", (e) => start(e.clientX, e.clientY));
this.canvas.addEventListener("pointermove", (e) => move(e.clientX, e.clientY));
this.canvas.addEventListener("pointerup", end);
this.canvas.addEventListener("pointerleave", end);
}
/** Feed it a level out of GET /floorplan/presence, plus the payload's `unplaced`. */
setLevel(level, unplaced) {
this.level = level;
this.unplaced = unplaced || [];
if (this.options.photoUrl) this._preloadPhotos();
this.draw();
}
_preloadPhotos() {
const people = [];
for (const room of (this.level && this.level.rooms) || []) {
for (const person of room.occupants || []) people.push(person);
}
people.push(...this.unplaced);
for (const person of people) {
if (!person.has_photo || this.photos.has(person.id)) continue;
const url = this.options.photoUrl(person.id);
if (!url) continue;
const image = new Image();
image.crossOrigin = "anonymous";
image.onload = () => {
this.photos.set(person.id, image);
this.draw();
};
// A missing photo is not an error: the initial-on-colour fallback is the
// designed rendering, not a degraded one.
image.onerror = () => this.photos.set(person.id, null);
image.src = url;
this.photos.set(person.id, undefined); // in flight
}
}
/** Normalised plan coords (0..1, plus a height) -> screen pixels. */
_project(x, y, z) {
const yaw = this.options.yaw * DEG;
const pitch = this.options.pitch * DEG;
// Centre the plan on the origin so orbiting rotates the house rather than
// swinging it out of frame.
const cx = x - 0.5;
const cy = y - 0.5;
const rx = cx * Math.cos(yaw) - cy * Math.sin(yaw);
const ry = cx * Math.sin(yaw) + cy * Math.cos(yaw);
const scale = Math.min(this.width, this.height) * 0.78;
return {
x: this.width / 2 + rx * scale,
y: this.height / 2 + ry * scale * Math.sin(pitch) - (z || 0) * scale * Math.cos(pitch),
};
}
/** Depth key for painter's-algorithm sorting: further from the camera draws first. */
_depth(points) {
const yaw = this.options.yaw * DEG;
let sum = 0;
for (const [x, y] of points) {
sum += (x - 0.5) * Math.sin(yaw) + (y - 0.5) * Math.cos(yaw);
}
return sum / points.length;
}
_roomState(room) {
// THREE states, never two. A room the plan has drawn but whose area HA never
// reports is neither occupied nor confirmed-empty, and drawing it as empty is a
// quiet lie — "nobody is in the study" and "nothing can see the study" are
// different sentences.
if ((room.occupants || []).length) return "occupied";
if (!room.ha_area_id) return "unknown";
return "empty";
}
draw() {
const { ctx } = this;
ctx.clearRect(0, 0, this.width, this.height);
if (!this.level || !(this.level.rooms || []).length) {
ctx.fillStyle = "rgba(234,220,255,0.5)";
ctx.font = "16px system-ui, sans-serif";
ctx.textAlign = "center";
ctx.fillText("No rooms drawn on this level yet.", this.width / 2, this.height / 2);
return;
}
const height = this.options.wallHeight;
const rooms = (this.level.rooms || [])
.map((room) => {
let points = [];
try {
points = typeof room.points === "string" ? JSON.parse(room.points) : room.points || [];
} catch (err) {
points = [];
}
return { room, points, depth: this._depth(points.length ? points : [[0.5, 0.5]]) };
})
.filter((entry) => entry.points.length >= 3)
.sort((a, b) => a.depth - b.depth);
for (const { room, points } of rooms) {
const state = this._roomState(room);
this._drawWalls(points, height, state);
this._drawFloor(points, height, state, room);
}
// Markers last and in the same order, so a marker is never hidden behind a wall
// drawn after it.
for (const { room, points } of rooms) {
this._drawOccupants(room, points, height);
}
this._drawUnplacedShelf();
}
_drawWalls(points, height, state) {
const { ctx } = this;
// Each wall quad gets its own depth so the far ones do not paint over the near.
const walls = [];
for (let i = 0; i < points.length; i++) {
const a = points[i];
const b = points[(i + 1) % points.length];
walls.push({ a, b, depth: this._depth([a, b]) });
}
walls.sort((p, q) => p.depth - q.depth);
for (const { a, b } of walls) {
const p1 = this._project(a[0], a[1], 0);
const p2 = this._project(b[0], b[1], 0);
const p3 = this._project(b[0], b[1], height);
const p4 = this._project(a[0], a[1], height);
ctx.beginPath();
ctx.moveTo(p1.x, p1.y);
ctx.lineTo(p2.x, p2.y);
ctx.lineTo(p3.x, p3.y);
ctx.lineTo(p4.x, p4.y);
ctx.closePath();
ctx.fillStyle =
state === "occupied" ? "rgba(210,190,240,0.30)" : "rgba(150,130,190,0.13)";
ctx.fill();
ctx.strokeStyle =
state === "occupied" ? "rgba(255,255,255,0.55)" : "rgba(160,140,200,0.28)";
ctx.lineWidth = 1;
ctx.stroke();
}
}
_drawFloor(points, height, state, room) {
const { ctx } = this;
ctx.beginPath();
points.forEach(([x, y], i) => {
const p = this._project(x, y, height);
if (i === 0) ctx.moveTo(p.x, p.y);
else ctx.lineTo(p.x, p.y);
});
ctx.closePath();
// Occupancy reads in LIGHTNESS, not hue — the occupant dots already use colour to
// mean *who*, and making the room fill compete on the same channel is how both
// stop being readable.
if (state === "occupied") ctx.fillStyle = "rgba(255,252,245,0.90)";
else ctx.fillStyle = "rgba(28,18,44,0.85)";
ctx.fill();
ctx.strokeStyle =
state === "occupied" ? "rgba(255,255,255,0.9)" : "rgba(160,140,200,0.45)";
ctx.lineWidth = state === "occupied" ? 2 : 1;
// The third state is drawn dashed: one extra call, and it is the difference
// between "empty" and "nothing can see this room".
ctx.setLineDash(state === "unknown" ? [5, 4] : []);
ctx.stroke();
ctx.setLineDash([]);
const centre = this._centroid(points, height);
ctx.fillStyle = state === "occupied" ? "rgba(30,18,45,0.75)" : "rgba(200,180,235,0.45)";
ctx.font = "600 11px system-ui, sans-serif";
ctx.textAlign = "center";
ctx.fillText(room.name || "", centre.x, centre.y + 4);
}
_centroid(points, z) {
let x = 0;
let y = 0;
for (const [px, py] of points) {
x += px;
y += py;
}
return this._project(x / points.length, y / points.length, z);
}
_drawOccupants(room, points, height) {
const occupants = room.occupants || [];
if (!occupants.length) return;
const centre = this._centroid(points, height);
const radius = 17;
const spread = Math.min(occupants.length - 1, 3) * (radius + 4);
occupants.slice(0, 4).forEach((person, index) => {
const x = centre.x - spread / 2 + index * (radius + 4);
// Floated above the floor so the marker reads as standing in the room rather
// than as painted on it — a flat sprite on the floor is unreadable at a glance,
// which is the only way this view is ever used.
this._drawMarker(person, x, centre.y - radius - 14, radius);
});
if (occupants.length > 4) {
const { ctx } = this;
ctx.fillStyle = "rgba(30,18,45,0.8)";
ctx.font = "600 11px system-ui, sans-serif";
ctx.fillText(`+${occupants.length - 4}`, centre.x + spread / 2 + radius, centre.y - radius - 10);
}
}
_drawMarker(person, x, y, radius) {
const { ctx } = this;
const colour = person.color || "#c084fc";
const photo = this.photos.get(person.id);
ctx.save();
ctx.beginPath();
ctx.arc(x, y, radius, 0, Math.PI * 2);
ctx.closePath();
if (photo) {
ctx.clip();
ctx.drawImage(photo, x - radius, y - radius, radius * 2, radius * 2);
ctx.restore();
} else {
ctx.fillStyle = colour;
ctx.fill();
ctx.restore();
// Dark glyph on the person's colour — every value in identity's palette is
// mid-to-bright, so a white letter would vanish on the amber.
ctx.fillStyle = "#101014";
ctx.font = `700 ${Math.round(radius)}px system-ui, sans-serif`;
ctx.textAlign = "center";
ctx.textBaseline = "middle";
ctx.fillText(person.initial || "?", x, y + 1);
ctx.textBaseline = "alphabetic";
}
// The ring stays even behind a photo: it is what ties this marker to the same
// person's marker on the watch and in the admin panel.
ctx.beginPath();
ctx.arc(x, y, radius, 0, Math.PI * 2);
ctx.strokeStyle = colour;
ctx.lineWidth = 3;
ctx.stroke();
}
/**
* Everyone who is home but not locatable, along the bottom. A shelf rather than a
* hidden list because these are the people you are most often looking for — the
* whole reason somebody walks up to this screen.
*/
_drawUnplacedShelf() {
if (!this.unplaced.length) return;
const { ctx } = this;
const radius = 14;
const y = this.height - radius - 10;
let x = radius + 12;
ctx.fillStyle = "rgba(234,220,255,0.55)";
ctx.font = "12px system-ui, sans-serif";
ctx.textAlign = "left";
ctx.fillText("Home, room unknown:", 12, y - radius - 6);
for (const person of this.unplaced.slice(0, 8)) {
this._drawMarker(person, x, y, radius);
x += radius * 2 + 8;
}
}
}
global.Floorplan3D = Floorplan3D;
})(typeof window !== "undefined" ? window : globalThis);

49
render/media-visualiser/nowplaying.html Normal file
View File

@ -0,0 +1,49 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no">
<title>Now playing</title>
<link rel="stylesheet" href="visualiser.css">
</head>
<body>
<!--
The now-playing screen every endpoint can show: a circular spectrum behind the album
art, coloured from the cover, with lyrics underneath when the track has them.
Config from URL query params, never baked in — the same rule as every other frontend
here, so this stays a static asset with no secret in git:
?ha=http://ha:8123&token=<HA long-lived token>&entity=media_player.living_room
The source is Home Assistant's own media_player state, which is the one thing every
endpoint in this house can reach and which already knows what is playing everywhere.
A kiosk with local audio can additionally call visualiser.attachAudio(<audio element>)
to upgrade itself to the reactive tier; without that it renders the synthetic one,
which is the honest default for a panel showing what a DIFFERENT room is playing.
-->
<div id="stage">
<canvas id="ring"></canvas>
<div id="centre">
<img id="cover" alt="" hidden>
<div id="cover-fallback" aria-hidden="true">♪</div>
</div>
<div id="meta">
<h1 id="title">—</h1>
<p id="artist"></p>
<p id="tier-note" class="tier-note"></p>
</div>
<div id="lyrics" hidden>
<div id="lyrics-lines"></div>
</div>
<p id="error" class="error" hidden></p>
</div>
<script src="visualiser.js"></script>
<script src="nowplaying.js"></script>
</body>
</html>

173
render/media-visualiser/nowplaying.js Normal file
View File

@ -0,0 +1,173 @@
/*
* nowplaying — wires visualiser.js to Home Assistant's media_player state.
*
* HA is the source because it is the one thing every endpoint here can reach and it
* already knows what is playing in every room. That also means a panel can show ANOTHER
* room's music, which is the normal case and the reason the synthetic tier exists.
*/
"use strict";
const params = new URLSearchParams(location.search);
const HA = (params.get("ha") || "").replace(/\/$/, "");
const TOKEN = params.get("token") || "";
const ENTITY = params.get("entity") || "";
// How often to ask HA. Two seconds is plenty: the ring animates locally from position,
// so polling only has to catch track CHANGES, not drive the animation.
const POLL_MS = Number(params.get("poll") || 2000);
const $ = (id) => document.getElementById(id);
if (!HA || !TOKEN || !ENTITY) {
$("error").hidden = false;
$("error").textContent =
"Not configured — needs ?ha=&token=&entity= in the URL. See nowplaying.html.";
throw new Error("nowplaying: missing ?ha=/&token=/&entity=");
}
const visualiser = new MediaVisualiser($("ring"), { background: "#0a0510" });
visualiser.start();
let currentTrackKey = "";
// Position is interpolated between polls rather than asked for constantly: HA reports
// media_position with the timestamp it was measured at, so the honest current position
// is that value plus the time since. Polling for it instead would make the progress arc
// tick in two-second jumps.
let positionBase = { ms: 0, at: Date.now(), playing: false };
function haGet(path) {
return fetch(`${HA}${path}`, { headers: { Authorization: `Bearer ${TOKEN}` } }).then((res) => {
if (!res.ok) throw new Error(`${res.status} ${res.statusText}`);
return res.json();
});
}
function poll() {
haGet(`/api/states/${encodeURIComponent(ENTITY)}`)
.then((state) => {
$("error").hidden = true;
render(state);
})
.catch((err) => {
$("error").hidden = false;
$("error").textContent = `Home Assistant: ${err.message}`;
})
.finally(() => setTimeout(poll, POLL_MS));
}
function render(state) {
const a = state.attributes || {};
const playing = state.state === "playing";
const key = `${a.media_title || ""}|${a.media_artist || ""}|${a.entity_picture || ""}`;
$("title").textContent = a.media_title || (state.state === "off" ? "Nothing playing" : "—");
$("artist").textContent = [a.media_artist, a.media_album_name].filter(Boolean).join(" — ");
// HA gives media_position together with media_position_updated_at, which is what makes
// interpolation correct rather than a guess.
const updatedAt = a.media_position_updated_at ? Date.parse(a.media_position_updated_at) : Date.now();
positionBase = { ms: (a.media_position || 0) * 1000, at: updatedAt, playing };
if (key !== currentTrackKey) {
currentTrackKey = key;
loadArtwork(a.entity_picture);
loadLyrics(a);
}
visualiser.setTrack({
duration_ms: (a.media_duration || 0) * 1000,
position_ms: currentPosition(),
playing,
// Some sources expose a tempo; most do not. Null means the synthetic tier uses its
// default rate rather than pretending to know the BPM.
tempo: a.media_tempo || null,
});
$("tier-note").textContent =
visualiser.tier === "reactive"
? ""
: "visual rhythm — not an audio analysis";
}
function currentPosition() {
if (!positionBase.playing) return positionBase.ms;
return positionBase.ms + (Date.now() - positionBase.at);
}
// Keep the arc and the lyric highlight moving between polls.
setInterval(() => {
visualiser.setTrack({ position_ms: currentPosition() });
highlightLyric();
}, 250);
function loadArtwork(picture) {
const cover = $("cover");
if (!picture) {
cover.hidden = true;
$("cover-fallback").hidden = false;
return;
}
const url = picture.startsWith("http") ? picture : `${HA}${picture}`;
const image = new Image();
// Needed for paletteFrom(): without it the canvas is tainted and getImageData throws,
// which the SDK handles by keeping the default palette — this just makes the good
// path possible when HA sends the header.
image.crossOrigin = "anonymous";
image.onload = () => {
cover.src = image.src;
cover.hidden = false;
$("cover-fallback").hidden = true;
visualiser.setArtwork(image);
};
image.onerror = () => {
cover.hidden = true;
$("cover-fallback").hidden = false;
};
image.src = url;
}
// --- lyrics ---------------------------------------------------------------------------
// ABSENT IS THE NORMAL CASE. Most tracks in most libraries have no lyrics, so the layout
// is designed for "no lyrics" with lyrics as the addition — not a gap where they would be.
let lyricLines = [];
let lyricPlain = "";
function loadLyrics(attributes) {
lyricLines = [];
lyricPlain = "";
const raw = attributes.media_lyrics || attributes.lyrics || "";
if (!raw) {
$("lyrics").hidden = true;
return;
}
lyricLines = MediaVisualiser.parseLrc(raw);
if (lyricLines.length) {
$("lyrics-lines").innerHTML = lyricLines
.map((l, i) => `<p data-i="${i}">${escapeHtml(l.text)}</p>`)
.join("");
} else {
// Plain text: shown, never auto-scrolled at a guessed rate. A guessed scroll is
// wrong within ten seconds and stays wrong for the rest of the song.
lyricPlain = raw;
$("lyrics-lines").innerHTML = `<pre class="plain">${escapeHtml(raw)}</pre>`;
}
$("lyrics").hidden = false;
}
function highlightLyric() {
if (!lyricLines.length) return;
const index = MediaVisualiser.currentLyricIndex(lyricLines, currentPosition());
const lines = $("lyrics-lines").children;
for (let i = 0; i < lines.length; i++) {
lines[i].classList.toggle("current", Number(lines[i].dataset.i) === index);
}
const active = $("lyrics-lines").querySelector(".current");
if (active) active.scrollIntoView({ block: "center", behavior: "smooth" });
}
function escapeHtml(s) {
return String(s ?? "").replace(/[&<>"']/g, (c) =>
({ "&": "&amp;", "<": "&lt;", ">": "&gt;", '"': "&quot;", "'": "&#39;" }[c])
);
}
poll();

108
render/media-visualiser/visualiser.css Normal file
View File

@ -0,0 +1,108 @@
/* media-visualiser styling. Dark by necessity: this is a full-screen surface on a TV or
* a panel in a room where the lights may be off, and the ring's colours come from album
* art that has to read against it. */
* { box-sizing: border-box; }
html, body {
margin: 0;
height: 100%;
background: #0a0510;
color: #eadcff;
font-family: system-ui, sans-serif;
overflow: hidden;
}
#stage {
position: relative;
height: 100%;
display: grid;
grid-template-rows: 1fr auto auto;
place-items: center;
padding: 4vh 4vw;
}
/* The ring fills the stage and everything else sits on top of it. */
#ring {
position: absolute;
inset: 0;
width: 100%;
height: 100%;
}
#centre {
position: relative;
grid-row: 1;
display: grid;
place-items: center;
}
#cover, #cover-fallback {
width: min(34vh, 34vw);
height: min(34vh, 34vw);
border-radius: 50%;
object-fit: cover;
box-shadow: 0 12px 40px rgba(0, 0, 0, 0.6);
}
#cover-fallback {
display: grid;
place-items: center;
font-size: min(12vh, 12vw);
color: rgba(234, 220, 255, 0.35);
background: rgba(255, 255, 255, 0.05);
}
#meta { position: relative; grid-row: 2; text-align: center; max-width: 90vw; }
#meta h1 { margin: 0; font-size: clamp(20px, 3.4vh, 40px); }
#meta p { margin: 4px 0 0; color: #a98fc4; font-size: clamp(14px, 2vh, 22px); }
/* The synthetic tier says so, quietly and permanently. It is a mood light, and the
* moment somebody believes it is a spectrum every bass drop it misses is a bug. */
.tier-note {
font-size: 12px !important;
opacity: 0.45;
letter-spacing: 0.04em;
}
/* Lyrics are the addition, never the layout's assumption — most tracks have none, and
* this row simply collapses when they do. */
#lyrics {
position: relative;
grid-row: 3;
width: min(700px, 92vw);
max-height: 24vh;
overflow-y: auto;
text-align: center;
scrollbar-width: none;
mask-image: linear-gradient(transparent, #000 18%, #000 82%, transparent);
}
#lyrics::-webkit-scrollbar { display: none; }
#lyrics-lines p {
margin: 6px 0;
font-size: clamp(15px, 2.2vh, 24px);
color: rgba(234, 220, 255, 0.38);
transition: color 240ms ease, transform 240ms ease;
}
#lyrics-lines p.current {
color: #fff;
transform: scale(1.06);
}
#lyrics-lines .plain {
white-space: pre-wrap;
font-family: inherit;
font-size: clamp(14px, 1.9vh, 20px);
color: rgba(234, 220, 255, 0.6);
text-align: center;
}
.error {
position: absolute;
bottom: 12px;
left: 0;
right: 0;
text-align: center;
color: #ff8080;
font-size: 14px;
}

338
render/media-visualiser/visualiser.js Normal file
View File

@ -0,0 +1,338 @@
/*
* media-visualiser — a circular spectrum behind a now-playing screen, coloured from the
* album art, with lyrics under the cover.
*
* Vendored, dependency-free, no build step — the same choice as digest-canvas-sdk and
* canvas-sdk. Copied into each host that needs it rather than served from one place, so
* a kiosk with no route to the container host still renders.
*
* THE TWO TIERS, WHICH ARE THE WHOLE DESIGN
* ------------------------------------------
* CAVA reads an audio stream. Most endpoints do not have one: a kitchen panel showing
* what the LIVING ROOM is playing has no audio to analyse, and never will. A design
* that assumes real audio works on one screen and shows a dead circle on the rest,
* which is worse than not having it — a dead visualiser reads as broken, not as absent.
*
* reactive — audio is local. Real FFT, via WebAudio's AnalyserNode or a `cava -r`
* feed pushed in by the host's agent. Bars are the actual spectrum.
* synthetic — everywhere else. The ring breathes from track POSITION and tempo. It is
* a mood light, it is honest about being one (see `tier` on the object and
* the `data-tier` attribute on the canvas), and nobody watching from
* across a kitchen can tell.
*
* The synthetic tier must never claim to be the reactive one. The moment somebody
* believes it is a spectrum, every bass drop it does not match becomes a bug report.
*/
"use strict";
(function (global) {
const TAU = Math.PI * 2;
// --- palette ---------------------------------------------------------------------
/**
* Dominant colours from an image, as [{r,g,b}]. Done on a 64x64 downscale because
* this is a palette, not a photograph — full resolution costs time and changes
* nothing.
*/
function paletteFrom(image, count) {
const size = 64;
const canvas = document.createElement("canvas");
canvas.width = canvas.height = size;
const ctx = canvas.getContext("2d", { willReadFrequently: true });
ctx.drawImage(image, 0, 0, size, size);
let data;
try {
data = ctx.getImageData(0, 0, size, size).data;
} catch (err) {
// A cross-origin cover taints the canvas and getImageData throws. That is a
// configuration problem (art served from another origin without CORS), not a
// reason to have no visualiser — fall back to the default palette.
return null;
}
// Bucket in a coarse RGB grid. 4 bits per channel is enough to group "the same
// colour" without merging colours a person would call different.
const buckets = new Map();
for (let i = 0; i < data.length; i += 4) {
const r = data[i], g = data[i + 1], b = data[i + 2];
if (data[i + 3] < 128) continue;
// REJECT NEAR-GREYS AND NEAR-BLACKS BEFORE RANKING. Album art is full of them,
// and a naive palette from a dark cover is four indistinguishable dark greys —
// i.e. a visualiser that looks switched off.
const max = Math.max(r, g, b), min = Math.min(r, g, b);
if (max < 40) continue; // near-black
if (max - min < 24) continue; // near-grey: no hue to speak of
const key = ((r >> 4) << 8) | ((g >> 4) << 4) | (b >> 4);
const entry = buckets.get(key) || { r: 0, g: 0, b: 0, n: 0 };
entry.r += r; entry.g += g; entry.b += b; entry.n += 1;
buckets.set(key, entry);
}
const ranked = [...buckets.values()]
.sort((a, b) => b.n - a.n)
.slice(0, count || 4)
.map((e) => ({ r: Math.round(e.r / e.n), g: Math.round(e.g / e.n), b: Math.round(e.b / e.n) }));
return ranked.length ? ranked : null;
}
function relativeLuminance({ r, g, b }) {
const f = (c) => {
const v = c / 255;
return v <= 0.03928 ? v / 12.92 : Math.pow((v + 0.055) / 1.055, 2.4);
};
return 0.2126 * f(r) + 0.7152 * f(g) + 0.0722 * f(b);
}
/**
* Lift a colour until it clears the background it is drawn on.
*
* This is the step that gets skipped, and skipping it is why so many album-art
* visualisers are invisible on dark covers: a visualiser you cannot see is the same
* as no visualiser at all.
*/
function ensureContrast(colour, backgroundLuminance, minRatio) {
let { r, g, b } = colour;
for (let i = 0; i < 24; i++) {
const l = relativeLuminance({ r, g, b });
const ratio = (Math.max(l, backgroundLuminance) + 0.05) / (Math.min(l, backgroundLuminance) + 0.05);
if (ratio >= (minRatio || 3)) break;
r = Math.min(255, Math.round(r * 1.12 + 8));
g = Math.min(255, Math.round(g * 1.12 + 8));
b = Math.min(255, Math.round(b * 1.12 + 8));
}
return { r, g, b };
}
const css = ({ r, g, b }, alpha) =>
alpha === undefined ? `rgb(${r},${g},${b})` : `rgba(${r},${g},${b},${alpha})`;
// --- the visualiser ----------------------------------------------------------------
class MediaVisualiser {
/**
* @param {HTMLCanvasElement} canvas
* @param {object} options { bars, background, minContrast }
*/
constructor(canvas, options) {
this.canvas = canvas;
this.ctx = canvas.getContext("2d");
this.options = Object.assign({ bars: 96, background: "#0a0510", minContrast: 3 }, options || {});
this.palette = [{ r: 192, g: 132, b: 252 }, { r: 255, g: 62, b: 200 }];
this.levels = new Float32Array(this.options.bars);
this.targets = new Float32Array(this.options.bars);
this.tier = "synthetic";
this.track = { position_ms: 0, duration_ms: 0, tempo: null, playing: false };
this.analyser = null;
this.running = false;
this._resize();
window.addEventListener("resize", () => this._resize());
}
_resize() {
const rect = this.canvas.getBoundingClientRect();
const dpr = Math.min(window.devicePixelRatio || 1, 2);
this.canvas.width = Math.max(1, Math.round(rect.width * dpr));
this.canvas.height = Math.max(1, Math.round(rect.height * dpr));
this.ctx.setTransform(dpr, 0, 0, dpr, 0, 0);
this.width = rect.width;
this.height = rect.height;
}
/** Colour the ring from an already-loaded <img> of the cover. */
setArtwork(image) {
const found = paletteFrom(image, 4);
if (!found) return false;
const bg = relativeLuminance(this._backgroundRgb());
this.palette = found.map((c) => ensureContrast(c, bg, this.options.minContrast));
return true;
}
_backgroundRgb() {
const hex = String(this.options.background).replace("#", "");
return {
r: parseInt(hex.slice(0, 2), 16) || 0,
g: parseInt(hex.slice(2, 4), 16) || 0,
b: parseInt(hex.slice(4, 6), 16) || 0,
};
}
/** Reactive tier: drive the ring from a real WebAudio graph. */
attachAudio(mediaElementOrStream) {
try {
const AudioContextCtor = global.AudioContext || global.webkitAudioContext;
if (!AudioContextCtor) return false;
const audioCtx = new AudioContextCtor();
const source =
mediaElementOrStream instanceof MediaStream
? audioCtx.createMediaStreamSource(mediaElementOrStream)
: audioCtx.createMediaElementSource(mediaElementOrStream);
const analyser = audioCtx.createAnalyser();
analyser.fftSize = 512;
analyser.smoothingTimeConstant = 0.7;
source.connect(analyser);
if (!(mediaElementOrStream instanceof MediaStream)) analyser.connect(audioCtx.destination);
this.analyser = analyser;
this.bins = new Uint8Array(analyser.frequencyBinCount);
this.tier = "reactive";
this.canvas.dataset.tier = "reactive";
return true;
} catch (err) {
// Autoplay policy, a cross-origin element, no AudioContext — all end here, and
// all mean the same thing: fall back to synthetic rather than showing nothing.
return false;
}
}
/**
* Reactive tier, remote feed: levels pushed in by the host's agent from `cava -r`.
* An array of 0..1 values. Same tier as WebAudio because it is the same claim —
* these numbers came from the actual audio.
*/
pushLevels(levels) {
if (!levels || !levels.length) return;
this.tier = "reactive";
this.canvas.dataset.tier = "reactive";
for (let i = 0; i < this.targets.length; i++) {
const source = levels[Math.floor((i / this.targets.length) * levels.length)];
this.targets[i] = Math.max(0, Math.min(1, source || 0));
}
}
/** Track state, for the synthetic tier and for the progress arc. */
setTrack(track) {
this.track = Object.assign(this.track, track || {});
if (!this.analyser && this.tier !== "reactive") {
this.canvas.dataset.tier = "synthetic";
}
}
start() {
if (this.running) return;
this.running = true;
const frame = (now) => {
if (!this.running) return;
this._step(now);
requestAnimationFrame(frame);
};
requestAnimationFrame(frame);
}
stop() {
this.running = false;
}
_step(now) {
const n = this.targets.length;
if (this.analyser) {
this.analyser.getByteFrequencyData(this.bins);
for (let i = 0; i < n; i++) {
// Log-ish bin mapping: linear FFT bins put almost everything in the first
// eighth of the ring, which looks like a fault rather than like music.
const t = i / n;
const bin = Math.floor(Math.pow(t, 1.7) * (this.bins.length - 1));
this.targets[i] = this.bins[bin] / 255;
}
} else if (this.tier !== "reactive") {
// SYNTHETIC. Derived from position and tempo — a slow travelling wave with a
// beat-rate pulse. Deliberately smooth: anything jittery invites the comparison
// with real audio that this tier cannot win.
const bpm = this.track.tempo || 100;
const beat = (now / 1000) * (bpm / 60);
const pulse = 0.5 + 0.5 * Math.sin(beat * TAU);
const energy = this.track.playing ? 0.45 + 0.35 * pulse : 0.06;
for (let i = 0; i < n; i++) {
const phase = (i / n) * TAU * 3 + now / 1400;
this.targets[i] = Math.max(0, energy * (0.55 + 0.45 * Math.sin(phase)));
}
}
// One smoother for both tiers, so switching between them does not jump.
for (let i = 0; i < n; i++) {
this.levels[i] += (this.targets[i] - this.levels[i]) * 0.22;
}
this._draw();
}
_draw() {
const { ctx, width, height } = this;
const cx = width / 2;
const cy = height / 2;
const inner = Math.min(width, height) * 0.30;
const maxBar = Math.min(width, height) * 0.17;
ctx.clearRect(0, 0, width, height);
const n = this.levels.length;
for (let i = 0; i < n; i++) {
const angle = (i / n) * TAU - Math.PI / 2;
const level = this.levels[i];
const length = 2 + level * maxBar;
const colour = this.palette[i % this.palette.length];
ctx.strokeStyle = css(colour, 0.25 + level * 0.75);
ctx.lineWidth = Math.max(2, (TAU * inner) / n - 2);
ctx.lineCap = "round";
ctx.beginPath();
ctx.moveTo(cx + Math.cos(angle) * inner, cy + Math.sin(angle) * inner);
ctx.lineTo(cx + Math.cos(angle) * (inner + length), cy + Math.sin(angle) * (inner + length));
ctx.stroke();
}
// Progress as a thin arc just inside the ring — the one piece of information a
// now-playing screen is actually asked for from across a room.
const { position_ms: pos, duration_ms: dur } = this.track;
if (dur > 0) {
ctx.strokeStyle = css(this.palette[0], 0.9);
ctx.lineWidth = 3;
ctx.lineCap = "butt";
ctx.beginPath();
ctx.arc(cx, cy, inner - 10, -Math.PI / 2, -Math.PI / 2 + TAU * Math.min(1, pos / dur));
ctx.stroke();
}
}
}
// --- lyrics --------------------------------------------------------------------------
/**
* Parse an LRC document into [{ms, text}]. Returns [] for anything that is not LRC,
* which is how the caller tells synced from plain: if this comes back empty and there
* was text, the text is plain and must NOT be auto-scrolled to a guessed rate — that
* is wrong within ten seconds and stays wrong.
*/
function parseLrc(text) {
const lines = [];
const re = /\[(\d+):(\d+)(?:[.:](\d+))?\]/g;
for (const raw of String(text || "").split(/\r?\n/)) {
let match;
const stamps = [];
re.lastIndex = 0;
while ((match = re.exec(raw)) !== null) {
const centis = match[3] ? parseInt(match[3].padEnd(3, "0").slice(0, 3), 10) : 0;
stamps.push(parseInt(match[1], 10) * 60000 + parseInt(match[2], 10) * 1000 + centis);
}
const body = raw.replace(re, "").trim();
if (stamps.length && body) stamps.forEach((ms) => lines.push({ ms, text: body }));
}
return lines.sort((a, b) => a.ms - b.ms);
}
/** Index of the line that should be highlighted at `positionMs`, or -1. */
function currentLyricIndex(lines, positionMs) {
let index = -1;
for (let i = 0; i < lines.length; i++) {
if (lines[i].ms <= positionMs) index = i;
else break;
}
return index;
}
global.MediaVisualiser = MediaVisualiser;
global.MediaVisualiser.paletteFrom = paletteFrom;
global.MediaVisualiser.ensureContrast = ensureContrast;
global.MediaVisualiser.parseLrc = parseLrc;
global.MediaVisualiser.currentLyricIndex = currentLyricIndex;
})(typeof window !== "undefined" ? window : globalThis);

5
tools/build-door-panel-iso.sh
View File

@ -62,6 +62,10 @@ ENABLE_VOICE_SATELLITE="$CORE_KIOSK_VOICE_SATELLITE"
VOICE_SATELLITE_NAME="$CORE_KIOSK_FRIENDLY_NAME"
VOICE_WAKE_WORD="$CORE_KIOSK_WAKE_WORD"
DOOR_PANEL_NAME="$CORE_KIOSK_FRIENDLY_NAME"
# The HA area this device physically sits in, from the kiosk's `room` in
# CoreSystemConfig.json. The agent publishes it as suggested_area so HA files
# the device in the right room by itself — see docs/rooms-and-endpoints.md.
DOOR_PANEL_ROOM="${CORE_KIOSK_ROOM:-}"
IDENTITY_WEB_URL="$CORE_IDENTITY_WEB_URL"
IDENTITY_URL="$CORE_IDENTITY_URL"
IDENTITY_TOKEN="$CORE_IDENTITY_TOKEN"
@ -184,6 +188,7 @@ cat > "$INCLUDES/etc/door-panel-agent/config.env" <<EOF
# here; change CoreSystemConfig.json at the repo root and rebuild.
KIOSK_USERNAME=${KIOSK_USERNAME}
DOOR_PANEL_NAME=${DOOR_PANEL_NAME}
DOOR_PANEL_ROOM=${DOOR_PANEL_ROOM}
MQTT_BROKER_HOST=${MQTT_BROKER_HOST}
MQTT_BROKER_PORT=${MQTT_BROKER_PORT}

5
tools/build-kitchen-display-iso.sh
View File

@ -61,6 +61,10 @@ ENABLE_VOICE_SATELLITE="$CORE_KIOSK_VOICE_SATELLITE"
VOICE_SATELLITE_NAME="$CORE_KIOSK_FRIENDLY_NAME"
VOICE_WAKE_WORD="$CORE_KIOSK_WAKE_WORD"
KITCHEN_DISPLAY_NAME="$CORE_KIOSK_FRIENDLY_NAME"
# The HA area this device physically sits in, from the kiosk's `room` in
# CoreSystemConfig.json. The agent publishes it as suggested_area so HA files
# the device in the right room by itself — see docs/rooms-and-endpoints.md.
KITCHEN_DISPLAY_ROOM="${CORE_KIOSK_ROOM:-}"
PANTRY_WEB_URL="$CORE_PANTRY_WEB_URL"
PANTRY_VISION_URL="$CORE_PANTRY_VISION_URL"
PANTRY_VISION_TOKEN="$CORE_PANTRY_VISION_TOKEN"
@ -185,6 +189,7 @@ cat > "$INCLUDES/etc/kitchen-display-agent/config.env" <<EOF
# hand-edit here; change CoreSystemConfig.json and rebuild.
KIOSK_USERNAME=${KIOSK_USERNAME}
KITCHEN_DISPLAY_NAME=${KITCHEN_DISPLAY_NAME}
KITCHEN_DISPLAY_ROOM=${KITCHEN_DISPLAY_ROOM}
MQTT_BROKER_HOST=${MQTT_BROKER_HOST}
MQTT_BROKER_PORT=${MQTT_BROKER_PORT}

12
tools/build-thin-client-iso.sh
View File

@ -59,6 +59,10 @@ ENABLE_VOICE_SATELLITE="$CORE_KIOSK_VOICE_SATELLITE"
VOICE_SATELLITE_NAME="$CORE_KIOSK_FRIENDLY_NAME"
VOICE_WAKE_WORD="$CORE_KIOSK_WAKE_WORD"
THINCLIENT_NAME="$CORE_KIOSK_FRIENDLY_NAME"
# The HA area this device physically sits in, from the kiosk's `room` in
# CoreSystemConfig.json. The agent publishes it as suggested_area so HA files
# the device in the right room by itself — see docs/rooms-and-endpoints.md.
THINCLIENT_ROOM="${CORE_KIOSK_ROOM:-}"
HA_URL="$CORE_HA_URL"
DIGEST_WEB_URL="$CORE_DIGEST_WEB_URL"
ADMIN_WEB_URL="$CORE_ADMIN_WEB_URL"
@ -176,6 +180,13 @@ install -m 0755 "${CONFIGS_DIR}/greetd/kiosk-session" "$INCLUDES/usr/local/bin/
install -m 0755 "${CONFIGS_DIR}/sway/digest-browser" "$INCLUDES/usr/local/bin/digest-browser"
install -m 0755 "${CONFIGS_DIR}/sway/admin-browser" "$INCLUDES/usr/local/bin/admin-browser"
install -m 0755 "${CONFIGS_DIR}/sway/capture-view" "$INCLUDES/usr/local/bin/capture-view"
# Bound to the remote's power/sleep buttons — see the Sway config's remote-control
# section for why those turn the display off rather than the machine.
install -m 0755 "${CONFIGS_DIR}/sway/display-toggle" "$INCLUDES/usr/local/bin/display-toggle"
# fleet-bootstrap: fetch this machine's published monitoring script and run it. Opt-in —
# with no /etc/fleet-bootstrap.conf it exits 0 and does nothing. See tools/fleet-bootstrap.sh
# for what enabling it actually grants (root execution from a central service).
install -m 0755 "${CORE_REPO_ROOT}/tools/fleet-bootstrap.sh" "$INCLUDES/usr/local/bin/fleet-bootstrap"
install -m 0755 "${CONFIGS_DIR}/wayvnc/start-wayvnc" "$INCLUDES/usr/local/bin/start-wayvnc"
install -m 0644 "${CONFIGS_DIR}/mpv/mpv.conf" "$INCLUDES/home/${KIOSK_USERNAME}/.config/mpv/mpv.conf"
@ -262,6 +273,7 @@ cat > "$INCLUDES/etc/thinclient-agent/config.env" <<EOF
# here; change CoreSystemConfig.json at the repo root and rebuild.
KIOSK_USERNAME=${KIOSK_USERNAME}
THINCLIENT_NAME=${THINCLIENT_NAME}
THINCLIENT_ROOM=${THINCLIENT_ROOM}
MQTT_BROKER_HOST=${MQTT_BROKER_HOST}
MQTT_BROKER_PORT=${MQTT_BROKER_PORT}

5
tools/build-touch-panel-iso.sh
View File

@ -64,6 +64,10 @@ ENABLE_VOICE_SATELLITE="$CORE_KIOSK_VOICE_SATELLITE"
VOICE_SATELLITE_NAME="$CORE_KIOSK_FRIENDLY_NAME"
VOICE_WAKE_WORD="$CORE_KIOSK_WAKE_WORD"
TOUCHPANEL_NAME="$CORE_KIOSK_FRIENDLY_NAME"
# The HA area this device physically sits in, from the kiosk's `room` in
# CoreSystemConfig.json. The agent publishes it as suggested_area so HA files
# the device in the right room by itself — see docs/rooms-and-endpoints.md.
TOUCHPANEL_ROOM="${CORE_KIOSK_ROOM:-}"
HA_URL="$CORE_HA_URL"
# ---------------------------------------------------------------------------
@ -206,6 +210,7 @@ cat > "$INCLUDES/etc/touchpanel-agent/config.env" <<EOF
# here; change CoreSystemConfig.json at the repo root and rebuild.
KIOSK_USERNAME=${KIOSK_USERNAME}
TOUCHPANEL_NAME=${TOUCHPANEL_NAME}
TOUCHPANEL_ROOM=${TOUCHPANEL_ROOM}
MQTT_BROKER_HOST=${MQTT_BROKER_HOST}
MQTT_BROKER_PORT=${MQTT_BROKER_PORT}

75
tools/config-export.py
View File

@ -116,23 +116,62 @@ def main(argv: list[str]) -> int:
# --- Secrets ---
secrets = cfg.get("secrets", {})
for key in ("identity_token", "pantry_vision_token", "transit_token", "mqtt_username",
"mqtt_password", "ha_token", "opnsense_api_key", "opnsense_api_secret",
"mqtt_password", "ha_token", "checkmk_username", "checkmk_secret",
"workshop_token", "gitea_url", "gitea_token", "gitea_owner",
"freeipa_bind_password",
"ssh_authorized_key", "kiosk_password", "admin_password_hash"):
emit(f"CORE_{key.upper()}", secrets.get(key, ""))
# --- The existing OPNsense firewall (digest-engine's network digest reads its IDS) ---
# Emitted as scalars rather than as a JSON blob because the ISO builder writes them
# into IDSconf.json field by field, the same way it writes every generated env file:
# one place holds the values, and nothing downstream re-parses a nested structure.
opnsense = cfg.get("opnsense", {}) or {}
emit("CORE_OPNSENSE_BASE_URL", str(opnsense.get("base_url", "")).rstrip("/"))
emit("CORE_OPNSENSE_VERIFY_TLS", opnsense.get("verify_tls", True))
# Space-separated for shell iteration; the builder turns it back into a JSON array.
emit("CORE_OPNSENSE_INTERFACES", " ".join(str(i) for i in opnsense.get("interfaces", []) or []))
emit("CORE_OPNSENSE_MAX_ALERTS_SCANNED", opnsense.get("max_alerts_scanned", 5000))
emit("CORE_OPNSENSE_TOP_SIGNATURES", opnsense.get("top_signatures", 8))
emit("CORE_OPNSENSE_TOP_HOSTS", opnsense.get("top_hosts", 5))
emit("CORE_OPNSENSE_PACKET_CAPTURE_REFERENCE", opnsense.get("packet_capture_reference", ""))
# --- The household's OPNsense firewalls, and CheckMK ---
# The firewall list goes out as ONE JSON blob, unlike everything else here, and the
# reason is that it is a list: emitting scalars per firewall would mean
# CORE_OPNSENSE_0_BASE_URL and a shell loop reassembling them, which is a parser
# nobody wants to own. Consumers write it straight into IDSconf.json.
firewalls = cfg.get("opnsense", []) or []
if isinstance(firewalls, dict):
firewalls = [firewalls] # the old single-object shape; see validate-config.py
keys = (cfg.get("secrets", {}) or {}).get("opnsense_keys", {}) or {}
resolved = []
for fw in firewalls:
if not isinstance(fw, dict) or not str(fw.get("base_url") or "").strip():
continue
pair = keys.get(str(fw.get("name") or ""), {}) or {}
resolved.append({
"name": fw.get("name", "main"),
"base_url": str(fw.get("base_url", "")).rstrip("/"),
"api_key": pair.get("api_key", ""),
"api_secret": pair.get("api_secret", ""),
"verify_tls": fw.get("verify_tls", True),
"interfaces": fw.get("interfaces", []) or [],
"max_alerts_scanned": fw.get("max_alerts_scanned", 5000),
"top_signatures": fw.get("top_signatures", 8),
"top_hosts": fw.get("top_hosts", 5),
"packet_capture_reference": fw.get("packet_capture_reference", ""),
})
emit("CORE_OPNSENSE_JSON", json.dumps({"firewalls": resolved}))
emit("CORE_OPNSENSE_COUNT", len(resolved))
checkmk = cfg.get("checkmk", {}) or {}
emit("CORE_CHECKMK_BASE_URL", str(checkmk.get("base_url", "")).rstrip("/"))
emit("CORE_CHECKMK_SITE", checkmk.get("site", ""))
emit("CORE_CHECKMK_VERIFY_TLS", checkmk.get("verify_tls", True))
emit("CORE_CHECKMK_ONLY_PROBLEMS", checkmk.get("only_problems", True))
emit("CORE_CHECKMK_MAX_ROWS", checkmk.get("max_rows", 200))
# --- External identity provider (declaration only — nothing implements SSO yet) ---
idp = cfg.get("identity_provider", {}) or {}
emit("CORE_IDP_ISSUER_URL", str(idp.get("issuer_url", "")).rstrip("/"))
emit("CORE_IDP_REALM", idp.get("realm", ""))
emit("CORE_IDP_CLIENT_ID", idp.get("client_id", ""))
emit("CORE_IDP_PROTECTED_HOSTS", " ".join(str(h) for h in idp.get("protected_hosts", []) or []))
# --- FreeIPA (declaration only — the mirror is not implemented yet) ---
ipa = cfg.get("freeipa", {}) or {}
for key in ("server", "domain", "base_dn", "bind_dn", "household_group",
"chore_exempt_group", "admin_group"):
emit(f"CORE_FREEIPA_{key.upper()}", ipa.get(key, ""))
emit("CORE_FREEIPA_VERIFY_TLS", ipa.get("verify_tls", True))
emit("CORE_FREEIPA_SYNC_INTERVAL_MINUTES", ipa.get("sync_interval_minutes", 60))
# --- Enable flags ---
for flag, value in (cfg.get("container_host", {}).get("enable", {}) or {}).items():
@ -174,6 +213,13 @@ def main(argv: list[str]) -> int:
emit("CORE_KIOSK_TYPE", kiosk["type"])
emit("CORE_KIOSK_HOSTNAME", kiosk["hostname"])
emit("CORE_KIOSK_FRIENDLY_NAME", kiosk["friendly_name"])
# The HA area this device physically lives in. Baked into the agent, published
# as `suggested_area` in its MQTT discovery, so HA files the device in the right
# room without anybody dragging it there in the UI — and so every per-room
# feature (voice "turn the lights off in here", the floorplan, the workshop
# assistant) has one answer to "what is in this room". Empty is allowed and
# means "don't suggest"; see docs/rooms-and-endpoints.md.
emit("CORE_KIOSK_ROOM", kiosk.get("room", ""))
emit("CORE_KIOSK_USERNAME", kiosk["kiosk_username"])
emit("CORE_KIOSK_VOICE_SATELLITE", kiosk.get("voice_satellite", False))
emit("CORE_KIOSK_ENABLE_INSTALLER", kiosk.get("enable_installer", False))
@ -192,6 +238,7 @@ def main(argv: list[str]) -> int:
endpoint = matches[0]
emit("CORE_AUDIO_HOSTNAME", endpoint["hostname"])
emit("CORE_AUDIO_FRIENDLY_NAME", endpoint["friendly_name"])
emit("CORE_AUDIO_ROOM", endpoint.get("room", ""))
emit("CORE_AUDIO_ARCH", endpoint["arch"])
return 0

100
tools/fleet-bootstrap.sh Executable file
View File

@ -0,0 +1,100 @@
#!/bin/sh
# fleet-bootstrap — fetch this machine's published fleet script, run it, report back.
#
# Installed to /usr/local/bin/fleet-bootstrap on every endpoint, driven by a systemd
# timer. What it is FOR: getting each machine into CheckMK monitoring without anybody
# SSHing into eleven hosts and pasting the same installer.
#
# THIS IS REMOTE CODE EXECUTION, AND IT IS SUPPOSED TO BE. There is no way to
# "distribute a script to the endpoints" that is not. So the honesty is in the
# constraints rather than in pretending otherwise:
#
# 1. It runs the PUBLISHED version only. An upload is a draft and is invisible here
# until a human publishes it deliberately — see workshop/fleet.py.
# 2. It verifies the sha256 the server sent against the body it received, and refuses
# to run on a mismatch. That is not protection against a malicious server (the
# server chose both), it is protection against a truncated download, which is the
# failure that actually happens.
# 3. It runs a version ONCE. The recorded state file is what stops a timer from
# re-running an installer every fifteen minutes forever.
# 4. It reports what it ran, pass or fail. A script that was served is not a script
# that succeeded, and only the endpoint knows which.
# 5. It is opt-in per host: no WORKSHOP_URL, no execution.
#
# The whole capability is worth one sentence in your own words before enabling it:
# anyone who can publish to that service can run code as root on every machine here.
# Treat WORKSHOP_TOKEN accordingly — it is the most powerful credential in this project.
set -eu
CONFIG="${FLEET_CONFIG:-/etc/fleet-bootstrap.conf}"
[ -f "$CONFIG" ] && . "$CONFIG"
WORKSHOP_URL="${WORKSHOP_URL:-}"
WORKSHOP_TOKEN="${WORKSHOP_TOKEN:-}"
FLEET_PLATFORM="${FLEET_PLATFORM:-}"
STATE_DIR="${FLEET_STATE_DIR:-/var/lib/fleet-bootstrap}"
if [ -z "$WORKSHOP_URL" ] || [ -z "$WORKSHOP_TOKEN" ] || [ -z "$FLEET_PLATFORM" ]; then
echo "fleet-bootstrap: not configured (need WORKSHOP_URL, WORKSHOP_TOKEN, FLEET_PLATFORM in $CONFIG)" >&2
exit 0 # Not an error: an unconfigured host is a host that opted out.
fi
command -v curl >/dev/null 2>&1 || { echo "fleet-bootstrap: curl is required" >&2; exit 1; }
mkdir -p "$STATE_DIR"
HOSTNAME_NOW="$(hostname)"
RESPONSE="$(curl -fsS -H "Authorization: Bearer ${WORKSHOP_TOKEN}" \
"${WORKSHOP_URL%/}/fleet/script/${FLEET_PLATFORM}" 2>/dev/null || true)"
if [ -z "$RESPONSE" ]; then
# No published script for this platform yet is the normal state before somebody
# uploads one. Silence beats a daily error mail about a thing nobody has done yet.
echo "fleet-bootstrap: nothing published for ${FLEET_PLATFORM}"
exit 0
fi
# python3 rather than jq: every image in this project already has python3, and the
# script body is JSON-escaped text that a shell parser would mangle.
VERSION="$(printf '%s' "$RESPONSE" | python3 -c 'import json,sys;print(json.load(sys.stdin).get("version",""))')"
EXPECTED_SHA="$(printf '%s' "$RESPONSE" | python3 -c 'import json,sys;print(json.load(sys.stdin).get("sha256",""))')"
SCRIPT_FILE="${STATE_DIR}/${FLEET_PLATFORM}.v${VERSION}.sh"
printf '%s' "$RESPONSE" | python3 -c 'import json,sys;sys.stdout.write(json.load(sys.stdin).get("body",""))' > "$SCRIPT_FILE"
ACTUAL_SHA="$(sha256sum "$SCRIPT_FILE" | cut -d" " -f1)"
if [ "$ACTUAL_SHA" != "$EXPECTED_SHA" ]; then
echo "fleet-bootstrap: checksum mismatch (expected $EXPECTED_SHA, got $ACTUAL_SHA) — refusing to run" >&2
rm -f "$SCRIPT_FILE"
exit 1
fi
STATE_FILE="${STATE_DIR}/ran-${FLEET_PLATFORM}"
if [ -f "$STATE_FILE" ] && [ "$(cat "$STATE_FILE")" = "$ACTUAL_SHA" ]; then
echo "fleet-bootstrap: v${VERSION} already ran here"
exit 0
fi
echo "fleet-bootstrap: running ${FLEET_PLATFORM} v${VERSION} (${ACTUAL_SHA})"
chmod 0700 "$SCRIPT_FILE"
set +e
OUTPUT="$(sh "$SCRIPT_FILE" 2>&1)"
RC=$?
set -e
if [ "$RC" -eq 0 ]; then
# Recorded only on success, so a failed run is retried on the next timer rather than
# being remembered as done.
printf '%s' "$ACTUAL_SHA" > "$STATE_FILE"
fi
# Report either way. The failure case is the one worth having on the admin page.
curl -fsS -X POST -H "Authorization: Bearer ${WORKSHOP_TOKEN}" -H "Content-Type: application/json" \
-d "$(python3 -c 'import json,sys
print(json.dumps({"hostname": sys.argv[1], "platform": sys.argv[2],
"version": int(sys.argv[3]) if sys.argv[3] else None,
"sha256": sys.argv[4], "ok": sys.argv[5] == "0",
"detail": sys.argv[6][-2000:]}))' \
"$HOSTNAME_NOW" "$FLEET_PLATFORM" "$VERSION" "$ACTUAL_SHA" "$RC" "$OUTPUT")" \
"${WORKSHOP_URL%/}/fleet/report" >/dev/null 2>&1 || \
echo "fleet-bootstrap: ran, but could not report back" >&2
exit "$RC"

167
tools/setup-container-host.sh
View File

@ -85,6 +85,23 @@ ENABLE_GALLERY_SMB="${ENABLE_GALLERY_SMB:-false}"
GALLERY_SMB_USERNAME="${GALLERY_SMB_USERNAME:-gallery}"
GALLERY_SMB_PASSWORD="${GALLERY_SMB_PASSWORD:-}" # <-- SET THIS before flipping the toggle above
# --- Photo web frontend (Immich) — browsing the same photos the SMB share serves ---
# The share is right for bulk copy and for keeping the files openable without any of
# this; it is hopeless for FINDING a photo. Immich adds search, albums, timeline and
# faces over the same directory. Off by default: it brings a Postgres and a Redis with
# it, which is the largest single addition to this stack's footprint.
ENABLE_PHOTOS_WEB="${ENABLE_PHOTOS_WEB:-false}"
PHOTOS_WEB_PORT="${PHOTOS_WEB_PORT:-2283}"
PHOTOS_DB_PASSWORD="${PHOTOS_DB_PASSWORD:-}" # <-- SET THIS before flipping the toggle above
# --- Workshop assistant (project notebook + knowledge store + health poller) ---
ENABLE_WORKSHOP="${ENABLE_WORKSHOP:-false}"
WORKSHOP_PORT="${WORKSHOP_PORT:-8102}"
WORKSHOP_WEB_PORT="${WORKSHOP_WEB_PORT:-8103}"
WORKSHOP_SRC="${WORKSHOP_SRC:-/opt/smart-home/src/workshop}"
WORKSHOP_SMB_USERNAME="${WORKSHOP_SMB_USERNAME:-workshop}"
WORKSHOP_SMB_PASSWORD="${WORKSHOP_SMB_PASSWORD:-}" # <-- SET THIS before flipping the toggle above
# --- Scheduled backups (restic) — off by default until you pick a target ---
# Set ENABLE_BACKUPS=true and RESTIC_REPOSITORY to a local path (e.g. an
# external/USB drive mount, or a NAS mount), or a remote target restic
@ -397,6 +414,24 @@ if [[ "$ENABLE_GALLERY_SMB" == "true" ]]; then
# skips idle-gallery mode with nothing to show, same as an unreachable share).
mkdir -p "$BASE_DIR"/gallery
fi
if [[ "$ENABLE_PHOTOS_WEB" == "true" ]]; then
# Immich's own library and database. Separate from the gallery share, which it only
# ever reads: two writers to one photo tree with different ideas of the layout is how
# a collection gets quietly reorganised.
mkdir -p "$BASE_DIR"/photos/{db,library}
fi
if [[ "$ENABLE_WORKSHOP" == "true" ]]; then
# data/ holds the two SQLite files (work, and never-pruned knowledge); workspace/ is
# the writable SMB share the assistant files artefacts into.
mkdir -p "$BASE_DIR"/workshop/{data,workspace}
if [[ ! -f "$BASE_DIR/workshop/workshop.env" ]]; then
cp "$WORKSHOP_SRC/workshop.env.example" "$BASE_DIR/workshop/workshop.env"
chmod 600 "$BASE_DIR/workshop/workshop.env"
echo " Seeded $BASE_DIR/workshop/workshop.env from the template — fill in a real"
echo " WORKSHOP_TOKEN (openssl rand -hex 32). CheckMK, the firewall list and Gitea"
echo " are all optional; each is refused with a clear message while unset."
fi
fi
if [[ "$ENABLE_DIGEST_ENGINE" == "true" ]]; then
mkdir -p "$BASE_DIR"/digest/{output,data}
if [[ "$ENABLE_WHATSAPP_INGEST" == "true" ]]; then
@ -433,7 +468,12 @@ if [[ "$ENABLE_ADMIN_CANVAS" == "true" ]]; then
fi
fi
if [[ "$ENABLE_PANTRY_VISION" == "true" ]]; then
mkdir -p "$BASE_DIR"/pantry-vision
# data/ holds ONE file: doorway.py's hint database (what an appliance camera thinks
# it saw at a door, with a timestamp). Everything that is actually inventory still
# lives in Grocy — see pantry-vision/doorway.py on why an observation and a stock
# row are deliberately not kept in the same place. Losing this directory costs the
# household its recent sightings and nothing else.
mkdir -p "$BASE_DIR"/pantry-vision/data
# Same handling as admin-canvas.env above: seed from the committed template on
# first run, 600, never committed (repo .gitignore covers *.env).
if [[ ! -f "$BASE_DIR/pantry-vision/pantry-vision.env" ]]; then
@ -444,8 +484,9 @@ if [[ "$ENABLE_PANTRY_VISION" == "true" ]]; then
fi
fi
if [[ "$ENABLE_IDENTITY" == "true" ]]; then
# Unlike pantry-vision (stateless, all state lives in Grocy), identity owns its
# own SQLite DB and registration photos — both need a persistent bind mount.
# pantry-vision keeps only doorway hints locally (everything that is inventory is
# Grocy's); identity is different in kind — it owns its own SQLite DB and the
# registration photos, and neither is reconstructible from anywhere else.
mkdir -p "$BASE_DIR"/identity/data/photos
if [[ ! -f "$BASE_DIR/identity/identity.env" ]]; then
cp "$IDENTITY_SRC/identity.env.example" "$BASE_DIR/identity/identity.env"
@ -748,6 +789,118 @@ if [[ "$ENABLE_PORTAINER" == "true" ]]; then
"
fi
# The workshop workspace is a SECOND SHARE ON THE SAME SAMBA CONTAINER, not a second
# container: Samba serves many shares on one port, and a second container would fight
# this one for 445 — the same class of collision already recorded as open decision #31.
# It is the one WRITABLE share in this stack (the gallery is read only = yes), so it
# gets its own volume and its own account; see workshop/README.md for why that matters
# when the thing writing into it is an LLM.
WORKSHOP_SHARE_ENV=""
WORKSHOP_SHARE_VOLUME=""
if [[ "$ENABLE_WORKSHOP" == "true" && "$ENABLE_GALLERY_SMB" == "true" ]]; then
WORKSHOP_SHARE_ENV="
- ACCOUNT_${WORKSHOP_SMB_USERNAME}=${WORKSHOP_SMB_PASSWORD}
- SAMBA_VOLUME_CONFIG_workshop=path = /shares/workshop; valid users = ${WORKSHOP_SMB_USERNAME}; guest ok = no; read only = no; browseable = yes"
WORKSHOP_SHARE_VOLUME="
- ${BASE_DIR}/workshop/workspace:/shares/workshop"
elif [[ "$ENABLE_WORKSHOP" == "true" ]]; then
echo " NOTE: ENABLE_WORKSHOP is on but ENABLE_GALLERY_SMB is off, so there is no Samba"
echo " container to hang the workspace share on. The workshop service works fine —"
echo " its notebook is in SQLite — but artefacts will only be reachable through the"
echo " container filesystem until you enable the SMB container too."
fi
# Immich: search/albums/faces over the SAME directory the SMB share exports. Two views
# of one pile of files, which is the point — the share is right for bulk copy and for
# keeping the photos openable with no software at all, and hopeless for finding one.
# It is mounted READ-ONLY here on purpose: Immich's own upload path would put files
# where its database expects them rather than where the share's layout puts them, and
# two writers to one tree with different ideas of the layout is how a photo collection
# gets quietly reorganised. Import from the share, write nothing back.
PHOTOS_WEB_BLOCK=""
if [[ "$ENABLE_PHOTOS_WEB" == "true" ]]; then
if [[ -z "$PHOTOS_DB_PASSWORD" ]]; then
echo "ERROR: ENABLE_PHOTOS_WEB=true but PHOTOS_DB_PASSWORD is empty." >&2
exit 1
fi
PHOTOS_WEB_BLOCK="
immich-db:
image: ghcr.io/immich-app/postgres:14-vectorchord0.4.3-pgvectors0.2.0
container_name: immich-db
restart: unless-stopped
environment:
- POSTGRES_USER=immich
- POSTGRES_PASSWORD=${PHOTOS_DB_PASSWORD}
- POSTGRES_DB=immich
- TZ=${TIMEZONE}
volumes:
- ${BASE_DIR}/photos/db:/var/lib/postgresql/data
immich-redis:
image: docker.io/valkey/valkey:8-bookworm
container_name: immich-redis
restart: unless-stopped
immich-server:
image: ghcr.io/immich-app/immich-server:release
container_name: immich-server
restart: unless-stopped
depends_on:
- immich-db
- immich-redis
ports:
- \"${PHOTOS_WEB_PORT}:2283\"
environment:
- DB_HOSTNAME=immich-db
- DB_USERNAME=immich
- DB_PASSWORD=${PHOTOS_DB_PASSWORD}
- DB_DATABASE_NAME=immich
- REDIS_HOSTNAME=immich-redis
- TZ=${TIMEZONE}
volumes:
- ${BASE_DIR}/photos/library:/usr/src/app/upload
# The gallery, read-only. Add it in Immich as an External Library.
- ${BASE_DIR}/gallery:/mnt/gallery:ro
"
fi
# workshop: the project notebook, the knowledge store and the health poller.
# /data holds two SQLite files (work, and never-pruned knowledge); /workspace is the
# writable SMB share. See workshop/README.md.
WORKSHOP_BLOCK=""
WORKSHOP_WEB_BLOCK=""
if [[ "$ENABLE_WORKSHOP" == "true" ]]; then
WORKSHOP_BLOCK="
workshop:
build: ${WORKSHOP_SRC}
image: smart-home/workshop:local
container_name: workshop
restart: unless-stopped
ports:
- \"${WORKSHOP_PORT}:${WORKSHOP_PORT}\"
volumes:
- ${BASE_DIR}/workshop/data:/data
- ${BASE_DIR}/workshop/workspace:/workspace
env_file:
- ${BASE_DIR}/workshop/workshop.env
environment:
- WORKSHOP_PORT=${WORKSHOP_PORT}
- TZ=${TIMEZONE}
"
WORKSHOP_WEB_BLOCK="
workshop-web:
image: nginx:alpine
container_name: workshop-web
restart: unless-stopped
ports:
- \"${WORKSHOP_WEB_PORT}:80\"
volumes:
- ${WORKSHOP_SRC}/frontend:/usr/share/nginx/html:ro
environment:
- TZ=${TIMEZONE}
"
fi
GALLERY_SMB_BLOCK=""
if [[ "$ENABLE_GALLERY_SMB" == "true" ]]; then
# servercontainers/samba over dperson/samba: dperson's image has gone without
@ -765,9 +918,9 @@ if [[ "$ENABLE_GALLERY_SMB" == "true" ]]; then
environment:
- TZ=${TIMEZONE}
- ACCOUNT_${GALLERY_SMB_USERNAME}=${GALLERY_SMB_PASSWORD}
- SAMBA_VOLUME_CONFIG_gallery=path = /shares/gallery; valid users = ${GALLERY_SMB_USERNAME}; guest ok = no; read only = yes; browseable = yes
- SAMBA_VOLUME_CONFIG_gallery=path = /shares/gallery; valid users = ${GALLERY_SMB_USERNAME}; guest ok = no; read only = yes; browseable = yes${WORKSHOP_SHARE_ENV}
volumes:
- ${BASE_DIR}/gallery:/shares/gallery
- ${BASE_DIR}/gallery:/shares/gallery${WORKSHOP_SHARE_VOLUME}
cap_add:
- NET_ADMIN
"
@ -882,6 +1035,8 @@ if [[ "$ENABLE_PANTRY_VISION" == "true" ]]; then
- grocy
ports:
- \"${PANTRY_VISION_PORT}:${PANTRY_VISION_PORT}\"
volumes:
- ${BASE_DIR}/pantry-vision/data:/data
env_file:
- ${BASE_DIR}/pantry-vision/pantry-vision.env
environment:
@ -1241,7 +1396,7 @@ ${FRIGATE_DEVICES}
- PUID=1000
- PGID=1000
- TZ=${TIMEZONE}
${MEALIE_BLOCK}${NODERED_BLOCK}${NETDATA_BLOCK}${HOMEPAGE_BLOCK}${NTFY_BLOCK}${PORTAINER_BLOCK}${GALLERY_SMB_BLOCK}${DIGEST_ENGINE_BLOCK}${DIGEST_WEB_BLOCK}${WHATSAPP_BRIDGE_BLOCK}${ADMIN_CANVAS_BLOCK}${ADMIN_WEB_BLOCK}${PANTRY_VISION_BLOCK}${PANTRY_WEB_BLOCK}${IDENTITY_BLOCK}${IDENTITY_WEB_BLOCK}${PROXY_BLOCK}${TRASH_CALENDAR_BLOCK}${TRANSIT_BLOCK}${TRANSIT_SYNC_BLOCK}${OTP_BLOCK}${CHORES_BLOCK}${MUSIC_ASSISTANT_BLOCK}
${MEALIE_BLOCK}${NODERED_BLOCK}${NETDATA_BLOCK}${HOMEPAGE_BLOCK}${NTFY_BLOCK}${PORTAINER_BLOCK}${GALLERY_SMB_BLOCK}${DIGEST_ENGINE_BLOCK}${DIGEST_WEB_BLOCK}${WHATSAPP_BRIDGE_BLOCK}${ADMIN_CANVAS_BLOCK}${ADMIN_WEB_BLOCK}${PANTRY_VISION_BLOCK}${PANTRY_WEB_BLOCK}${IDENTITY_BLOCK}${IDENTITY_WEB_BLOCK}${PROXY_BLOCK}${TRASH_CALENDAR_BLOCK}${TRANSIT_BLOCK}${TRANSIT_SYNC_BLOCK}${OTP_BLOCK}${CHORES_BLOCK}${MUSIC_ASSISTANT_BLOCK}${PHOTOS_WEB_BLOCK}${WORKSHOP_BLOCK}${WORKSHOP_WEB_BLOCK}
EOF
# ---------------------------------------------------------------------------

186
tools/validate-config.py
View File

@ -261,22 +261,140 @@ def validate_secrets(cfg: dict, rep: Report) -> None:
"empty — every kiosk will connect to Mosquitto anonymously. Fine while "
"allow_anonymous is on; revisit before that changes")
# OPNsense's API key pair. Half-configured is the failure worth catching: an
# address with no credentials (or credentials with no address) looks configured
# and produces a network digest that silently reports nothing.
opnsense_url = str(_get(cfg, "opnsense.base_url", "") or "").strip()
opnsense_key = str(_get(cfg, "secrets.opnsense_api_key", "") or "").strip()
opnsense_secret = str(_get(cfg, "secrets.opnsense_api_secret", "") or "").strip()
if opnsense_url and not (opnsense_key and opnsense_secret):
rep.error("secrets.opnsense_api_key",
"opnsense.base_url is set, so both opnsense_api_key and "
"opnsense_api_secret are needed. OPNsense mints them: System -> Access "
"-> Users -> API keys; give that user only the 'Services: Intrusion "
"Detection' privilege")
if (opnsense_key or opnsense_secret) and not opnsense_url:
rep.warn("opnsense.base_url",
"empty, but an OPNsense API key is set — nothing will use it until this "
"points at the firewall (e.g. https://opnsense.home.lan)")
# OPNsense: a LIST of firewalls now, each with its own key pair keyed by name.
# Half-configured is the failure worth catching: an address with no credentials (or
# credentials with no address) looks configured and produces a network digest that
# silently reports nothing — which is indistinguishable from a quiet network.
firewalls = _get(cfg, "opnsense", []) or []
if isinstance(firewalls, dict):
# The single-object shape this key used to have. Accepted so an existing config
# keeps validating, and reported once so it gets migrated rather than lingering.
rep.warn("opnsense",
"is a single object — it is a list of firewalls now, so more than one can "
"be watched. Wrap it in [ ] and give it a `name`; the old shape still works "
"for one firewall but nothing new reads it")
firewalls = [firewalls]
keys = _get(cfg, "secrets.opnsense_keys", {}) or {}
seen_names: set[str] = set()
for index, fw in enumerate(firewalls):
where = f"opnsense[{index}]"
if not isinstance(fw, dict):
rep.error(where, "must be an object")
continue
name = str(fw.get("name") or "").strip()
if not name:
rep.error(f"{where}.name", "every firewall needs a name — it is what tells two "
"of them apart in the digest and in the health table")
elif name in seen_names:
rep.error(f"{where}.name", f"'{name}' is used twice; names must be unique")
else:
seen_names.add(name)
url = str(fw.get("base_url") or "").strip()
pair = keys.get(name, {}) if isinstance(keys, dict) else {}
api_key = str((pair or {}).get("api_key") or "").strip()
api_secret = str((pair or {}).get("api_secret") or "").strip()
if url and not (api_key and api_secret):
rep.error(f"secrets.opnsense_keys.{name or index}",
f"{where}.base_url is set, so this firewall needs both api_key and "
"api_secret. OPNsense mints them: System -> Access -> Users -> API keys; "
"give that user only the 'Services: Intrusion Detection' privilege")
if (api_key or api_secret) and not url:
rep.warn(f"{where}.base_url",
"empty, but an API key is set for it — nothing will use that key until "
"this points at the firewall (e.g. https://opnsense.home.lan)")
# CheckMK, same half-configured check and the same reasoning.
cmk_url = str(_get(cfg, "checkmk.base_url", "") or "").strip()
cmk_user = str(_get(cfg, "secrets.checkmk_username", "") or "").strip()
cmk_secret = str(_get(cfg, "secrets.checkmk_secret", "") or "").strip()
if cmk_url and not (cmk_user and cmk_secret):
rep.error("secrets.checkmk_username",
"checkmk.base_url is set, so both checkmk_username and checkmk_secret are "
"needed. Use a Guest-role automation user and its AUTOMATION SECRET, not a "
"login password — the read-only guarantee comes from that role")
if cmk_url and not str(_get(cfg, "checkmk.site", "") or "").strip():
rep.error("checkmk.site", "required — it is the path segment in every CheckMK API URL")
if (cmk_user or cmk_secret) and not cmk_url:
rep.warn("checkmk.base_url", "empty, but CheckMK credentials are set — nothing polls it yet")
# The workshop assistant's own token, on the same rule as every other service token.
if _get(cfg, "container_host.enable.workshop", False) and not (
_get(cfg, "secrets.workshop_token", "") or ""):
rep.error("secrets.workshop_token",
"required because container_host.enable.workshop is true. "
"Generate one with: openssl rand -hex 32")
# Gitea is optional even with workshop on — the assistant simply refuses repo
# creation without it — but half a config is worth a word.
gitea_url = str(_get(cfg, "secrets.gitea_url", "") or "").strip()
gitea_token = str(_get(cfg, "secrets.gitea_token", "") or "").strip()
if bool(gitea_url) != bool(gitea_token):
rep.warn("secrets.gitea_url",
"gitea_url and gitea_token only work as a pair — repo creation stays "
"refused until both are set")
if gitea_token and not str(_get(cfg, "secrets.gitea_owner", "") or "").strip():
rep.warn("secrets.gitea_owner",
"empty, so repos are created under the token's OWN account. Prefer a "
"dedicated 'workshop-bot' user scoped to one organisation — the blast "
"radius of a leaked env file should be one org, not everything you own")
# KEYCLOAK AND FREEIPA ARE ONE DECISION. Keycloak federates FreeIPA as its user
# store; it is not a place people are created. Configuring one without the other
# would mean a second, parallel set of household accounts — the exact outcome a
# directory exists to prevent — so either alone is an error, not a warning.
issuer = str(_get(cfg, "identity_provider.issuer_url", "") or "").strip()
ipa_server = str(_get(cfg, "freeipa.server", "") or "").strip()
if issuer and not ipa_server:
rep.error("freeipa.server",
"identity_provider.issuer_url is set, but Keycloak is never deployed without "
"FreeIPA in this project — it federates the directory rather than being its "
"own user store. Configure both, or neither")
if ipa_server and not issuer:
rep.error("identity_provider.issuer_url",
"freeipa.server is set, but FreeIPA here exists to be federated by Keycloak — "
"the two are one decision. Configure both, or neither")
if issuer:
if not issuer.startswith(("http://", "https://")):
rep.error("identity_provider.issuer_url", f"{issuer!r} is not a URL")
elif issuer.startswith("http://"):
rep.warn("identity_provider.issuer_url",
"is plain HTTP — an OIDC issuer over HTTP means tokens in cleartext on "
"the LAN")
if ipa_server:
for field in ("domain", "base_dn", "bind_dn"):
if not str(_get(cfg, f"freeipa.{field}", "") or "").strip():
rep.error(f"freeipa.{field}", "required once freeipa.server is set")
if not (_get(cfg, "secrets.freeipa_bind_password", "") or ""):
rep.error("secrets.freeipa_bind_password",
"required once freeipa.server is set. Use a dedicated service account "
"with READ-ONLY access — the mirror never writes to the directory")
if not str(_get(cfg, "freeipa.household_group", "") or "").strip():
rep.error("freeipa.household_group",
"required — without it every service account in the directory becomes "
"a household member")
# The USR_HA_<parameter> convention. A WARNING, not an error: an existing
# directory may already have its own naming, and renaming groups in FreeIPA is
# not something a config file gets to force. The prefix is what makes "which
# groups does the smart home read?" answerable with one filter.
for field, expected in (("household_group", "USR_HA_household"),
("chore_exempt_group", "USR_HA_chore_exempt"),
("admin_group", "USR_HA_admins")):
value = str(_get(cfg, f"freeipa.{field}", "") or "").strip()
if value and not value.startswith("USR_HA_"):
rep.warn(f"freeipa.{field}",
f"{value!r} does not follow the USR_HA_<parameter> convention "
f"(expected {expected!r}). It will still work — this is about being "
"able to audit which groups the smart home reads")
if issuer or ipa_server:
rep.warn("identity_provider",
"NEITHER SSO NOR THE DIRECTORY MIRROR IS IMPLEMENTED YET — these blocks record "
"the decision so the proxy config and identity can be pointed at them later. "
"They change no behaviour today")
if not (_get(cfg, "secrets.ha_token", "") or ""):
rep.warn("secrets.ha_token",
@ -324,6 +442,34 @@ def validate_proxy(cfg: dict, rep: Report) -> None:
"which is why the kiosk images still use plain HTTP; see proxy/README.md")
# An HA area_id: lowercase, digits, underscores. Home Assistant slugifies area names
# into exactly this shape, and it is also what identity's floorplan rooms store in
# `ha_area_id`. Validating it here is what keeps ONE room vocabulary across the
# project instead of "Kitchen" in one file and "kitchen" in another — see
# docs/rooms-and-endpoints.md.
AREA_ID_RE = re.compile(r"^[a-z0-9_]+$")
def _check_room(value, where: str, rep) -> str:
"""Returns the room, or "" — and reports why it isn't usable if it isn't.
Missing is a WARNING, not an error: a household that hasn't decided its room names
yet must still be able to build an image. What it loses is automatic area
assignment in HA, which is a nuisance to fix by hand, not a broken device.
"""
if value is None or value == "":
rep.warn(where, "no room set — HA won't file this device in an area by itself, "
"and every per-room feature (voice 'in here', the floorplan, "
"room-scoped assistants) will treat it as unplaced")
return ""
if not isinstance(value, str) or not AREA_ID_RE.match(value):
rep.error(where, f"{value!r} is not an HA area_id — lowercase letters, digits and "
"underscores only, e.g. 'living_room'")
return ""
return value
def validate_kiosks(cfg: dict, rep: Report) -> None:
kiosks = _get(cfg, "kiosks")
if kiosks is None:
@ -334,6 +480,8 @@ def validate_kiosks(cfg: dict, rep: Report) -> None:
return
hostnames: dict[str, int] = {}
# hostname -> room, so the summary below can say what ended up where.
rooms: dict[str, str] = {}
for index, kiosk in enumerate(kiosks):
where = f"kiosks[{index}]"
if not isinstance(kiosk, dict):
@ -353,6 +501,9 @@ def validate_kiosks(cfg: dict, rep: Report) -> None:
hostnames[hostname] = index
if not kiosk.get("friendly_name"):
rep.error(f"{where}.friendly_name", "missing — this is the name shown on the HA device")
rooms[hostname if isinstance(hostname, str) else where] = _check_room(
kiosk.get("room"), f"{where}.room", rep
)
if not kiosk.get("kiosk_username"):
rep.error(f"{where}.kiosk_username", "missing")
@ -378,6 +529,9 @@ def validate_kiosks(cfg: dict, rep: Report) -> None:
f"must be one of {sorted(ARCHITECTURES)}, got {endpoint.get('arch')!r}")
if not endpoint.get("friendly_name"):
rep.error(f"{where}.friendly_name", "missing")
rooms[hostname if isinstance(hostname, str) else where] = _check_room(
endpoint.get("room"), f"{where}.room", rep
)
# Cross-check: a kiosk that talks to a disabled service will build fine and then
# fail at runtime with a connection error, which is exactly the class of "works on

19
workshop/Dockerfile Normal file
View File

@ -0,0 +1,19 @@
# workshop — the project notebook behind the workshop assistant (docs/workshop-assistant.md).
# `restart: unless-stopped`, same as every other write service here.
FROM python:3.11-slim
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1
# git is the one runtime dependency this service has beyond the stdlib — git_ops.py
# shells out to it. Installed explicitly rather than assumed present in the base image.
RUN apt-get update && apt-get install -y --no-install-recommends git \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
# stdlib only (sqlite3 + http.server) — no requirements.txt, same call as
# admin-canvas/server.py and pantry-vision/server.py.
COPY server.py knowledge.py gitea.py git_ops.py ./
CMD ["python", "server.py"]

344
workshop/README.md Normal file
View File

@ -0,0 +1,344 @@
# workshop
The project notebook behind the workshop/office assistant — **step 1 of
[`docs/workshop-assistant.md`](../docs/workshop-assistant.md), and deliberately only
step 1.**
That note ranks four capabilities (conversational planning, spec research, guide
lookup, camera identification) and says to build this one first: it is the only one
with no perception problem in it, and it is where all the others land. An identified
mainboard is worth nothing until there is a project to attach it to.
## Two databases and a share
| | Holds | Lifetime |
|---|---|---|
| **`workshop.db`** | projects, notes, decisions — a record of **work** | scoped to a project; over when it's over |
| **`workshop-knowledge.db`** | workflow instructions, facts, project learnings, **hardware inventory** — a record of **what is true** | **forever. Nothing here is ever pruned** |
| **`/workspace`** (SMB) | datasheets, generated diagrams, photos, long-form notes | whatever you leave there |
Two files rather than four more tables because they have different lifetimes and
different blast radius: rebuilding the project store must not take the standing
instruction about how you solder with it.
**Nothing in the knowledge store expires.** Every other store in this stack has a
retention window — `doorway.py` keeps sightings 30 days, `digest-engine`'s archive 400
— because stale observations are worse than none. This one is the opposite kind of
data: the entire point is that you tell it once. A retention policy would be a policy
of forgetting the thing the feature exists to remember, so there is no cutoff and no
cleanup pass anywhere in `knowledge.py`. Corrections are per-row `DELETE`s — no
expiry does not mean nothing is ever wrong.
### The knowledge tables
- **`workflow_notes`** — retrieved by **activity**. "When I'm soldering, flux first."
Activity `*` means always. Re-adding the same instruction *updates* rather than
duplicating, since telling it twice is the exact problem the table solves.
- **`facts`** — retrieved by **keyword**. Device specs, URLs, part numbers. Re-adding
the same subject overwrites: a spec you looked up again is usually one you had wrong.
- **`project_knowledge`** — retrieved by **project**, and deliberately not a foreign
key into `workshop.db`. What you learned building the NAS stays true after the NAS
is gone.
- **`hardware`** — what you own, where it is, and what it's promised to.
**`GET /context?activity=&q=&project=`** is what makes these a memory rather than four
lists: one call returns the standing instructions, the relevant facts, the project's
learnings and its hardware. An unknown activity still returns the always-notes —
"I don't know what you're doing" is no reason to forget how you like to work.
### There is exactly one inventory
An earlier cut had a `parts` table in the project store. It's gone. Two inventories is
the failure this repo's own docs keep warning about: the one you didn't update becomes
a lie, and you find out by buying a part you already own. What you *own* is knowledge;
what a project *needs* is a claim on it, which is `hardware.project_slug`.
**Four statuses, and the middle two are the point:**
| | Means | Can I use it right now? |
|---|---|---|
| `available` | on the shelf, unpromised | yes |
| `assigned` | **reserved** for a project, still on the shelf | yes, if you're willing to change your mind |
| `in_use` | **installed and working** somewhere | not without taking something apart |
| `retired` | dead, sold, given away | no — kept so "didn't I have one?" has an answer |
Naming a project *is* the assignment: `{"project_slug": "nas-build"}` moves an item to
`assigned` by itself, and clearing it releases to `available` — an item assigned to a
project while still reading `available` is how an inventory starts lying. **The one
exception: `in_use` is never silently downgraded by a re-assignment.** Saying which
project something is in must not turn a thing you'd have to unscrew into a thing the
list says you can just take.
### Why a share and not blobs in the database
Mixing them gets the worst of both: a database you can't browse and files nothing can
query. Keeping artefacts as files means **every one of them outlives this service** — a datasheet in the
share is a PDF you can open from a laptop with no API, no export step, and no
dependence on this project still existing in two years.
Consequently this service **never serves file contents**. `GET /projects/<slug>/files`
is a directory listing; opening the file is the share's job, and a second copy of the
bytes over HTTP would just be a second place for them to go stale.
## The workspace is the one writable share in this stack
The photo gallery share (`gallery-smb`) is `read only = yes` on purpose. This one
can't be — the assistant writes into it — and that difference is the whole security
story:
- **Its own volume**, not a subdirectory of anything else, so the blast radius of a
bad path is "the workshop workspace" and is recoverable.
- **Its own SMB account.** Guest off, same as the gallery.
- **Nothing on the host ever executes anything found in it.** The moment it becomes a
place for scripts, it is a different security question than this component answers.
- **Slugs are validated, never sanitised** (`SLUG_RE` in `server.py`). A project slug
is a directory name on that share, which makes it the one input here with a
filesystem consequence. Silently rewriting `../../etc` to `etc` would let two
different requests write to one directory with nobody finding out, so a bad slug is
a 400.
`_ensure_project_dir()` is best-effort: an unmounted or read-only share does **not**
stop a project being created, because the notebook is in SQLite and that is the part
that matters. Every response that touches the workspace carries `available`, so the
API says plainly whether the share is working instead of implying it by silence.
## Specs carry their source, or they are marked as not having one
`specs` always sits next to `source_url`, in both `hardware` and `facts`. This is the schema half of the
rule in the feasibility note:
> **Every spec must be quoted from a fetched document, with its URL, and never
> generated from the model's memory.**
A hallucinated TDP wastes an afternoon; a hallucinated pinout destroys hardware. A
spec you read off the part in your hand is a perfectly good source and is accepted
without a URL — it is just recorded as unsourced, so the UI can mark it and a later
research pass knows which specs still need a document behind them. **Never populate
`specs` from an LLM's memory.** That is not a style preference here, it is the reason
the column pair exists.
## Why decisions are their own table
"What did I decide last Tuesday, and why" is the question a project notebook exists to
answer, and a pile of notes answers it worst. A decision is a question, an answer, and
a *because* — the last one being the column the table is for, because an answer
without its reasoning is something you will re-litigate in three weeks.
## API
All endpoints are bearer-token gated; an unset `WORKSHOP_TOKEN` rejects everything.
| Endpoint | What it does |
|---|---|
| `GET /projects[?room=<area_id>]` | every project, or the ones belonging to a room |
| `POST /projects` | `{"name", "slug"?, "room"?, "summary"?}` → creates it and its workspace directories |
| `GET /projects/<slug>` | the project with its notes, decisions, durable knowledge and claimed hardware |
| `POST /projects/<slug>` | `{"name"?, "status"?, "summary"?, "room"?}` — status is `active`/`parked`/`done` |
| `GET /projects/<slug>/files` | directory listing of the workspace, never contents |
| `POST /projects/<slug>/notes` | `{"body", "author"?}` — `author` distinguishes what you wrote from what the assistant wrote |
| `POST /projects/<slug>/decisions` | `{"question", "answer", "because"?}` |
| `POST /projects/<slug>/knowledge` | `{"body", "keywords"?}` — a durable learning, not a log entry |
| `POST /projects/<slug>/repo` | `{"name"?, "description"?, "private"?}` → creates a Gitea repo, applies branch protection, sets up `code/` and records the URL |
| `GET /projects/<slug>/git` | branch, dirty files, last five commits |
| `POST /projects/<slug>/commit` | `{"message", "push"?}` — commit and push. Reported separately: a failed push is not a failed commit |
| `POST /projects/<slug>/branch` | `{"name"}` — create and switch. There is no delete counterpart, on purpose |
| `POST /projects/<slug>/scrub-request` | `{"path"?, "note"?}` → the manual commands to remove something from history. **Executes nothing** |
| `GET /hardware[?q=&status=&project=]` | the inventory |
| `POST /hardware` | `{"designation", "kind"?, "quantity"?, "storage_location"?, "status"?, "project_slug"?, "specs"?, "source_url"?, "notes"?}` |
| `POST /hardware/<id>` | edit anything, including assignment |
| `GET /knowledge/workflow[?activity=]` · `POST /knowledge/workflow` | standing instructions |
| `GET /knowledge/facts[?q=]` · `POST /knowledge/facts` | keyword-retrieved specifics |
| `GET /knowledge/projects[?project=&q=]` | durable project learnings |
| `GET /context[?activity=&q=&project=]` | **everything that applies right now, in one call** |
| `GET /health` | CheckMK + every firewall: current state, how long it's held, the problems |
| `GET /cameras` | network camera names + go2rtc stream ids |
| `GET /fleet` | every slot, its versions, and what each endpoint reported |
| `GET /fleet/script/<platform>` | the **published** script for a platform (what endpoints fetch) |
| `POST /fleet/upload` | `{"platform", "body", "note"?}` → a new **draft** version |
| `POST /fleet/publish` | `{"platform", "version"}` → make it the one endpoints run |
| `POST /fleet/report` | an endpoint saying what it ran and whether it worked |
| `DELETE /hardware/<id>`, `DELETE /knowledge/{workflow,facts,projects}/<id>` | corrections |
`parked` is not `done`: a shelved project whose parts are still allocated to it is
exactly what you want to find *before* buying those parts again.
`room` is an HA `area_id`, the same vocabulary as everywhere else in this project —
see [`docs/rooms-and-endpoints.md`](../docs/rooms-and-endpoints.md). It is what lets a
room-scoped assistant answer "what am I working on" with the projects of the room it
was asked in.
## Configure
```sh
cp workshop/workshop.env.example /opt/smart-home/workshop/workshop.env
openssl rand -hex 32 # WORKSHOP_TOKEN
chmod 600 /opt/smart-home/workshop/workshop.env
```
## The web interface
`frontend/` — a static page for **editing the hardware inventory by hand**: search,
filter by status or project, inline edit, add, delete. Same shape as
`pantry-vision/frontend` (vanilla JS, no build step, `?api=&token=` from the URL).
It edits the inventory and nothing else, on purpose. Every other table records
something that was *said or decided*, and those are only wrong if someone recorded
them wrong. The inventory describes **physical reality**, which drifts without telling
anybody — so it's the one table that needs somewhere a person can sit down and fix it.
Deliberately **not** the purple/magenta holo theme from the feasibility note: that's
for the canvas surfaces the assistant draws on, where the look is the point. This is a
form you correct data in, and a glow behind a text field is a cost with no benefit.
## Gitea
`POST /projects/<slug>/repo` creates a repository and records its URL on the project.
Needs `GITEA_URL` + `GITEA_TOKEN` (Settings → Applications → Generate New Token, scope
`write:repository`); refused with a clear message while unset.
### The assistant writes history. It never rewrites it.
**Allowed, unattended:** commit, push, branch, tag, merge — everything that *adds*.
The worst case is a bad commit you revert.
**Never, by any path:** force-push, rebase, amend, `reset --hard`, `filter-branch`,
`filter-repo`, branch or tag deletion, reflog expiry, `gc --prune`. Rewriting has no
undo, and the whole point is that a rollback is always feasible.
That refusal lives in `git_ops.py` as a deny-list checked on every git invocation —
but **client-side refusal is a policy, not a control.** The enforcement is Gitea
**branch protection**, applied automatically at repo creation (`enable_force_push:
false`, `enable_delete: false`), which rejects the push regardless of what asked for
it, including a human who typed `--force` out of habit. Check
`repo.protection.applied` in the create response; never assume it's on because the
repo exists.
**When history genuinely has to be scrubbed** — a committed API token —
`POST /projects/<slug>/scrub-request` returns the commands **for you to run by hand**
from the repo on the SMB share. It executes nothing. The manual step *is* the safety
mechanism: the person running `git filter-repo` can read it first and take a copy;
a service doing it on a timer cannot. The instructions lead with *rotate the secret
first*, because the commit was already pushed and a scrub that makes you feel finished
without rotating is worse than no scrub.
The working tree is **`code/`** inside the project workspace, never the workspace root
— so no git operation of any kind has a path to the datasheets and photos beside it.
Repository deletion isn't implemented at all, even though the token permits it.
Use a **dedicated Gitea user** ("workshop-bot") scoped to one organisation via
`GITEA_OWNER`, not a token on your own account. The blast radius of a leaked env file
is then one org of generated repos.
An existing repo of the same name is a **success, not a conflict** — asking twice means
you wanted it to exist, and erroring there would make every retry look like a failure
while leaving the caller with no URL to record.
## Infrastructure health
`health.py` polls **CheckMK** and **every OPNsense firewall** every few minutes and
keeps ~30 days of samples in `infra_status`.
**Why here and not in digest-engine**, which already reads one firewall: the digest
runs four times a day, and "is the NAS disk failing right now" is not a question with a
six-hour answer. This service is always on, so it polls; `digest-engine`'s
`ingest/infra_health.py` reads *this* over HTTP. One poller, two consumers — and the
digest gets **"critical since Tuesday"** instead of a snapshot it can't compare to
anything.
Three states, not two: `ok`, `problem`, and **`unreachable`**. "I could not ask" is a
different fact from "I asked and it is broken", and collapsing them is how a display
shows green for a machine that has been off all day. Every failure is written as a row
rather than raised — silence has to be visible.
**Multiple firewalls are the point.** `OPNSENSE_JSON` is a list and every row carries
its firewall's `name`, all the way to the display and the digest prompt. A household
with two firewalls must never be told "the IDS is running".
Read-only, and not by promise: the CheckMK user should hold the **Guest** role, which
cannot acknowledge, downtime or reschedule. From OPNsense this reads exactly one
endpoint (`GET /api/ids/service/status`); the alert query stays in digest-engine, which
already does the paging properly — two implementations of one read would eventually
give two different answers about one log file.
Unlike the rest of the knowledge store, these samples **do** expire (30 days). A
service state from three weeks ago is an observation, not a fact.
## Cameras
`GET /cameras` returns names and go2rtc stream ids; the frontend's Cameras tab embeds
go2rtc's own player per stream. **This service never proxies video** — a Python HTTP
server in the path of an H.264 stream is how a working camera starts stuttering.
Network cameras only. Every USB camera in this project is aimed at one fixed thing at
an angle useless for anything else.
## Fleet scripts — the monitoring-agent admin surface
The **Fleet scripts** tab holds one script per kind of machine, uploaded through the
browser and fetched by each endpoint's `fleet-bootstrap` timer
(`tools/fleet-bootstrap.sh`). It is how every machine in the house gets a CheckMK
agent without SSHing into eleven hosts and pasting the same installer.
| Slot | Covers | Fetched by a device? |
|---|---|---|
| `debian-x86` | thin clients, touch panel, kitchen display, door panel, workshop | yes |
| `raspbian-arm` | arm64 audio endpoints, any future Pi | yes |
| `container-host` | the Docker host — agent *plus* container and disk checks | yes |
| `llm-host` | the GPU machine — different packages, and the only host where VRAM is worth checking | yes |
| `esphome` | voice satellites, BLE proxies, RuView nodes | **no** |
| `network-appliance` | OPNsense, switches, cameras | **no** |
The last two **cannot run a script** — a microcontroller has no shell. Their slots hold
the CheckMK-*server* side (SNMP config, a special agent that polls them), and the UI
says so. They exist because "are the ESPs monitored?" deserves a visible answer rather
than being quietly out of scope.
Slots are a **fixed set**, not a free-form file list: each endpoint knows which slot is
its own without being told, and empty slots are *listed* — the thing someone setting
this up needs to see is which platforms are still uncovered.
### The rules that make this safe enough to have
This is remote code execution by design; there is no version of "distribute scripts to
the endpoints" that isn't. So the honesty is in the constraints:
- **Upload is not deploy.** An upload is a *draft*; endpoints only ever fetch the
*published* version, and publishing is a second, deliberate click. Same
propose-then-confirm rule as pantry-vision's stock writes — and it matters more here,
because what you are confirming runs as root on every machine in the house.
- **This service never executes anything.** It stores text and serves text. There is no
"deploy now" button, because a button that runs a script on eleven machines at once
is a button that breaks eleven machines at once.
- **Scripts live in SQLite, not on the SMB share** — the one place the "artefacts go on
the share" rule is wrong. The share is writable by anyone with the SMB password, and
a file executed as root on every endpoint must not be writable by that path: it would
make a share credential a whole-fleet code-execution credential with no history.
- **Every version is kept**, so rolling back is picking an older row — the same
append-only reasoning `git_ops.py` uses.
- **The endpoint verifies the sha256** before running, runs a version once, and
**reports back pass or fail**. A script that was *served* is not a script that
*succeeded*, and only the endpoint knows which. The admin page flags a host running
an older version than the one now published.
- **Opt-in per host**: no `/etc/fleet-bootstrap.conf`, no execution.
Worth saying in your own words before enabling it: **anyone who can publish here can
run code as root on every machine in the house.** `WORKSHOP_TOKEN` is the most powerful
credential in this project.
## Not built yet
Everything past step 1 of the feasibility note, and it is worth being explicit because
this component is the foundation the rest attaches to:
1. **The canvas `svg` window kind and its server-side renderers** (Graphviz first) —
the diagram half of `docs/workshop-assistant.md`. The frontend here is an inventory
editor plus cameras plus health; it is not yet the assistant's display surface.
2. **Spec research, guide lookup, camera identification.** Steps 2–6 of that doc. All
write into this schema; none exist.
3. **Nothing has run against a real deployment.** Covered by smoke tests against
temporary databases (projects, knowledge, hardware assignment including the
`in_use` exception, git commit/branch/deny-list, an unreachable health target), and
that is the whole of the testing. In particular: **the CheckMK API shape and the
Gitea branch-protection payload are written from documentation**, and both have
version-sensitive field names.

320
workshop/fleet.py Normal file
View File

@ -0,0 +1,320 @@
"""Fleet scripts — one uploaded script per platform, fetched by the endpoints.
WHAT THIS IS FOR
----------------
Getting every machine in the house into CheckMK monitoring means running an agent
installer on each of them, and the installer is not the same file on a Debian x86 thin
client, an arm64 Raspberry Pi audio endpoint and the GPU host. So: one slot per
platform, uploaded through the admin UI, fetched by each endpoint over HTTP. Every
machine in the house gets a slot, including the two kinds that cannot run a script at
all — see PLATFORMS.
The slots are a FIXED SET, not a free-form list of files. A fixed set means every
endpoint knows which slot is its own without being told, the UI can say plainly which
platforms are covered and which are still empty, and nobody has to invent a naming
convention that then has to be kept in two places.
THIS SERVICE NEVER EXECUTES ANYTHING
-------------------------------------
It stores text and it serves text. Execution happens on the endpoint, by that
endpoint's own systemd unit, under that endpoint's own root — which is exactly where
the decision to run it belongs. Nothing here reaches out to a machine, and there is no
"deploy now" button, because a button that runs a script on eleven machines at once is
a button that breaks eleven machines at once.
UPLOAD IS NOT DEPLOY
--------------------
An upload creates a new **draft** version. Endpoints only ever fetch the **published**
one, and publishing is a second, deliberate action. This is the same propose-then-
confirm rule pantry-vision applies to stock writes and identity applies to merges, and
it matters more here than in either: the thing being confirmed will run as root on
every machine in the house.
WHY THE SCRIPTS LIVE IN SQLITE AND NOT ON THE SMB SHARE
--------------------------------------------------------
The workspace share is writable by anyone with the SMB password. A file that will be
executed as root on every endpoint must not be writable by that path — it would mean a
share credential is silently a whole-fleet code-execution credential, with no version
history and no record of who changed what. In the database each version is immutable,
checksummed, and attributed. This is the one place where "artefacts go on the share"
is the wrong rule.
WHAT AN ENDPOINT SEES, AND WHAT IT REPORTS
-------------------------------------------
`GET /fleet/script/<platform>` returns the published body plus its sha256 and version.
The endpoint runs it and POSTs back what it ran. That report is the only way to answer
"is this machine actually monitored yet", and it is deliberately reported by the
endpoint rather than assumed by this service: a script that was served is not a script
that succeeded.
"""
from __future__ import annotations
import hashlib
import logging
import os
import sqlite3
from datetime import datetime, timezone
from pathlib import Path
LOG = logging.getLogger("workshop.fleet")
DB_PATH = Path(os.environ.get("WORKSHOP_KNOWLEDGE_DB_PATH", "/data/workshop-knowledge.db"))
# One slot per KIND OF MACHINE in the house — every machine gets monitored, so every
# machine needs a slot. Fixed set on purpose (see the module docstring): each endpoint
# knows which slot is its own without being told, and the UI can show which platforms
# are still uncovered, which is the thing somebody setting this up needs to see.
#
# The split is by "would the installer actually differ", not by role. A thin client and
# a touch panel are both Debian x86 running the same agent, so they share a slot; the
# container host does not, because a Docker host wants the agent plus container checks
# and its own plugin set. Splitting a slot later is easy; merging two that people have
# already written different scripts into is not.
PLATFORMS = {
"debian-x86": {
"label": "Debian / x86_64 kiosk endpoints",
"covers": "thin clients, touch panel, kitchen display, door panel, workshop display",
"runs_on_device": True,
},
"raspbian-arm": {
"label": "Raspberry Pi OS / arm64 endpoints",
"covers": "the arm64 audio endpoints, and any future Pi",
"runs_on_device": True,
},
"container-host": {
"label": "Docker / container host",
"covers": "the Phase 1 machine running the whole compose stack — the agent plus "
"container and disk checks, which is a different install from a kiosk's",
"runs_on_device": True,
},
"llm-host": {
"label": "LLM / GPU host",
"covers": "the Ollama machine — a different package set, and the one host where "
"GPU temperature and VRAM are worth checking at all",
"runs_on_device": True,
},
"esphome": {
"label": "ESP32 devices (voice satellites, BLE proxies, RuView nodes)",
"covers": "microcontrollers. THEY CANNOT RUN A SHELL SCRIPT — this slot holds the "
"CheckMK-side piece instead (an active check or a special agent that "
"polls their ESPHome/MQTT state from the monitoring server). The slot "
"exists so 'have we got the ESPs monitored?' has a visible answer "
"rather than being quietly out of scope",
# Nothing fetches this slot: no endpoint of this kind runs an agent.
"runs_on_device": False,
},
"network-appliance": {
"label": "Firewalls, switches and network cameras",
"covers": "OPNsense and anything else with no place to install an agent. Same "
"shape as the ESP slot: the script belongs on the CheckMK server "
"(SNMP or a special agent), not on the device",
# Nothing fetches this slot: no endpoint of this kind runs an agent.
"runs_on_device": False,
},
}
MAX_SCRIPT_BYTES = int(os.environ.get("WORKSHOP_MAX_SCRIPT_KB", "256")) * 1024
def _now() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def init_db() -> None:
with sqlite3.connect(DB_PATH, timeout=10) as conn:
conn.executescript(
"""
-- Every version ever uploaded, kept. Rolling back is picking an older row,
-- which is only possible if nothing deletes them — the same append-only
-- reasoning git_ops.py applies to history, for the same reason: what runs
-- as root on the fleet must always be recoverable.
CREATE TABLE IF NOT EXISTS fleet_scripts (
id INTEGER PRIMARY KEY,
platform TEXT NOT NULL,
version INTEGER NOT NULL,
body TEXT NOT NULL,
-- Shown in the UI and returned to the endpoint, which records what it
-- ran. This is how "what is actually on that machine" stops being a
-- guess.
sha256 TEXT NOT NULL,
note TEXT,
-- Upload is not deploy: a new row starts unpublished and endpoints
-- never see it until somebody publishes it deliberately.
published INTEGER NOT NULL DEFAULT 0,
uploaded_at TEXT NOT NULL,
published_at TEXT
);
CREATE UNIQUE INDEX IF NOT EXISTS fleet_scripts_version
ON fleet_scripts (platform, version);
-- What each endpoint says it ran. REPORTED, never assumed: a script that
-- was served is not a script that succeeded, and the difference is the
-- whole point of having this table.
CREATE TABLE IF NOT EXISTS fleet_reports (
id INTEGER PRIMARY KEY,
hostname TEXT NOT NULL,
platform TEXT NOT NULL,
version INTEGER,
sha256 TEXT,
ok INTEGER NOT NULL DEFAULT 0,
detail TEXT,
reported_at TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS fleet_reports_host ON fleet_reports (hostname, reported_at);
"""
)
def _sha256(body: str) -> str:
return hashlib.sha256(body.encode("utf-8")).hexdigest()
def upload(platform: str, body: str, note: str = "") -> dict:
"""Store a new draft version. Never published by this call."""
if platform not in PLATFORMS:
return {"ok": False, "reason": "bad_platform",
"message": f"Unknown platform. Known: {', '.join(sorted(PLATFORMS))}."}
body = str(body or "")
if not body.strip():
return {"ok": False, "reason": "bad_field", "message": "The script is empty."}
if len(body.encode("utf-8")) > MAX_SCRIPT_BYTES:
return {"ok": False, "reason": "too_large",
"message": f"Scripts are capped at {MAX_SCRIPT_BYTES // 1024} KB. Something "
"bigger than that is a package, not a bootstrap script."}
digest = _sha256(body)
with sqlite3.connect(DB_PATH, timeout=10) as conn:
conn.row_factory = sqlite3.Row
existing = conn.execute(
"SELECT id, version, published FROM fleet_scripts WHERE platform = ? AND sha256 = ? "
"ORDER BY version DESC LIMIT 1", (platform, digest)
).fetchone()
if existing:
# Byte-identical to a version already stored. Storing it again would create
# two versions nobody can tell apart, and the honest answer to "did this
# change?" is no.
return {"ok": True, "unchanged": True, "version": existing["version"],
"sha256": digest, "published": bool(existing["published"]),
"message": f"Identical to version {existing['version']} — nothing stored."}
row = conn.execute(
"SELECT MAX(version) AS v FROM fleet_scripts WHERE platform = ?", (platform,)
).fetchone()
version = int((row["v"] or 0)) + 1
conn.execute(
"INSERT INTO fleet_scripts (platform, version, body, sha256, note, published, "
"uploaded_at) VALUES (?, ?, ?, ?, ?, 0, ?)",
(platform, version, body, digest, str(note or "").strip() or None, _now()),
)
LOG.info("workshop: stored %s script v%d (%s), unpublished", platform, version, digest[:12])
return {"ok": True, "version": version, "sha256": digest, "published": False,
"message": "Stored as a draft. Endpoints will not fetch it until you publish it."}
def publish(platform: str, version: int) -> dict:
"""Make one version the one endpoints fetch. Exactly one per platform is live."""
if platform not in PLATFORMS:
return {"ok": False, "reason": "bad_platform", "message": "Unknown platform."}
with sqlite3.connect(DB_PATH, timeout=10) as conn:
conn.row_factory = sqlite3.Row
row = conn.execute(
"SELECT id FROM fleet_scripts WHERE platform = ? AND version = ?", (platform, version)
).fetchone()
if row is None:
return {"ok": False, "reason": "no_such_version", "message": "No such version."}
# Unpublish the rest first: two published versions of one platform would make
# "what do the endpoints run" unanswerable, which is the question this exists
# to answer.
conn.execute("UPDATE fleet_scripts SET published = 0 WHERE platform = ?", (platform,))
conn.execute(
"UPDATE fleet_scripts SET published = 1, published_at = ? WHERE id = ?",
(_now(), row["id"]),
)
LOG.info("workshop: published %s script v%d", platform, version)
return {"ok": True, "platform": platform, "version": version}
def published(platform: str) -> dict | None:
"""What an endpoint of this platform should run, or None."""
if platform not in PLATFORMS:
return None
with sqlite3.connect(DB_PATH, timeout=10) as conn:
conn.row_factory = sqlite3.Row
row = conn.execute(
"SELECT version, body, sha256, note, published_at FROM fleet_scripts "
"WHERE platform = ? AND published = 1", (platform,)
).fetchone()
return {k: row[k] for k in row.keys()} if row else None
def overview() -> dict:
"""Every slot, whether it is filled, and which endpoints have reported.
Empty slots are listed rather than omitted — "no script for arm64 yet" is the thing
somebody setting this up needs to see, and a UI that only lists what exists cannot
show it.
"""
slots = []
with sqlite3.connect(DB_PATH, timeout=10) as conn:
conn.row_factory = sqlite3.Row
for platform, meta in PLATFORMS.items():
versions = [
{k: r[k] for k in r.keys()}
for r in conn.execute(
"SELECT version, sha256, note, published, uploaded_at, published_at "
"FROM fleet_scripts WHERE platform = ? ORDER BY version DESC LIMIT 10",
(platform,),
)
]
live = next((v for v in versions if v["published"]), None)
slots.append({
"platform": platform,
"label": meta["label"],
"covers": meta["covers"],
# False means no device fetches this — the script belongs on the CheckMK
# server instead. The UI says so rather than showing a slot that looks
# broken because nothing ever reports against it.
"runs_on_device": meta.get("runs_on_device", True),
"published_version": live["version"] if live else None,
"published_sha256": live["sha256"] if live else None,
"versions": versions,
})
reports = [
{k: r[k] for k in r.keys()}
for r in conn.execute(
"SELECT hostname, platform, version, sha256, ok, detail, MAX(reported_at) "
"AS reported_at FROM fleet_reports GROUP BY hostname ORDER BY hostname"
)
]
# The cross-check that makes the page worth opening: a host that ran an older
# version than the one now published is drifted, and nothing else here would say so.
live_by_platform = {s["platform"]: s["published_version"] for s in slots}
for report in reports:
expected = live_by_platform.get(report["platform"])
report["current"] = expected is not None and report["version"] == expected
return {"slots": slots, "reports": reports}
def report(payload: dict) -> dict:
"""An endpoint saying what it ran and how it went."""
hostname = str(payload.get("hostname") or "").strip()
platform = str(payload.get("platform") or "").strip()
if not hostname or platform not in PLATFORMS:
return {"ok": False, "reason": "bad_field",
"message": "'hostname' and a known 'platform' are both required."}
try:
version = int(payload.get("version"))
except (TypeError, ValueError):
version = None
with sqlite3.connect(DB_PATH, timeout=10) as conn:
conn.execute(
"INSERT INTO fleet_reports (hostname, platform, version, sha256, ok, detail, "
"reported_at) VALUES (?, ?, ?, ?, ?, ?, ?)",
(hostname, platform, version, str(payload.get("sha256") or "") or None,
1 if payload.get("ok") else 0, str(payload.get("detail") or "")[:2000], _now()),
)
LOG.info("workshop: %s reported %s v%s (ok=%s)", hostname, platform, version,
bool(payload.get("ok")))
return {"ok": True}

496
workshop/frontend/app.js Normal file
View File

@ -0,0 +1,496 @@
// workshop inventory editor. Vanilla JS, no framework, no build step — same choice as
// pantry-vision/frontend and the canvas SDKs.
//
// Config comes from URL query params (?api=...&token=...), never hardcoded: this file
// is a generic static asset served read-only, with no secret in it.
//
// WHY THIS PAGE EXISTS AT ALL, given the assistant can write the same rows: the
// inventory is the one table that describes *physical reality* — where a thing is,
// whether it is still on the shelf — and physical reality drifts without telling
// anybody. Every other table here records something that was said or decided, and
// those are only ever wrong if someone recorded them wrong. This one goes stale by
// itself, so it needs somewhere a person can sit down and fix it.
"use strict";
const params = new URLSearchParams(location.search);
const API = (params.get("api") || "").replace(/\/$/, "");
const TOKEN = params.get("token") || "";
if (!API || !TOKEN) {
document.body.innerHTML =
'<p class="error" style="padding:24px">workshop not configured — missing ?api=&token= in the URL.</p>';
throw new Error("workshop: missing ?api=/&token= query params");
}
const $ = (id) => document.getElementById(id);
const STATUS_LABELS = {
available: "Available",
assigned: "Reserved",
in_use: "In use",
retired: "Retired",
};
// The sentence under each status, shown on the row. Reserved and in-use look similar
// in a list and mean very different things when you are deciding whether you can grab
// something right now, so the list says which out loud rather than relying on a colour.
const STATUS_HINTS = {
available: "on the shelf, unpromised",
assigned: "spoken for, but still on the shelf",
in_use: "installed and working — taking it back means taking something apart",
retired: "dead, sold or given away",
};
function api(path, options) {
options = options || {};
options.headers = Object.assign({ Authorization: `Bearer ${TOKEN}` }, options.headers || {});
return fetch(`${API}${path}`, options).then((res) =>
res.json().catch(() => ({})).then((body) => {
if (!res.ok) throw new Error(body.message || body.error || `${res.status} ${res.statusText}`);
return body;
})
);
}
const send = (path, body, method) =>
api(path, {
method: method || "POST",
headers: { "Content-Type": "application/json" },
body: body === undefined ? undefined : JSON.stringify(body),
});
function escapeHtml(s) {
return String(s ?? "").replace(/[&<>"']/g, (c) =>
({ "&": "&amp;", "<": "&lt;", ">": "&gt;", '"': "&quot;", "'": "&#39;" }[c])
);
}
function setStatus(text, isError) {
$("status").textContent = text || "";
$("status").className = isError ? "error" : "muted";
}
// --- state -------------------------------------------------------------------------
let hardware = [];
let projects = [];
function load() {
const query = new URLSearchParams();
if ($("search").value.trim()) query.set("q", $("search").value.trim());
if ($("filter-status").value) query.set("status", $("filter-status").value);
if ($("filter-project").value) query.set("project", $("filter-project").value);
return api(`/hardware?${query}`)
.then((data) => {
hardware = data.hardware || [];
render();
})
.catch((err) => {
$("list").innerHTML = `<p class="error">Could not load: ${escapeHtml(err.message)}</p>`;
});
}
function loadProjects() {
return api("/projects")
.then((data) => {
projects = data.projects || [];
const options =
'<option value="">— none —</option>' +
projects
.map((p) => `<option value="${escapeHtml(p.slug)}">${escapeHtml(p.name)}</option>`)
.join("");
$("f-project").innerHTML = options;
$("filter-project").innerHTML =
'<option value="">Any project</option>' +
projects
.map((p) => `<option value="${escapeHtml(p.slug)}">${escapeHtml(p.name)}</option>`)
.join("");
})
.catch(() => {
// A workshop with no projects yet is the normal first-run state, not an error.
});
}
function render() {
const counts = hardware.reduce((acc, h) => {
acc[h.status] = (acc[h.status] || 0) + 1;
return acc;
}, {});
$("counts").textContent = hardware.length
? Object.keys(STATUS_LABELS)
.filter((s) => counts[s])
.map((s) => `${counts[s]} ${STATUS_LABELS[s].toLowerCase()}`)
.join(" · ")
: "";
if (!hardware.length) {
$("list").innerHTML =
'<p class="muted">Nothing here yet. "Add hardware" records what you own and — the part that ' +
'actually saves time — where you put it.</p>';
return;
}
$("list").innerHTML = hardware.map(renderRow).join("");
wireRows();
}
function renderRow(item) {
const projectOptions =
'<option value="">— none —</option>' +
projects
.map(
(p) =>
`<option value="${escapeHtml(p.slug)}"${p.slug === item.project_slug ? " selected" : ""}>${escapeHtml(
p.name
)}</option>`
)
.join("");
const statusOptions = Object.keys(STATUS_LABELS)
.map(
(s) =>
`<option value="${s}"${s === item.status ? " selected" : ""}>${escapeHtml(STATUS_LABELS[s])}</option>`
)
.join("");
return `<article class="item status-${escapeHtml(item.status)}" data-id="${item.id}">
<div class="item-head">
<input class="designation" type="text" value="${escapeHtml(item.designation)}" aria-label="Designation">
<input class="kind" type="text" value="${escapeHtml(item.kind)}" placeholder="kind" aria-label="Kind">
<input class="quantity" type="number" min="0" step="1" value="${escapeHtml(item.quantity)}" aria-label="Quantity">
</div>
<div class="item-body">
<label class="field">Where
<input class="location" type="text" value="${escapeHtml(item.storage_location)}" placeholder="drawer 3, blue box">
</label>
<label class="field">Status
<select class="status">${statusOptions}</select>
</label>
<label class="field">Project
<select class="project">${projectOptions}</select>
</label>
</div>
<p class="hint">${escapeHtml(STATUS_HINTS[item.status] || "")}</p>
<div class="item-actions">
<button class="save primary">Save</button>
<button class="delete danger">Delete</button>
<span class="row-status muted"></span>
</div>
</article>`;
}
function wireRows() {
$("list").querySelectorAll(".item").forEach((el) => {
const id = Number(el.dataset.id);
const note = el.querySelector(".row-status");
// Changing the project is the edit people come here to make, so it saves on the
// spot rather than waiting for a Save nobody remembers to press. Everything else
// is free text mid-typing and waits.
el.querySelector(".project").addEventListener("change", () => saveRow(el, id, note));
el.querySelector(".status").addEventListener("change", () => saveRow(el, id, note));
el.querySelector(".save").addEventListener("click", () => saveRow(el, id, note));
el.querySelector(".delete").addEventListener("click", () => {
const name = el.querySelector(".designation").value.trim();
if (!window.confirm(`Delete ${name}? This removes the record, not the thing.`)) return;
send(`/hardware/${id}`, undefined, "DELETE")
.then(() => load())
.catch((err) => {
note.textContent = err.message;
note.className = "row-status error";
});
});
});
}
function saveRow(el, id, note) {
const body = {
designation: el.querySelector(".designation").value.trim(),
kind: el.querySelector(".kind").value.trim(),
quantity: Number(el.querySelector(".quantity").value) || 0,
storage_location: el.querySelector(".location").value.trim(),
// Both are sent together so the server's own coupling rule decides the outcome —
// including the one that matters, that naming a project on something already
// `in_use` does not quietly demote it to merely reserved.
status: el.querySelector(".status").value,
project_slug: el.querySelector(".project").value || null,
};
note.textContent = "Saving…";
note.className = "row-status muted";
send(`/hardware/${id}`, body)
.then(() => load())
.catch((err) => {
note.textContent = err.message;
note.className = "row-status error";
});
}
// --- add ---------------------------------------------------------------------------
$("add-btn").addEventListener("click", () => {
$("add-form").hidden = !$("add-form").hidden;
if (!$("add-form").hidden) $("f-designation").focus();
});
$("add-cancel").addEventListener("click", () => {
$("add-form").hidden = true;
});
$("add-form").addEventListener("submit", (event) => {
event.preventDefault();
setStatus("Adding…");
send("/hardware", {
designation: $("f-designation").value.trim(),
kind: $("f-kind").value.trim(),
quantity: Number($("f-quantity").value) || 1,
storage_location: $("f-location").value.trim(),
status: $("f-status").value,
project_slug: $("f-project").value || null,
notes: $("f-notes").value.trim(),
})
.then(() => {
$("add-form").reset();
$("add-form").hidden = true;
setStatus("");
return load();
})
.catch((err) => setStatus(err.message, true));
});
// --- filters -----------------------------------------------------------------------
let searchTimer = null;
$("search").addEventListener("input", () => {
clearTimeout(searchTimer);
searchTimer = setTimeout(load, 200);
});
$("filter-status").addEventListener("change", load);
$("filter-project").addEventListener("change", load);
// --- Tabs ---------------------------------------------------------------------------
document.querySelectorAll(".tab").forEach((tab) => {
tab.addEventListener("click", () => {
document.querySelectorAll(".tab").forEach((t) => t.classList.toggle("active", t === tab));
document.querySelectorAll(".panel").forEach((p) => p.classList.toggle("active", p.id === tab.dataset.tab));
if (tab.dataset.tab === "cameras") loadCameras();
if (tab.dataset.tab === "fleet") loadFleet();
});
});
// --- Cameras --------------------------------------------------------------------------
// go2rtc's own stream page, in an iframe, one per camera. Deliberately NOT a hand-rolled
// WebRTC client: go2rtc already ships a player that negotiates WebRTC and falls back to
// MSE on its own, and reimplementing that here would be a second thing to keep working
// against a moving browser target. The workshop service never proxies the video — a
// Python HTTP server in the path of an H.264 stream is how a working camera starts
// stuttering.
function loadCameras() {
const el = $("camera-grid");
api("/cameras")
.then((data) => {
const cameras = data.cameras || [];
const base = (data.go2rtc_url || "").replace(/\/$/, "");
if (!cameras.length || !base) {
el.innerHTML =
'<p class="muted">No cameras configured — set WORKSHOP_CAMERAS and GO2RTC_URL in ' +
"workshop.env. Both are needed: the names live here, the streams live in go2rtc.</p>";
return;
}
el.innerHTML = cameras
.map(
(cam) => `<figure class="camera">
<iframe src="${escapeHtml(base)}/stream.html?src=${encodeURIComponent(cam.stream)}&mode=webrtc"
loading="lazy" allowfullscreen title="${escapeHtml(cam.name)}"></iframe>
<figcaption>${escapeHtml(cam.name)}</figcaption>
</figure>`
)
.join("");
})
.catch((err) => {
el.innerHTML = `<p class="error">Could not load cameras: ${escapeHtml(err.message)}</p>`;
});
}
// --- Fleet scripts --------------------------------------------------------------------
// The admin surface for getting every machine into CheckMK. Two deliberate properties
// show up in this UI rather than being buried: an empty slot is LISTED (the thing a
// person setting this up needs to see is which platforms are still uncovered), and
// publish is a separate button from upload.
function loadFleet() {
api("/fleet")
.then((data) => {
$("fleet-slots").innerHTML = (data.slots || []).map(renderSlot).join("");
wireFleet();
const reports = data.reports || [];
$("fleet-reports").innerHTML = reports.length
? reports
.map(
(r) => `<div class="item ${r.ok ? (r.current ? "status-available" : "status-assigned") : "status-retired"}">
<div class="item-head">
<strong>${escapeHtml(r.hostname)}</strong>
<span class="muted">${escapeHtml(r.platform)} · v${escapeHtml(r.version ?? "?")}</span>
<span class="muted">${r.ok ? "ok" : "FAILED"}${
r.ok && !r.current ? " · behind the published version" : ""
}</span>
</div>
<p class="hint">${escapeHtml(relativeTimeish(r.reported_at))}${
r.detail ? ` — ${escapeHtml(String(r.detail).slice(-200))}` : ""
}</p>
</div>`
)
.join("")
: '<p class="muted">Nothing has reported yet. An endpoint appears here the first ' +
"time its timer runs, which is also how you find out the timer is working.</p>";
})
.catch((err) => {
$("fleet-slots").innerHTML = `<p class="error">Could not load: ${escapeHtml(err.message)}</p>`;
});
}
function renderSlot(slot) {
const live = slot.published_version;
const versions = (slot.versions || [])
.map(
(v) => `<li>
v${v.version} · <code>${escapeHtml(String(v.sha256).slice(0, 12))}</code>
${v.published ? '<strong>· published</strong>' : `<button class="row-btn publish-btn"
data-platform="${escapeHtml(slot.platform)}" data-version="${v.version}">Publish</button>`}
${v.note ? `<span class="muted"> — ${escapeHtml(v.note)}</span>` : ""}
</li>`
)
.join("");
return `<article class="item ${live ? "status-available" : "status-retired"}">
<div class="item-head">
<strong>${escapeHtml(slot.label)}</strong>
<span class="muted">${escapeHtml(slot.covers)}</span>
</div>
<p class="hint">${
live
? `Published: v${live} · <code>${escapeHtml(String(slot.published_sha256).slice(0, 12))}</code>`
: "<strong>Nothing published for this platform yet.</strong>"
}</p>
${
slot.runs_on_device
? ""
: '<p class="hint"><strong>These devices cannot run a script.</strong> This slot holds the ' +
"CheckMK-<em>server</em> side instead — an SNMP config or a special agent that polls them. " +
"Nothing fetches it, so no endpoint will ever report against it; the slot exists so " +
'"are these monitored?" has a visible answer.</p>'
}
<label class="field">Paste the script
<textarea class="script-body" rows="6" data-platform="${escapeHtml(slot.platform)}"
placeholder="#!/bin/sh&#10;# CheckMK agent install for ${escapeHtml(slot.label)}"></textarea>
</label>
<div class="item-actions">
<input class="script-note" type="text" placeholder="what changed (optional)">
<button class="row-btn upload-btn" data-platform="${escapeHtml(slot.platform)}">Upload as draft</button>
<span class="row-status muted"></span>
</div>
${versions ? `<ul class="versions">${versions}</ul>` : ""}
</article>`;
}
function wireFleet() {
$("fleet-slots").querySelectorAll(".upload-btn").forEach((btn) =>
btn.addEventListener("click", () => {
const card = btn.closest(".item");
const note = card.querySelector(".row-status");
const body = card.querySelector(".script-body").value;
if (!body.trim()) {
note.textContent = "Nothing to upload.";
return;
}
note.textContent = "Uploading…";
send("/fleet/upload", {
platform: btn.dataset.platform,
body,
note: card.querySelector(".script-note").value.trim(),
})
.then((r) => {
note.textContent = r.message || "Stored as a draft.";
return loadFleet();
})
.catch((err) => {
note.textContent = err.message;
note.className = "row-status error";
});
})
);
$("fleet-slots").querySelectorAll(".publish-btn").forEach((btn) =>
btn.addEventListener("click", () => {
if (
!window.confirm(
`Publish v${btn.dataset.version} for ${btn.dataset.platform}?\n\n` +
"Every endpoint of this platform will run it as root the next time its timer fires."
)
)
return;
send("/fleet/publish", {
platform: btn.dataset.platform,
version: Number(btn.dataset.version),
})
.then(() => loadFleet())
.catch((err) => window.alert(err.message));
})
);
}
// --- System health --------------------------------------------------------------------
// One pill in the header rather than a page you have to remember to open: health you
// only see when you go looking for it is health you find out about from the failure.
const HEALTH_LABELS = { ok: "All OK", problem: "Problems", unreachable: "Unreachable" };
function loadHealth() {
return api("/health")
.then((data) => {
const pill = $("health-pill");
if (!data.configured) {
// Nothing being watched and everything being fine look identical from here, and
// only one of them is good news — so this says which.
pill.textContent = "health: not configured";
pill.className = "pill unknown";
$("health-body").innerHTML =
'<p class="muted">No CheckMK server and no firewalls configured. Nothing is being ' +
"watched, which is not the same as nothing being wrong.</p>";
return;
}
const overall = data.overall || "ok";
pill.textContent = HEALTH_LABELS[overall] || overall;
pill.className = `pill ${overall}`;
$("health-body").innerHTML = (data.results || [])
.map((row) => {
const since = row.since ? ` — since ${escapeHtml(relativeTimeish(row.since))}` : "";
const problems = (row.detail || [])
.map((p) => `<li>${escapeHtml([p.host, p.service, p.state, p.output].filter(Boolean).join(" · "))}</li>`)
.join("");
return `<article class="health-row ${escapeHtml(row.state)}">
<h3>${escapeHtml(row.target)} <span class="muted">(${escapeHtml(row.source)})</span></h3>
<p>${escapeHtml(row.state)}${since}</p>
<p class="muted">${escapeHtml(row.summary)}</p>
${problems ? `<ul>${problems}</ul>` : ""}
</article>`;
})
.join("") || '<p class="muted">No samples yet — the poller runs every few minutes.</p>';
})
.catch(() => {
$("health-pill").textContent = "health: unknown";
$("health-pill").className = "pill unknown";
});
}
function relativeTimeish(iso) {
const then = Date.parse(iso || "");
if (!Number.isFinite(then)) return iso || "an unknown time";
const mins = Math.round((Date.now() - then) / 60000);
if (mins < 60) return `${mins} min ago`;
const hours = Math.round(mins / 60);
return hours < 48 ? `${hours}h ago` : `${Math.round(hours / 24)}d ago`;
}
$("health-pill").addEventListener("click", () => {
loadHealth();
$("health-dialog").showModal();
});
loadHealth();
setInterval(loadHealth, 60000);
loadProjects().then(load);

123
workshop/frontend/index.html Normal file
View File

@ -0,0 +1,123 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Workshop</title>
<link rel="stylesheet" href="style.css">
</head>
<body>
<!--
workshop's frontend. Vanilla JS, no build step, no framework — the same
"vendored, dependency-free" choice as the digest/admin canvas SDKs and
pantry-vision's kiosk page. Config (API URL + bearer token) arrives as URL query
params, never baked in, so this stays a static asset with no secret in git.
Scope: editing the hardware inventory by hand. Everything else the workshop service
holds — projects, decisions, workflow notes, facts — is written by the assistant or
over the API; the inventory is the one table a person needs to sit down and correct,
because it is the one that describes physical reality and drifts from it.
-->
<header id="top">
<h1>Workshop</h1>
<nav id="tabs">
<button class="tab active" data-tab="inventory">Inventory</button>
<button class="tab" data-tab="cameras">Cameras</button>
<button class="tab" data-tab="fleet">Fleet scripts</button>
</nav>
<span id="counts" class="muted"></span>
<!-- System health, always visible, never a separate screen you have to remember to
open. One dot: green/amber/red, with the detail behind a click. -->
<button id="health-pill" class="pill" title="System health">…</button>
</header>
<dialog id="health-dialog">
<h2>System health</h2>
<div id="health-body"><p class="muted">Loading…</p></div>
<form method="dialog"><button class="primary">Close</button></form>
</dialog>
<main>
<section id="inventory" class="panel active">
<div id="controls">
<input id="search" type="search" placeholder="Search designation, kind or location…" autocomplete="off">
<select id="filter-status">
<option value="">Any status</option>
<option value="available">Available — on the shelf, unpromised</option>
<option value="assigned">Reserved — spoken for, still on the shelf</option>
<option value="in_use">In use — installed and working</option>
<option value="retired">Retired — dead, sold, given away</option>
</select>
<select id="filter-project"><option value="">Any project</option></select>
<button id="add-btn" class="primary">Add hardware</button>
</div>
<p id="status" class="muted"></p>
<form id="add-form" hidden>
<div class="grid">
<label>What is printed on it
<input id="f-designation" type="text" required placeholder="LSI 9211-8i">
</label>
<label>Kind
<input id="f-kind" type="text" placeholder="HBA, PSU, RAM…">
</label>
<label>How many
<input id="f-quantity" type="number" min="0" step="1" value="1">
</label>
<label>Where it is
<input id="f-location" type="text" placeholder="drawer 3, antistatic bag">
</label>
<label>Status
<select id="f-status">
<option value="available">Available</option>
<option value="assigned">Reserved for a project</option>
<option value="in_use">In use</option>
<option value="retired">Retired</option>
</select>
</label>
<label>Project
<select id="f-project"><option value="">— none —</option></select>
</label>
<label class="wide">Notes
<input id="f-notes" type="text" placeholder="anything worth knowing next time you pick it up">
</label>
</div>
<div class="actions">
<button type="button" id="add-cancel">Cancel</button>
<button type="submit" class="primary">Add it</button>
</div>
</form>
<div id="list"><p class="muted">Loading…</p></div>
</section>
<section id="fleet" class="panel">
<p class="muted">
One monitoring-agent script per platform, fetched by each endpoint's
<code>fleet-bootstrap</code> timer. <strong>Uploading is not deploying</strong> —
a new upload is a draft until you publish it, because the thing you are confirming
runs as root on every machine in the house.
</p>
<div id="fleet-slots"><p class="muted">Loading…</p></div>
<h2>What the endpoints report</h2>
<p class="muted">
Reported by each machine after it runs, never assumed here: a script that was
served is not a script that succeeded.
</p>
<div id="fleet-reports"></div>
</section>
<section id="cameras" class="panel">
<p class="muted">
The network cameras, straight from go2rtc. The USB cameras in this house are all
aimed at one fixed thing at an angle useless for anything else, so they are not
here.
</p>
<div id="camera-grid" class="camera-grid"><p class="muted">Loading…</p></div>
</section>
</main>
<script src="app.js"></script>
</body>
</html>

196
workshop/frontend/style.css Normal file
View File

@ -0,0 +1,196 @@
/* workshop inventory editor styling.
*
* Deliberately NOT the purple/magenta holo theme from docs/workshop-assistant.md: that
* is for the canvas surfaces the assistant draws on, where the look is the point. This
* is a form you sit at and correct data in, and a glow behind a text field is a cost
* with no benefit. The theme belongs on the display; the editor belongs to your eyes.
*/
* { box-sizing: border-box; }
html, body {
margin: 0;
min-height: 100%;
background: #14101c;
color: #e9e2f5;
font-family: system-ui, sans-serif;
}
#top {
display: flex;
align-items: baseline;
gap: 16px;
padding: 18px 24px;
border-bottom: 1px solid rgba(200, 120, 255, 0.22);
}
#top h1 { margin: 0; font-size: 20px; }
main { padding: 20px 24px 60px; max-width: 1100px; }
.muted { color: #a98fc4; font-size: 14px; }
.error { color: #ff8080; font-size: 14px; }
#controls {
display: flex;
flex-wrap: wrap;
gap: 10px;
margin-bottom: 14px;
}
#controls input[type="search"] { flex: 1; min-width: 220px; }
input, select, button {
min-height: 42px;
padding: 6px 10px;
font-size: 15px;
border-radius: 8px;
border: 1px solid rgba(200, 120, 255, 0.24);
background: #1d1629;
color: #e9e2f5;
}
button { cursor: pointer; }
button.primary { background: #c084fc; color: #17101f; border-color: transparent; font-weight: 600; }
button.danger { border-color: rgba(255, 120, 120, 0.4); color: #ff9c9c; }
#add-form {
border: 1px solid rgba(200, 120, 255, 0.24);
border-radius: 12px;
padding: 16px;
margin-bottom: 18px;
background: rgba(28, 12, 44, 0.5);
}
.grid {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(210px, 1fr));
gap: 12px;
}
.grid label, .field {
display: flex;
flex-direction: column;
gap: 4px;
font-size: 13px;
color: #a98fc4;
}
.grid label.wide { grid-column: 1 / -1; }
.actions { display: flex; gap: 10px; justify-content: flex-end; margin-top: 14px; }
/* --- rows ------------------------------------------------------------------------ */
.item {
border: 1px solid rgba(200, 120, 255, 0.18);
border-left-width: 5px;
border-radius: 12px;
padding: 14px;
margin-bottom: 12px;
background: rgba(28, 12, 44, 0.35);
}
/* Status carried on the left edge, so a long list is scannable without reading every
* row — and reinforced by the sentence under each one, because reserved and in-use are
* the pair people confuse and a colour alone would not settle it. */
.status-available { border-left-color: #35d488; }
.status-assigned { border-left-color: #e0a000; }
.status-in_use { border-left-color: #ff3ec8; }
.status-retired { border-left-color: #5b4a6e; opacity: 0.7; }
.item-head { display: flex; gap: 10px; flex-wrap: wrap; }
.item-head .designation { flex: 2; min-width: 200px; font-weight: 600; }
.item-head .kind { flex: 1; min-width: 120px; }
.item-head .quantity { width: 90px; }
.item-body {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));
gap: 12px;
margin-top: 10px;
}
.item .hint { margin: 8px 0 0; font-size: 13px; color: #a98fc4; }
.item-actions { display: flex; align-items: center; gap: 10px; margin-top: 12px; }
.row-status { font-size: 13px; }
/* --- Tabs, cameras, health ---------------------------------------------------------- */
#tabs { display: flex; gap: 6px; }
.tab {
min-height: 36px;
padding: 4px 14px;
background: transparent;
border-color: transparent;
color: #a98fc4;
}
.tab.active { background: rgba(200, 120, 255, 0.18); color: #e9e2f5; }
.panel { display: none; }
.panel.active { display: block; }
/* The health pill sits in the header and is never a page you have to remember to open:
* health you only see when you go looking for it is health you learn about from the
* failure instead. Colour AND text, because a coloured dot alone is a guess. */
.pill {
margin-left: auto;
min-height: 34px;
padding: 2px 14px;
border-radius: 999px;
font-size: 13px;
font-weight: 600;
}
.pill.ok { background: rgba(53, 212, 136, 0.18); border-color: #35d488; color: #9ff0c4; }
.pill.problem { background: rgba(224, 160, 0, 0.18); border-color: #e0a000; color: #ffd98a; }
.pill.unreachable { background: rgba(255, 90, 90, 0.18); border-color: #ff5a5a; color: #ffb3b3; }
.pill.unknown { background: rgba(255, 255, 255, 0.08); color: #a98fc4; }
#health-dialog {
border: 1px solid rgba(200, 120, 255, 0.3);
border-radius: 14px;
background: #1a1226;
color: #e9e2f5;
max-width: 640px;
width: 90vw;
}
#health-dialog::backdrop { background: rgba(0, 0, 0, 0.6); }
.health-row { border-left: 4px solid #5b4a6e; padding-left: 12px; margin: 14px 0; }
.health-row.ok { border-left-color: #35d488; }
.health-row.problem { border-left-color: #e0a000; }
.health-row.unreachable { border-left-color: #ff5a5a; }
.health-row h3 { margin: 0 0 4px; font-size: 16px; }
.health-row p { margin: 2px 0; font-size: 14px; }
.health-row ul { margin: 6px 0 0; padding-left: 18px; font-size: 13px; color: #a98fc4; }
.camera-grid {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(320px, 1fr));
gap: 12px;
}
.camera { margin: 0; }
.camera iframe {
width: 100%;
aspect-ratio: 16 / 9;
border: 1px solid rgba(200, 120, 255, 0.24);
border-radius: 10px;
background: #000;
}
.camera figcaption { font-size: 14px; color: #a98fc4; padding-top: 6px; }
/* --- Fleet scripts ------------------------------------------------------------------ */
.script-body {
width: 100%;
font-family: ui-monospace, monospace;
font-size: 13px;
padding: 10px;
border-radius: 8px;
border: 1px solid rgba(200, 120, 255, 0.24);
background: #120d1b;
color: #e9e2f5;
resize: vertical;
}
.script-note { flex: 1; min-width: 160px; }
.versions { margin: 10px 0 0; padding-left: 18px; font-size: 13px; color: #a98fc4; }
.versions li { margin: 4px 0; }
.versions code { color: #c084fc; }
#fleet h2 { font-size: 17px; margin: 22px 0 4px; }

261
workshop/git_ops.py Normal file
View File

@ -0,0 +1,261 @@
"""Git for workshop projects: the assistant may write history, never rewrite it.
THE RULE, STATED ONCE
---------------------
**Everything that ADDS to history is allowed. Nothing that REMOVES from it is.**
Commit, push, branch, tag, merge: yes, unattended. Force-push, rebase, amend, reset
--hard, filter-branch, filter-repo, branch deletion, tag deletion, reflog expiry, gc
--prune: never, by any path, for any reason, including a good one.
The reasoning is a rollback guarantee. If history is append-only, then whatever the
assistant did wrong is *recoverable by looking at an earlier commit* — the worst case
is a bad commit you revert. The moment rewriting is on the table, the worst case
becomes work that no longer exists anywhere, and no amount of care makes that
recoverable after the fact.
WHEN HISTORY GENUINELY HAS TO BE SCRUBBED
------------------------------------------
It does happen — an API token committed by accident is the real case, and leaving it
in history is worse than the rewrite. So this module **writes the commands out and
hands them to you**, and you run them yourself from the repository on the SMB share.
See scrub_instructions().
That is not theatre. The person running `git filter-repo` can see what is about to
happen, has the repo in front of them, and can take a copy first. A service doing it
on a timer at 3am cannot. The manual step *is* the safety mechanism, and automating it
would remove the only thing making it safe.
CLIENT-SIDE REFUSAL IS A POLICY, NOT A CONTROL
-----------------------------------------------
Everything in this file is a convention that a different process with the same token
could ignore. The actual guarantee is **Gitea branch protection**, which is applied on
repo creation (see gitea.py's protect_branch) and which refuses force-pushes and
deletions server-side. This module and that protection say the same thing twice, on
purpose: one of them is the intent, the other is the enforcement.
"""
from __future__ import annotations
import logging
import os
import re
import shlex
import subprocess
from pathlib import Path
LOG = logging.getLogger("workshop.git")
GIT_AUTHOR_NAME = os.environ.get("WORKSHOP_GIT_AUTHOR_NAME", "workshop assistant")
GIT_AUTHOR_EMAIL = os.environ.get("WORKSHOP_GIT_AUTHOR_EMAIL", "workshop@localhost")
GIT_TIMEOUT = float(os.environ.get("WORKSHOP_GIT_TIMEOUT", "120"))
GIT_DEFAULT_BRANCH = os.environ.get("WORKSHOP_GIT_BRANCH", "main")
# Every one of these either destroys history or can be made to. Matched against the
# whole argument list of any git invocation this module builds, as a backstop against
# a future caller assembling one from parameters — the deny-list is cheap and the
# thing it prevents is not recoverable.
FORBIDDEN_ARGS = {
"--force", "-f", "--force-with-lease", "--force-if-includes",
"filter-branch", "filter-repo", "rebase", "reset", "gc", "prune", "reflog",
"--amend", "--hard", "--delete", "--prune", "-D", "--allow-unrelated-histories",
}
# `git push --delete <ref>` and `git branch -d` are spelled without the tokens above in
# some forms, so subcommands that only ever remove are refused outright.
FORBIDDEN_SUBCOMMANDS = {"filter-branch", "filter-repo", "rebase", "reset", "gc", "reflog", "replace"}
class HistoryRewriteRefused(RuntimeError):
"""Raised when a command would remove history. Never caught-and-continued."""
def _check(args: list[str]) -> None:
if args and args[0] in FORBIDDEN_SUBCOMMANDS:
raise HistoryRewriteRefused(f"`git {args[0]}` rewrites history and is never run by this service")
for arg in args:
if arg in FORBIDDEN_ARGS:
raise HistoryRewriteRefused(f"`{arg}` can remove history and is never passed by this service")
def _git(repo: Path, args: list[str]) -> tuple[int, str, str]:
_check(args)
env = {
**os.environ,
"GIT_AUTHOR_NAME": GIT_AUTHOR_NAME,
"GIT_AUTHOR_EMAIL": GIT_AUTHOR_EMAIL,
"GIT_COMMITTER_NAME": GIT_AUTHOR_NAME,
"GIT_COMMITTER_EMAIL": GIT_AUTHOR_EMAIL,
# Never sit waiting for a passphrase or a username on a service with no tty.
"GIT_TERMINAL_PROMPT": "0",
}
proc = subprocess.run(
["git", *args], cwd=repo, env=env, capture_output=True, text=True, timeout=GIT_TIMEOUT
)
return proc.returncode, proc.stdout.strip(), proc.stderr.strip()
def _code_dir(project_dir: Path) -> Path:
"""The git working tree is `code/`, NOT the project root.
The rest of the workspace — datasheets, photos, generated diagrams — deliberately
stays outside version control. They are large, binary, and already backed by the
share; more importantly, keeping them out means no git operation of any kind has a
path to them. `git clean` cannot reach a datasheet that was never in the repo.
"""
return project_dir / "code"
def ensure_repo(project_dir: Path, remote_url: str = "") -> dict:
"""Initialise `code/` as a repository if it isn't one, and set its remote."""
code = _code_dir(project_dir)
try:
code.mkdir(parents=True, exist_ok=True)
except OSError as exc:
return {"ok": False, "reason": "workspace_unavailable", "message": str(exc)}
if not (code / ".git").is_dir():
rc, _, err = _git(code, ["init", "-b", GIT_DEFAULT_BRANCH])
if rc != 0:
return {"ok": False, "reason": "git_error", "message": err}
LOG.info("workshop: initialised git repo at %s", code)
if remote_url:
rc, out, _ = _git(code, ["remote"])
if "origin" in out.split():
_git(code, ["remote", "set-url", "origin", remote_url])
else:
_git(code, ["remote", "add", "origin", remote_url])
return {"ok": True, "path": str(code)}
def status(project_dir: Path) -> dict:
code = _code_dir(project_dir)
if not (code / ".git").is_dir():
return {"ok": True, "initialised": False, "path": str(code)}
rc, changed, _ = _git(code, ["status", "--porcelain"])
# `branch --show-current`, not `rev-parse --abbrev-ref HEAD`: on a repo with no
# commits yet the latter reports the literal string "HEAD", which reads on screen
# as a detached head rather than as "brand new, nothing committed".
_, branch, _ = _git(code, ["branch", "--show-current"])
_, log, _ = _git(code, ["log", "-5", "--pretty=%h %s"])
return {
"ok": rc == 0,
"initialised": True,
"path": str(code),
"branch": branch,
"dirty": bool(changed),
"changed_files": [line[3:] for line in changed.splitlines() if line[3:]],
"recent": log.splitlines(),
}
def commit(project_dir: Path, message: str, push: bool = True) -> dict:
"""Stage everything under `code/`, commit, and push. Additive only.
No `--amend`, ever, including for "just a typo in the message" — an amend after a
push is a force-push waiting to happen, and a typo in a commit message costs
nothing compared to the rule staying simple enough to be true.
"""
code = _code_dir(project_dir)
if not (code / ".git").is_dir():
return {"ok": False, "reason": "not_initialised", "message": "No repository yet — create one first."}
message = str(message or "").strip()
if not message:
return {"ok": False, "reason": "bad_field", "message": "A commit message is required."}
rc, _, err = _git(code, ["add", "-A"])
if rc != 0:
return {"ok": False, "reason": "git_error", "message": err}
rc, changed, _ = _git(code, ["status", "--porcelain"])
if not changed:
return {"ok": True, "committed": False, "message": "Nothing to commit."}
rc, out, err = _git(code, ["commit", "-m", message])
if rc != 0:
return {"ok": False, "reason": "git_error", "message": err or out}
result = {"ok": True, "committed": True, "detail": out.splitlines()[:2]}
if push:
rc, out, err = _git(code, ["push", "-u", "origin", "HEAD"])
result["pushed"] = rc == 0
if rc != 0:
# A failed push is not a failed commit — the work is safe locally on the
# share, which is the whole reason committing and pushing are reported
# separately rather than as one boolean.
result["push_error"] = err or out
LOG.warning("workshop: commit succeeded but push failed: %s", err or out)
return result
def branch(project_dir: Path, name: str) -> dict:
"""Create and switch to a branch. Creating only — there is no delete counterpart
in this module, because deleting a branch is how commits stop being reachable."""
code = _code_dir(project_dir)
if not re.match(r"^[A-Za-z0-9][A-Za-z0-9._/-]{0,100}$", str(name or "")):
return {"ok": False, "reason": "bad_field", "message": "That isn't a usable branch name."}
rc, out, err = _git(code, ["switch", "-c", name])
return {"ok": rc == 0, "detail": out or err}
def scrub_instructions(project_dir: Path, secret_path: str = "", note: str = "") -> dict:
"""The commands to remove something from history — printed, never run.
Called when something that must not be in history is found in it: a token, a key, a
password. This service will not do it. What it does is write out exactly what to
run, from the repository on the share, so the person doing it can read it first and
take a copy.
The instructions deliberately lead with "rotate the secret first". Scrubbing a
leaked token from git is the *second* thing to do — the commit was pushed, and a
token that has been on a server for an hour must be assumed compromised whether or
not the history is cleaned afterwards. A scrub that makes people feel finished
without rotating is worse than no scrub.
"""
code = _code_dir(project_dir)
target = shlex.quote(secret_path) if secret_path else "PATH/TO/FILE"
return {
"ok": True,
"manual": True,
"why": (
"Removing anything from history is never done by this service — see git_ops.py. "
"Run these yourself, from the repository on the share, so you can read them "
"first and take a copy."
),
"note": note or "",
"repo_path": str(code),
"steps": [
{
"order": 1,
"do": "ROTATE THE SECRET FIRST.",
"why": "It was pushed. Assume it is compromised whether or not you clean the history. "
"Everything below is damage limitation, not a fix.",
"command": "",
},
{
"order": 2,
"do": "Take a copy of the repository before touching it.",
"why": "A rewrite is the one operation with no undo. A copy is the undo.",
"command": f"cp -a {shlex.quote(str(code))} {shlex.quote(str(code) + '.backup')}",
},
{
"order": 3,
"do": "Remove the file from every commit.",
"why": "git-filter-repo is the maintained tool; filter-branch is deprecated and slower.",
"command": f"cd {shlex.quote(str(code))} && git filter-repo --invert-paths --path {target}",
},
{
"order": 4,
"do": "Re-add the remote and force-push.",
"why": "filter-repo drops the remote on purpose, so that this step has to be deliberate.",
"command": "git remote add origin <URL> && git push --force --all && git push --force --tags",
},
{
"order": 5,
"do": "Turn branch protection back on in Gitea if step 4 required turning it off.",
"why": "Protection is what stops this from being possible unattended. Leaving it off "
"quietly removes the guarantee the whole arrangement rests on.",
"command": "",
},
],
}

209
workshop/gitea.py Normal file
View File

@ -0,0 +1,209 @@
"""Gitea repositories for workshop projects.
The workshop assistant produces things that want version control — firmware, scripts,
KiCad files, a config it has been iterating on — and this household already runs Gitea.
So a project can have a repository, created on request, recorded against the project.
WHAT THIS DOES
--------------
Creates repositories, reports what exists, and — the part that matters —
**applies branch protection at creation so history cannot be rewritten.** Committing
and pushing live in `git_ops.py`; the assistant does both, unattended.
The line is not "may it write to git" but "may it *remove* anything": adding to
history is allowed because the worst case is a bad commit you revert, while rewriting
history has no undo. `protect_branch()` below is what makes that a control rather than
a promise — Gitea refuses the force-push regardless of what asked for it, including a
human who typed `--force` out of habit.
The git working tree is `code/` inside the project workspace, never the workspace
root, so no git operation of any kind has a path to the datasheets and photos sitting
beside it.
THE TOKEN
---------
Gitea needs an **API token** with permission to create repositories — Settings →
Applications → Generate New Token, scope `write:repository` (plus `write:user` if you
want it to create repos under your own account rather than an organisation). That
token can create and delete repositories, so:
- It lives in `workshop.env`, 600, like every other credential in this project.
- **Repository deletion is not implemented here at all**, and neither is any history
rewrite. The token can technically do both; nothing in this service gives them a
path. When history genuinely has to be scrubbed — a committed token — see
`git_ops.scrub_instructions()`, which writes the commands out for you to run by
hand.
- Prefer a token on a **dedicated Gitea user** ("workshop-bot") with access to one
organisation, over one on your own account with access to everything you own. The
blast radius of a leaked env file is then one org of generated repos.
"""
from __future__ import annotations
import json
import logging
import os
import re
import urllib.error
import urllib.request
LOG = logging.getLogger("workshop.gitea")
GITEA_URL = os.environ.get("GITEA_URL", "").rstrip("/")
GITEA_TOKEN = os.environ.get("GITEA_TOKEN", "")
# Which account or organisation new repositories belong to. Empty means the token's own
# user, which works and is the worse default — see the module docstring.
GITEA_OWNER = os.environ.get("GITEA_OWNER", "").strip()
GITEA_PRIVATE = os.environ.get("GITEA_PRIVATE_REPOS", "true").strip().lower() != "false"
GITEA_TIMEOUT = float(os.environ.get("GITEA_TIMEOUT", "15"))
# Gitea's own constraint on repository names, applied here so a bad name is a clear
# refusal from this service rather than a 422 from somewhere else.
REPO_NAME_RE = re.compile(r"^[A-Za-z0-9._-]{1,100}$")
def configured() -> bool:
return bool(GITEA_URL and GITEA_TOKEN)
def _request(method: str, path: str, payload: dict | None = None) -> tuple[int, dict]:
data = json.dumps(payload).encode() if payload is not None else None
req = urllib.request.Request(f"{GITEA_URL}/api/v1{path}", data=data, method=method)
req.add_header("Content-Type", "application/json")
# Gitea accepts `Authorization: token <t>`; the bearer form is not universally
# honoured across versions, so use the documented one.
req.add_header("Authorization", f"token {GITEA_TOKEN}")
try:
with urllib.request.urlopen(req, timeout=GITEA_TIMEOUT) as resp:
body = resp.read()
return resp.status, (json.loads(body) if body else {})
except urllib.error.HTTPError as exc:
body = exc.read()
try:
parsed = json.loads(body) if body else {}
except ValueError:
parsed = {"message": body.decode("utf-8", "replace")[:300]}
return exc.code, parsed
def create_repo(name: str, description: str = "", private: bool | None = None) -> dict:
"""Create a repository, or report the one that is already there.
**An existing repository of that name is a success, not a conflict.** Asking twice
means the second caller wanted a repo to exist, which it does — and returning an
error there would make every retry look like a failure while leaving the assistant
with no URL to record.
"""
if not configured():
return {
"ok": False,
"reason": "not_configured",
"message": "GITEA_URL and GITEA_TOKEN are not set — see workshop.env.example.",
}
name = str(name or "").strip()
if not REPO_NAME_RE.match(name):
return {
"ok": False,
"reason": "bad_name",
"message": "A repository name is letters, digits, dot, dash or underscore.",
}
payload = {
"name": name,
"description": str(description or "")[:255],
"private": GITEA_PRIVATE if private is None else bool(private),
# An empty repo is the right starting point: the workspace already has the
# files, and an auto-generated README is the first merge conflict.
"auto_init": False,
}
path = f"/orgs/{GITEA_OWNER}/repos" if GITEA_OWNER else "/user/repos"
status, body = _request("POST", path, payload)
if status in (200, 201):
LOG.info("workshop: created Gitea repository %s", body.get("full_name") or name)
summary = _repo_summary(body)
summary["protection"] = protect_branch(body.get("owner", {}).get("login", ""), name)
return {"ok": True, "created": True, "repo": summary}
if status == 409:
existing = get_repo(name)
if existing.get("ok"):
return {"ok": True, "created": False, "repo": existing["repo"]}
LOG.warning("workshop: Gitea repo creation failed (%s): %s", status, body.get("message"))
return {
"ok": False,
"reason": "gitea_error",
"status": status,
"message": body.get("message") or f"Gitea returned {status}.",
}
def protect_branch(owner: str, repo: str, branch: str = "") -> dict:
"""Refuse force-pushes and branch deletion on the default branch, server-side.
THIS IS THE ENFORCEMENT. `git_ops.py` refuses to rewrite history, but that is a
convention any process holding the same token could ignore. Branch protection is
the control: Gitea rejects the push regardless of what asked for it, including a
human at a terminal who typed --force out of habit.
Applied at creation because retrofitting it means a window where it wasn't on, and
that window is exactly when a new repo gets its messy first pushes.
Best-effort: a failure here is reported, not raised. A repo without protection is
still a usable repo, and refusing to create one would trade a real capability for a
guarantee that git_ops.py already provides in the normal case. **The response says
which you got** — never assume protection is on because a repo exists.
VERIFY against your Gitea: the branch-protection payload's field names have changed
across versions (`branch_name` vs `rule_name` in particular), and this is written
from the documented shape, not against a live instance.
"""
if not owner:
return {"applied": False, "reason": "owner unknown"}
payload = {
"rule_name": branch or "main",
"branch_name": branch or "main",
"enable_push": True,
# The two that matter.
"enable_force_push": False,
"enable_delete": False,
}
status, body = _request("POST", f"/repos/{owner}/{repo}/branch_protections", payload)
if status in (200, 201):
return {"applied": True, "branch": payload["rule_name"]}
LOG.warning(
"workshop: could not apply branch protection to %s/%s (%s): %s — history is protected "
"only by this service's own refusal until you set it in Gitea by hand",
owner, repo, status, body.get("message"),
)
return {"applied": False, "status": status, "message": body.get("message")}
def get_repo(name: str) -> dict:
if not configured():
return {"ok": False, "reason": "not_configured", "message": "Gitea is not configured."}
owner = GITEA_OWNER
if not owner:
status, user = _request("GET", "/user")
if status != 200:
return {"ok": False, "reason": "gitea_error", "message": "Could not resolve the token's own user."}
owner = user.get("login", "")
status, body = _request("GET", f"/repos/{owner}/{name}")
if status == 200:
return {"ok": True, "repo": _repo_summary(body)}
return {"ok": False, "reason": "not_found", "message": f"No repository {owner}/{name}."}
def _repo_summary(body: dict) -> dict:
"""Only the fields anything here needs. Gitea's repo object is ~60 keys, and
storing or echoing all of them would make this service's API shape hostage to
Gitea's."""
return {
"name": body.get("name"),
"full_name": body.get("full_name"),
"html_url": body.get("html_url"),
"clone_url": body.get("clone_url"),
"ssh_url": body.get("ssh_url"),
"private": body.get("private"),
}

344
workshop/health.py Normal file
View File

@ -0,0 +1,344 @@
"""Infrastructure health: CheckMK and every OPNsense firewall, polled on a timer.
WHY THIS LIVES IN `workshop` AND NOT IN `digest-engine`
--------------------------------------------------------
Both consume it, and it was tempting to put the polling in the digest since the digest
already reads one firewall. That would have been wrong in one specific way: the digest
runs **four times a day**, and "is the NAS disk failing right now" is not a question
with a six-hour answer. So the poller lives in the always-on service, keeps a small
history, and the digest reads *that* — one poller, two consumers, and the digest gets
trend ("this has been critical since Tuesday") instead of a snapshot it cannot compare
to anything.
WHAT IT POLLS, AND WHAT IT WILL NEVER DO
-----------------------------------------
- **CheckMK**: `GET /check_mk/api/1.0/domain-types/{host,service}/collections/all`,
read-only, with a Guest-role automation user. Never acknowledges, never downtimes,
never reschedules. Those are the endpoints a monitoring integration is *expected* to
call, and the reason not to is that an assistant silencing an alert is
indistinguishable from the alert being fixed.
- **OPNsense**: `GET /api/ids/service/status` only — is Suricata running. The alert
*query* stays in digest-engine, which already does it properly; duplicating the
paging logic here would give the household two different answers about the same log.
MULTIPLE FIREWALLS ARE THE POINT
---------------------------------
`OPNSENSE_JSON` carries a list, and every row this module writes carries the firewall's
`name`. A household with a main and a DMZ firewall must never be told "the IDS is
running" — the honest sentence is "main is running, dmz has not answered since 14:20",
and that is only possible if the name survives all the way to the display.
EVERY FAILURE IS A ROW, NOT AN EXCEPTION
-----------------------------------------
A target that cannot be reached is recorded as `unreachable` with the error attached.
That is the whole point of a health poller: silence must be visible. A poll that
raises and logs would leave the last good row in place and the display would keep
showing green for a machine that has been off for a day.
"""
from __future__ import annotations
import base64
import json
import logging
import os
import sqlite3
import threading
import time
import urllib.error
import urllib.parse
import urllib.request
from datetime import datetime, timedelta, timezone
from pathlib import Path
LOG = logging.getLogger("workshop.health")
# Same database as the rest of the knowledge store: this is a table about the house's
# machines, which is exactly the "durable facts" category knowledge.py holds.
DB_PATH = Path(os.environ.get("WORKSHOP_KNOWLEDGE_DB_PATH", "/data/workshop-knowledge.db"))
CHECKMK_URL = os.environ.get("CHECKMK_BASE_URL", "").rstrip("/")
CHECKMK_SITE = os.environ.get("CHECKMK_SITE", "cmk").strip()
CHECKMK_USERNAME = os.environ.get("CHECKMK_USERNAME", "")
CHECKMK_SECRET = os.environ.get("CHECKMK_SECRET", "")
CHECKMK_ONLY_PROBLEMS = os.environ.get("CHECKMK_ONLY_PROBLEMS", "true").lower() != "false"
CHECKMK_MAX_ROWS = int(os.environ.get("CHECKMK_MAX_ROWS", "200"))
POLL_INTERVAL = int(os.environ.get("WORKSHOP_HEALTH_INTERVAL_SECONDS", "300"))
HTTP_TIMEOUT = float(os.environ.get("WORKSHOP_HEALTH_TIMEOUT", "20"))
# Long enough to see a pattern ("this flaps every night"), short enough that the table
# stays small. Unlike knowledge.py's own tables, samples DO expire: a service state
# from three weeks ago is an observation, not a fact, and observations go stale — the
# same distinction doorway.py draws.
RETENTION_DAYS = int(os.environ.get("WORKSHOP_HEALTH_RETENTION_DAYS", "30"))
# CheckMK's numeric states, which appear in the API as integers with no labels.
HOST_STATES = {0: "up", 1: "down", 2: "unreachable"}
SERVICE_STATES = {0: "ok", 1: "warning", 2: "critical", 3: "unknown"}
# What counts as "needs a human". `unknown` is included deliberately: a check that
# cannot report is not a check that passed.
BAD_SERVICE_STATES = {1, 2, 3}
def _now() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def firewalls() -> list[dict]:
"""The firewall list, from the JSON blob the config export writes."""
raw = os.environ.get("OPNSENSE_JSON", "").strip()
if not raw:
return []
try:
parsed = json.loads(raw)
except ValueError:
LOG.warning("workshop: OPNSENSE_JSON is not valid JSON — no firewalls will be polled")
return []
entries = parsed.get("firewalls") if isinstance(parsed, dict) else parsed
return [f for f in (entries or []) if isinstance(f, dict) and f.get("base_url")]
def init_db() -> None:
with sqlite3.connect(DB_PATH, timeout=10) as conn:
conn.executescript(
"""
-- One row per target per poll. A time series, not a current-state table:
-- "the NAS has been critical since Tuesday" is the sentence worth being
-- able to say, and a single-row-per-target design cannot say it.
CREATE TABLE IF NOT EXISTS infra_status (
id INTEGER PRIMARY KEY,
-- 'checkmk' | 'opnsense'
source TEXT NOT NULL,
-- Which instance. The firewall's name, or the CheckMK site. NEVER empty:
-- a household with two firewalls must never be told "the IDS is running".
target TEXT NOT NULL,
-- ok | problem | unreachable. Three, not two: "I could not ask" is a
-- different fact from "I asked and it is broken", and collapsing them is
-- how a display shows green for a machine that has been off all day.
state TEXT NOT NULL,
summary TEXT NOT NULL,
-- The problem rows themselves, as JSON, so the display can list them
-- without a second query and the digest can quote them.
detail TEXT,
problem_count INTEGER NOT NULL DEFAULT 0,
checked_at TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS infra_by_target ON infra_status (source, target, checked_at);
"""
)
def _record(source: str, target: str, state: str, summary: str,
detail: list | None = None, problem_count: int = 0) -> None:
try:
with sqlite3.connect(DB_PATH, timeout=10) as conn:
conn.execute(
"INSERT INTO infra_status (source, target, state, summary, detail, problem_count, "
"checked_at) VALUES (?, ?, ?, ?, ?, ?, ?)",
(source, target, state, summary, json.dumps(detail or []), problem_count, _now()),
)
cutoff = (datetime.now(timezone.utc) - timedelta(days=RETENTION_DAYS)).isoformat()
conn.execute("DELETE FROM infra_status WHERE checked_at < ?", (cutoff.replace("+00:00", "Z"),))
except sqlite3.Error:
LOG.warning("workshop: could not record health for %s/%s", source, target, exc_info=True)
# --- CheckMK ----------------------------------------------------------------------
def _checkmk_get(path: str) -> tuple[int, dict]:
url = f"{CHECKMK_URL}/{CHECKMK_SITE}/check_mk/api/1.0{path}"
req = urllib.request.Request(url)
# CheckMK's own scheme: `Authorization: Bearer <user> <automation secret>`. It is
# not an OAuth bearer despite the word, which is why this is built by hand.
req.add_header("Authorization", f"Bearer {CHECKMK_USERNAME} {CHECKMK_SECRET}")
req.add_header("Accept", "application/json")
try:
with urllib.request.urlopen(req, timeout=HTTP_TIMEOUT) as resp:
return resp.status, json.loads(resp.read() or b"{}")
except urllib.error.HTTPError as exc:
return exc.code, {"detail": exc.read().decode("utf-8", "replace")[:200]}
except (urllib.error.URLError, TimeoutError, OSError, ValueError) as exc:
return 0, {"detail": str(exc)}
def poll_checkmk() -> dict | None:
"""Hosts and services that are not OK. None when CheckMK isn't configured."""
if not (CHECKMK_URL and CHECKMK_USERNAME and CHECKMK_SECRET):
return None
target = CHECKMK_SITE or "checkmk"
columns = "&columns=name&columns=state&columns=description&columns=plugin_output&columns=host_name"
query = "?query=" + urllib.parse.quote(json.dumps(
{"op": "!=", "left": "state", "right": "0"}
)) if CHECKMK_ONLY_PROBLEMS else ""
status, hosts = _checkmk_get(f"/domain-types/host/collections/all{query}{columns}")
if status != 200:
message = hosts.get("detail") or f"HTTP {status}"
_record("checkmk", target, "unreachable", f"CheckMK did not answer: {message}")
return {"source": "checkmk", "target": target, "state": "unreachable", "summary": message}
status, services = _checkmk_get(f"/domain-types/service/collections/all{query}{columns}")
service_rows = services.get("value", []) if status == 200 else []
problems = []
for row in (hosts.get("value") or [])[:CHECKMK_MAX_ROWS]:
extensions = row.get("extensions", {})
state = extensions.get("state")
if state:
problems.append({
"kind": "host",
"host": extensions.get("name") or row.get("title"),
"state": HOST_STATES.get(state, str(state)),
})
for row in service_rows[:CHECKMK_MAX_ROWS]:
extensions = row.get("extensions", {})
state = extensions.get("state")
if state in BAD_SERVICE_STATES:
problems.append({
"kind": "service",
"host": extensions.get("host_name"),
"service": extensions.get("description"),
"state": SERVICE_STATES.get(state, str(state)),
"output": (extensions.get("plugin_output") or "")[:200],
})
state = "problem" if problems else "ok"
summary = (
f"{len(problems)} problem(s): "
+ ", ".join(f"{p.get('host')}{'/' + p['service'] if p.get('service') else ''} {p['state']}"
for p in problems[:5])
if problems else "everything CheckMK watches is OK"
)
_record("checkmk", target, state, summary, problems, len(problems))
return {"source": "checkmk", "target": target, "state": state, "summary": summary,
"problems": problems}
# --- OPNsense ---------------------------------------------------------------------
def poll_firewall(firewall: dict) -> dict:
"""Is Suricata running on this firewall? Read-only, one endpoint.
Deliberately NOT the alert query — digest-engine owns that, including the paging
and the window logic, and two implementations of the same read would eventually
give the household two different answers about one log file.
"""
name = str(firewall.get("name") or "opnsense")
base = str(firewall.get("base_url") or "").rstrip("/")
auth = base64.b64encode(
f"{firewall.get('api_key', '')}:{firewall.get('api_secret', '')}".encode()
).decode()
req = urllib.request.Request(f"{base}/api/ids/service/status")
req.add_header("Authorization", f"Basic {auth}")
context = None
if firewall.get("verify_tls") is False:
# OPNsense ships a self-signed certificate. Off is a real choice a household
# makes; it is recorded in the row's summary so nobody later mistakes a
# working poll for a verified one.
import ssl
context = ssl._create_unverified_context() # noqa: S323
try:
with urllib.request.urlopen(req, timeout=HTTP_TIMEOUT, context=context) as resp:
body = json.loads(resp.read() or b"{}")
except Exception as exc: # noqa: BLE001 — every failure is a row, see the docstring
_record("opnsense", name, "unreachable", f"{name} did not answer: {exc}")
return {"source": "opnsense", "target": name, "state": "unreachable", "summary": str(exc)}
running = str(body.get("status", "")).lower() == "running"
state = "ok" if running else "problem"
summary = (
f"Suricata is running on {name}" if running
else f"Suricata is NOT running on {name} (status: {body.get('status', 'unknown')}) — "
f"this firewall's IDS section of the digest will be empty, which is not the same "
f"as quiet"
)
_record("opnsense", name, state, summary, [body], 0 if running else 1)
return {"source": "opnsense", "target": name, "state": state, "summary": summary}
# --- the round ---------------------------------------------------------------------
def poll_all() -> dict:
results = []
checkmk = poll_checkmk()
if checkmk:
results.append(checkmk)
for firewall in firewalls():
results.append(poll_firewall(firewall))
return {"checked_at": _now(), "results": results}
def latest() -> dict:
"""The most recent sample per target, plus how long each state has held.
`since` is what makes this worth reading twice: "critical" tells you to look,
"critical since Tuesday 06:00" tells you whether it is new. Computed by walking
back through the samples until the state changes.
"""
try:
with sqlite3.connect(DB_PATH, timeout=10) as conn:
conn.row_factory = sqlite3.Row
rows = conn.execute(
"SELECT source, target, state, summary, detail, problem_count, MAX(checked_at) "
"AS checked_at FROM infra_status GROUP BY source, target ORDER BY source, target"
).fetchall()
results = []
for row in rows:
entry = {k: row[k] for k in row.keys()}
entry["detail"] = json.loads(entry.get("detail") or "[]")
changed = conn.execute(
"SELECT MAX(checked_at) AS t FROM infra_status WHERE source = ? AND target = ? "
"AND state != ?",
(row["source"], row["target"], row["state"]),
).fetchone()
entry["since"] = None
if changed and changed["t"]:
later = conn.execute(
"SELECT MIN(checked_at) AS t FROM infra_status WHERE source = ? AND "
"target = ? AND state = ? AND checked_at > ?",
(row["source"], row["target"], row["state"], changed["t"]),
).fetchone()
entry["since"] = later["t"] if later else None
results.append(entry)
except sqlite3.Error:
LOG.warning("workshop: could not read infra_status", exc_info=True)
return {"results": [], "available": False}
worst = "ok"
for entry in results:
if entry["state"] == "unreachable":
worst = "unreachable"
elif entry["state"] == "problem" and worst == "ok":
worst = "problem"
return {
"results": results,
"available": True,
# One word for the display's overlay to key off, so it does not re-derive
# "is anything wrong" from a list in four different places.
"overall": worst,
"configured": bool(CHECKMK_URL) or bool(firewalls()),
}
def start_poller() -> None:
"""Poll in the background for as long as the service runs.
A daemon thread rather than a systemd timer or a cron container: this service is
already always-on, the work is two HTTP calls, and a separate scheduler would be a
second thing to deploy and a second place for the credentials to live.
"""
if not (CHECKMK_URL or firewalls()):
LOG.info("workshop: no CheckMK and no firewalls configured — health polling is off")
return
def loop():
while True:
try:
poll_all()
except Exception: # noqa: BLE001 — a poller that dies is worse than a bad poll
LOG.warning("workshop: health poll round failed", exc_info=True)
time.sleep(max(60, POLL_INTERVAL))
threading.Thread(target=loop, name="health-poller", daemon=True).start()
LOG.info("workshop: health polling every %ds (checkmk=%s, firewalls=%d)",
max(60, POLL_INTERVAL), bool(CHECKMK_URL), len(firewalls()))

524
workshop/knowledge.py Normal file
View File

@ -0,0 +1,524 @@
"""workshop knowledge — the part the assistant is supposed to *learn*, in its own
database, kept forever.
WHY THIS IS A SECOND DATABASE AND NOT FOUR MORE TABLES IN workshop.db
---------------------------------------------------------------------
`workshop.db` is a **record of work**: this project, these decisions, this log. It is
scoped to things that happen and then are over. `knowledge.db` is a **record of what
is true**, and it outlives every project in it — the standing instruction about how you
solder, the pinout you looked up in March, the fact that the good multimeter lives in
the third drawer. Those two have different lifetimes, different backup value, and
different blast radius when one is wrong, and keeping them in one file would mean the
first `DROP`/rebuild of a project store took the second with it.
**NOTHING HERE IS EVER PRUNED.** Every other store in this project has a retention
window — `doorway.py` keeps sightings 30 days, `digest-engine`'s archive keeps 400 —
because stale observations are worse than none. This is the opposite kind of data: the
whole point is that you tell it once. A retention policy here would be a policy of
forgetting the thing the feature exists to remember, so there is deliberately no
cutoff, no cleanup pass, and no `created_at <` anywhere in this file.
THE FOUR TABLES, AND WHY EACH IS SEPARATE
------------------------------------------
They are all "things the assistant should know", but they are retrieved by different
keys, and that is what makes them different tables rather than one `facts` table with
a `type` column:
- `workflow_notes` — retrieved by **activity**. Standing instructions: "when I'm
soldering, always X". Surfaced whether or not anyone asked.
- `facts` — retrieved by **keyword**. Specifics: device specs, URLs, part
numbers. Surfaced when the subject comes up.
- `project_knowledge` — retrieved by **project**. What was learned *about this build*
that outlives its log entries.
- `hardware` — retrieved by **what you own**. Where a thing is stored, and what
it is currently promised to.
`GET /context` is what ties them together and is the endpoint that makes this a
learning system rather than four lists: given an activity, a subject and a project, it
returns everything that applies, so the assistant is told the standing considerations
instead of being reminded of them.
HARDWARE IS HERE, NOT IN workshop.db
-------------------------------------
An earlier cut had a `parts` table in the project store. That was a second inventory,
and two inventories is exactly the failure this repo's own docs keep warning about —
the one you didn't update becomes a lie, and you find out by buying a part you already
own. What you own is knowledge (durable, yours, survives every project); what a project
*needs* is a claim on it, which is the `project_slug` column here.
"""
from __future__ import annotations
import json
import logging
import os
import re
import sqlite3
from datetime import datetime, timezone
from pathlib import Path
LOG = logging.getLogger("workshop.knowledge")
DB_PATH = Path(os.environ.get("WORKSHOP_KNOWLEDGE_DB_PATH", "/data/workshop-knowledge.db"))
# What a piece of hardware can be. The middle two are the distinction that earns this
# column, and conflating them is the mistake worth spending four lines to avoid:
#
# available — on the shelf, unpromised. Fair game.
# assigned — RESERVED for a project. Still physically on the shelf, still something
# you could pick up, but spoken for. This is "I plan to use it for that".
# in_use — ACTUALLY INSTALLED and working somewhere. Getting it back means taking
# something apart, and probably means that something stops working.
# retired — dead, sold, or given away. Kept rather than deleted so "didn't I have
# one of those?" has an answer other than silence.
#
# Why they are not one "unavailable": the question you ask at 23:00 is "can I use this
# right now?", and "it's reserved for the NAS I haven't started" and "it's in the NAS,
# which is serving the house" are wildly different answers. The first is a decision you
# can revisit in a second; the second is an evening's work and an outage.
HARDWARE_STATUSES = ("available", "assigned", "in_use", "retired")
_SPLIT_RE = re.compile(r"[,;]+")
def _now() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def _keywords(value) -> str:
"""Keywords stored as a comma-separated, lowercased, de-duplicated string.
Not a join table. A join table is the correct schema and the wrong tool here: this
is a single-household knowledge base where the largest realistic query is "does any
row mention 'esp32'", and a LIKE against a normalised string answers that without
three tables and a migration. Lowercasing on write is what makes the match
case-insensitive without a function index.
"""
if isinstance(value, str):
parts = _SPLIT_RE.split(value)
elif isinstance(value, (list, tuple)):
parts = [str(v) for v in value]
else:
return ""
seen: list[str] = []
for part in parts:
word = " ".join(str(part).strip().lower().split())
if word and word not in seen:
seen.append(word)
return ",".join(seen)
def _db() -> sqlite3.Connection:
DB_PATH.parent.mkdir(parents=True, exist_ok=True)
conn = sqlite3.connect(DB_PATH, timeout=10)
conn.row_factory = sqlite3.Row
return conn
def init_db() -> None:
with _db() as conn:
conn.executescript(
"""
-- 1. THINGS TO ALWAYS NOTE ABOUT HOW I WORK -------------------------
-- Standing instructions tied to an activity, so the same considerations
-- do not have to be repeated every session. This is the table that stops
-- the assistant being told twice.
CREATE TABLE IF NOT EXISTS workflow_notes (
id INTEGER PRIMARY KEY,
-- The activity this applies to: 'soldering', 'pcb-design', 'laptop
-- repair'. The literal '*' means ALWAYS — advice that is true of every
-- session, which is deliberately a value rather than a second table so
-- that "always" is retrieved by the same query as everything else.
activity TEXT NOT NULL,
instruction TEXT NOT NULL,
-- Why the instruction exists. Optional, and the field most worth
-- filling: an instruction whose reason is recorded can be re-evaluated
-- when circumstances change, and one without a reason gets followed
-- forever or dropped for the wrong reasons.
why TEXT,
-- 1 = mention it every time; 2 = mention when relevant. Two levels on
-- purpose. A five-point scale invites tuning that nobody ever does, and
-- the only distinction that changes behaviour is "always say this" vs
-- "have this ready".
importance INTEGER NOT NULL DEFAULT 2,
keywords TEXT NOT NULL DEFAULT '',
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);
-- 2. SPECIFIC THINGS, FOUND BY KEYWORD ------------------------------
-- Device specs, URLs, part numbers, "the NAS is 192.168.30.40".
CREATE TABLE IF NOT EXISTS facts (
id INTEGER PRIMARY KEY,
subject TEXT NOT NULL,
body TEXT NOT NULL,
-- Where it came from. Same rule as the project store's parts: a fact
-- with no source is a fact somebody typed, which is fine and is
-- recorded as such. What must never happen is a spec arriving here
-- from a model's memory wearing a URL it did not read.
source_url TEXT,
keywords TEXT NOT NULL DEFAULT '',
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);
-- 3. WHAT WAS LEARNED ABOUT A PROJECT -------------------------------
-- Distinct from workshop.db's `notes`, and the distinction is lifetime:
-- a note is the running log of a session ("board arrived, started on the
-- PSU"), this is what remains true afterwards ("this board's BIOS needs
-- CSM off or the HBA won't post"). The log is disposable; this is not.
CREATE TABLE IF NOT EXISTS project_knowledge (
id INTEGER PRIMARY KEY,
-- The project's slug in workshop.db. Deliberately NOT a foreign key —
-- separate database, and more importantly the knowledge should survive
-- the project being deleted. What you learned building the NAS is still
-- true when the NAS is gone.
project_slug TEXT NOT NULL,
body TEXT NOT NULL,
keywords TEXT NOT NULL DEFAULT '',
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);
-- 4. WHAT I OWN, AND WHERE IT IS ------------------------------------
CREATE TABLE IF NOT EXISTS hardware (
id INTEGER PRIMARY KEY,
-- What is printed on it — the label-first principle from
-- docs/workshop-assistant.md, in the schema.
designation TEXT NOT NULL,
kind TEXT,
quantity INTEGER NOT NULL DEFAULT 1,
-- WHERE IT IS PHYSICALLY. Free text on purpose: "third drawer, blue
-- box", "under the bench", "lent to Linus". Any structure imposed here
-- would be a structure somebody has to maintain, and the value is
-- entirely in it being written down at all.
storage_location TEXT,
status TEXT NOT NULL DEFAULT 'available',
-- The claim. Setting this is what "I plan to use it for that" means,
-- and it is why you can ask what is already spoken for before ordering
-- more. Nulled when it goes back to available.
project_slug TEXT,
specs TEXT,
source_url TEXT,
notes TEXT,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS workflow_by_activity ON workflow_notes (activity);
CREATE INDEX IF NOT EXISTS knowledge_by_project ON project_knowledge (project_slug);
CREATE INDEX IF NOT EXISTS hardware_by_status ON hardware (status);
CREATE INDEX IF NOT EXISTS hardware_by_project ON hardware (project_slug);
"""
)
def _row(row) -> dict:
return {k: row[k] for k in row.keys()}
def _like(term: str) -> str:
return f"%{term.strip().lower()}%"
# --- 1. workflow notes ------------------------------------------------------------
def list_workflow_notes(activity: str = "") -> dict:
"""Notes for an activity, plus the always-notes. Always-first, then by importance.
An empty activity returns everything, because "show me what you think you know
about how I work" has to be answerable — a knowledge base you cannot audit is one
you stop trusting the moment it says something odd.
"""
with _db() as conn:
if activity:
rows = conn.execute(
"SELECT * FROM workflow_notes WHERE activity = '*' OR activity = ? OR keywords LIKE ? "
"ORDER BY (activity = '*') DESC, importance ASC, id ASC",
(activity.strip().lower(), _like(activity)),
).fetchall()
else:
rows = conn.execute(
"SELECT * FROM workflow_notes ORDER BY (activity = '*') DESC, importance ASC, id ASC"
).fetchall()
return {"workflow_notes": [_row(r) for r in rows]}
def add_workflow_note(payload: dict) -> dict:
instruction = str(payload.get("instruction") or "").strip()
if not instruction:
return {"ok": False, "reason": "bad_field", "message": "'instruction' is required."}
activity = str(payload.get("activity") or "*").strip().lower() or "*"
try:
importance = int(payload.get("importance", 2))
except (TypeError, ValueError):
importance = 2
importance = 1 if importance <= 1 else 2
with _db() as conn:
# A duplicate instruction for the same activity is almost always somebody
# telling it the same thing twice — which is the exact problem this table
# exists to solve, so it updates rather than accumulating near-identical rows.
existing = conn.execute(
"SELECT id FROM workflow_notes WHERE activity = ? AND lower(instruction) = lower(?)",
(activity, instruction),
).fetchone()
if existing:
conn.execute(
"UPDATE workflow_notes SET why = COALESCE(?, why), importance = ?, keywords = ?, "
"updated_at = ? WHERE id = ?",
(str(payload.get("why") or "").strip() or None, importance,
_keywords(payload.get("keywords")), _now(), existing["id"]),
)
return {"ok": True, "id": existing["id"], "updated": True}
cur = conn.execute(
"INSERT INTO workflow_notes (activity, instruction, why, importance, keywords, created_at, "
"updated_at) VALUES (?, ?, ?, ?, ?, ?, ?)",
(activity, instruction, str(payload.get("why") or "").strip() or None, importance,
_keywords(payload.get("keywords")), _now(), _now()),
)
return {"ok": True, "id": cur.lastrowid, "updated": False}
# --- 2. facts ---------------------------------------------------------------------
def list_facts(query: str = "") -> dict:
with _db() as conn:
if query:
rows = conn.execute(
"SELECT * FROM facts WHERE lower(subject) LIKE ? OR keywords LIKE ? OR lower(body) LIKE ? "
"ORDER BY subject",
(_like(query), _like(query), _like(query)),
).fetchall()
else:
rows = conn.execute("SELECT * FROM facts ORDER BY subject").fetchall()
return {"facts": [_row(r) for r in rows]}
def add_fact(payload: dict) -> dict:
subject = str(payload.get("subject") or "").strip()
body = str(payload.get("body") or "").strip()
if not subject or not body:
return {"ok": False, "reason": "bad_field", "message": "'subject' and 'body' are both required."}
body_value = body
if isinstance(payload.get("body"), (dict, list)):
body_value = json.dumps(payload["body"])
with _db() as conn:
existing = conn.execute("SELECT id FROM facts WHERE lower(subject) = lower(?)", (subject,)).fetchone()
if existing:
# Facts are corrected far more often than they are duplicated — a spec you
# looked up again is usually a spec you got wrong the first time.
conn.execute(
"UPDATE facts SET body = ?, source_url = ?, keywords = ?, updated_at = ? WHERE id = ?",
(body_value, str(payload.get("source_url") or "").strip() or None,
_keywords(payload.get("keywords")), _now(), existing["id"]),
)
return {"ok": True, "id": existing["id"], "updated": True}
cur = conn.execute(
"INSERT INTO facts (subject, body, source_url, keywords, created_at, updated_at) "
"VALUES (?, ?, ?, ?, ?, ?)",
(subject, body_value, str(payload.get("source_url") or "").strip() or None,
_keywords(payload.get("keywords")), _now(), _now()),
)
return {"ok": True, "id": cur.lastrowid, "updated": False}
# --- 3. project knowledge ---------------------------------------------------------
def list_project_knowledge(project_slug: str = "", query: str = "") -> dict:
clauses, params = [], []
if project_slug:
clauses.append("project_slug = ?")
params.append(project_slug.strip().lower())
if query:
clauses.append("(lower(body) LIKE ? OR keywords LIKE ?)")
params.extend([_like(query), _like(query)])
where = f"WHERE {' AND '.join(clauses)}" if clauses else ""
with _db() as conn:
rows = conn.execute(
f"SELECT * FROM project_knowledge {where} ORDER BY project_slug, id DESC", params
).fetchall()
return {"project_knowledge": [_row(r) for r in rows]}
def add_project_knowledge(project_slug: str, payload: dict) -> dict:
body = str(payload.get("body") or "").strip()
if not body:
return {"ok": False, "reason": "bad_field", "message": "'body' is required."}
with _db() as conn:
cur = conn.execute(
"INSERT INTO project_knowledge (project_slug, body, keywords, created_at, updated_at) "
"VALUES (?, ?, ?, ?, ?)",
(project_slug.strip().lower(), body, _keywords(payload.get("keywords")), _now(), _now()),
)
return {"ok": True, "id": cur.lastrowid}
# --- 4. hardware inventory --------------------------------------------------------
def list_hardware(query: str = "", status: str = "", project_slug: str = "") -> dict:
clauses, params = [], []
if query:
clauses.append("(lower(designation) LIKE ? OR lower(COALESCE(kind,'')) LIKE ? "
"OR lower(COALESCE(storage_location,'')) LIKE ?)")
params.extend([_like(query), _like(query), _like(query)])
if status:
clauses.append("status = ?")
params.append(status.strip().lower())
if project_slug:
clauses.append("project_slug = ?")
params.append(project_slug.strip().lower())
where = f"WHERE {' AND '.join(clauses)}" if clauses else ""
with _db() as conn:
rows = conn.execute(
f"SELECT * FROM hardware {where} ORDER BY status, designation", params
).fetchall()
return {"hardware": [_row(r) for r in rows]}
def add_hardware(payload: dict) -> dict:
designation = str(payload.get("designation") or "").strip()
if not designation:
return {
"ok": False,
"reason": "bad_field",
"message": "'designation' is required — what is printed on the thing.",
}
status = str(payload.get("status") or "").strip().lower()
project_slug = str(payload.get("project_slug") or "").strip().lower()
# Naming a project IS the assignment. Requiring a separate status field to agree
# would be a second thing to get wrong, so the two are derived from each other.
if not status:
status = "assigned" if project_slug else "available"
if status not in HARDWARE_STATUSES:
return {"ok": False, "reason": "bad_field",
"message": f"status must be one of {', '.join(HARDWARE_STATUSES)}."}
try:
quantity = max(0, int(payload.get("quantity", 1)))
except (TypeError, ValueError):
quantity = 1
specs = payload.get("specs")
with _db() as conn:
cur = conn.execute(
"INSERT INTO hardware (designation, kind, quantity, storage_location, status, project_slug, "
"specs, source_url, notes, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
(
designation,
str(payload.get("kind") or "").strip() or None,
quantity,
str(payload.get("storage_location") or "").strip() or None,
status,
project_slug or None,
json.dumps(specs) if isinstance(specs, (dict, list)) else (specs or None),
str(payload.get("source_url") or "").strip() or None,
str(payload.get("notes") or "").strip() or None,
_now(), _now(),
),
)
return {"ok": True, "id": cur.lastrowid}
def update_hardware(hardware_id: int, payload: dict) -> dict:
"""Edit a piece of hardware — including the one edit that matters, assigning it.
`{"project_slug": "nas-build"}` moves it to `assigned` on its own; an explicit
`{"project_slug": null}` releases it back to `available`. That coupling is
deliberate: an item assigned to a project while still reading `available` is how an
inventory starts lying, and there is no case where you want those two apart.
**`in_use` is never overwritten by a re-assignment**, which is the one exception
and the reason the statuses are distinct at all. Naming the project of something
that is currently installed and working means "this is what it's in", not "please
demote it to reserved" — and silently downgrading it would turn a thing you'd have
to unscrew into a thing the list says you can just take.
"""
updates: list[tuple[str, object]] = []
if "project_slug" in payload:
slug = str(payload.get("project_slug") or "").strip().lower()
updates.append(("project_slug", slug or None))
if "status" not in payload:
with _db() as conn:
row = conn.execute("SELECT status FROM hardware WHERE id = ?", (hardware_id,)).fetchone()
currently_in_use = row is not None and row["status"] == "in_use"
if not (currently_in_use and slug):
updates.append(("status", "assigned" if slug else "available"))
if "status" in payload:
status = str(payload.get("status") or "").strip().lower()
if status not in HARDWARE_STATUSES:
return {"ok": False, "reason": "bad_field",
"message": f"status must be one of {', '.join(HARDWARE_STATUSES)}."}
updates.append(("status", status))
# Releasing something must not leave a stale claim behind.
if status == "available" and "project_slug" not in payload:
updates.append(("project_slug", None))
for field in ("designation", "kind", "storage_location", "source_url", "notes"):
if field in payload:
value = str(payload[field] or "").strip()
if field == "designation" and not value:
return {"ok": False, "reason": "bad_field", "message": "'designation' cannot be empty."}
updates.append((field, value or None))
if "quantity" in payload:
try:
updates.append(("quantity", max(0, int(payload["quantity"]))))
except (TypeError, ValueError):
return {"ok": False, "reason": "bad_field", "message": "'quantity' must be a number."}
if "specs" in payload:
specs = payload["specs"]
updates.append(("specs", json.dumps(specs) if isinstance(specs, (dict, list)) else (specs or None)))
if not updates:
return {"ok": False, "reason": "bad_field", "message": "Nothing to update."}
with _db() as conn:
if conn.execute("SELECT 1 FROM hardware WHERE id = ?", (hardware_id,)).fetchone() is None:
return {"ok": False, "reason": "no_such_item", "message": "No hardware with that id."}
for column, value in updates:
conn.execute(f"UPDATE hardware SET {column} = ? WHERE id = ?", (value, hardware_id))
conn.execute("UPDATE hardware SET updated_at = ? WHERE id = ?", (_now(), hardware_id))
row = conn.execute("SELECT * FROM hardware WHERE id = ?", (hardware_id,)).fetchone()
return {"ok": True, "hardware": _row(row)}
def delete_row(table: str, row_id: int) -> dict:
"""Corrections. The only deletion path, and it is per-row on purpose.
"No retention period" means nothing expires by itself; it does not mean nothing is
ever wrong. A knowledge base you cannot correct becomes one you route around.
"""
if table not in ("workflow_notes", "facts", "project_knowledge", "hardware"):
return {"ok": False, "reason": "bad_field", "message": "Unknown table."}
with _db() as conn:
cur = conn.execute(f"DELETE FROM {table} WHERE id = ?", (row_id,))
if cur.rowcount == 0:
return {"ok": False, "reason": "no_such_item", "message": "Nothing with that id."}
return {"ok": True}
# --- the payoff -------------------------------------------------------------------
def context(activity: str = "", query: str = "", project_slug: str = "") -> dict:
"""Everything that applies right now, in one call.
This is the endpoint that makes the four tables a memory rather than four lists.
An assistant starting a conversation asks it once with whatever it knows —
the room's activity, the subject at hand, the open project — and gets the standing
instructions it would otherwise have to be told again, the specifics it would
otherwise look up, and what hardware is already promised elsewhere.
Everything is best-effort and additive: an unknown activity returns the always-
notes rather than nothing, because "I have no idea what you're doing" is not a
reason to forget how you like to work.
"""
result = {
"activity": activity,
"project_slug": project_slug,
"workflow_notes": list_workflow_notes(activity)["workflow_notes"],
"facts": list_facts(query)["facts"] if query else [],
"project_knowledge": (
list_project_knowledge(project_slug)["project_knowledge"] if project_slug else []
),
"hardware": [],
}
if project_slug:
result["hardware"] = list_hardware(project_slug=project_slug)["hardware"]
elif query:
result["hardware"] = list_hardware(query=query)["hardware"]
return result

683
workshop/server.py Normal file
View File

@ -0,0 +1,683 @@
"""workshop — the project notebook behind the workshop/office assistant, from
docs/workshop-assistant.md.
STEP 1 OF THAT DOC, AND DELIBERATELY ONLY STEP 1. The feasibility note ranks four
capabilities and says to build this one first, because it is the only one with no
perception problem in it and because it is where everything else lands: an identified
mainboard is worth nothing until there is a project to attach it to. Camera
identification, spec research and guide lookup all arrive later as *writers into this
database*, not as separate stores.
THE SPLIT THAT SHAPES THIS FILE
--------------------------------
**SQLite is the index; the SMB share is the filing cabinet.** State that has to be
queried — projects, decisions, what step you are on — lives in the database.
(What you *own* is knowledge, not work product: it lives in knowledge.py's separate,
never-pruned database, because there must be exactly one inventory.)
Artefacts — datasheets, generated diagrams, photos of the board, notes you want to
open from a laptop — live as ordinary files under WORKSPACE_DIR, which is exported
read-write over SMB. Mixing the two gets you the worst of both: a database you cannot
browse and files nothing can query.
That is also why this service never returns file *contents* for anything in the
workspace. It lists what is there and it hands back paths; opening the file is the
share's job, and a second copy of the bytes over HTTP would be a second place for them
to be stale. `GET /projects/<slug>/files` is a directory listing, not a file server.
WHY DECISIONS ARE THEIR OWN TABLE
----------------------------------
"What did I decide last Tuesday, and why" is the single question a project notebook
exists to answer, and it is the one a pile of notes answers worst. A decision has a
question, an answer, and a reason, and it is worth the four columns to keep those
apart — the same argument this project already makes for every "flag the reasoning,
not just the outcome" README in the repo.
SECURITY BOUNDARY, same as every other service here: one bearer token, checked on
every request including the GETs, failing closed when unset. This one holds no
credentials and no personal data, but it *writes files into a share*, so see
_safe_slug() for the one rule that keeps that from becoming a path-traversal
question.
"""
from __future__ import annotations
import json
import logging
import os
import re
import sqlite3
import sys
from datetime import datetime, timezone
from http import HTTPStatus
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from urllib.parse import parse_qs, urlsplit
import fleet
import git_ops
import gitea
import health
import knowledge
LOG = logging.getLogger("workshop")
TOKEN = os.environ.get("WORKSHOP_TOKEN", "")
DB_PATH = Path(os.environ.get("WORKSHOP_DB_PATH", "/data/workshop.db"))
# The SMB-exported workspace. Every project gets a directory under here; see
# docs/workshop-assistant.md for the layout and for why it is a share at all.
WORKSPACE_DIR = Path(os.environ.get("WORKSHOP_WORKSPACE_DIR", "/workspace"))
PROJECT_SUBDIRS = ("notes", "datasheets", "diagrams", "photos", "scratch")
MAX_JSON_BYTES = 256 * 1024
# A slug is a directory name on a writable share, which makes it the one piece of
# user input in this service with a filesystem consequence. Anchored, no dots, no
# separators — "../../etc" cannot survive this, and neither can a name that only
# differs from another by case on a case-insensitive mount.
SLUG_RE = re.compile(r"^[a-z0-9][a-z0-9-]{0,62}$")
def _now() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def _safe_slug(value: str) -> str | None:
"""A slug, or None. Never raises, never sanitises-and-continues.
Sanitising is the tempting option and the wrong one: silently turning `../x` into
`x` means two different requests write to one directory, and the person who typed
the first one never finds out. A bad slug is a 400.
"""
slug = str(value or "").strip().lower()
return slug if SLUG_RE.match(slug) else None
def _slugify(name: str) -> str:
"""A first-guess slug from a project name, for the common case where nobody
supplied one. Not authoritative — the result still goes through _safe_slug()."""
slug = re.sub(r"[^a-z0-9]+", "-", str(name or "").strip().lower()).strip("-")
return slug[:63] or "project"
# --- storage ----------------------------------------------------------------------
def _db() -> sqlite3.Connection:
DB_PATH.parent.mkdir(parents=True, exist_ok=True)
conn = sqlite3.connect(DB_PATH, timeout=10)
conn.row_factory = sqlite3.Row
conn.execute("PRAGMA foreign_keys = ON")
return conn
def init_db() -> None:
with _db() as conn:
conn.executescript(
"""
CREATE TABLE IF NOT EXISTS projects (
id INTEGER PRIMARY KEY,
slug TEXT NOT NULL UNIQUE,
name TEXT NOT NULL,
-- The HA area this project belongs to, so a room-scoped assistant can
-- answer "what am I working on" with the projects of the room it was
-- asked in. An area_id, the same vocabulary as everything else in this
-- project — see docs/rooms-and-endpoints.md.
room TEXT,
-- active | parked | done. Parked is not done: a shelved project whose
-- parts are still allocated to it is exactly what you want to find
-- before buying those parts again.
status TEXT NOT NULL DEFAULT 'active',
summary TEXT,
-- The project's Gitea repository, once one exists. A URL, not a flag:
-- what anybody actually wants from this column is something to click
-- or clone, and "true" would send every reader to Gitea to search.
repo_url TEXT,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS notes (
id INTEGER PRIMARY KEY,
project_id INTEGER NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
body TEXT NOT NULL,
-- Who or what wrote this. 'human' or an assistant identifier — kept so
-- a note the model wrote is never mistaken for something you said, the
-- same distinction digest-engine draws between a source's claim and a
-- fact.
author TEXT NOT NULL DEFAULT 'human',
created_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS decisions (
id INTEGER PRIMARY KEY,
project_id INTEGER NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
question TEXT NOT NULL,
answer TEXT NOT NULL,
-- The column this table exists for. An answer without its reasoning is
-- a thing you will re-litigate in three weeks.
because TEXT,
created_at TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS notes_by_project ON notes (project_id, created_at);
CREATE INDEX IF NOT EXISTS decisions_by_project ON decisions (project_id, created_at);
"""
)
def _project_dir(slug: str) -> Path:
return WORKSPACE_DIR / "projects" / slug
def _ensure_project_dir(slug: str) -> list[str]:
"""Create the project's workspace directories. Best-effort by design.
An unmounted or read-only share must not stop a project from being created — the
notebook is in SQLite and is the part that matters, and a share that comes back
later can have its directories made then (every write path calls this). Returns
the subdirectories that exist, so the API can say plainly whether the share is
working rather than implying it is.
"""
made = []
for sub in PROJECT_SUBDIRS:
try:
(_project_dir(slug) / sub).mkdir(parents=True, exist_ok=True)
made.append(sub)
except OSError:
LOG.warning("workshop: could not create %s/%s in the workspace", slug, sub, exc_info=True)
return made
def _row(row) -> dict:
return {k: row[k] for k in row.keys()}
def list_projects(room: str = "") -> dict:
with _db() as conn:
if room:
rows = conn.execute(
"SELECT * FROM projects WHERE room = ? ORDER BY updated_at DESC", (room,)
).fetchall()
else:
rows = conn.execute("SELECT * FROM projects ORDER BY updated_at DESC").fetchall()
return {"projects": [_row(r) for r in rows]}
def create_project(payload: dict) -> dict:
name = str(payload.get("name") or "").strip()
if not name:
return {"ok": False, "reason": "bad_field", "message": "'name' is required."}
slug = _safe_slug(payload.get("slug") or _slugify(name))
if slug is None:
return {
"ok": False,
"reason": "bad_slug",
"message": "A slug is lowercase letters, digits and hyphens — it is also a "
"directory name on the share, so it can't contain dots or slashes.",
}
room = str(payload.get("room") or "").strip()
with _db() as conn:
if conn.execute("SELECT 1 FROM projects WHERE slug = ?", (slug,)).fetchone():
return {"ok": False, "reason": "duplicate", "message": f"A project called {slug!r} already exists."}
conn.execute(
"INSERT INTO projects (slug, name, room, status, summary, created_at, updated_at) "
"VALUES (?, ?, ?, 'active', ?, ?, ?)",
(slug, name, room or None, str(payload.get("summary") or "").strip() or None, _now(), _now()),
)
row = conn.execute("SELECT * FROM projects WHERE slug = ?", (slug,)).fetchone()
made = _ensure_project_dir(slug)
LOG.info("workshop: created project %s (workspace dirs: %s)", slug, ", ".join(made) or "none")
return {"ok": True, "project": _row(row), "workspace": _workspace_status(slug, made)}
def _workspace_status(slug: str, made: list[str]) -> dict:
"""Says plainly whether the share is usable, rather than implying it is by silence."""
return {
"path": str(_project_dir(slug)),
"subdirs": made,
"available": len(made) == len(PROJECT_SUBDIRS),
}
def get_project(slug: str) -> dict | None:
with _db() as conn:
row = conn.execute("SELECT * FROM projects WHERE slug = ?", (slug,)).fetchone()
if row is None:
return None
project = _row(row)
pid = row["id"]
project["notes"] = [_row(r) for r in conn.execute(
"SELECT * FROM notes WHERE project_id = ? ORDER BY created_at DESC", (pid,))]
project["decisions"] = [_row(r) for r in conn.execute(
"SELECT * FROM decisions WHERE project_id = ? ORDER BY created_at DESC", (pid,))]
# Hardware lives in the knowledge database, not here — see knowledge.py's docstring
# on why there is exactly one inventory. A project reads its claims from it.
project["hardware"] = knowledge.list_hardware(project_slug=slug)["hardware"]
project["knowledge"] = knowledge.list_project_knowledge(slug)["project_knowledge"]
project["workspace"] = _workspace_status(slug, [
s for s in PROJECT_SUBDIRS if (_project_dir(slug) / s).is_dir()
])
return project
def update_project(slug: str, payload: dict) -> dict:
updates = []
if "name" in payload:
name = str(payload["name"] or "").strip()
if not name:
return {"ok": False, "reason": "bad_field", "message": "'name' cannot be empty."}
updates.append(("name", name))
if "status" in payload:
status = str(payload["status"] or "").strip().lower()
if status not in ("active", "parked", "done"):
return {"ok": False, "reason": "bad_field", "message": "status is active, parked or done."}
updates.append(("status", status))
for field in ("summary", "room"):
if field in payload:
updates.append((field, str(payload[field] or "").strip() or None))
if not updates:
return {"ok": False, "reason": "bad_field", "message": "Nothing to update."}
with _db() as conn:
if conn.execute("SELECT 1 FROM projects WHERE slug = ?", (slug,)).fetchone() is None:
return {"ok": False, "reason": "no_such_project", "message": "No project with that slug."}
for column, value in updates:
conn.execute(f"UPDATE projects SET {column} = ? WHERE slug = ?", (value, slug))
conn.execute("UPDATE projects SET updated_at = ? WHERE slug = ?", (_now(), slug))
row = conn.execute("SELECT * FROM projects WHERE slug = ?", (slug,)).fetchone()
return {"ok": True, "project": _row(row)}
def _project_id(conn, slug: str) -> int | None:
row = conn.execute("SELECT id FROM projects WHERE slug = ?", (slug,)).fetchone()
return int(row["id"]) if row else None
def add_note(slug: str, payload: dict) -> dict:
body = str(payload.get("body") or "").strip()
if not body:
return {"ok": False, "reason": "bad_field", "message": "'body' is required."}
author = str(payload.get("author") or "human").strip() or "human"
with _db() as conn:
pid = _project_id(conn, slug)
if pid is None:
return {"ok": False, "reason": "no_such_project", "message": "No project with that slug."}
conn.execute(
"INSERT INTO notes (project_id, body, author, created_at) VALUES (?, ?, ?, ?)",
(pid, body, author, _now()),
)
conn.execute("UPDATE projects SET updated_at = ? WHERE id = ?", (_now(), pid))
return {"ok": True}
def add_decision(slug: str, payload: dict) -> dict:
question = str(payload.get("question") or "").strip()
answer = str(payload.get("answer") or "").strip()
if not question or not answer:
return {"ok": False, "reason": "bad_field", "message": "'question' and 'answer' are both required."}
with _db() as conn:
pid = _project_id(conn, slug)
if pid is None:
return {"ok": False, "reason": "no_such_project", "message": "No project with that slug."}
conn.execute(
"INSERT INTO decisions (project_id, question, answer, because, created_at) VALUES (?, ?, ?, ?, ?)",
(pid, question, answer, str(payload.get("because") or "").strip() or None, _now()),
)
conn.execute("UPDATE projects SET updated_at = ? WHERE id = ?", (_now(), pid))
return {"ok": True}
def create_project_repo(slug: str, payload: dict) -> dict:
"""Give a project a Gitea repository and remember where it is.
The repo name defaults to the project slug — the slug is already the project's
stable, filesystem-safe identity, and having the directory on the share, the row in
the database and the repository all answer to one name is worth more than letting
each pick its own.
"""
with _db() as conn:
row = conn.execute("SELECT * FROM projects WHERE slug = ?", (slug,)).fetchone()
if row is None:
return {"ok": False, "reason": "no_such_project", "message": "No project with that slug."}
name = str(payload.get("name") or slug).strip()
description = str(payload.get("description") or row["summary"] or f"Workshop project: {row['name']}")
result = gitea.create_repo(name, description, payload.get("private"))
if not result.get("ok"):
return result
url = result["repo"].get("html_url")
# Set up code/ as a working tree pointed at the new remote, so the first commit
# does not need a second call. Best-effort: a repo with no local tree yet is fine.
result["local"] = git_ops.ensure_repo(_project_dir(slug), result["repo"].get("clone_url") or "")
with _db() as conn:
conn.execute("UPDATE projects SET repo_url = ?, updated_at = ? WHERE slug = ?", (url, _now(), slug))
LOG.info("workshop: project %s -> %s", slug, url)
return result
def cameras() -> dict:
"""The network cameras the workshop display may show.
Network, not USB: every USB camera in this project is aimed at one fixed thing (an
item held up to the kitchen display, an appliance door) at an angle that is useless
for anything else. The cameras worth putting on a workshop screen are the ones
already on the network and already ingested by go2rtc/Frigate.
"""
raw = os.environ.get("WORKSHOP_CAMERAS", "").strip()
entries = []
for item in raw.split(","):
item = item.strip()
if not item or ":" not in item:
continue
name, _, stream = item.partition(":")
if name.strip() and stream.strip():
entries.append({"name": name.strip(), "stream": stream.strip()})
return {"cameras": entries, "go2rtc_url": os.environ.get("GO2RTC_URL", "").rstrip("/")}
def list_files(slug: str) -> dict:
"""A directory listing of the project's workspace — names and sizes, never bytes.
The share serves the files; this says what is there. Symlinks are reported but
never followed, and nothing outside the project directory is ever listed, because
the whole point of the slug rule is that this path is not user-controlled beyond
one directory name.
"""
base = _project_dir(slug)
result: dict[str, list] = {}
for sub in PROJECT_SUBDIRS:
entries = []
directory = base / sub
try:
for entry in sorted(directory.iterdir(), key=lambda p: p.name):
try:
stat = entry.stat()
entries.append({
"name": entry.name,
"bytes": stat.st_size,
"modified_at": datetime.fromtimestamp(stat.st_mtime, timezone.utc)
.isoformat().replace("+00:00", "Z"),
"is_dir": entry.is_dir(),
})
except OSError:
continue
except OSError:
# Missing or unreadable: an empty list plus `available: false` below, not
# an error — a share that is temporarily not mounted must not look like a
# project that has no files.
pass
result[sub] = entries
return {
"path": str(base),
"available": base.is_dir(),
"files": result,
}
# --- HTTP -------------------------------------------------------------------------
class Handler(BaseHTTPRequestHandler):
server_version = "workshop/1"
def log_message(self, format, *args): # noqa: A002
LOG.info("%s - %s", self.address_string(), format % args)
def _authorized(self) -> bool:
return bool(TOKEN) and self.headers.get("Authorization", "") == f"Bearer {TOKEN}"
def _respond(self, status: HTTPStatus, payload) -> None:
body = json.dumps(payload).encode("utf-8")
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(body)))
self.send_header("Access-Control-Allow-Origin", "*")
self.end_headers()
self.wfile.write(body)
def _json_body(self) -> dict | None:
try:
length = int(self.headers.get("Content-Length") or 0)
if length > MAX_JSON_BYTES:
raise ValueError(f"body too large ({length} bytes)")
payload = json.loads(self.rfile.read(length) or b"{}")
except (ValueError, json.JSONDecodeError) as exc:
self._respond(HTTPStatus.BAD_REQUEST, {"error": f"bad request body: {exc}"})
return None
if not isinstance(payload, dict):
self._respond(HTTPStatus.BAD_REQUEST, {"error": "body must be a JSON object"})
return None
return payload
def _slug_or_400(self, raw: str) -> str | None:
slug = _safe_slug(raw)
if slug is None:
self._respond(HTTPStatus.BAD_REQUEST, {"error": f"{raw!r} is not a valid project slug"})
return slug
def _result(self, result: dict) -> None:
if result.get("ok"):
self._respond(HTTPStatus.OK, result)
return
status = {
"no_such_project": HTTPStatus.NOT_FOUND,
"no_such_item": HTTPStatus.NOT_FOUND,
"no_such_version": HTTPStatus.NOT_FOUND,
"duplicate": HTTPStatus.CONFLICT,
}.get(str(result.get("reason")), HTTPStatus.BAD_REQUEST)
self._respond(status, result)
def do_OPTIONS(self): # noqa: N802
self.send_response(HTTPStatus.NO_CONTENT)
self.send_header("Access-Control-Allow-Origin", "*")
self.send_header("Access-Control-Allow-Headers", "Authorization, Content-Type")
self.send_header("Access-Control-Allow-Methods", "GET, POST, DELETE, OPTIONS")
self.end_headers()
def do_GET(self): # noqa: N802
if not self._authorized():
self._respond(HTTPStatus.UNAUTHORIZED, {"error": "missing or invalid bearer token"})
return
split = urlsplit(self.path)
path, query = split.path, parse_qs(split.query)
project_match = re.match(r"^/projects/([^/]+)$", path)
files_match = re.match(r"^/projects/([^/]+)/files$", path)
git_match = re.match(r"^/projects/([^/]+)/git$", path)
fleet_script_match = re.match(r"^/fleet/script/([a-z0-9-]+)$", path)
if path == "/projects":
self._respond(HTTPStatus.OK, list_projects((query.get("room") or [""])[0].strip()))
elif path == "/hardware":
self._respond(HTTPStatus.OK, knowledge.list_hardware(
(query.get("q") or [""])[0].strip(),
(query.get("status") or [""])[0].strip(),
(query.get("project") or [""])[0].strip(),
))
elif path == "/knowledge/workflow":
self._respond(HTTPStatus.OK, knowledge.list_workflow_notes(
(query.get("activity") or [""])[0].strip()))
elif path == "/knowledge/facts":
self._respond(HTTPStatus.OK, knowledge.list_facts((query.get("q") or [""])[0].strip()))
elif path == "/knowledge/projects":
self._respond(HTTPStatus.OK, knowledge.list_project_knowledge(
(query.get("project") or [""])[0].strip(), (query.get("q") or [""])[0].strip()))
elif path == "/fleet":
self._respond(HTTPStatus.OK, fleet.overview())
elif fleet_script_match:
# What an endpoint of this platform should run. Serves the PUBLISHED version
# only — a draft is invisible here by design.
entry = fleet.published(fleet_script_match.group(1))
if entry is None:
self._respond(HTTPStatus.NOT_FOUND,
{"error": "no published script for that platform"})
else:
self._respond(HTTPStatus.OK, entry)
elif path == "/cameras":
# Names and stream ids only — this service never proxies video. Putting a
# Python HTTP server in the path of an H.264 stream is how a working camera
# becomes a stuttering one; the kiosk talks to go2rtc directly.
self._respond(HTTPStatus.OK, cameras())
elif path == "/health":
# The workshop display's overlay reads this. `overall` is one word so the
# display never re-derives "is anything wrong" from a list.
self._respond(HTTPStatus.OK, health.latest())
elif path == "/context":
# The payoff: everything the assistant should already know, in one call.
self._respond(HTTPStatus.OK, knowledge.context(
(query.get("activity") or [""])[0].strip(),
(query.get("q") or [""])[0].strip(),
(query.get("project") or [""])[0].strip(),
))
elif project_match:
slug = self._slug_or_400(project_match.group(1))
if slug is None:
return
project = get_project(slug)
if project is None:
self._respond(HTTPStatus.NOT_FOUND, {"error": "no such project"})
else:
self._respond(HTTPStatus.OK, project)
elif git_match:
slug = self._slug_or_400(git_match.group(1))
if slug is not None:
self._respond(HTTPStatus.OK, git_ops.status(_project_dir(slug)))
elif files_match:
slug = self._slug_or_400(files_match.group(1))
if slug is not None:
self._respond(HTTPStatus.OK, list_files(slug))
else:
self._respond(HTTPStatus.NOT_FOUND, {"error": "no such endpoint"})
def do_DELETE(self): # noqa: N802
"""Corrections. "No retention period" means nothing expires by itself — it does
not mean nothing is ever wrong, and a knowledge base you cannot correct is one
people route around."""
if not self._authorized():
self._respond(HTTPStatus.UNAUTHORIZED, {"error": "missing or invalid bearer token"})
return
path = urlsplit(self.path).path
match = re.match(r"^/(hardware|knowledge/workflow|knowledge/facts|knowledge/projects)/(\d+)$", path)
if not match:
self._respond(HTTPStatus.NOT_FOUND, {"error": "no such endpoint"})
return
table = {
"hardware": "hardware",
"knowledge/workflow": "workflow_notes",
"knowledge/facts": "facts",
"knowledge/projects": "project_knowledge",
}[match.group(1)]
self._result(knowledge.delete_row(table, int(match.group(2))))
def do_POST(self): # noqa: N802
if not self._authorized():
self._respond(HTTPStatus.UNAUTHORIZED, {"error": "missing or invalid bearer token"})
return
path = urlsplit(self.path).path
note_match = re.match(r"^/projects/([^/]+)/notes$", path)
knowledge_match = re.match(r"^/projects/([^/]+)/knowledge$", path)
repo_match = re.match(r"^/projects/([^/]+)/repo$", path)
commit_match = re.match(r"^/projects/([^/]+)/commit$", path)
branch_match = re.match(r"^/projects/([^/]+)/branch$", path)
scrub_match = re.match(r"^/projects/([^/]+)/scrub-request$", path)
hardware_match = re.match(r"^/hardware/(\d+)$", path)
decision_match = re.match(r"^/projects/([^/]+)/decisions$", path)
project_match = re.match(r"^/projects/([^/]+)$", path)
payload = self._json_body()
if payload is None:
return
if path == "/projects":
self._result(create_project(payload))
elif path == "/hardware":
self._result(knowledge.add_hardware(payload))
elif hardware_match:
self._result(knowledge.update_hardware(int(hardware_match.group(1)), payload))
elif path == "/fleet/upload":
self._result(fleet.upload(payload.get("platform", ""), payload.get("body", ""),
payload.get("note", "")))
elif path == "/fleet/publish":
# The second, deliberate action. Upload is not deploy — see fleet.py.
try:
version = int(payload.get("version"))
except (TypeError, ValueError):
self._respond(HTTPStatus.BAD_REQUEST, {"error": "'version' is required"})
return
self._result(fleet.publish(payload.get("platform", ""), version))
elif path == "/fleet/report":
self._result(fleet.report(payload))
elif path == "/knowledge/workflow":
self._result(knowledge.add_workflow_note(payload))
elif path == "/knowledge/facts":
self._result(knowledge.add_fact(payload))
elif note_match:
slug = self._slug_or_400(note_match.group(1))
if slug is not None:
self._result(add_note(slug, payload))
elif repo_match:
slug = self._slug_or_400(repo_match.group(1))
if slug is not None:
self._result(create_project_repo(slug, payload))
elif commit_match:
slug = self._slug_or_400(commit_match.group(1))
if slug is not None:
self._result(git_ops.commit(
_project_dir(slug), payload.get("message", ""), bool(payload.get("push", True))))
elif branch_match:
slug = self._slug_or_400(branch_match.group(1))
if slug is not None:
self._result(git_ops.branch(_project_dir(slug), payload.get("name", "")))
elif scrub_match:
# Never executes anything. Returns the commands for a human to run from the
# repository on the share — see git_ops.scrub_instructions().
slug = self._slug_or_400(scrub_match.group(1))
if slug is not None:
self._result(git_ops.scrub_instructions(
_project_dir(slug), payload.get("path", ""), payload.get("note", "")))
elif knowledge_match:
slug = self._slug_or_400(knowledge_match.group(1))
if slug is not None:
self._result(knowledge.add_project_knowledge(slug, payload))
elif decision_match:
slug = self._slug_or_400(decision_match.group(1))
if slug is not None:
self._result(add_decision(slug, payload))
elif project_match:
slug = self._slug_or_400(project_match.group(1))
if slug is not None:
self._result(update_project(slug, payload))
else:
self._respond(HTTPStatus.NOT_FOUND, {"error": "no such endpoint"})
def main() -> int:
logging.basicConfig(
level=os.environ.get("LOG_LEVEL", "INFO").upper(),
format="%(asctime)s %(levelname)-7s %(name)s: %(message)s",
)
if not TOKEN:
LOG.error("WORKSHOP_TOKEN is not set — every request will be rejected until it is.")
init_db()
knowledge.init_db()
health.init_db()
fleet.init_db()
health.start_poller()
try:
(WORKSPACE_DIR / "projects").mkdir(parents=True, exist_ok=True)
except OSError:
LOG.warning(
"workshop: %s is not writable — the notebook still works, but nothing can be "
"filed on the share until it is",
WORKSPACE_DIR, exc_info=True,
)
port = int(os.environ.get("WORKSHOP_PORT", "8102"))
server = ThreadingHTTPServer(("0.0.0.0", port), Handler)
LOG.info("workshop listening on :%d (db: %s, workspace: %s)", port, DB_PATH, WORKSPACE_DIR)
try:
server.serve_forever()
except KeyboardInterrupt:
pass
return 0
if __name__ == "__main__":
sys.exit(main())

141
workshop/workshop.env.example Normal file
View File

@ -0,0 +1,141 @@
# workshop configuration template.
#
# Copy to /opt/smart-home/workshop/workshop.env on the container host, fill in, and
# chmod 600 it. Same never-commit handling as every other *.env here.
# ---------------------------------------------------------------------------
# Auth — required. Fails closed (rejects every request) while empty.
# openssl rand -hex 32
# ---------------------------------------------------------------------------
WORKSHOP_TOKEN=
# ---------------------------------------------------------------------------
# Storage. THE SPLIT THAT MATTERS: the database is the index, the workspace is the
# filing cabinet. Queryable state (projects, parts, decisions) goes in SQLite;
# artefacts (datasheets, generated diagrams, photos, notes you want to open from a
# laptop) are ordinary files under the workspace, which is exported read-write over
# SMB. See docs/workshop-assistant.md.
#
# The workspace is the ONE writable share in this stack — the photo gallery share is
# deliberately read-only — so it gets its own volume and its own SMB account. Nothing
# on the host ever executes anything found in it.
# ---------------------------------------------------------------------------
WORKSHOP_DB_PATH=/data/workshop.db
WORKSHOP_WORKSPACE_DIR=/workspace
# ---------------------------------------------------------------------------
# Run behaviour
# ---------------------------------------------------------------------------
# 8098 is identity_web — see CoreSystemConfig.json's ports block, which the validator
# checks for duplicates.
WORKSHOP_PORT=8102
LOG_LEVEL=INFO
# ---------------------------------------------------------------------------
# Knowledge store — the second database, and the one that is NEVER pruned.
#
# workshop.db is a record of WORK (projects, decisions, the log). This is a record of
# WHAT IS TRUE: standing instructions about how you work, facts and specs by keyword,
# what was learned about a project, and the hardware inventory. Every other store in
# this stack has a retention window because stale observations are worse than none;
# this one is the opposite — the whole point is that you tell it once. There is no
# cutoff and no cleanup pass, by design.
# ---------------------------------------------------------------------------
WORKSHOP_KNOWLEDGE_DB_PATH=/data/workshop-knowledge.db
# ---------------------------------------------------------------------------
# Gitea (optional; repo creation is refused while unset).
#
# Lets a project be given a git repository. The service CREATES repos and reports
# them — it never pushes, commits or clones, and it has no delete path at all, even
# though the token technically permits one.
#
# The token: Gitea → Settings → Applications → Generate New Token, scope
# `write:repository` (plus `write:user` if GITEA_OWNER is left empty so repos are made
# under the token's own account).
#
# STRONGLY PREFERRED: a dedicated Gitea user ("workshop-bot") with access to ONE
# organisation, named in GITEA_OWNER — rather than a token on your own account, which
# can reach everything you own. The blast radius of a leaked env file is then one org
# of generated repos.
# ---------------------------------------------------------------------------
GITEA_URL=
GITEA_TOKEN=
GITEA_OWNER=
# Generated repos are private unless this is explicitly "false".
GITEA_PRIVATE_REPOS=true
GITEA_TIMEOUT=15
# ---------------------------------------------------------------------------
# Git. The assistant commits and pushes unattended; it NEVER rewrites history.
#
# Allowed: commit, push, branch, tag, merge — everything that ADDS to history, because
# the worst case is a bad commit you revert.
# Never, by any path: force-push, rebase, amend, reset --hard, filter-branch,
# filter-repo, branch/tag deletion, reflog expiry, gc --prune. Rewriting has no undo.
#
# That refusal lives in git_ops.py, but the real enforcement is Gitea BRANCH
# PROTECTION, applied automatically when a repo is created (enable_force_push: false,
# enable_delete: false). Client-side refusal is a policy; branch protection is a
# control. Check the create-repo response's `repo.protection.applied` — do not assume
# it is on because the repo exists.
#
# When history genuinely has to be scrubbed (a committed API token), POST
# /projects/<slug>/scrub-request returns the commands to run BY HAND from the repo on
# the SMB share. The manual step is the safety mechanism; automating it would remove
# the only thing making it safe.
#
# The working tree is code/ inside the project workspace — never the workspace root —
# so no git operation can reach the datasheets and photos beside it.
# ---------------------------------------------------------------------------
WORKSHOP_GIT_AUTHOR_NAME=workshop assistant
WORKSHOP_GIT_AUTHOR_EMAIL=workshop@localhost
WORKSHOP_GIT_BRANCH=main
WORKSHOP_GIT_TIMEOUT=120
# ---------------------------------------------------------------------------
# Infrastructure health — CheckMK + every OPNsense firewall, polled here.
#
# WHY HERE AND NOT IN digest-engine: the digest runs four times a day, and "is the NAS
# disk failing right now" is not a question with a six-hour answer. This service is
# always on, so it polls and keeps ~30 days of samples; digest-engine reads THIS over
# HTTP (its ENABLE_INFRA_HEALTH_INGEST). One poller, two consumers — and the digest
# gets "critical since Tuesday" instead of a snapshot it can't compare to anything.
#
# READ-ONLY, and not by promise: the CheckMK user should have the Guest role, which
# cannot acknowledge, downtime or reschedule. From OPNsense this reads exactly one
# endpoint (GET /api/ids/service/status — is Suricata running); the alert query stays
# in digest-engine, which already does the paging properly.
#
# OPNSENSE_JSON is written by tools/config-export.py from the `opnsense` LIST in
# CoreSystemConfig.json — every firewall carries its own name, and that name reaches
# the digest and the display, so "the IDS is running" can never stand in for two
# firewalls of which one is down.
# ---------------------------------------------------------------------------
CHECKMK_BASE_URL=
CHECKMK_SITE=cmk
CHECKMK_USERNAME=
CHECKMK_SECRET=
CHECKMK_ONLY_PROBLEMS=true
CHECKMK_MAX_ROWS=200
OPNSENSE_JSON=
WORKSHOP_HEALTH_INTERVAL_SECONDS=300
WORKSHOP_HEALTH_TIMEOUT=20
WORKSHOP_HEALTH_RETENTION_DAYS=30
# ---------------------------------------------------------------------------
# Network cameras the workshop display may show.
#
# The USB webcams in this project are fixed-purpose (a kitchen item held to a lens, a
# doorway) and their angles are useless for anything else — so what the workshop shows
# is the NETWORK cameras, via the go2rtc/Frigate this stack already runs.
#
# Format: name:stream_id[,name:stream_id...] where stream_id is the camera's name in
# go2rtc. The frontend builds a WebRTC/MSE URL from it; nothing here proxies video,
# because putting a Python HTTP server in the path of an H.264 stream is how you turn
# a working camera into a stuttering one.
# ---------------------------------------------------------------------------
WORKSHOP_CAMERAS=
# Base URL of go2rtc's own web API, reachable from the BROWSER (not from this
# container) — the kiosk connects to it directly.
GO2RTC_URL=