SmartestHome/hosts/kitchen-display/README.md

# Sway kitchen display

Phase 17 of `docs/project-plan.md`. Builds a Debian 12 live ISO for a single-purpose
fridge/pantry display: hold a grocery item up to its camera, confirm what the vision
model thinks it is and when it'll go bad, put it away — then browse the resulting
inventory (soonest-to-expire first) and Grocy's recipes, all touch-driven.

**The camera-vision and Grocy read/write logic does not live in this directory** —
that's [`../../pantry-vision/`](../../pantry-vision/README.md), a container-host
service this device's kiosk browser calls directly. This host is deliberately thin:
a Sway session, one Chromium kiosk window, and `kitchen-display-agent`, which only
ever controls *which screen is showing* (Scan / Inventory / Recipes, plus
Registration — see below) over MQTT — the same "the agent controls the surface, a
separate write API owns the content" split `hosts/thin-client`'s admin canvas already
established.

**This is also one of the camera-equipped endpoints [`../../identity/`](../../identity/README.md)'s
person registration flow can run on** — "register me as `<name>`" by voice (needs
`ENABLE_VOICE_SATELLITE=true` and a real mic on this specific unit) or the
touchscreen form either way. See identity's own README for the actual design
(multi-phone support, anti-spoofing); this host only adds the mic/kiosk-page plumbing.

**A third, simpler sibling of `../thin-client/` and `../touch-panel/`**, not a variant
of either — one workspace, one app, no dock, no music, no general browsing. It reuses
the thin client's live-build toolchain and `configs/`+`agent/` split.

| | |
|---|---|
| Compositor | Sway, one workspace, one Chromium kiosk window |
| Autologin | greetd, straight into `/usr/local/bin/kiosk-session` |
| Touch input | Native Wayland `wl_touch`, same assumption as `../touch-panel/` |
| Camera | Browser-native `getUserMedia()` inside the Chromium kiosk page — no separate native capture app on this device at all |
| Content | `pantry-vision`'s frontend (`../../pantry-vision/frontend/`) by default; `identity`'s registration page on demand |
| Voice | wyoming-satellite + openWakeWord — **opt-in, off by default**, needed for "register me as `<name>`" |
| Remote control (HA/LLM) | `kitchen-display-agent` — **Show scan / Show inventory / Show recipes / Show registration** buttons only |
| Remote control (human) | SSH only — no wayvnc, same scope decision as `../touch-panel/` |

## Hardware

**No specific unit has been chosen.** What's assumed: a touchscreen (native
`wl_touch`, same caveat as `../touch-panel/README.md`) mounted somewhere near the
fridge/pantry, with a USB webcam pointed at wherever items get held up — either
built into the panel or a separate camera on a short cable/gooseneck mount so the
angle can be adjusted independently of the screen. See §1.15 of
`docs/project-plan.md` for the (unverified, non-specific) hardware line item.

## Before you build

Deploy `pantry-vision`/`pantry-web` first (`ENABLE_PANTRY_VISION` in
`hosts/container-host/scripts/setup-container-host.sh`) — this image builds and boots
fine without it, but every tab will show a connection error until it exists. Then edit
the `# CONFIGURATION` block at the top of
[`scripts/build-kitchen-display-iso.sh`](scripts/build-kitchen-display-iso.sh):

| Variable | What to put in it |
|---|---|
| `MQTT_BROKER_HOST` | LAN IP of the container host running Mosquitto |
| `PANTRY_WEB_URL` | The `pantry-web` service's URL, e.g. `http://192.168.1.10:8096` |
| `PANTRY_VISION_URL` | The `pantry-vision` API's URL, e.g. `http://192.168.1.10:8095` — read by `app.js` from the kiosk window's own URL query params (see `pantry-kiosk`'s comment), never baked into the served frontend files |
| `PANTRY_VISION_TOKEN` | **Must exactly match** `PANTRY_VISION_TOKEN` in `pantry-vision/pantry-vision.env` on the container host — there's no way for this repo to push that value between the two hosts for you |
| `IDENTITY_WEB_URL` / `IDENTITY_URL` / `IDENTITY_TOKEN` | Same shape as the `PANTRY_*` ones above, for `../../identity/` (`ENABLE_IDENTITY` in `setup-container-host.sh`) — `IDENTITY_TOKEN` must match `identity/identity.env`'s own token |
| `ENABLE_VOICE_SATELLITE` | `false` unless this specific unit has a real microphone attached — see `../thin-client/README.md`'s identical flag for the same per-image reasoning |
| `KIOSK_USERNAME` / `IMAGE_HOSTNAME` / `KITCHEN_DISPLAY_NAME` | Per-device identity |
| `SSH_AUTHORIZED_KEY` | Optional — password auth is disabled and there is no wayvnc |

## Build

```sh
sudo ./scripts/build-kitchen-display-iso.sh
```

Same directory-split convention as every other host in this project: `configs/` and
`agent/` are human-edited and git-tracked; `live-build/config/includes.chroot/` is
generated, gitignored, never hand-edited.

## Why the camera has no native capture app

`../touch-panel/` and `../thin-client/` both shell out to native processes for their
media (mpv, the Spotify Flatpak). This device doesn't need that: Chromium's own
`getUserMedia()` API, called from `pantry-vision/frontend/app.js`, talks to the
webcam directly and hands a captured frame straight to `POST /identify` as a blob —
no intermediate script, no temp file, no separate process to supervise. The one thing
this needs from the kiosk launch itself is `--use-fake-ui-for-media-stream`
(`configs/sway/pantry-kiosk`), which auto-accepts the camera permission prompt that
would otherwise sit unanswered on a screen nobody is there to click "Allow" on.

## Home Assistant entities

`kitchen-display-agent` publishes four buttons on connect: **Show scan**, **Show
inventory**, **Show recipes**, **Show registration** — the first three kill and
relaunch the kiosk Chromium window at `PANTRY_WEB_URL/index.html?...#<tab>`; the
fourth relaunches it at `IDENTITY_WEB_URL/register.html?...` instead (a different
backend, `identity`, behind the same "Show X" shape — see
`kitchen_display_agent/main.py`'s `on_show()`). That's the entire MQTT surface; there
is no media_player, no workspace select (one workspace), no capture/audio/
remote-desktop entities the way the thin client has. Reading or writing the actual
inventory, or registering a person, is a **separate** path — Home Assistant/the LLM
calling `pantry-vision`'s or `identity`'s own published API directly, not through
this agent, exactly mirroring the split `hosts/thin-client`'s admin canvas already
established between "which screen is showing" (MQTT, this agent) and "what's
actually on it" (a dedicated write API). Voice registration in particular never
touches this agent at all — see `../../identity/README.md`'s worked HA intent-script
example.

### Security boundary

Same shape and reasoning as every other host's agent in this project:
`kitchen_display_agent/mqtt_discovery.py` is the entire inbound MQTT control surface
of this machine — LLM tool call → HA service call → MQTT → this agent, no HTTP
listener, no websocket, no exposed Sway IPC socket, no VNC. `pantry-vision` is a
**separate, deliberately published** service with its own bearer-token boundary — see
`../../pantry-vision/README.md`'s "A real network listener, unlike admin-canvas"
section for why that one's trust model is different on purpose.

## Manual verification still outstanding

None of this has been run on hardware. In rough order — on top of everything already
flagged as unverified in `../../pantry-vision/README.md` (the Grocy API assumptions,
the vision model's accuracy/latency, which this device's whole usefulness rides on):

1. The ISO builds at all.
2. greetd lands in Sway with no login prompt.
3. **Touch input as native `wl_touch`** — same open item as `../touch-panel/`.
4. **Camera enumeration and `--use-fake-ui-for-media-stream`** — whether Chromium
   picks the right `/dev/videoN` when more than one video device is present (a
   built-in laptop-style webcam plus a dedicated one, say), and whether that flag
   actually suppresses the permission prompt on this Chromium/Wayland combination
   specifically (it's a well-known flag for automated testing, not confirmed here
   against a real kiosk session).
5. `kitchen-display-agent` connects to Mosquitto and the device appears in HA.
6. End-to-end scan latency: camera capture → `/identify` → a usable proposal on
   screen, with a real vision model on real LLM-host hardware. This is the number
   that decides whether the whole "hold it up, wait, confirm" workflow feels usable
   or like standing at the fridge for 30 seconds waiting on an LLM.
7. Whether `?api=...&token=...` survives being passed through `--app=` URL escaping
   correctly for every character that can appear in `PANTRY_VISION_URL` — the
   `sed`-based percent-encoding in `configs/sway/pantry-kiosk` handles `/` and `:`
   only, which is enough for a bare `http://host:port` but not for a URL with a path
   or query string of its own.
8. Idle-blank timeout (20 minutes, `configs/sway/config`) and touch-resume — untested.
9. Switching between `pantry-kiosk` and `identity-kiosk` (both `configs/sway/`)
   actually replaces the window rather than leaving two Chromium instances up — both
   scripts kill any chromium process, not just their own profile, specifically
   because this device has only one workspace; never exercised against a real
   in-progress scan or registration getting interrupted mid-flow by the other.
10. Everything in `../../identity/README.md`'s own verification list, especially
    `TRUSTED_ENTITY_PREFIXES` — registration will not find a candidate at all until
    that's set to match your real HA entity IDs.