# Sway touch panel

Phase 16 of `docs/project-plan.md`. Builds a Debian 12 live ISO for a touch-driven
wall/counter panel: autologin into Sway, three fixed apps (Spotify, Home Assistant,
a general web browser) — four where the panel asks for **Discord** — switched with a
finger via an always-on dock, and also controllable by Home Assistant/the local LLM
over MQTT.

**A different device from `../thin-client/`**, not a variant of it. The thin client is
a couch-distance media station whose primary control surface is HA/MQTT and wayvnc,
deliberately with no on-screen bars. This one is touched directly, so it needs the
opposite: a persistent, finger-sized on-screen dock, native touchscreen input, and no
gesture/remote-control machinery. It reuses the thin client's live-build toolchain and
`configs/`+`agent/` directory-split convention, not its live-build tree — same
relationship as `../audio-endpoint/`'s amd64 build to the thin client's.

What ends up on the image:

| | |
|---|---|
| Compositor | Sway, workspaces `1:spotify` / `2:home` / `3:web` (+ `4:discord` where enabled), always-on touch dock |
| Autologin | greetd, `default_session` straight into `/usr/local/bin/kiosk-session` |
| Touch input | Native Wayland `wl_touch` preferred; degrades to a usable single-touch pointer if the hardware only exposes an emulated-mouse HID interface — see below |
| Music | Full Spotify GUI client (Flathub `com.spotify.Client`) — not a headless Connect receiver |
| Chat / voice | Discord (Flathub `com.discordapp.Discord`), **only where `enable_discord` is set** — see below |
| Home dashboard | Chromium in kiosk mode, `--app=$HA_URL`, auto-restart if it crashes |
| Web browser | Firefox, minimal chrome (back/forward/reload/address bar), general browsing |
| On-screen keyboard | wvkbd, toggled from the dock — see the caveat below, no auto-show |
| Remote control (HA/LLM) | `touchpanel-agent`, a systemd service publishing HA MQTT-discovery entities |
| Remote control (human) | SSH only — **no wayvnc on this image**, see below |

## Before you build

> **No touch-panel hardware has been chosen.** Nothing here has been booted on real
> metal or a real touchscreen. See "Touch input: two tiers" below for how this image
> handles either way a real touchscreen might show up to Linux.

Edit the `# CONFIGURATION` block at the top of
[`tools/build-touch-panel-iso.sh`](../../tools/build-touch-panel-iso.sh):

| Variable | What to put in it |
|---|---|
| `MQTT_BROKER_HOST` | LAN IP of the container host running Mosquitto (Phase 1) |
| `HA_URL` | Home Assistant URL, e.g. `http://192.168.1.10:8123` — this is what loads in the Home workspace |
| `KIOSK_USERNAME` | Autologin account name (`kiosk`) |
| `TOUCHPANEL_NAME` / `IMAGE_HOSTNAME` | Per-panel identity; each touch panel needs its own |
| `SSH_AUTHORIZED_KEY` | Optional. Password auth is disabled and there is no wayvnc, so without a key the only admin path is the local console. |

## Build

```sh
sudo -E tools/build-touch-panel-iso.sh
```

Same directory-split convention as the thin client: `configs/` and `agent/` are the
human-edited, git-tracked source of truth; `live-build/config/includes.chroot/` is
**generated** — wiped and rebuilt on every run, gitignored, never hand-edited. The only
templated tokens are `@KIOSK_USERNAME@`/`@KEYBOARD_LAYOUT@`; everything else the
in-chroot hooks need comes from `/etc/touchpanel-agent/config.env`.

## Touch input: two tiers

Not every touchscreen tells Linux the truth about being a touchscreen. Some cheap USB
controllers implement single-touch by emulating an absolute-position HID **mouse**
(a click at wherever the finger is) instead of a proper HID **digitizer**. udev reads
the device's capability bits to decide `ID_INPUT_TOUCHSCREEN` vs. `ID_INPUT_MOUSE`,
gets it wrong for exactly this class of hardware, and libinput then hands sway a
`type:pointer` device instead of `type:touch` — every tap becomes a mouse click, not
a touch event. This image handles both cases rather than assuming the good one:

1. **Native `wl_touch`** (`input type:touch` in `configs/sway/config`) — full
   multi-touch, works with no extra setup. This is what you get if the hardware
   reports correctly.
2. **Fallback: `input type:pointer`** — active on this image regardless, so a
   misclassified touchscreen is still usable out of the box: `accel_profile flat` +
   `pointer_accel 0` keep the absolute tap position from being distorted by mouse-
   acceleration curves meant for relative motion, `map_to_output "*"` keeps the
   coordinate space aligned to the screen, and `seat seat0 hide_cursor 1` hides the
   cursor within ~1ms of it stopping so a tap doesn't leave a visible arrow sitting on
   screen. The ceiling here is single-touch tap/click-drag — no pinch, no true
   multi-touch gestures — which is enough for the dock and most kiosk UI, not enough
   for e.g. pinch-zooming a map in the web browser workspace.

**The real fix, once real hardware is known**:
`configs/udev/99-touchscreen-override.rules` re-tags the specific device by its USB
vendor/product ID so udev (and therefore libinput, and therefore sway) treats it as a
genuine touchscreen — restoring full `wl_touch` semantics instead of settling for the
pointer fallback's ceiling. It ships as an inert `0000:0000` template that matches no
real hardware; that file's own header comment has the exact `udevadm`/`libinput
list-devices` commands to find your device's real IDs and confirm the fix took. Fill
it in and rebuild — or edit it directly on a booted image and
`udevadm control --reload-rules && udevadm trigger && swaymsg reload`, no rebuild
needed to test it.

Quick check on a booted image: `libinput list-devices | grep -A2 Capabilities` — if it
lists `touch`, you're on tier 1 already and don't need the udev override at all.

## Why there's no wayvnc on this image

The thin client's primary control surface is deliberately remote (HA/MQTT + wayvnc)
because nobody is standing at a couch-distance media station. This panel is the
opposite: it's mounted somewhere and touched directly, so remote *view/control* isn't
the primary need the way it is for the thin client — SSH already covers "something's
broken, let me fix it from a shell." If a real need for a remote-view channel shows up
once this is on real hardware, adding wayvnc back is a small, isolated change (copy
`../thin-client/configs/wayvnc/` and its two hooks) — deliberately left out for now
rather than built against a guess.

## Home Assistant login

The Chromium kiosk window's profile (`~/.config/touchpanel-chromium-ha`) is persistent
across restarts specifically so logging into Home Assistant once survives a reboot —
unlike the thin client's digest/admin kiosk Firefox profiles, which are stateless
pages re-copied on every launch. Log in on first boot (tap the dock's Home button if
the window doesn't already have focus) and it should stay logged in after that,
**unverified** — depends on HA's own session cookie lifetime and whether Chromium's
Wayland profile persistence behaves the way assumed.

## Spotify: full GUI, not a headless Connect receiver

`../thin-client/` and `../audio-endpoint/` both run `spotifyd`/`librespot` — a
Spotify Connect *receiver* with no interface of its own, controlled from the phone
app or HA. This panel runs the real, official Spotify Linux client instead
(`configs/spotify/spotify-launch`, Flathub `com.spotify.Client`), because the whole
point of a touch panel is a screen you interact with directly: browsing your library,
searching, picking a playlist with a finger. **Spotify Premium is required for
playback either way** — same requirement as the headless receivers.

Login is interactive, on the device, on first launch, and persists in the Flatpak's
own data directory (`~/.var/app/com.spotify.Client`) — nothing here wipes or reseeds
it.

`touchpanel_agent/mpris_bridge.py` bridges Spotify's MPRIS interface (via
`playerctl`) into an HA `media_player`-shaped set of entities, identical in shape to
the thin client's own bridge, just pointed at `spotify` instead of `mpv`/`spotifyd` as
the player name — **unverified that the Spotify Flatpak's MPRIS bus name is actually
`spotify`** (some Flatpak-sandboxed apps register under a `flatpak.<app-id>`-shaped
name instead). If Playback state never updates, check `playerctl -l` on the booted
panel and adjust `PLAYER_PRIORITY` in `mpris_bridge.py`.

## Touch dock

An always-visible bar reserved at the bottom of the screen (`configs/eww/`, layer-shell
`:exclusive true` so nothing ever tiles under or draws over it) with four buttons —
five on a panel with Discord: **Spotify**, **Home**, **Web**, (**Discord**), and
**Keyboard**. The workspace ones call `swaymsg
workspace` directly — fixed constants in `eww.yuck`, nothing from MQTT/HA is ever
interpolated into them, same rule as the thin client's now-playing widget. This is the
touch-first equivalent of the thin client's HA **Screen** select entity — both exist,
one for a finger, one for the LLM.

Session-scoped (`exec_always` in `configs/sway/config`), not part of
`touchpanel-agent`, for the identical reason the thin client's now-playing widget
isn't: compositor UI should die with the compositor, not need a restart-independent
lifecycle or open a second inbound control channel into the security-sensitive agent.

**If `eww` isn't installed** (it's not in Debian bookworm main —
`0500-eww-dock.hook.chroot` tries apt and otherwise leaves a documented placeholder),
there is no on-screen app switcher at all. The panel still boots and shows Home; HA/
LLM-driven app switching via `touchpanel-agent`'s **Screen** select and **Show
\<app\>** buttons still works either way, since that's a separate control path.

## Discord — a per-panel fourth app

Off by default; turned on per panel with `enable_discord` on that kiosk in
`CoreSystemConfig.json`. `tools/validate-config.py` rejects it on any other image
type — no other build installs the Flatpak or has a workspace for it, so the flag
would otherwise be silently ignored on a panel that boots looking fine.

**What it is for.** The Loggia panel is the balcony one, and the point is that a voice
call survives stepping outside for a cigarette: you move the call onto this machine on
the way out and back onto the desktop on the way in. That is a *switch device* action,
which is why Discord starts at session boot and stays logged in rather than launching
on demand — an app that needs fifteen seconds and a login is not one you use on the
way past. It is also why the HA **Show Discord** button and the dock button both
resolve to focus-the-running-app, exactly like Home and Spotify.

Three things follow from it being an Electron/XWayland app rather than a native
Wayland one:

- Sway matches it on `class="discord"`, **not** `app_id` — an `app_id` rule would
  silently never fire.
- Touch arrives as emulated pointer events, so it lands in the same tier-2 behaviour
  described under *Touch input* below: tap and drag work, two-finger scrolling in the
  message list may not. Fine for join / mute / leave, which is the job.
- It deliberately does **not** get `inhibit_idle`. A call would otherwise hold the
  panel's display on for an hour in an empty room; the screen blanks and the call
  keeps running, because sway's idle timeout powers off the output, it does not
  suspend the machine or the audio stream.

Which audio device the call actually uses is Discord's own setting, on-device, once —
this image does not manage it, and `touchpanel-agent` has no entity for it.

## On-screen keyboard — no auto-show, by design

`configs/keyboard/toggle-keyboard` toggles `wvkbd` on and off. There is deliberately
**no automatic show-on-text-field-focus** — that needs the `text-input-v3`/
`virtual-keyboard-v1` Wayland protocols wired all the way through each app via a shell
component (the way Phosh does it for squeekboard), which this image doesn't run.
Instead: tap the dock's Keyboard button before typing, tap it again to dismiss. This
is a known, accepted limitation, not a bug to chase before the panel is otherwise
useful — see the verification list below if you want to pursue proper auto-show later.

`wvkbd`, not `squeekboard`, was picked specifically because it doesn't assume that
shell integration exists; see `0700-onscreen-keyboard.hook.chroot`'s comment. It is
also not in Debian bookworm main, so that hook has the same apt-first/documented-
placeholder shape as `eww` and the thin client's `spotifyd`/`librespot` install.

## Home Assistant entities

`touchpanel-agent` publishes MQTT-discovery configs on connect. Under the MQTT
integration you should get one device per touch panel with:

- **Show Spotify**, **Show Home**, **Show web browser**, and on a Discord panel
  **Show Discord** (buttons) — switch workspace
  and, for Spotify/Home, focus the already-running app rather than relaunching it
  (both are meant to stay open and stateful, unlike the thin client's stateless
  digest/admin kiosk pages)
- **Screen** (select) — `1:spotify` / `2:home` / `3:web`, plus `4:discord` on a panel
  that has it. The options are derived from the app table rather than fixed, so a
  dropdown never offers a workspace this image has no app for
- **Playback state** (sensor, with track metadata as attributes), **Volume**
  (number), and play/pause / next / previous / stop (buttons), bridged from
  Spotify's own MPRIS interface
- A **media_player** discovery payload — see the thin client README's identical
  caveat: stock Home Assistant's MQTT integration has no `media_player` platform: the
  button/number/sensor entities above are what give playback control on a plain
  install; the HACS *MQTT Media Player* integration is what picks up this one.

### Security boundary

Identical shape and reasoning to `../thin-client/README.md`'s section of the same
name: `touchpanel_agent/mqtt_discovery.py` is the **entire** inbound control surface
of this machine. The LLM never gets a direct network path here — the only chain is
*LLM tool call → HA service call → MQTT → touchpanel-agent*. No HTTP listener, no
websocket server, no exposed Sway IPC socket, no VNC (this image has none at all, see
above). New control features belong as additional MQTT entities, not as a second
listener.

## Manual verification still outstanding

None of this has been run on hardware. In rough order:

1. The ISO builds at all (`lb build` is network-heavy and can fail on mirror hiccups).
2. greetd lands in Sway with no login prompt, on vt1, with getty@tty1 masked.
3. **Which tier the real touchscreen lands in** — see "Touch input: two tiers" above.
   Both `input type:touch` and the `input type:pointer` fallback are shipped and
   should both work without a rebuild; what's unverified is whether the fallback's
   `accel_profile flat`/`pointer_accel 0`/`hide_cursor` combination actually feels
   good on real absolute-position-emulated-as-mouse hardware (as opposed to just
   being technically functional), and whether `99-touchscreen-override.rules`'
   `ATTRS{idVendor}`/`ATTRS{idProduct}` match actually walks up to the right USB
   parent on a given controller's specific kernel driver.
4. `touchpanel-agent` connects to Mosquitto and the device appears in HA.
5. **Flathub app ID `com.spotify.Client` is correct** — flagged for verification in
   `0300-flatpak-spotify.hook.chroot`, same as the thin client's Steam Link ID.
5b. **Flathub app ID `com.discordapp.Discord` is correct**, on a Discord panel —
   flagged in `0350-flatpak-discord.hook.chroot` for the same reason.
5c. **Discord's real X11 `class`** — assumed `discord`, which is what the sway
   fullscreen rule and the agent's focus criteria both match on. `swaymsg -t
   get_tree` on the booted panel is the way to check; if it is capitalised or
   suffixed, the app opens in a window that never goes fullscreen and **Show
   Discord** switches workspace without focusing anything.
6. Spotify login persists in `~/.var/app/com.spotify.Client` across a reboot.
7. Spotify's real MPRIS bus name — assumed `spotify`, not confirmed (see the Spotify
   section above); `playerctl -l` on the booted panel is the way to check.
8. **Chromium's Wayland `app_id` for a `--app=` kiosk window** — assumed to start with
   `chromium` (`configs/sway/config`'s `for_window` rule and
   `touchpanel_agent/main.py`'s `focus_criteria` both match on it). If focusing the
   Home workspace's window from the dock/HA stops working, check the real app_id with
   `swaymsg -t get_tree` and correct both places.
9. HA login persistence in the Chromium kiosk profile across a reboot (see above).
10. `eww` and `wvkbd` are not in Debian bookworm main — both hooks try apt and
    otherwise leave a documented placeholder (dock absent / keyboard absent
    respectively); neither has been checked against the real bookworm archive.
11. **The Firefox extension IDs in `configs/firefox/policies.json`** — same unverified
    caveat as the thin client's copy; see that README's identical item.
12. **Phase 16's own "reactive path" check**: power off the container host and confirm
    the panel still boots to Spotify + the browser (Home will show a connection
    error, which is the expected/acceptable failure mode — unlike the thin client,
    there's no local-media fallback to fall back to here since this device has no
    local media source at all).
13. Keyboard layout defaults to German (`KEYBOARD_LAYOUT="de"`) — same per-image,
    no-runtime-override handling as the thin client's identical setting.
14. Dock touch-target sizing (`configs/eww/eww.scss`, ~84px buttons) — a starting
    guess, not measured against a real finger on real glass at the real screen DPI
    chosen once hardware exists.
