diff --git a/.gitignore b/.gitignore index 6b58209..881ba2d 100644 --- a/.gitignore +++ b/.gitignore @@ -114,3 +114,5 @@ hosts/*/live-build/chroot/ hosts/*/live-build/cache/ hosts/*/live-build/binary/ hosts/*/live-build/*.iso +config/generated-*.yaml +hosts/audio-endpoint/rpi-image-gen/config/generated-*.yaml diff --git a/CoreSystemConfig.json.template b/CoreSystemConfig.json.template index 9c3c638..ed51f0f 100644 --- a/CoreSystemConfig.json.template +++ b/CoreSystemConfig.json.template @@ -183,7 +183,8 @@ ], "build": { - "_comment": "Where finished images land, relative to the repo root. Gitignored — see .gitignore — because these carry every secret in this file.", - "output_dir": "iso-out" + "_comment": "Where finished images land, relative to the repo root. Gitignored — see .gitignore — because these carry every secret in this file. arm64_prebake bakes each arm64 audio endpoint's hostname into its own image, so every device in the household is named by the build rather than typed into Raspberry Pi Imager afterwards; it costs one full rpi-image-gen run per room. Set it false to build a single generic .img and name each unit at flash time instead.", + "output_dir": "iso-out", + "arm64_prebake": true } } diff --git a/docs/project-plan.md b/docs/project-plan.md index 8f568d2..014bccc 100644 --- a/docs/project-plan.md +++ b/docs/project-plan.md @@ -534,7 +534,7 @@ New hardware: §1.13. Two separate build pipelines, arm64 and amd64 — see belo 2. **Two build pipelines, not one image with a flag**: the HiFiBerry Amp2 (arm64's assumed audio hardware) is a Raspberry Pi GPIO HAT with no amd64 equivalent — an x86 mini PC instead uses a USB DAC/amp (e.g. Fosi Audio V3, USB Audio Class, no driver needed). arm64 uses **rpi-image-gen** (the Raspberry Pi Foundation's current officially-recommended tool for custom images, superseding pi-gen for this use case) to produce a real flashable `.img`. amd64 reuses **the thin client's own live-build toolchain** rather than a third, unfamiliar one — a new, much smaller live-build tree with the entire graphical/kiosk stack stripped out, headless boot straight to `multi-user.target`. 3. **Spotify Connect install is apt-first with a documented fallback, reused identically on both variants** from `hosts/thin-client/live-build/config/hooks/normal/0500-spotify-connect.hook.chroot`'s own already-established logic (neither `spotifyd` nor `librespot` is in Debian bookworm main) — not two independent judgment calls about the same package. 4. **Audio output is direct ALSA, not PipeWire**, on both variants — deliberately different from the thin client, since this is a single-purpose headless appliance with no multi-app mixing need and no desktop session to run a per-user PipeWire daemon in the first place. -5. **Per-room identity differs by variant, each using whichever mechanism its own platform actually supports well**: arm64 builds one generic image and sets hostname/Wi-Fi per physical unit via Raspberry Pi Imager's OS Customisation dialog (works on any `.img`, no rebuild needed); amd64 has no equivalent tool for a generic x86 ISO, so it bakes `IMAGE_HOSTNAME` in at build time instead, one build per room, matching the thin client's own `THINCLIENT_NAME` convention. Both converge on the same mechanism at the software layer: `spotify-connect-start` reads `$(hostname)` fresh at every service start, regardless of which point in the pipeline set it. +5. ~~Per-room identity differs by variant~~ — **converged** (`tools/`): **both** variants now bake the hostname in at build time from `CoreSystemConfig.json`'s `audio_endpoints` array, one image per room, so no device in the household gets its identity typed in after the build. arm64 was the last exception (a generic `.img` named in Raspberry Pi Imager's OS Customisation dialog at flash time), and it was worth converging because it was the only remaining hand-entry point *and* it failed silently when mistyped — `spotify-connect-start` reads `$(hostname)`, so a typo yields a Spotify Connect device under the wrong room name rather than an error. **The tradeoff is explicit**: one full `rpi-image-gen` run per arm64 room (qemu-emulated on an x86 builder) instead of one for all; `build.arm64_prebake: false` restores the single-generic-image behaviour. Wi-Fi and SSH keys are still set in Imager on arm64 — only the hostname moved into the build. Both converge on the same mechanism at the software layer: `spotify-connect-start` reads `$(hostname)` fresh at every service start, regardless of which point in the pipeline set it. 6. **rpi-image-gen's exact config/layer YAML schema is the single highest-risk unverified part of this phase** — written from the tool's documented structure, not hands-on validated (no ARM build environment available). See `hosts/audio-endpoint/README.md`'s prominent callout before a real build. 7. Neither image gives Home Assistant any control or visibility over these endpoints — no MQTT, no HA entities. Deliberate scope decision matching the "per-room independent" choice, not an oversight; revisit as a separate addition if remote control/monitoring from HA is wanted later. 8. Nothing built or flashed on real hardware — no Raspberry Pi, no HiFiBerry Amp2, no x86 test box available. See the itemized unverified list in `hosts/audio-endpoint/README.md`. diff --git a/hosts/audio-endpoint/README.md b/hosts/audio-endpoint/README.md index dcb607f..78af49c 100644 --- a/hosts/audio-endpoint/README.md +++ b/hosts/audio-endpoint/README.md @@ -11,7 +11,7 @@ paths genuinely differ (see below) — not one image with a flag. | Audio hardware | HiFiBerry Amp2 HAT (GPIO, 60W, drives 4–8Ω passive speakers directly) | A USB DAC/amp, e.g. Fosi Audio V3 (USB Audio Class, drives passive speakers directly) | | Build tool | [rpi-image-gen](https://github.com/raspberrypi/rpi-image-gen) | live-build (same tool `hosts/thin-client/` uses) | | Output | A bootable `.img` | A bootable live ISO | -| Per-room identity | Set per physical unit, after building, via Raspberry Pi Imager | Baked in at build time — one build per room | +| Per-room identity | Baked in at build time from `CoreSystemConfig.json` — one build per room (set `build.arm64_prebake: false` to get one generic image named in Pi Imager instead) | Baked in at build time — one build per room | Both variants share one thing: `configs/spotify-connect.service` + `configs/spotify-connect-start`, an architecture-independent systemd @@ -89,14 +89,28 @@ a `.img` under that clone's `work/` directory. ### Per-room identity -**Build one generic image, not one per room.** Raspberry Pi Imager's own **OS -Customisation** dialog (gear icon, or Ctrl+Shift+X) works on *any* `.img`, -including local custom ones — hostname, Wi-Fi SSID/password, and an SSH key, all -set per-flash without touching the image itself. Use it on this image, once per -physical unit, before writing the SD card. `spotify-connect-start` reads -`$(hostname)` at every service start, so whatever hostname you set in Imager *is* -that unit's Spotify Connect device name in the picker — no separate config file, -no rebuild per room. +**The hostname is baked in, one image per room** — same as every other image in +this repo. `tools/build-audio-endpoint-image-arm64.sh` substitutes the endpoint's +hostname from `CoreSystemConfig.json`'s `audio_endpoints` array into a generated +copy of `config/audio-endpoint.yaml`, and `build-all.sh` builds one per entry. + +That's a change from the original design, which built one generic `.img` and left +identity to Raspberry Pi Imager's **OS Customisation** dialog at flash time. Imager +still works and is still the right tool for Wi-Fi and SSH keys (neither is baked +in) — but hostname was worth moving into the build for two reasons: + +- It was the **only** identity in the household still typed in by hand after a + build, which is the exact step `CoreSystemConfig.json` exists to remove. +- It failed *quietly* when mistyped. `spotify-connect-start` reads `$(hostname)` + at every service start, so a typo doesn't error — it just puts a Spotify Connect + device with the wrong room name in the picker, and you find out when you cast to + the wrong room. + +**The cost is real**: one full `rpi-image-gen` run per room instead of one for all +of them, and on an x86 build host each of those is a qemu-emulated arm64 bootstrap. +If you have several arm64 rooms and a slow builder, set `"arm64_prebake": false` in +the config's `build` section — that restores the single generic image, and +`build-all.sh` will build it once no matter how many arm64 rooms are listed. ### Hardware setup diff --git a/hosts/audio-endpoint/rpi-image-gen/config/audio-endpoint.yaml b/hosts/audio-endpoint/rpi-image-gen/config/audio-endpoint.yaml index 6bb7ba1..f8ecb3d 100644 --- a/hosts/audio-endpoint/rpi-image-gen/config/audio-endpoint.yaml +++ b/hosts/audio-endpoint/rpi-image-gen/config/audio-endpoint.yaml @@ -53,10 +53,17 @@ hooks: chroot: - ../scripts/install-spotify-connect.sh -# Left at the tool's own default — Raspberry Pi Imager's OS Customisation -# dialog overrides this per physical unit after the image is built (see -# hosts/audio-endpoint/README.md's "Per-room identity" section). Do not -# hardcode a room name here; spotify-connect-start reads $(hostname) at -# service-start time, whatever it ends up being set to per unit. +# REWRITTEN PER ROOM AT BUILD TIME. tools/build-audio-endpoint-image-arm64.sh +# substitutes the endpoint's hostname from CoreSystemConfig.json into the `set:` +# line below and writes a generated-.yaml alongside this file; this +# value is only the fallback for a hand-run build. +# +# Don't edit it to a room name — that would be a second place a room name lives, +# which is the duplication the whole tools/ build system exists to remove. Add +# the room to the config's "audio_endpoints" array instead. +# +# The hostname matters beyond identification: spotify-connect-start reads +# $(hostname) at service-start time, so this is also the room's Spotify Connect +# device name. hostname: set: audio-endpoint diff --git a/tools/README.md b/tools/README.md index 5163893..14875ec 100644 --- a/tools/README.md +++ b/tools/README.md @@ -106,11 +106,16 @@ to indicate the first was lost. Per-type builders take a hostname when several of that type exist, and refuse with the list of real ones if you omit it or get it wrong. -**The one deliberate exception is the arm64 audio endpoint.** Raspberry Pi Imager sets -hostname and Wi-Fi per unit at flash time, so one generic `.img` serves every room and -`build-all.sh` builds it once no matter how many arm64 entries are listed. amd64 has no -equivalent tool for a generic x86 ISO, so it bakes the hostname in and genuinely needs -one ISO per room — see `docs/project-plan.md` Phase 15.5. +This holds for **both** audio-endpoint architectures: the arm64 image used to be +generic (named per unit in Raspberry Pi Imager at flash time) and is now pre-named +like everything else, because it was the last identity in the household typed in after +a build — and one that failed *silently* when mistyped, since `spotify-connect-start` +reads `$(hostname)` and a typo just yields a Spotify device under the wrong room name. + +The cost is one full `rpi-image-gen` run per arm64 room, qemu-emulated on an x86 +builder. Set `"arm64_prebake": false` in the config's `build` section to go back to a +single generic `.img`, built once regardless of how many arm64 rooms are listed. Wi-Fi +and SSH keys are still set in Imager for arm64 either way — only the hostname moved. ## The ISOs contain secrets diff --git a/tools/build-all.sh b/tools/build-all.sh index 788fdbf..44608e2 100755 --- a/tools/build-all.sh +++ b/tools/build-all.sh @@ -137,13 +137,12 @@ if [[ "$MODE" == "all" || "$MODE" == "kiosks" ]]; then run_build "${ktype} — ${khost}" "$builder" "$khost" done - # The two audio-endpoint architectures are asymmetric on purpose, and it changes how - # many images each needs (docs/project-plan.md Phase 15.5): - # amd64 — no x86 equivalent of Pi Imager's OS Customisation exists, so the hostname - # is baked in at build time. One ISO PER ROOM. - # arm64 — Raspberry Pi Imager sets hostname/Wi-Fi per unit at flash time, so one - # generic .img serves every room. Built ONCE no matter how many are listed. - arm64_done="false" + # Both architectures now produce ONE PRE-NAMED IMAGE PER ROOM, so every device in + # the household is identified by the build rather than by something typed in + # afterwards. arm64 used to be the exception (one generic .img, hostname set in + # Raspberry Pi Imager at flash time); baking it in costs a full rpi-image-gen run per + # room, which is the tradeoff `build.arm64_prebake: false` exists to undo. + arm64_generic_done="false" for row in "${AUDIO_ROWS[@]:-}"; do [[ -n "$row" ]] || continue IFS=$'\t' read -r aarch ahost _ <<< "$row" @@ -153,13 +152,19 @@ if [[ "$MODE" == "all" || "$MODE" == "kiosks" ]]; then "${SCRIPT_DIR}/build-audio-endpoint-iso-amd64.sh" "$ahost" ;; arm64) - if [[ "$arm64_done" == "true" ]]; then - core_log "arm64 audio image already built — ${ahost} flashes the same .img (hostname set in Pi Imager)" - continue + if [[ "$CORE_ARM64_PREBAKE" != "true" ]]; then + # Generic mode: one image for every arm64 room, named in Pi Imager. + if [[ "$arm64_generic_done" == "true" ]]; then + core_log "arm64 generic image already built — ${ahost} flashes the same .img (name it in Pi Imager)" + continue + fi + run_build "audio endpoint (arm64, generic image)" \ + "${SCRIPT_DIR}/build-audio-endpoint-image-arm64.sh" "$ahost" + arm64_generic_done="true" + else + run_build "audio endpoint (arm64) — ${ahost}" \ + "${SCRIPT_DIR}/build-audio-endpoint-image-arm64.sh" "$ahost" fi - run_build "audio endpoint (arm64, generic image)" \ - "${SCRIPT_DIR}/build-audio-endpoint-image-arm64.sh" "$ahost" - arm64_done="true" ;; *) core_warn "Unknown audio endpoint arch '${aarch}' — skipping ${ahost}" diff --git a/tools/build-audio-endpoint-image-arm64.sh b/tools/build-audio-endpoint-image-arm64.sh index ed6f6f4..c4f4f45 100755 --- a/tools/build-audio-endpoint-image-arm64.sh +++ b/tools/build-audio-endpoint-image-arm64.sh @@ -32,7 +32,17 @@ core_select_audio_endpoint arm64 "${1:-}" AUDIO_ENDPOINT_DIR="${CORE_REPO_ROOT}/hosts/audio-endpoint" RPI_IMAGE_GEN_DIR="${AUDIO_ENDPOINT_DIR}/rpi-image-gen" SHARED_CONFIGS_DIR="$AUDIO_ENDPOINT_DIR/configs" -BUILD_CONFIG="$RPI_IMAGE_GEN_DIR/config/audio-endpoint.yaml" +BUILD_CONFIG_TEMPLATE="$RPI_IMAGE_GEN_DIR/config/audio-endpoint.yaml" +# Per-endpoint build config, generated below. Kept out of the tracked config/ dir so a +# build never leaves a room-specific file lying where the template belongs. +if [[ "${CORE_ARM64_PREBAKE:-true}" == "true" ]]; then + BUILD_CONFIG="$RPI_IMAGE_GEN_DIR/config/generated-${CORE_AUDIO_HOSTNAME}.yaml" + IMAGE_INSTANCE="$CORE_AUDIO_HOSTNAME" +else + # Generic mode: build the template unchanged, name each unit in Pi Imager. + BUILD_CONFIG="$BUILD_CONFIG_TEMPLATE" + IMAGE_INSTANCE="generic" +fi # --------------------------------------------------------------------------- # Sanity checks @@ -44,8 +54,8 @@ for f in spotify-connect.service spotify-connect-start; do fi done -if [ ! -f "$BUILD_CONFIG" ]; then - echo "Missing $BUILD_CONFIG" >&2 +if [ ! -f "$BUILD_CONFIG_TEMPLATE" ]; then + echo "Missing $BUILD_CONFIG_TEMPLATE" >&2 exit 1 fi @@ -54,6 +64,50 @@ if ! command -v git >/dev/null 2>&1; then exit 1 fi +# --------------------------------------------------------------------------- +# 0. Generate this room's build config — PRE-NAMED, not generic. +# +# The arm64 image used to be built once and handed to Raspberry Pi Imager, whose OS +# Customisation dialog set hostname/Wi-Fi per physical unit at flash time. That worked, +# but it made this the ONE image in the household where identity is typed in by hand +# after the build — exactly the step CoreSystemConfig.json exists to remove. It also +# failed quietly when mistyped: spotify-connect-start reads $(hostname) at service +# start, so a typo doesn't error, it just puts a Spotify Connect device with the wrong +# room name on the network. +# +# So the hostname is baked in here instead, per endpoint, same as every other image. +# The cost is honest and worth stating: one full rpi-image-gen run PER ROOM rather than +# one for all of them, and on an x86 build host that means qemu-emulated arm64 +# bootstrapping each time. Set "arm64_prebake": false in the config's build section to +# go back to a single generic image and do it in Imager. +# --------------------------------------------------------------------------- +if [[ "${CORE_ARM64_PREBAKE:-true}" == "true" ]]; then +core_log "Generating build config for ${CORE_AUDIO_HOSTNAME}" +python3 - "$BUILD_CONFIG_TEMPLATE" "$BUILD_CONFIG" "$CORE_AUDIO_HOSTNAME" <<'PYEOF' +import re, sys +template, dest, hostname = sys.argv[1], sys.argv[2], sys.argv[3] +text = open(template).read() + +# Only the hostname value is rewritten. Deliberately a targeted substitution rather +# than a YAML load/dump: this file is full of explanatory comments about an unverified +# schema, and round-tripping it through a YAML library would silently strip every one +# of them. It also avoids a PyYAML dependency in the build path. +new, n = re.subn(r"(?m)^(hostname:\n(?:\s+#.*\n)*\s+set:\s*).*$", + lambda m: m.group(1) + hostname, text) +if n != 1: + sys.exit(f"error: expected exactly one 'hostname:/set:' block in {template}, found {n}. " + "The template's shape changed — fix this substitution rather than shipping " + "an image named after the wrong room.") + +header = (f"# GENERATED by tools/build-audio-endpoint-image-arm64.sh for '{hostname}'.\n" + f"# Do not edit — edit config/audio-endpoint.yaml (the template) or\n" + f"# CoreSystemConfig.json, and rebuild.\n") +open(dest, "w").write(header + new) +PYEOF +else + core_log "arm64_prebake is false — building the generic image (name each unit in Pi Imager)" +fi + # --------------------------------------------------------------------------- # 1. Fetch rpi-image-gen if not already present # --------------------------------------------------------------------------- @@ -78,19 +132,33 @@ cd "$RPI_IMAGE_GEN_SRC" # script and config/audio-endpoint.yaml is unaffected either way. ./rpi-image-gen build -c "$BUILD_CONFIG" +# Publish alongside every other image. The search is best-effort because +# rpi-image-gen's output path is part of the unverified surface — if it finds nothing, +# core_publish_image says so and the image is still wherever the tool put it. +# Newest .img wins, rather than `-newer $BUILD_CONFIG`: a build that finishes in the +# same filesystem timestamp granularity as the config write is not strictly "newer", +# and that comparison silently found nothing rather than failing visibly. +IMG="$(find "$RPI_IMAGE_GEN_SRC/work" -name '*.img' -printf '%T@ %p\n' 2>/dev/null \ + | sort -rn | head -1 | cut -d' ' -f2-)" +if [[ -n "$IMG" ]]; then + IMG="$(core_publish_image "$IMG" "audio-endpoint" "$IMAGE_INSTANCE")" +else + core_warn "No .img found under $RPI_IMAGE_GEN_SRC/work — check rpi-image-gen's own output + above for where it wrote the image; this wrapper could not locate it." +fi + echo echo "=== Done (if rpi-image-gen's invocation above matched the real tool) ===" -echo "Image should be under $RPI_IMAGE_GEN_SRC/work/ — check rpi-image-gen's own" -echo "build output above for the exact path; this wrapper does not parse it." echo echo "Next steps:" echo " 1. Flash with Raspberry Pi Imager (GUI) or:" echo " rpi-imager --cli .img /dev/" -echo " 2. BEFORE writing, use Imager's OS Customisation (gear icon / Ctrl+Shift+X)" -echo " to set THIS unit's hostname (= its Spotify Connect device name — see" -echo " the README's 'Per-room identity' section), Wi-Fi SSID/password, and" -echo " an SSH key if you want remote access. This is per physical unit, not" -echo " per build — the same .img is reused for every room." +echo " 2. The hostname is ALREADY SET to '${CORE_AUDIO_HOSTNAME}' in this image — it is" +echo " built for one room, not generic, so there is nothing to type into Imager's" +echo " OS Customisation dialog for it. That hostname is also the Spotify Connect" +echo " device name (spotify-connect-start reads \$(hostname) at service start)." +echo " STILL DO set Wi-Fi and an SSH key in Imager if this unit needs them —" +echo " those are not baked in, see README.md's 'Per-room identity' section." echo " 3. Attach the HiFiBerry Amp2 HAT and passive speakers before first boot." echo " 4. First boot checklist (see README.md for the full unverified list):" echo " systemctl status spotify-connect" diff --git a/tools/config-export.py b/tools/config-export.py index f5b2f71..365d22c 100755 --- a/tools/config-export.py +++ b/tools/config-export.py @@ -127,6 +127,7 @@ def main(argv: list[str]) -> int: emit("CORE_VOICE_WAKE_WORD", cfg.get("voice", {}).get("wake_word", "ok_nabu")) emit("CORE_BUILD_OUTPUT_DIR", cfg.get("build", {}).get("output_dir", "iso-out")) + emit("CORE_ARM64_PREBAKE", cfg.get("build", {}).get("arm64_prebake", True)) # --- The selected kiosk, if one was asked for --- if kiosk_hostname: