SmartestHome/tools

tools — the build system

Every build and setup script for this project, driven by one config file at the repo root.

cp CoreSystemConfig.json.template CoreSystemConfig.json
$EDITOR CoreSystemConfig.json
tools/validate-config.py            # check it before you commit to a long build
sudo -E tools/build-all.sh          # build everything

That's the whole workflow.

Why this exists

Before it, the same facts lived in six places. The container host's IP was written into four ISO builders; IDENTITY_TOKEN had to match by hand across three; every service URL was a hand-typed string with a port in it. Any one of them could be wrong, and the symptom was always the same and always late: an image that boots fine and then can't reach something, discovered after a 40-minute build and a reboot.

Two changes fix that class of bug:

1. Nothing is written twice. No script in this directory contains an IP address, a port or a token. They read CoreSystemConfig.json.
2. Anything derivable is derived. You give the subnet prefix once and one last octet per host. Every address and every service URL is computed from those.

The twinned pair

The container host and the LLM host are built as a matched set, and the twinning is mechanical rather than a matter of remembering:

network.subnet_prefix   192.168.30
container_host.ip_last_octet    12   ->  192.168.30.12
llm_host.ip_last_octet          13   ->  192.168.30.13

  ...so the container host's OLLAMA_HOST is http://192.168.30.13:11434
     because it was COMPUTED from the LLM host's octet in the same build,
     not because someone typed the same address into two files.

Move the LLM host to .21 and rebuild: the container host's Ollama URL follows on its own. Change the subnet prefix and both halves move, along with every kiosk's service URLs. Neither image can be built pointing at an address the other one isn't using.

Both halves are stamped with the same SMARTHOME_PAIR_ID in /etc/smarthome-build, so two USB sticks found in a drawer months later can be checked against each other. The ID is a hash of the config's meaning, not its bytes — reformatting the JSON doesn't change it, moving a host does.

The validator

validate-config.py runs before every build and refuses to start on an error, so a mistake costs seconds instead of an hour. It's stdlib-only on purpose: it has to be able to run on a fresh checkout before anything is installed.

What it catches, beyond missing and malformed values:

Check	Why it matters
Duplicate ports	Two services on one port means a container that silently fails to bind, or whichever won last boot answering. Miserable to diagnose from the symptom — and it's how music_assistant (default 8095) collides with pantry_vision, which Compose's own check never catches because Music Assistant runs network_mode: host
Both core hosts on one address	The twinning's single assumption
A host colliding with the gateway	—
Duplicate kiosk hostnames	They identify devices on the network and in HA
Placeholder or low-entropy tokens	Caught before the length check, so padding changeme out to 32 characters doesn't sneak past
A private key pasted where the public key goes	—
A kiosk that needs a disabled service	A door panel built against enable.identity: false builds fine and fails at runtime
Wi-Fi SSID without a PSK, MQTT user without a password	—
192.168.0.x / 192.168.1.x subnets (warning)	Collides with typical café and hotel LANs, which breaks a WireGuard split tunnel routing that range — see docs/network-integration.md §2.1

Warnings print but don't block. Errors block and nothing is written.

What's here

Script	Builds
build-all.sh	Everything. The normal entry point — --core, --kiosks, --dry-run
build-core-pair.sh	The twinned container host + LLM host
build-container-host-iso.sh	Just the container host
build-llm-host-iso.sh	Just the LLM host
build-door-panel-iso.sh etc.	One kiosk; takes a hostname when several of a type are configured
build-audio-endpoint-iso-amd64.sh	An amd64 audio endpoint (mini PC, live-build → .iso)
build-audio-endpoint-image-arm64.sh	An arm64 audio endpoint (Pi + HiFiBerry, rpi-image-gen → .img)
setup-container-host.sh	The container-host setup itself, run by its ISO's first-boot unit (or by hand)
setup-llm-host.sh	Same, for the LLM host
generate-caddyfile.sh	The reverse proxy's config, derived like everything else
export-proxy-ca.sh	Fetch Caddy's internal root CA so browsers stop warning
generate-tokens.py	Fill in empty service tokens and write tokens.txt
validate-config.py	Check the config
config-export.py	Config → shell variables, deriving URLs. Where the twinning happens
lib/coreconfig.sh	The loader every builder sources

Building one image is supported but unusual: the images are a set that has to agree with itself, which is why build-all.sh is the default and a failure in one image doesn't abandon the rest.

One entry, one image

Kiosks and audio endpoints come from arrays in the config, and every entry gets its own artifact in iso-out/, named smarthome-<hostname>-<pairid>.iso. That naming is load-bearing rather than cosmetic: live-build always writes the same filename into the same per-host tree, so two thin clients would otherwise have the second silently overwrite the first — leaving one ISO carrying the second room's hostname and nothing to indicate the first was lost.

Per-type builders take a hostname when several of that type exist, and refuse with the list of real ones if you omit it or get it wrong.

This holds for both audio-endpoint architectures: the arm64 image used to be generic (named per unit in Raspberry Pi Imager at flash time) and is now pre-named like everything else, because it was the last identity in the household typed in after a build — and one that failed silently when mistyped, since spotify-connect-start reads $(hostname) and a typo just yields a Spotify device under the wrong room name.

The cost is one full rpi-image-gen run per arm64 room, qemu-emulated on an x86 builder. Set "arm64_prebake": false in the config's build section to go back to a single generic .img, built once regardless of how many arm64 rooms are listed. Wi-Fi and SSH keys are still set in Imager for arm64 either way — only the hostname moved.

Tokens fill themselves in

Leave identity_token, pantry_vision_token and transit_token empty and the first build generates them, writes them back into the config, and drops a tokens.txt next to it with the values and what each is for.

Writing them back is the part that matters. A token is only useful because two machines agree on it — the container host runs identity with it, and every kiosk image is built with the same value baked into its URLs. Generating fresh randomness per build would produce a door panel that cannot talk to the service it was built for. So blanks are filled once, persisted, and never overwritten.

Only tokens this project can legitimately invent are generated. ha_token isn't (only Home Assistant can mint one, and not until it's running), nor is mqtt_password (it has to match Mosquitto), nor admin_password_hash (needs mkpasswd), nor ssh_authorized_key (a generated key would have no private half you hold). tokens.txt lists those too, with the reason, so an empty field is never a mystery.

Both tokens.txt and the filled-in config are gitignored.

HTTPS

proxy.enabled puts a Caddy reverse proxy in front of this repo's own services: one hostname, a real HTTP→HTTPS redirect, and the API on the same origin as the page.

That last part isn't cosmetic — serving the admin panel over HTTPS while its ?api= still pointed at http://…:8097 would have every call blocked as mixed content. And the reason to want HTTPS at all is that the admin panel's URL carries IDENTITY_TOKEN, which grants administrative access to the person registry and to the device grants that decide whether a smart lock opens. See proxy/README.md.

The ISOs contain secrets

This is deliberate — burning everything in is what makes installation unattended, with no env files to edit on a freshly-booted host. It also means every ISO is a credential: Wi-Fi PSK, service tokens, MQTT and HA credentials, all readable by anyone holding the stick.

.gitignore covers CoreSystemConfig.json and iso-out/, so neither can be committed by accident. Wiping old USB sticks is on you.

Two things can't be burnt in

Neither exists at build time, so both need a human afterwards:

1. HA_TOKEN — a Long-Lived Access Token from Home Assistant's own UI, which doesn't exist until HA has been started and an account created. Put it in the config and rebuild, or edit identity.env on the container host.
2. TRUSTED_ENTITY_PREFIXES — the real entity_id prefixes your Private BLE Device setup produces (Developer Tools → States). The shipped default is a guess and it's the highest-risk unknown in Phase 6.

Adding a service or a kiosk

• A new port: add it to ports and reference it in config-export.py's derived URLs. The duplicate check covers it from then on.
• Another audio endpoint: add an entry to audio_endpoints with its arch. build-all.sh builds every entry, so listing both an amd64 and an arm64 one gets you both — they're separate toolchains producing different artifacts, not one image that runs on both.
• Another kiosk of an existing type: add an entry to kiosks with its own hostname. build-all.sh picks it up; the per-type builder takes the hostname as an argument.
• A new kiosk type: add it to KIOSK_TYPES in validate-config.py and add a build-<type>-iso.sh. build-all.sh finds it by naming convention.

Manual verification still outstanding

1. No ISO has ever been built with this. lb build needs live-build, root, and a long network fetch; none of that has been run. What has been tested is everything up to that point: config validation and URL derivation (43 checks), and the builders' generated artifacts — env files, preseed, network config, /etc/hosts, first-boot units, build stamps — produced by the real code paths with only lb stubbed (44 checks). The lb config/lb build invocations themselves are unverified.
2. The preseed files are written from Debian's documented shape, not tested. An unattended install that gets a preseed key wrong typically stops at an interactive prompt rather than failing loudly, so budget for a monitor on the first install.
3. partman-auto/disk erases the configured disk without confirmation. That is what unattended means, and it's why install_disk is worth double-checking against the actual machine you boot it on.
4. Static addressing assumes eth0. Debian's predictable interface naming may well call it enp3s0 on your hardware, in which case /etc/network/interfaces.d/smarthome needs the real name.
5. The kiosk builders' migration is untested end-to-end. Their config blocks now read from CoreSystemConfig.json, but the body of each script is unchanged from when it worked with hand-edited constants — so the risk is confined to the mapping, not to image contents.