diff --git a/README.md b/README.md index 1c628b5..36c5a52 100644 --- a/README.md +++ b/README.md @@ -81,8 +81,8 @@ chores/ Presence/calendar-driven household chore nudging + - [ ] RuView node configs — `firmware/ruview/` documents the real upstream project ([github.com/ruvnet/ruview](https://github.com/ruvnet/ruview), integrated not forked) + a per-room provisioning wrapper + `automations.yaml.example` (sleep → dim lights, possible-distress → whole-household alert, concurrent elevated heart rate → colored lighting, bathroom occupancy → an external door indicator). **Every automation's entity_id is an unconfirmed placeholder**, and the concurrent-two-person-heart-rate rule rests on an unconfirmed assumption about RuView's multi-target vital-sign capability — see `firmware/ruview/README.md` §5–6 and `docs/project-plan.md` open decisions #32–33 before relying on any of it - [ ] Frigate peephole camera config (real RTSP details) - [ ] Grocy kiosk (Pi + touchscreen) setup -- [ ] LLM host (Ollama) setup script -- [ ] CalDAV / Nextcloud calendar integration notes +- [ ] LLM host (Ollama) setup script — `hosts/llm-host/` written: auto-detects a GPU vs CPU-only tier, runs Ollama as a pinned container (no `curl | sh` into root), pulls the Phase 3 text model + a vision model for `pantry-vision`/`chores`, and takes a reasoned position on Assist-vs-digest GPU contention (`MAX_LOADED_MODELS=1` — swap predictably rather than OOM mid-request). **Never run — no Debian machine, no GPU, no Ollama server**; the script is syntax-checked and its generated compose validated as YAML for both tiers, and that's the whole of the testing. The vision-model pick is still unmade (open decision #18), and the contention defaults are reasoned, not measured (#4) — see `hosts/llm-host/README.md` +- [x] CalDAV / Nextcloud calendar integration notes — `docs/caldav-integration.md`: the four independent clients (digest-engine read-only, chores' busy-check read-only, trash-calendar create-only, HA's own bridge), why they share one Nextcloud **app password** and what that costs (rotation touches three env files; **the read-only invariant is a code property, not a permission boundary** — an app password can't be scoped read-only or to one calendar), `trash-calendar`'s UID-prefix ownership invariant, and the two traps that cost real debugging time (unexpanded recurrence reporting a meeting on the day it was created; `CALDAV_VERIFY_TLS=false`). Nextcloud itself is pre-existing — nothing here deploys it - [x] Sway thin-client ISO (live-build) + thinclient-agent — built, not yet boot-tested on real hardware; RDP replaced by wayvnc (resolved), remaining open items (mic-enabled rooms, exact hardware target, wayvnc password provisioning) in `docs/project-plan.md` §4 - [ ] Thin-client follow-ups in progress: fullscreen-aware now-playing widget (cover art + controls), minimal Firefox chrome + uBlock Origin/SponsorBlock, persistent audio-output selection, outbound RDP/VNC client (`rdp-vnc.json`), HA mobile-app browser remote control (text input + mouse buttons), capture-card ("receiver box") video source selection on a new `5:capture` workspace, idle-gallery weather/clock overlay (clock always, weather via a new `smarthome/weather/current` MQTT topic an HA automation has to publish) — built, not yet tried against real capture-card hardware or a real weather automation, see `hosts/thin-client/README.md` - [x] Quarter-daily digest engine (mail/Signal/Telegram/Discord/WhatsApp, news, financial ingestion; LLM synthesis; digest-canvas SDK) — built and wired into `setup-container-host.sh` (`ENABLE_DIGEST_ENGINE`, off by default), not yet run against real credentials; household/calendar ingest (CalDAV/Grocy) still needs a real data source wired in, see `docs/project-plan.md` §4 diff --git a/docs/caldav-integration.md b/docs/caldav-integration.md new file mode 100644 index 0000000..22a58f6 --- /dev/null +++ b/docs/caldav-integration.md @@ -0,0 +1,159 @@ +# CalDAV / Nextcloud calendar integration + +Notes for [Phase 8](project-plan.md) and for the three services in this repo that +talk to the household calendar directly. + +**This repo does not deploy Nextcloud.** It's listed in the project plan's hardware +table as an *already running* household service (`§ "Nextcloud instance | Calendar +backend (CalDAV sync target)"`), and `setup-container-host.sh` never touches it. +Everything below assumes you already have one and are pointing this stack at it. + +Like `network-integration.md`, this is **guidance, not automation** — nothing under +this repo configures Nextcloud or Home Assistant for you. + +## Who talks to the calendar, and in which direction + +Four independent clients, and the direction of each matters more than the count: + +| Client | Direction | What it does | +|---|---|---| +| `digest-engine/ingest/caldav.py` | **read-only** | Pulls events in the digest window for the household section of the quarter-daily digest | +| `chores/check.py` (`_household_currently_busy()`) | **read-only** | Asks "is there a currently-active event whose summary matches a quiet keyword?" before nudging anyone | +| `trash-calendar/sync.py` | **write** (create only) | Writes bin-collection dates from the municipal ICS feed onto the household calendar | +| Home Assistant's CalDAV integration | read/write | The general-purpose bridge — `calendar.*` entities, `calendar.create_event` | + +These are **four separate connections to the same server**, not one shared client. That +is deliberate: `chores` runs as a oneshot timer job with no HTTP surface, `digest-engine` +runs on its own schedule, and neither should have to be up for the other to work. + +## One app password, not four + +All three of this repo's services deliberately read the **same** environment variable +names — `CALDAV_URL`, `CALDAV_USERNAME`, `CALDAV_PASSWORD`, `CALDAV_VERIFY_TLS` — and +expect the **same single Nextcloud app password** pasted into each of their env files. + +Create it at: **Nextcloud → Settings → Security → Devices & sessions → Create new app +password.** + +**Not the account password.** An app password is revocable on its own, scoped to this +one integration, and keeps the account password out of files on the container host. +It's also *mandatory* once two-factor authentication is on: the DAV endpoints have no +way to prompt for a second factor, so a 2FA account simply cannot authenticate to +CalDAV with its normal password. + +### What that choice actually costs + +Sharing one credential is the right call at household scale — four credentials to +create, rotate and revoke, for four clients that are all equally trusted, is +bookkeeping without a security gain. But be clear about the two consequences: + +1. **Rotating it means editing three files.** `digest-engine.env`, `chores.env`, and + `trash-calendar.env` on the container host, plus re-entering it in Home Assistant's + CalDAV integration. Nothing propagates it for you, and a service left with the old + value fails *quietly* — every one of these clients is built to degrade rather than + crash, so a stale credential looks like "the calendar had nothing to say." +2. **The read-only invariant is a code property, not a permission boundary.** A + Nextcloud app password cannot be scoped to read-only, nor to a single calendar. The + credential `digest-engine` holds is fully capable of deleting every event you own — + what stops it is that `ingest/caldav.py` only ever issues reads, an invariant + asserted in its own docstring and in the project plan's Phase 12 rule. If that + module ever grew a write, no permission on the server side would catch it. Treat + changes to those files accordingly. + +## What each service needs beyond the shared four + +| Variable | Used by | Meaning | +|---|---|---| +| `CALDAV_CALENDARS` | digest-engine | Which calendars to read (blank = all discovered) | +| `CALDAV_LOOKAHEAD_HOURS` | digest-engine | How far forward to look; the window is deliberately asymmetric (back over the digest window, forward over this) because a calendar is mostly useful forwards | +| `CALDAV_MAX_EVENTS` | digest-engine | Cap on events fed into the LLM context | +| `CALDAV_QUIET_KEYWORDS` | chores | Summary substrings that mean "don't nudge right now" — default `busy,meeting,call,movie,sleep` | +| `CALDAV_TARGET_CALENDAR` | trash-calendar | The **one** calendar it writes into, by display name | + +`CALDAV_TARGET_CALENDAR` being singular is intentional: the read paths can happily +span several calendars, but a writer that had to *guess* which of several calendars a +bin-collection event belongs in would eventually guess wrong, in someone else's +calendar. + +## The write path's ownership invariant + +`trash-calendar` is the only thing here that writes, and it constrains itself to +events it created: + +- Every event it creates gets a UID prefixed `smartesthome-trash-`, derived + deterministically from the source feed's own content. +- It only ever *creates* under that prefix, and only ever checks for existence before + creating. +- It never reads, modifies, or deletes anything else in the target calendar. +- A re-run with nothing new in the feed touches nothing at all. + +So the blast radius of a bug there is "duplicate or missing bin-day events", not +"someone's appointments are gone". If you add another writer later, copy this shape. + +## URL form + +Point `CALDAV_URL` at Nextcloud's **DAV root**: + +``` +https://cloud.example.com/remote.php/dav +``` + +The `caldav` library discovers the principal and its calendars from there. Nextcloud +also documents the per-user form +(`https:///remote.php/dav/principals/users//`); either works. + +## Two traps worth knowing before you debug them + +**Recurring events without expansion.** A weekly recurring event fetched without +`expand=True` comes back *once*, as its original master VEVENT carrying an RRULE — so +a naive client reports the meeting on the day it was first created, possibly years +ago, and nothing looks obviously broken. `digest-engine` passes `expand=True` and +retries for servers that reject it outright. Any new client needs the same. + +**`CALDAV_VERIFY_TLS=false` is for a self-signed internal cert, and nothing else.** +It disables certificate verification entirely, which on a LAN-only Nextcloud behind +your own CA is a reasonable trade, and on anything reachable beyond the LAN is a +straightforward man-in-the-middle hole. If your Nextcloud has a real certificate — +and it should, Let's Encrypt is free — leave this `true`. + +## Home Assistant's own integration + +Separate from this repo's three clients, and set up in HA's UI rather than by anything +here: **Settings → Devices & Services → Add Integration → CalDAV**, same URL and app +password. + +Phase 8's own guidance still stands: **create recurring events directly in Nextcloud, +not through HA**, and gate delete/move actions behind a confirmation step. HA's CalDAV +bridge is good at reading and at creating simple one-off events; recurrence rules are +where the impedance mismatch between the integration and the server shows up. + +## Failure behaviour + +Every client here fails soft, which is correct and also means an outage is quiet: + +| Client | If Nextcloud is unreachable | +|---|---| +| digest-engine | Household section renders without calendar content | +| chores | `_household_currently_busy()` returns "not busy" — **fails open on purpose**, so a broken calendar check can never be the reason chores stop being nudged | +| trash-calendar | The sync run is skipped; tomorrow's timer tries again | + +There is no alerting on any of this. A Nextcloud that's been down for a week looks +identical to a week with no calendar events — the same "silence isn't a signal" gap +noted in the project plan's open decision #38. + +## Still unverified + +1. **None of this has been run against a real Nextcloud instance** from this repo. + `digest-engine/ingest/caldav.py` is written against the `caldav` library's + documented API (sourced and dated in its own docstring); `chores`' busy-check and + `trash-calendar`'s write path have never been executed against a live server at all. +2. **`CALDAV_TARGET_CALENDAR` matching is by display name**, which is whatever the + calendar is called in Nextcloud's UI — not a URL or an ID. Renaming the calendar + silently breaks the trash sync. +3. **Nobody has checked what happens when the app password is revoked** mid-operation + — expected to surface as an auth error each client swallows into its normal + degrade path, i.e. silently, but that's reasoning rather than observation. +4. **The quiet-keyword list is a guess at how this household actually labels events** + (`busy,meeting,call,movie,sleep`). It matches on substrings of the summary, so an + event called "Call with the bank" pauses every chore nudge in the house for its + duration — which may or may not be what you want. diff --git a/docs/network-integration.md b/docs/network-integration.md index b8bd593..9d1dd87 100644 --- a/docs/network-integration.md +++ b/docs/network-integration.md @@ -180,6 +180,17 @@ column shows which are opt-in vs. always-on with the base stack. `ports:` mapping in the generated compose file at all, reachable only from other containers on the compose network (i.e. Home Assistant), by design. +### On the LLM host (a separate machine — `hosts/llm-host/`) + +| Port | Service | Auth | +|---|---|---| +| 11434 | Ollama | **None whatsoever.** Not a token, not a password — and the API is not read-only: it can pull and **delete** models as well as generate. Anyone who can reach this port can do all of it | + +That row is the strongest case in this table for §3's VLAN segmentation and §1's no- +port-forward rule. It is also why the LLM host belongs on the smart-home VLAN rather +than the trusted LAN: it accepts unauthenticated instructions from anything that can +reach it, which is exactly the property you want blast-radius boundaries around. + ## 5. What's still unverified here This entire document was written against `setup-container-host.sh`'s current diff --git a/docs/project-plan.md b/docs/project-plan.md index 811f31d..d93827a 100644 --- a/docs/project-plan.md +++ b/docs/project-plan.md @@ -273,10 +273,12 @@ real hardware" callouts for everything downstream of this.)* 3. Build plain HA automations: presence (RuView) on → light on at neutral default; off (with delay) → light off. **Validate this works with the LLM host powered off — this is your safety-net baseline.** ### Phase 3 — LLM host + conversation agent -1. Stand up the GPU host (or CPU fallback) with Ollama, pull Qwen2.5-14B-Instruct (or 7B/3B for CPU). +1. Stand up the GPU host (or CPU fallback) with Ollama, pull Qwen2.5-14B-Instruct (or 7B/3B for CPU). **Scripted**: `hosts/llm-host/scripts/setup-llm-host.sh` auto-detects the tier (`nvidia-smi` must both exist *and* succeed), installs Docker + the NVIDIA Container Toolkit, and runs Ollama as a pinned container rather than `curl | sh` into a root shell. It deliberately does **not** install the GPU driver — the most hardware/kernel-specific step on that box, and silently picking a version is how you get a machine that doesn't boot. 2. Add the Ollama integration in HA, point it at the LLM host over LAN. 3. Set up Wyoming faster-whisper + Piper, configure an Assist pipeline. 4. Test open-ended conversation and basic tool-calling before wiring into presence logic. +5. **Then power the LLM host off and confirm the house still works** — presence → light, the door panel, chore nudges (plain template instead of LLM-phrased), digest skipping rather than erroring. This is the testing-checklist guardrail below, and it's the entire reason this is a separate machine: it has to be able to be off. Anything that *breaks* rather than degrading is a bug in the consumer, not in the LLM host. +6. **Ollama has no authentication of any kind**, and its API can pull and delete models, not only generate — so the network is the whole boundary. Smart-home VLAN, never port-forwarded; it's in `docs/network-integration.md`'s port table for that reason. ### Phase 4 — AI-managed brightness/color 1. Script triggered after the presence-automation light-on event, calling `ai_task.generate_data` with room/time/occupancy context, requesting structured JSON (`brightness`, `rgb_color`). @@ -438,10 +440,17 @@ who someone is, which this project already made `identity`'s job. 4. Use Grocy's own add/consume UI — no extra hardware/logic needed for the "in vs. out" problem. ### Phase 8 — Nextcloud calendar sync + +Written up in full in **`docs/caldav-integration.md`** — the four independent clients +and their directions, the shared-app-password decision and what it costs, and the +recurrence/TLS traps. Nextcloud itself is pre-existing; nothing in this repo deploys it. + 1. Add HA's CalDAV integration, point at Nextcloud's CalDAV URL. 2. Confirm read + write (`calendar.create_event`) both work. 3. Expose the calendar entity to the LLM's tool list. 4. Gate delete/move actions behind a confirmation step; create recurring events directly in Nextcloud, not via HA. +5. **One Nextcloud app password, shared by all four clients** (HA plus this repo's three), never the account password — revocable on its own, and mandatory anyway once 2FA is on, since DAV endpoints can't prompt for a second factor. Note the two consequences: rotating it means editing three env files *and* HA, and a stale credential fails **quietly** because every client here degrades rather than crashes. +6. **The read-only invariant on `digest-engine`/`chores` is a code property, not a permission boundary** — a Nextcloud app password cannot be scoped read-only or to a single calendar, so the credential those services hold could delete every event you own. What prevents it is that those modules only ever issue reads. Review changes to them with that in mind; the server will not catch a regression. ### Phase 9 — Observability, dashboard, backup 1. Deploy Netdata (or your team's eventual choice from the Netdata/Checkmk evaluation) for container/host monitoring. @@ -852,7 +861,7 @@ These need a decision before their respective implementation steps can be built 1. ~~RDP vs. VNC vs. desktop-environment swap~~ — **resolved**: wayvnc (VNC) replaces RDP for this project; Sway is kept. 2. ~~WhatsApp ingestion approach~~ — **resolved**: `whatsapp-bridge` (headful Chromium + `whatsapp-web.js` in an Xvfb virtual display), not Baileys. Still opt-in (`ENABLE_WHATSAPP_INGEST`); still recommend a secondary/non-critical number, since automating a personal account carries some risk even via the real web client. 3. **Mainstream news source list** — `digest-engine/feeds/curated-feeds.opml` ships with `marxist.com/feed/rss` (real) plus BBC World/Al Jazeera/Guardian World/DW as clearly-marked placeholders (Reuters/AP were skipped — both have restricted their public RSS and guessing a live URL seemed worse than an honest placeholder). Still needs the user's actual sign-off/edit. -4. **Ollama contention** — not yet resolved; `DIGEST_SCHEDULE` defaults to `00,06,12,18` (every 6h) in `setup-container-host.sh`, unadjusted for Assist-traffic overlap. Revisit once real usage patterns are known. +4. **Ollama contention** — **half resolved** (Phase 3, `hosts/llm-host/`). The *resource* half now has a defensible default: `OLLAMA_MAX_LOADED_MODELS=1` (a 14B text model and a vision model don't co-fit in 8–12GB, so force a predictable swap rather than VRAM thrash or an OOM mid-request), `OLLAMA_NUM_PARALLEL=1` (predictable latency for whoever is speaking to Assist over throughput nothing here needs), and `OLLAMA_KEEP_ALIVE=30m` (Ollama's 5m default makes a household that talks to Assist a few times an hour pay the model-load cost nearly every time). **Still open**: these are reasoned, not measured — nothing has run against a real GPU under concurrent load — and the *scheduling* half is untouched, `DIGEST_SCHEDULE` still defaults to `00,06,12,18` with no adjustment for real Assist-traffic overlap, which needs actual usage data to settle. 5. **Credential storage** — implemented as a git-ignored `.env` seeded from `digest-engine.env.example` (matches the restic-password precedent); `age`/`sops` was not built, considered adequate for now. 6. **Exact mic-enabled room list** — still needed; `ENABLE_VOICE_SATELLITE` in `build-thin-client-iso.sh` defaults to `false` per-image until rooms are chosen. 7. **Exact thin-client hardware target** — still needed; nothing in the built image assumes specific hardware, but Steam Link/Xwayland decode performance can't be validated without it. diff --git a/hosts/llm-host/README.md b/hosts/llm-host/README.md new file mode 100644 index 0000000..dfafc88 --- /dev/null +++ b/hosts/llm-host/README.md @@ -0,0 +1,146 @@ +# llm-host + +The Ollama machine, from [Phase 3 of the project plan](../../docs/project-plan.md). +A **separate physical host** from the container host, on purpose — see "Why a separate +machine" below. + +Everything in this project that wants inference calls this one server: + +| Caller | What it asks for | If this host is off | +|---|---|---| +| Home Assistant (Assist / AI Task) | Conversation, tool calls, Phase 4's brightness/colour JSON | Assist's LLM agent is unavailable; **presence → light still works**, on plain automations | +| `digest-engine` | Quarter-daily synthesis + the counter-run verification pass | The run is skipped; the timer doesn't error | +| `pantry-vision` | Grocery-item recognition from one photo | The scan flow reports it can't identify; Grocy is untouched | +| `chores` | Bin/dishes/litter vision checks, and reminder phrasing | Camera checks skip; nudges use the plain template | + +## The guardrail this host is built around + +**Every consumer must degrade to "unavailable", never to "broken", when this machine +is off.** The project plan's testing checklist states it directly — *"Does the reactive +path (presence → light on) work with the LLM host powered off? (It must.)"* + +That's the whole reason this is a separate box rather than more containers on the +Phase 1 host: it *can* be off — for power, for noise, because you pulled the GPU — and +the house still works. The setup script's closing output walks you through actually +testing that, and it's worth doing once for real rather than assuming. + +If something *breaks* rather than degrading when this host is down, that's a bug in +the consumer, not here. + +## Two tiers + +Auto-detected from whether `nvidia-smi` both exists **and** succeeds (a leftover driver +package on a machine whose card was pulled satisfies the first but not the second). +Override with `TIER` at the top of the script. + +| Tier | Model | Reality | +|---|---|---| +| `gpu` | `qwen2.5:14b-instruct` | What Phase 3 specifies | +| `cpu` | `qwen2.5:7b-instruct` | Single-digit tokens/sec. Enough to validate the entire pipeline end to end before buying a card — `docs/components.md`'s deliberate "skip GPU" fallback, not a failure mode | + +Plus a vision model (`llava` by default) for `pantry-vision` and `chores`. Set +`PULL_VISION_MODEL=false` to skip it and save several GB if you're not running those +camera paths yet. + +**The vision model choice is not a considered one.** `llava` is simply the default +those two services already ship with, and open decision #18 flags the pick as unmade +and completely unbenchmarked. If grocery recognition turns out too slow or too +inaccurate to be usable, this is the first knob to turn — `qwen2.5vl` and `moondream` +are the obvious alternatives to measure against. + +## Contention: interactive vs. batch on one GPU + +The real scheduling problem here (project plan open decision #4), and the script's +defaults take a position on it: + +- **Assist is interactive** — a person is standing in the room waiting. +- **`digest-engine` is batch** — every 6h, nobody watching. +- **The vision callers are occasional but want a *different model resident*.** + +| Setting | Default | Why | +|---|---|---| +| `OLLAMA_KEEP_ALIVE` | `30m` | Ollama's own default of 5m means a household that talks to Assist a few times an hour pays the model-load cost nearly every time. 30m keeps it warm through normal use | +| `OLLAMA_MAX_LOADED_MODELS` | `1` | **Deliberate.** A 14B text model and a vision model don't co-fit in 8–12GB; letting Ollama try produces VRAM thrash or an OOM mid-request instead of an honest swap. 1 means "swap predictably, pay the reload when vision is actually needed." Raise it only if you have the VRAM *and have checked* | +| `OLLAMA_NUM_PARALLEL` | `1` | Predictable latency for whoever is speaking, over throughput nothing here needs | + +This is a **reasoned default, not a measured one** — none of it has been run against a +real GPU under real concurrent load. The remaining half of open decision #4 (whether +`DIGEST_SCHEDULE`'s `00,06,12,18` overlaps real Assist usage) needs actual usage data +to settle; the settings above at least make the failure mode a predictable swap rather +than an OOM. + +## Security: Ollama has no authentication + +None. Not a token, not a password. And its API is not read-only — it can **pull and +delete models**, not just generate. Anyone who can reach `:11434` can do all of that. + +The network is therefore the entire boundary: keep this host on the smart-home VLAN +and never port-forward it, exactly as `docs/network-integration.md` §1 says for +everything else. It's now in that document's port table for the same reason. + +## Why a container, not the native installer + +Ollama's official install is `curl -fsSL https://ollama.com/install.sh | sh`, which +pipes a fetched script straight into a root shell. The container path gives a pinned +image, an uninstall that's `docker rm`, and no arbitrary remote code executed as root +— the same reasoning behind every other component in this project running in Docker. + +The native install is a perfectly legitimate alternative, and on some GPU setups it's +less fuss than the NVIDIA Container Toolkit. If you go that way, the one thing you +**must** still do is set `OLLAMA_HOST=0.0.0.0:11434` in the systemd unit — see below. + +## The one configuration mistake that looks like a dead host + +Ollama binds `127.0.0.1` by default. In a container, that means the published port +forwards to a socket nothing is listening on, and **every caller gets a connection +refused that is indistinguishable from "the LLM host is powered off"** — which, given +that every consumer here is built to tolerate exactly that, degrades silently and +looks like nothing is wrong. + +The compose file sets `OLLAMA_HOST=0.0.0.0:11434` for this reason. Check it first if +inference is mysteriously "unavailable" everywhere at once. (This is the same class of +bug as `chores`' env template pointing at `127.0.0.1` for a sibling container — see +the project plan's open decision #38.) + +## Run it + +```sh +cd hosts/llm-host/scripts +sudo ./setup-llm-host.sh +``` + +Edit the variables at the top first — `BASE_DIR` above all, since models are large +(a 14B Q4 model is ~9GB, a vision model another 5–8GB) and it defaults to +`/opt/llm-host`. + +For the GPU tier, **the NVIDIA driver must already work** (`nvidia-smi` prints your +card). The script installs the Container Toolkit that lets Docker see the GPU, but +deliberately does not install the driver: that's the most hardware- and +kernel-specific step on this machine, and silently choosing a driver version for +someone is a good way to produce a box that doesn't boot. + +Afterwards the script prints exactly what to paste into HA and into each service's env +file on the container host. + +## Manual verification still outstanding + +1. **None of this has been run.** No Debian machine, no GPU, no Ollama server — the + script is syntax-checked and its generated compose file is validated as YAML for + both tiers, and that is the entire extent of the testing. Same honesty rule as + every other unbuilt host in this repo. +2. **The model tags are library names that upstream does rename.** `qwen2.5:14b-instruct` + and `llava` are written from Ollama's library as documented, not confirmed pullable + today. A failed pull is deliberately non-fatal — the server stays up and you fix the + tag by hand — but check if one fails. +3. **The NVIDIA Container Toolkit repo/apt steps are from NVIDIA's documented install**, + not run on a real machine. `nvidia-ctk runtime configure --runtime=docker` followed + by a Docker restart is the documented shape; verify against NVIDIA's current docs + before trusting it on hardware you care about. +4. **The contention defaults are unmeasured** — see the table above. +5. **Whether a 14B model at Q4 actually fits your card** is not checked anywhere. On + 8GB it will be tight-to-impossible; on 12GB+ it's comfortable. If it OOMs, drop to + the CPU tier's 7B tag on the GPU, which is the cheap first thing to try. +6. **No Wake-on-LAN.** If you want the digest timer to wake this host rather than skip + its run, that's a BIOS + `ethtool -s wol g` + a `wakeonlan` call from the + container host's timer — deliberately not scripted here, since it depends on + hardware that hasn't been chosen. diff --git a/hosts/llm-host/scripts/setup-llm-host.sh b/hosts/llm-host/scripts/setup-llm-host.sh new file mode 100755 index 0000000..4c15d9f --- /dev/null +++ b/hosts/llm-host/scripts/setup-llm-host.sh @@ -0,0 +1,321 @@ +#!/usr/bin/env bash +# +# LLM Host Setup — Ollama, docs/project-plan.md Phase 3 +# Target: Debian 12 (Bookworm), a SEPARATE physical machine from the container host. +# +# Stands up the one thing this host exists for: an Ollama server on the LAN that +# everything else in this project calls for inference — +# - Home Assistant's Ollama conversation integration (Assist, AI Task) [Phase 3/4] +# - digest-engine's quarter-daily synthesis + counter-run [Phase 12] +# - pantry-vision's grocery-item recognition (vision model) [Phase 17] +# - chores' bin/dishes/litter checks + reminder phrasing [Phase 20] +# +# THE GUARDRAIL THIS HOST IS BUILT AROUND: **everything above must degrade to +# "unavailable", never to "broken", when this machine is off.** docs/project-plan.md's +# testing checklist states it directly — "Does the reactive path (presence → light on) +# work with the LLM host powered off? (It must.)" — and every consumer above already +# falls back to a deterministic template, a plain lookup table, or simply skipping the +# run. Nothing here should ever become load-bearing for the reactive smart-home +# baseline. That is why this is a separate host at all: so it CAN be off. +# +# Two hardware tiers, auto-detected (override with TIER below): +# gpu — NVIDIA card present. Runs the 14B-class model per Phase 3. +# cpu — no GPU. Runs a 7B/3B model. Slower (single-digit tok/s), enough to validate +# the whole pipeline before spending money on a card, per docs/components.md's +# "Fallback: skip GPU" line. +# +# Run as: sudo ./setup-llm-host.sh +# +# EDIT THE VARIABLES BELOW BEFORE RUNNING. + +set -euo pipefail + +# --------------------------------------------------------------------------- +# CONFIGURATION — edit these before running +# --------------------------------------------------------------------------- +BASE_DIR="/opt/llm-host" # Config + model storage. Models are BIG (a 14B + # Q4 model is ~9GB, a vision model another 5-8GB) + # — make sure this lives on a disk with room. + +TIER="auto" # auto | gpu | cpu + +OLLAMA_PORT="11434" # Ollama's own default. + +# Models to pull, per tier. Phase 3 specifies Qwen2.5-14B-Instruct (GPU) or 7B/3B +# (CPU). Tags are Ollama library names — `ollama list` on a real host to confirm what +# you actually ended up with, since library tags do get renamed upstream. +GPU_TEXT_MODEL="qwen2.5:14b-instruct" +CPU_TEXT_MODEL="qwen2.5:7b-instruct" + +# The vision model, for pantry-vision (grocery items) and chores (bin/dishes/litter). +# NOT A CONSIDERED CHOICE — `llava` is the default those services already ship with, +# and docs/project-plan.md open decision #18 flags the pick as unmade and completely +# unbenchmarked. Treat this as "something to measure", not "the answer": if grocery +# recognition is too slow or too wrong to be usable, this is the first knob to turn +# (qwen2.5vl and moondream are the obvious alternatives to try). +VISION_MODEL="llava" + +PULL_VISION_MODEL="true" # false to skip — saves several GB if you're not + # running pantry-vision/chores camera checks yet. + +# --- Contention between interactive and batch callers ------------------------------ +# The real scheduling problem on one GPU (docs/project-plan.md open decision #4): +# Assist is INTERACTIVE (a person is standing there waiting), while digest-engine is +# BATCH (every 6h, nobody watching) and the vision callers are occasional but want a +# DIFFERENT model resident. Defaults below optimise for the interactive case, because +# that's the one where latency is felt: +# +# KEEP_ALIVE — how long a model stays resident after its last request. Ollama's own +# default is 5m, which means a household that talks to Assist a few times an hour +# pays the model-load cost almost every time. 30m keeps it warm through normal use. +OLLAMA_KEEP_ALIVE="30m" +# MAX_LOADED_MODELS — how many distinct models may be resident at once. **1 is +# deliberate on a single consumer GPU**: a 14B text model and a vision model do not +# fit together in 8-12GB, and letting Ollama try produces VRAM thrash or an OOM +# mid-request rather than an honest swap. 1 means "swap predictably, pay the reload +# cost when the vision model is actually needed." Raise it only if you have the VRAM +# to hold both and have checked that you do. +OLLAMA_MAX_LOADED_MODELS="1" +# NUM_PARALLEL — concurrent requests served per loaded model. 1 keeps latency +# predictable for whoever is speaking to Assist; higher trades that for throughput +# nothing in this project currently needs. +OLLAMA_NUM_PARALLEL="1" + +# --------------------------------------------------------------------------- +# End of configuration +# --------------------------------------------------------------------------- + +log() { echo -e "\n\033[1;34m==>\033[0m $*"; } +warn() { echo -e "\033[1;33m[warn]\033[0m $*" >&2; } +die() { echo -e "\033[1;31m[error]\033[0m $*" >&2; exit 1; } + +[[ $EUID -eq 0 ]] || die "Run this with sudo." + +# --------------------------------------------------------------------------- +# Tier detection +# --------------------------------------------------------------------------- +detect_tier() { + if [[ "$TIER" != "auto" ]]; then + echo "$TIER" + return + fi + # nvidia-smi existing AND succeeding are different things — a leftover driver + # package on a machine whose card has been pulled would satisfy `command -v` alone + # and send us down the GPU path to fail later at container start. + if command -v nvidia-smi >/dev/null 2>&1 && nvidia-smi >/dev/null 2>&1; then + echo "gpu" + else + echo "cpu" + fi +} + +# --------------------------------------------------------------------------- +# Docker — same install path as hosts/container-host/scripts/setup-container-host.sh. +# Ollama is run as a container rather than natively installed for the same reason +# everything else in this project is: no `curl | sh` into a root shell, a pinned +# image, and an uninstall that's `docker rm`. The native installer is a legitimate +# alternative (see README.md); it is not the default here. +# --------------------------------------------------------------------------- +install_docker() { + if command -v docker >/dev/null 2>&1; then + log "Docker already installed — skipping" + return + fi + log "Installing Docker" + apt-get update + apt-get install -y ca-certificates curl gnupg + install -m 0755 -d /etc/apt/keyrings + curl -fsSL https://download.docker.com/linux/debian/gpg \ + | gpg --dearmor -o /etc/apt/keyrings/docker.gpg + chmod a+r /etc/apt/keyrings/docker.gpg + echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \ +https://download.docker.com/linux/debian $(. /etc/os-release && echo "$VERSION_CODENAME") stable" \ + > /etc/apt/sources.list.d/docker.list + apt-get update + apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin +} + +# --------------------------------------------------------------------------- +# NVIDIA Container Toolkit — only what lets Docker see the GPU. It does NOT install +# the driver: a working `nvidia-smi` is a prerequisite this script checks for rather +# than tries to satisfy, because driver installation is the single most +# hardware/kernel-specific step on this host and silently picking a driver version +# for someone is a good way to produce an unbootable machine. +# --------------------------------------------------------------------------- +install_nvidia_toolkit() { + if ! nvidia-smi >/dev/null 2>&1; then + die "TIER=gpu but nvidia-smi doesn't work. Install the NVIDIA driver first + (Debian: enable non-free-firmware, then 'apt install nvidia-driver firmware-misc-nonfree', + reboot, confirm 'nvidia-smi' prints your card), or set TIER=cpu to run without a GPU." + fi + if command -v nvidia-ctk >/dev/null 2>&1; then + log "NVIDIA Container Toolkit already installed — skipping" + else + log "Installing NVIDIA Container Toolkit" + curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \ + | gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg + curl -fsSL https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list \ + | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' \ + > /etc/apt/sources.list.d/nvidia-container-toolkit.list + apt-get update + apt-get install -y nvidia-container-toolkit + fi + nvidia-ctk runtime configure --runtime=docker + systemctl restart docker +} + +# --------------------------------------------------------------------------- +# Compose file +# --------------------------------------------------------------------------- +write_compose() { + local tier="$1" + local gpu_block="" + + if [[ "$tier" == "gpu" ]]; then + gpu_block=" + deploy: + resources: + reservations: + devices: + - driver: nvidia + count: all + capabilities: [gpu]" + fi + + mkdir -p "$BASE_DIR/models" + + cat > "$BASE_DIR/docker-compose.yml" </dev/null 2>&1; then + return 0 + fi + sleep 2 + done + die "Ollama didn't come up within 2 minutes. Check: docker logs ollama" +} + +# --------------------------------------------------------------------------- +# Main +# --------------------------------------------------------------------------- +TIER_RESOLVED="$(detect_tier)" +log "Tier: ${TIER_RESOLVED}$([[ "$TIER" == "auto" ]] && echo " (auto-detected)")" + +if [[ "$TIER_RESOLVED" == "cpu" ]]; then + warn "Running CPU-only. Expect single-digit tokens/sec — usable for validating the + whole pipeline end to end, slow enough to be annoying for live Assist. This is + docs/components.md's deliberate 'skip the GPU for now' fallback, not a failure." +fi + +install_docker +[[ "$TIER_RESOLVED" == "gpu" ]] && install_nvidia_toolkit + +write_compose "$TIER_RESOLVED" + +log "Starting Ollama" +docker compose -f "$BASE_DIR/docker-compose.yml" up -d +wait_for_ollama + +if [[ "$TIER_RESOLVED" == "gpu" ]]; then + TEXT_MODEL="$GPU_TEXT_MODEL" +else + TEXT_MODEL="$CPU_TEXT_MODEL" +fi + +# A failed pull is deliberately not fatal — the server is up and useful, and the +# likeliest cause is a renamed upstream tag, which a human fixes in seconds and a +# script cannot guess at. +pull_model "$TEXT_MODEL" || true +if [[ "$PULL_VISION_MODEL" == "true" ]]; then + pull_model "$VISION_MODEL" || true +fi + +HOST_IP="$(hostname -I | awk '{print $1}')" + +log "Installed models:" +docker exec ollama ollama list || warn "Could not list models" + +cat < Devices & Services -> Add Integration + -> Ollama, URL http://${HOST_IP}:${OLLAMA_PORT}, model ${TEXT_MODEL}. + Then Settings -> Voice assistants -> your Assist pipeline -> Conversation agent. + + 2. On the CONTAINER host, in each service's env file: + digest-engine.env : OLLAMA_HOST=http://${HOST_IP}:${OLLAMA_PORT} + OLLAMA_MODEL=${TEXT_MODEL} + pantry-vision.env : OLLAMA_HOST=http://${HOST_IP}:${OLLAMA_PORT} + OLLAMA_VISION_MODEL=${VISION_MODEL} + chores.env : OLLAMA_HOST=http://${HOST_IP}:${OLLAMA_PORT} + OLLAMA_VISION_MODEL=${VISION_MODEL} + OLLAMA_TEXT_MODEL=${TEXT_MODEL} (optional, reminder phrasing) + + NOTE the full scheme+port form: those services build URLs by string + concatenation, so a bare IP will not work. + + 3. THEN DO THE THING THIS HOST EXISTS TO SURVIVE — power it off and confirm the + reactive baseline still works: + - presence -> light on/off still fires (Phase 2 automations, no LLM in the loop) + - the door panel still shows weather/who's-home + - chores still nudges (plain template instead of LLM-phrased wording) + - digest-engine skips its run rather than erroring the timer + docs/project-plan.md's testing checklist calls this out explicitly. If anything + above BREAKS rather than degrading, that's a bug in the consumer, not here. + + 4. SECURITY: Ollama has no authentication of any kind, and its API can pull and + DELETE models, not just generate. Anyone who can reach :${OLLAMA_PORT} can do all + of that. Keep this host on the smart-home VLAN, never port-forwarded — see + docs/network-integration.md. + +EOF