diff --git a/README.md b/README.md index 1c2096b..2e364b2 100644 --- a/README.md +++ b/README.md @@ -57,6 +57,16 @@ pantry-vision/ Kitchen-display backend: a photo held up to the cam write into Grocy stock; also proxies Grocy's inventory (soonest-expiring first) and recipes to the kiosk frontend (write API + frontend/ static serving) +trash-calendar/ Reads Kennelbach's personal trash-collection ICS feed, + writes matching events onto the shared CalDAV calendar + (daily systemd timer, read-only against the feed) +transit/ Public transit: GET /departures (static GTFS lookup) + and GET /plan (OpenTripPlanner proxy, voice-usable via + HA Assist) — weekly GTFS refresh via systemd timer +chores/ Presence/calendar-driven household chore nudging + a + passive fairness tally + camera-verified trash-bin/ + dishes/litter checks (systemd-timed oneshot, no + long-lived service, no LLM-picked assignment) ``` ## Status @@ -65,8 +75,8 @@ pantry-vision/ Kitchen-display backend: a photo held up to the cam - [x] Container host setup script v1 (HA, Mosquitto, Zigbee2MQTT USB, Frigate, Grocy) - [x] Node-RED + monitoring (Netdata) + dashboard (Homepage) + ntfy + Portainer added to compose stack - [x] Backup (restic) setup — scripted, off by default until a backup target is picked (`ENABLE_BACKUPS`) -- [ ] Bermuda / ESPHome BLE proxy configs -- [ ] RuView node configs +- [ ] Bermuda / ESPHome BLE proxy configs — `firmware/esphome-ble-proxy/` built (stock ESPHome `bluetooth_proxy` component), not yet flashed to real hardware, see that directory's README +- [ ] RuView node configs — `firmware/ruview/` documents the real upstream project ([github.com/ruvnet/ruview](https://github.com/ruvnet/ruview), integrated not forked) + a per-room provisioning wrapper + `automations.yaml.example` (sleep → dim lights, possible-distress → whole-household alert, concurrent elevated heart rate → colored lighting, bathroom occupancy → an external door indicator). **Every automation's entity_id is an unconfirmed placeholder**, and the concurrent-two-person-heart-rate rule rests on an unconfirmed assumption about RuView's multi-target vital-sign capability — see `firmware/ruview/README.md` §5–6 and `docs/project-plan.md` open decisions #32–33 before relying on any of it - [ ] Frigate peephole camera config (real RTSP details) - [ ] Grocy kiosk (Pi + touchscreen) setup - [ ] LLM host (Ollama) setup script @@ -80,6 +90,11 @@ pantry-vision/ Kitchen-display backend: a photo held up to the cam - [ ] Sway touch panel (`hosts/touch-panel/`) — touch-driven Sway image: full Spotify GUI (Flathub), a dedicated Home Assistant Chromium kiosk window, a general web browser, an always-on touch dock for app switching, an on-screen keyboard (toggled manually, no auto-show), and `touchpanel-agent` (HA MQTT control, same LLM-mediated-through-HA security model as the thin client) — built, **no touch-panel hardware chosen and nothing booted on real metal**, see `hosts/touch-panel/README.md` - [ ] Kitchen/fridge display + `pantry-vision` (`hosts/kitchen-display/`, `pantry-vision/`) — hold a grocery item up to the camera, an Ollama vision model proposes what it is and roughly how long it keeps, a human confirms (never auto-committed) before it's written into Grocy stock; the display then shows inventory sorted by soonest-to-expire, groceries running low, and Grocy's recipes — built and wired into `setup-container-host.sh` (`ENABLE_PANTRY_VISION`, off by default), **nothing run against a real camera, vision model, or Grocy instance** — the Grocy API call shapes in particular are written from documentation only, see `pantry-vision/README.md` and `hosts/kitchen-display/README.md` - [ ] `identity` + door panel (`identity/`, `hosts/door-panel/`) — the person <-> BLE-identifier registry: "register me as ``" by voice or touchscreen, multi-phone support (multiple identifiers per person), anti-spoofing (only allowlisted IRK-resolved/fixed-tag entities are ever accepted as candidates, never a raw MAC), device-less people (a "no device" flag plus a hand-operated Home/Away toggle — the concrete case: a grandmother without a smartphone), and an anonymous "Guest" path. Backs `hosts/door-panel/`'s weather+clothing/who's-home/groceries-running-low dashboard and `hosts/kitchen-display/`'s "Show registration" screen — built and wired into `setup-container-host.sh` (`ENABLE_IDENTITY`, off by default), **nothing run against a real HA instance, real Private BLE Device entities, or a real voice pipeline** — `TRUSTED_ENTITY_PREFIXES` above all needs checking against Developer Tools -> States, see `identity/README.md` and `hosts/door-panel/README.md` +- [ ] `identity` also corroborates presence from Frigate face recognition (Phase 20, Tapo pan/tilt cameras) — an OR-ed-in second signal only, **never** a registration signal; and gained two per-person chore-system settings (`chore_exempt`, `chore_reminder_style`, set via `POST /people//chore-settings`, no frontend for it yet) consumed by `chores/`, see `identity/README.md` +- [ ] `trash-calendar` + `transit` (Phase 19, Kennelbach AT trash pickup + Vorarlberg public transit) — built and wired into `setup-container-host.sh` (`ENABLE_TRASH_CALENDAR`/`ENABLE_TRANSIT`/`ENABLE_TRIP_PLANNING`, all off by default), **nothing run against a live ICS feed, a live GTFS feed, or a real OpenTripPlanner instance** — trip planning also needs a manually-built OTP graph this repo does not build for you, see `trash-calendar/README.md` and `transit/README.md`'s "Route planning scope" +- [ ] `chores` (Phase 20) — presence/calendar-driven household chore nudging: "I don't care who does it, as long as it gets done" — nudges whoever's home, redirects to someone else if a chore goes neglected, keeps a passive fairness tally that never feeds back into who gets nudged, and camera-checks trash bins/dishes/litter via Frigate + an Ollama vision model. Built and wired into `setup-container-host.sh` (`ENABLE_CHORES`, off by default, every-2-hours systemd timer), **no Tapo camera hardware chosen and nothing run against real hardware**, see `chores/README.md` +- [ ] Music Assistant (optional, additive multi-room audio) — wired into `setup-container-host.sh` (`ENABLE_MUSIC_ASSISTANT`, off by default), **its default port is an unverified guess that collides with `PANTRY_VISION_PORT`** if both are enabled together, see `docs/project-plan.md` open decision #31 +- [ ] `docs/network-integration.md` (OPNsense VLAN segmentation + why nothing here should be port-forwarded to the WAN) — written, not run against a real OPNsense instance ## Quick start diff --git a/chores/Dockerfile b/chores/Dockerfile new file mode 100644 index 0000000..b5479f9 --- /dev/null +++ b/chores/Dockerfile @@ -0,0 +1,17 @@ +# chores — oneshot, driven by a systemd timer (every ~2h, +/-30min jitter via +# RandomizedDelaySec), same shape as digest-engine/trash-calendar/transit-sync. +FROM python:3.11-slim + +ENV PYTHONUNBUFFERED=1 \ + PYTHONDONTWRITEBYTECODE=1 + +WORKDIR /app + +COPY requirements.txt ./ +RUN pip install --no-cache-dir -r requirements.txt + +COPY check.py ./ + +RUN mkdir -p /data + +CMD ["python", "check.py"] diff --git a/chores/README.md b/chores/README.md new file mode 100644 index 0000000..fe6964b --- /dev/null +++ b/chores/README.md @@ -0,0 +1,158 @@ +# chores + +Camera-verified household task distribution, from +[Phase 20 of the project plan](../docs/project-plan.md). + +**The principle, stated once because it drives every design choice here: "I don't +care who does it, as long as it gets done."** This is not an assignment system that +picks one fair person and waits on them — it's a nudge system that keeps redirecting +to whoever is actually around until the chore is done. Fairness is tracked for +comparison, not enforced by any algorithm — see "The tally is passive" below. + +Runs every ~2 hours via a systemd timer with `RandomizedDelaySec=1800` (systemd's +own built-in jitter — the "+/-30 min in case something else is running" the phase +was specified with, not custom code). Three steps, in order — see `check.py`'s own +module docstring for the full reasoning on each: + +1. **Trash-day-eve** — reads the same personal collection-date feed + `trash-calendar` reads (`WASTE_ICS_URL`); if pickup is tomorrow, opens a "trash" + chore. +2. **Camera checks** (opt-in, needs `FRIGATE_URL` + `CAMERA_WATCHPOINTS`) — grabs a + Frigate snapshot per configured watch point (optionally moving a PTZ camera to a + preset first), asks an Ollama vision model a one-word question ("is this bin + FULL/PARTIAL/EMPTY", "is this counter DIRTY/CLEAN", "is there litter left out + here, YES/NO"), opens a chore on "needs attention," auto-closes one on "clear." +3. **Nudging** — ASAP, not on a fixed schedule: the first run after a chore opens + nudges whoever `identity` reports home right now (minus anyone `chore_exempt`, + see below). If the chore is still open `NEGLECT_THRESHOLD_HOURS` after the last + nudge (and the household calendar isn't showing a busy window), the nudge goes + to **someone different from who was last asked** — "the next person that walks + by" — rather than re-nagging the same person. `litter` chores are special-cased + to prefer whoever the camera most recently recognized nearby (a best-effort + "who left this" guess), since the point there is telling the actual person, not + just whoever's around — and `litter` also ignores `chore_exempt` entirely, see + below. Each nudge's wording is a plain template unless the target has a + `chore_reminder_style` set, see below. + +## Chore-exempt people — everyone except litter + +Set via `identity`'s `POST /people//chore-settings` (see identity/README.md, +no frontend for it yet — call the endpoint directly). A `chore_exempt` person is +dropped from the nudge rotation entirely — the "cousin visits often but doesn't owe +me chores" case. **Litter is the deliberate exception** (`_EXEMPTIONS_DONT_APPLY` +in `check.py`): an exempt person still gets told to put trash they left out into +the bin, because that isn't "doing a chore," it's cleaning up after yourself. If +everyone currently home is chore_exempt for a non-litter chore, that run just logs +and skips — the chore stays open until someone eligible is around. + +## Reminder tone is per-person and LLM-phrased, but never LLM-decided + +`identity`'s `chore_reminder_style` free-text field ("be assertive, don't let up" / +"be gentle, give me a few minutes of grace") is passed to `OLLAMA_TEXT_MODEL` (if +configured) purely to phrase the ntfy message text — see `_compose_message()`. This +is deliberately scoped narrower than the general chore-distribution LLM idea from +earlier in the project: it changes *how the reminder is worded*, never *who* gets +nudged or *when* — that decision stays presence/calendar-driven, per the module +docstring's stated principle. No `OLLAMA_TEXT_MODEL` configured, no style set for +that person, or the call fails/returns nothing for any reason: the plain +un-styled template is used verbatim, same wording as before this feature existed. + +## The tally is passive + +Every nudge is logged with who got it. **Nothing reads that log to decide who to +nudge next** — that decision is presence/calendar-driven only, per the stated +principle. The log exists purely so a household member can look at the numbers +later and judge fairness for themselves ("comparison for fairness's sake," not an +automated fairness algorithm) — `print_tally()` logs a rolling 30-day count each +run; nothing renders it anywhere yet, see "What's not built." + +## The calendar busy-check is household-wide, not per-person + +`_household_currently_busy()` looks for a currently-active event on the **one** +shared household calendar whose summary contains a configured keyword (`busy`, +`meeting`, `call`, `movie`, `sleep`, ...) and pauses nudging for that run if it +finds one. This is a real, honest limitation: it can't tell that only one person is +in that meeting and nudge someone else who's free — everyone's nudges pause +together. A real per-person calendar/availability model would need per-person +calendars, which this project doesn't have (see `docs/project-plan.md`'s open +decisions). Fails open (treats an unreachable/misconfigured calendar as "not busy") +so a broken calendar check can never be the reason chores stop getting nudged +entirely. + +## Never auto-completes from nudging + +Only a fresh camera check finding a watch point clear, or a manual close (not built +here — see "What's not built" below), ever marks a chore done. A nudge firing is +not the same as the chore being done; conflating the two would let something that +was repeatedly nudged but never actually done silently vanish. + +## Why this is a separate service, not folded into `identity` or `pantry-vision` + +It reads from `identity` (who's home, who's nearby, who was recently seen by a +camera) and reuses `trash-calendar`'s own ICS feed config and digest-engine's own +CalDAV credential names, but owns a genuinely different job — task state, +presence-driven nudging, notification escalation — that doesn't belong bolted onto +any of those services' own single responsibilities. Same reasoning as +`trash-calendar` itself being separate from `digest-engine`'s read-only calendar +ingestion. + +## What's not built + +- **No way to manually mark a chore done** — no HA button, no voice phrase, no API + endpoint. Right now the only way a chore closes is a camera re-check finding it + clear, or direct SQLite surgery. A real deployment probably wants an HA + button/voice "mark the trash as done" — deliberately left out of this pass rather + than guessed at without knowing how the household actually wants to interact with + it. +- **No per-person ntfy topics** — one shared `NTFY_TOPIC`, message text names who + it's for. Set up real per-person routing yourself if that's not enough. +- **No web UI / dashboard for chore history or the fairness tally** — + `chore_events` logs everything (created, nudged, auto_closed) and the tally is + logged each run, but nothing renders either anywhere yet. +- **No per-person calendar availability** — see "household-wide, not per-person" + above. +- **No re-check of `_compose_message()`'s LLM output** — whatever the model + returns (if anything) is sent as-is, no validation that it's actually on-topic, + on-tone, or even non-empty garbage beyond the plain empty-string fallback check. +- **No frontend for setting `chore_exempt`/`chore_reminder_style`** — set via a + direct `POST /people//chore-settings` call to `identity` until one exists. + +## Configure + +```sh +cp chores/chores.env.example /opt/smart-home/chores/chores.env +chmod 600 /opt/smart-home/chores/chores.env +$EDITOR /opt/smart-home/chores/chores.env +``` + +Works with nothing but `IDENTITY_URL`/`IDENTITY_TOKEN` and ntfy filled in — the +trash-day-eve check, calendar busy-check, and camera checks all individually no-op +when left unconfigured. + +## Manual verification still outstanding + +1. **Frigate's PTZ move-to-preset API shape** (`_frigate_snapshot()`'s + `POST /api//ptz/move/`) is assumed from Frigate's general PTZ + feature set, not confirmed against a real Tapo PTZ camera wired into Frigate — + see `docs/project-plan.md` §1.18/Phase 20's own callout. A wrong endpoint just + means the snapshot is taken from wherever the camera already was. +2. Whether the vision model's one-word FULL/PARTIAL/EMPTY/DIRTY/CLEAN/YES/NO + answers are actually reliable for a real bin/sink/hallway from a real camera + angle — completely unmeasured, same caveat as pantry-vision's own vision-model + accuracy note. +3. `_likely_culprit()`'s reliance on `identity`'s `face_seen_recently` field + assumes Frigate face-recognition presence corroboration is actually wired up + and working (`identity/README.md`'s own "Camera face recognition" section is + itself unverified) — until then, litter chores just fall back to "whoever's + around," same as every other chore type. +4. `_household_currently_busy()`'s CalDAV read has never been run against a real + Nextcloud instance from this specific code path (digest-engine's own + `ingest/caldav.py` is a separate, independently-tested read). +5. No camera hardware has been chosen (`docs/project-plan.md` §1.18) — nothing here + has been run against a real Tapo camera or Frigate PTZ integration at all. +6. `_compose_message()`'s LLM-phrased reminders have never been checked against a + real `OLLAMA_TEXT_MODEL` for whether the output actually respects a given + `chore_reminder_style` reliably, vs. just producing generically-toned text — + same "no measured accuracy" caveat as the vision checks above. A bad or + off-style result still gets sent (there's no re-check of the LLM's own output + here), just not blocked — see "What's not built" for why there's no re-check. diff --git a/chores/check.py b/chores/check.py new file mode 100644 index 0000000..83ab81a --- /dev/null +++ b/chores/check.py @@ -0,0 +1,531 @@ +"""chores — camera-verified household task distribution, from +docs/project-plan.md Phase 20. + +THE PRINCIPLE, stated once because it drives every design choice below: **"I don't +care who does it, as long as it gets done."** This is not an assignment system that +picks one fair person and waits on them — it's a nudge system that keeps redirecting +to whoever is actually around until the chore is done. Fairness is tracked for +comparison, not enforced by the algorithm — see "The tally is passive" below. + +Runs every ~2 hours (systemd timer, `RandomizedDelaySec=1800` gives the "+/-30 min +in case something else is running" jitter) and does three things, in order: + +1. **Trash-day-eve**: reads the same personal collection-date ICS feed + `trash-calendar` already reads (`WASTE_ICS_URL`, own env file, own read — this + script never writes to the calendar, `trash-calendar` owns that). If pickup is + tomorrow and no open "trash" chore exists yet, creates one. +2. **Camera checks** (optional, off unless `CAMERA_WATCHPOINTS` is configured — no + camera hardware has been chosen yet, see docs/project-plan.md §1.18): for each + configured watch point (a Frigate camera + optional PTZ preset), grabs a + snapshot via Frigate's own API and asks an Ollama vision model whether it shows + a full bin / dirty dishes. A "needs attention" result opens a chore if one isn't + already open; a "clear" result auto-closes one if it was. +3. **Nudging**: for every open chore, ASAP, not "wait for a schedule" — the first + run after a chore opens nudges whoever `identity` reports home right now (using + `identity`'s own room field to prefer someone actually near the relevant spot, + e.g. the kitchen for dishes, when that data is available). If the chore is + still open `NEGLECT_THRESHOLD_HOURS` after the last nudge, the household + calendar is checked for anyone currently marked busy (a lightweight, honest- + about-its-limits check — see `_household_currently_busy()`), and if nobody's in + a flagged busy window, the next available person — **preferring someone + different from who was last nudged**, "the next person that walks by" — gets + nudged instead. There is no fixed "assignee" who owns a chore; `last_nudged` + just remembers who to avoid re-nagging immediately and who to give tally credit + to if the chore resolves shortly after. + +`identity`'s `chore_exempt` flag (set via `POST /people//chore-settings`, see +identity/README.md) takes a person out of the nudge rotation entirely — a frequent +guest who isn't a household member doesn't owe chores. **`litter` is the one +exception** (`_EXEMPTIONS_DONT_APPLY` below): everyone, exempt or not, still gets +told to put trash they left out into the bin — that isn't "doing a chore," it's +cleaning up after yourself. + +`identity`'s `chore_reminder_style` free-text field (same endpoint) is passed to an +LLM that **phrases** the ntfy message in that person's preferred tone ("be +assertive," "be gentle, give me a few minutes of grace") — see `_compose_message()`. +This is wording only. It never touches who gets nudged or when, which stays +presence/calendar-driven per the principle above; if `OLLAMA_TEXT_MODEL` is unset, +the style is empty, or the call fails for any reason, the plain deterministic +template is used verbatim — same "never build against a guess" fallback discipline +as the vision checks above. + +## The tally is passive + +`chore_events`' `nudged` rows are the only record kept of who got asked about what. +**Nothing in the nudge logic above reads this tally to decide who to ask next** — +that decision is presence/calendar-driven only, exactly per the stated principle. +The tally exists purely so a household member can look at the numbers later and +judge fairness for themselves ("comparison for fairness's sake," not an automated +fairness algorithm). Attribution when a chore closes is a **best-effort heuristic** +(whoever was nudged most recently before it resolved) — nobody here actually +confirms who did it, see "What's not built." + +NEVER auto-completes a chore from the nudge step itself — only a fresh camera check +(finding the watch point clear) or a manual close (not built here, see README.md) +ever marks one done. A nudge firing is not the same as the chore being done; +conflating the two would let a repeatedly-nudged, never-actually-done chore quietly +disappear. +""" + +from __future__ import annotations + +import json +import logging +import os +import sqlite3 +import sys +import time +import urllib.request +from datetime import date, datetime, timedelta, timezone +from pathlib import Path + +LOG = logging.getLogger("chores") + +DB_PATH = Path(os.environ.get("CHORES_DB_PATH", "/data/chores.db")) + +IDENTITY_URL = os.environ.get("IDENTITY_URL", "").rstrip("/") +IDENTITY_TOKEN = os.environ.get("IDENTITY_TOKEN", "") + +FRIGATE_URL = os.environ.get("FRIGATE_URL", "").rstrip("/") +OLLAMA_HOST = os.environ.get("OLLAMA_HOST", "http://llm-host:11434").rstrip("/") +OLLAMA_VISION_MODEL = os.environ.get("OLLAMA_VISION_MODEL", "llava") +# Optional — phrases (never decides) reminders per-person, see _compose_message(). +# Unset means every reminder uses the plain template, same as before this existed. +OLLAMA_TEXT_MODEL = os.environ.get("OLLAMA_TEXT_MODEL", "").strip() + +NTFY_URL = os.environ.get("NTFY_URL", "").rstrip("/") +NTFY_TOPIC = os.environ.get("NTFY_TOPIC", "") + +NEGLECT_THRESHOLD_HOURS = float(os.environ.get("NEGLECT_THRESHOLD_HOURS", "4")) +WASTE_ICS_URL = os.environ.get("WASTE_ICS_URL", "").strip() + +# Household calendar busy-check (see _household_currently_busy()) — reuses +# digest-engine's own CALDAV_* credential names for the same "one Nextcloud app +# password" reasoning as trash-calendar, but this is a THIRD, independent read of +# it (never a write) — a lightweight, honestly-scoped check, not per-person +# availability, see README.md's limitation note. +CALDAV_URL = os.environ.get("CALDAV_URL", "").strip() +CALDAV_USERNAME = os.environ.get("CALDAV_USERNAME", "").strip() +CALDAV_PASSWORD = os.environ.get("CALDAV_PASSWORD", "") +CALDAV_VERIFY_TLS = os.environ.get("CALDAV_VERIFY_TLS", "true").strip().lower() == "true" +CALDAV_QUIET_KEYWORDS = [ + k.strip().lower() for k in os.environ.get("CALDAV_QUIET_KEYWORDS", "busy,meeting,call,movie,sleep").split(",") if k.strip() +] + +_CHORE_PROMPTS = { + "trash": None, # never camera-checked — trash-day-eve driven only, see module docstring + "bin_full": "Look at this photo of a trash/recycling bin. Answer with exactly one word: FULL, PARTIAL, or EMPTY.", + "dishes": "Look at this photo of a kitchen sink/counter area. Answer with exactly one word: DIRTY or CLEAN.", + "litter": ( + "Look at this photo of a household area. Is there any trash/garbage/litter left out " + "that does not belong there (not properly disposed of in a bin)? Answer with exactly " + "one word: YES or NO." + ), +} + +# Which watch points get "who was just seen here" culprit-attribution treatment +# (see _likely_culprit()) instead of the general "whoever's around" nudge — litter +# is specifically about telling whoever left it, not just whoever's nearby now. +_ATTRIBUTE_TO_RECENT_VIEWER = {"litter"} + +# Chore types where identity's chore_exempt flag does NOT apply — everyone still +# gets told to clean up litter they left out, exempt household member or not (see +# module docstring). Currently the same set as _ATTRIBUTE_TO_RECENT_VIEWER, but +# they mean different things — one is about attribution, this is about eligibility +# — so they're kept as separate names rather than reusing one for both purposes. +_EXEMPTIONS_DONT_APPLY = {"litter"} + + +def _now() -> str: + return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z") + + +def _db() -> sqlite3.Connection: + DB_PATH.parent.mkdir(parents=True, exist_ok=True) + conn = sqlite3.connect(DB_PATH) + conn.row_factory = sqlite3.Row + return conn + + +def init_db(conn: sqlite3.Connection) -> None: + conn.executescript( + """ + CREATE TABLE IF NOT EXISTS chores ( + id INTEGER PRIMARY KEY, + type TEXT NOT NULL, + status TEXT NOT NULL DEFAULT 'open', + assigned_to TEXT, + created_at TEXT NOT NULL, + assigned_at TEXT, + done_at TEXT, + reminder_count INTEGER NOT NULL DEFAULT 0, + last_reminder_at TEXT + ); + CREATE TABLE IF NOT EXISTS chore_events ( + id INTEGER PRIMARY KEY, + chore_id INTEGER, + event TEXT NOT NULL, + detail TEXT, + created_at TEXT NOT NULL + ); + """ + ) + + +def _log_event(conn, chore_id, event, detail="") -> None: + conn.execute( + "INSERT INTO chore_events (chore_id, event, detail, created_at) VALUES (?, ?, ?, ?)", + (chore_id, event, detail, _now()), + ) + + +def _open_chore(conn, chore_type: str): + return conn.execute("SELECT * FROM chores WHERE type = ? AND status = 'open'", (chore_type,)).fetchone() + + +def _create_chore(conn, chore_type: str) -> int: + cur = conn.execute( + "INSERT INTO chores (type, status, created_at) VALUES (?, 'open', ?)", (chore_type, _now()) + ) + assert cur.lastrowid is not None + _log_event(conn, cur.lastrowid, "created") + LOG.info("chores: created a new %r chore", chore_type) + return cur.lastrowid + + +def _close_chore(conn, chore_row) -> None: + conn.execute("UPDATE chores SET status = 'done', done_at = ? WHERE id = ?", (_now(), chore_row["id"])) + _log_event(conn, chore_row["id"], "auto_closed") + LOG.info("chores: %r chore #%d auto-closed (camera check came back clear)", chore_row["type"], chore_row["id"]) + + +# --- 1. Trash-day-eve ------------------------------------------------------------ +def check_trash_day(conn) -> None: + if not WASTE_ICS_URL: + return + from icalendar import Calendar as ICalendar + + try: + req = urllib.request.Request(WASTE_ICS_URL, headers={"User-Agent": "smartesthome-chores/1"}) + with urllib.request.urlopen(req, timeout=30) as resp: + raw = resp.read() + tomorrow = date.today() + timedelta(days=1) + pickup_tomorrow = False + for component in ICalendar.from_ical(raw).walk("VEVENT"): + dtstart = component.get("dtstart") + if dtstart is None: + continue + start = dtstart.dt + start_date = start.date() if isinstance(start, datetime) else start + if start_date == tomorrow: + pickup_tomorrow = True + break + except Exception: + LOG.warning("chores: could not check WASTE_ICS_URL for trash-day-eve", exc_info=True) + return + + if pickup_tomorrow and _open_chore(conn, "trash") is None: + _create_chore(conn, "trash") + + +# --- 2. Camera checks -------------------------------------------------------------- +def _watchpoints() -> list[tuple[str, str, str | None]]: + """CAMERA_WATCHPOINTS format: "type:camera[:preset],type:camera[:preset],...". + type must be a key in _CHORE_PROMPTS other than "trash" (bin_full, dishes). + """ + raw = os.environ.get("CAMERA_WATCHPOINTS", "").strip() + if not raw: + return [] + points = [] + for entry in raw.split(","): + parts = [p.strip() for p in entry.split(":")] + if len(parts) < 2 or parts[0] not in _CHORE_PROMPTS or parts[0] == "trash": + LOG.warning("chores: ignoring malformed CAMERA_WATCHPOINTS entry %r", entry) + continue + points.append((parts[0], parts[1], parts[2] if len(parts) > 2 else None)) + return points + + +def _frigate_snapshot(camera: str, preset: str | None) -> bytes | None: + # VERIFY: Frigate's PTZ-move-to-preset API shape is assumed, not confirmed + # against a real Frigate PTZ camera — see README.md. A failure here just means + # the snapshot is taken from wherever the camera already was, not a hard error. + if preset: + try: + move_req = urllib.request.Request( + f"{FRIGATE_URL}/api/{camera}/ptz/move/{preset}", method="POST" + ) + urllib.request.urlopen(move_req, timeout=10).close() + time.sleep(3) # give the camera time to physically move before snapshotting + except Exception: + LOG.warning("chores: could not move camera %r to preset %r (continuing anyway)", camera, preset, exc_info=True) + + try: + with urllib.request.urlopen(f"{FRIGATE_URL}/api/{camera}/latest.jpg", timeout=15) as resp: + return resp.read() + except Exception: + LOG.warning("chores: could not fetch snapshot for camera %r", camera, exc_info=True) + return None + + +def _ask_vision(image: bytes, prompt: str) -> str | None: + import base64 + + payload = { + "model": OLLAMA_VISION_MODEL, + "prompt": prompt, + "images": [base64.b64encode(image).decode("ascii")], + "stream": False, + "options": {"temperature": 0.1}, + } + try: + req = urllib.request.Request( + f"{OLLAMA_HOST}/api/generate", data=json.dumps(payload).encode(), headers={"Content-Type": "application/json"} + ) + with urllib.request.urlopen(req, timeout=90) as resp: + result = json.loads(resp.read()) + return (result.get("response") or "").strip().upper() + except Exception: + LOG.warning("chores: Ollama vision call failed", exc_info=True) + return None + + +def check_cameras(conn) -> None: + for chore_type, camera, preset in _watchpoints(): + if not FRIGATE_URL: + LOG.warning("chores: CAMERA_WATCHPOINTS configured but FRIGATE_URL is unset, skipping") + return + + image = _frigate_snapshot(camera, preset) + if image is None: + continue + + answer = _ask_vision(image, _CHORE_PROMPTS[chore_type]) + if answer is None: + continue + + needs_attention = ("FULL" in answer and "PARTIAL" not in answer) or "DIRTY" in answer or ( + chore_type == "litter" and "YES" in answer + ) + existing = _open_chore(conn, chore_type) + + if needs_attention and existing is None: + _create_chore(conn, chore_type) + elif not needs_attention and existing is not None: + _close_chore(conn, existing) + + +# --- 3. Nudging — presence/calendar-driven, "whoever's around," see module docstring +def _presence() -> list[dict]: + """Each dict: {"name", "home", "room", "face_seen_recently", "chore_exempt", + "chore_reminder_style"} — the raw shape identity's own /presence returns. Empty + list (never raises past this point) if identity is unreachable — nudging just + waits for the next run. + """ + if not IDENTITY_URL or not IDENTITY_TOKEN: + return [] + try: + req = urllib.request.Request(f"{IDENTITY_URL}/presence") + req.add_header("Authorization", f"Bearer {IDENTITY_TOKEN}") + with urllib.request.urlopen(req, timeout=10) as resp: + data = json.loads(resp.read()) + return [p for p in data.get("people", []) if p.get("home")] + except Exception: + LOG.warning("chores: could not reach identity for /presence", exc_info=True) + return [] + + +def _household_currently_busy() -> bool: + """A deliberately lightweight check, not per-person availability: is there a + CURRENTLY ACTIVE event on the one shared household calendar whose summary + contains one of CALDAV_QUIET_KEYWORDS? If so, nudges wait for the next run + rather than interrupting a meeting/movie/whoever's asleep. This is a real + limitation, not a bug: it can't tell that only ONE person is busy and nudge + someone else anyway — everyone's nudges pause together. See README.md. + Fails open (returns False, i.e. "not busy") on any error — a broken calendar + check must never be the reason chores stop getting nudged at all. + """ + if not (CALDAV_URL and CALDAV_USERNAME and CALDAV_PASSWORD) or not CALDAV_QUIET_KEYWORDS: + return False + try: + import caldav as caldav_lib + + now = datetime.now(timezone.utc) + with caldav_lib.DAVClient( + url=CALDAV_URL, username=CALDAV_USERNAME, password=CALDAV_PASSWORD, ssl_verify_cert=CALDAV_VERIFY_TLS + ) as client: + for calendar in client.principal().calendars(): + for item in calendar.search(start=now, end=now + timedelta(minutes=1), event=True, expand=True): + summary = str(item.icalendar_component.get("summary", "")).lower() + if any(keyword in summary for keyword in CALDAV_QUIET_KEYWORDS): + return True + except Exception: + LOG.warning("chores: calendar busy-check failed, proceeding as not-busy", exc_info=True) + return False + return False + + +def _likely_culprit(candidates: list[dict]) -> dict | None: + """For litter-type chores: prefer whoever was MOST RECENTLY seen by camera face + recognition — a best-effort "who was just here" guess, not a certainty. Falls + back to None (caller then treats it like any other chore) if nobody's + face_seen_recently. + """ + seen = [p for p in candidates if p.get("face_seen_recently")] + return seen[0] if seen else None + + +def nudge_open_chores(conn) -> None: + open_chores = conn.execute("SELECT * FROM chores WHERE status = 'open'").fetchall() + if not open_chores: + return + + home = _presence() + if not home: + LOG.info("chores: %d open chore(s), but nobody is home yet — nothing to do", len(open_chores)) + return + + if _household_currently_busy(): + LOG.info("chores: household calendar shows a busy/quiet window right now — skipping nudges this run") + return + + for chore in open_chores: + last_nudged = chore["assigned_to"] + last_nudged_at = chore["assigned_at"] + + if last_nudged_at is not None: + elapsed_hours = (datetime.now(timezone.utc) - datetime.fromisoformat(last_nudged_at.replace("Z", "+00:00"))).total_seconds() / 3600 + if elapsed_hours < NEGLECT_THRESHOLD_HOURS: + continue # not neglected yet — leave whoever was last nudged alone for now + + # chore_exempt people are out of the rotation entirely, EXCEPT litter — see + # _EXEMPTIONS_DONT_APPLY's module-level comment. + eligible = home if chore["type"] in _EXEMPTIONS_DONT_APPLY else [p for p in home if not p.get("chore_exempt")] + if not eligible: + LOG.info( + "chores: %r chore #%d open, but everyone home right now is chore_exempt — skipping", + chore["type"], chore["id"], + ) + continue + + # "The next person that walks by": prefer someone home right now who ISN'T + # who we last nudged (a real redirect, not the same person nagged again) — + # falls back to re-nudging the same person if they're genuinely the only + # one home. Litter gets a different preference first: whoever the camera + # most recently saw, since the point there is telling the actual culprit. + target = None + if chore["type"] in _ATTRIBUTE_TO_RECENT_VIEWER: + target = _likely_culprit(eligible) + if target is None: + different = [p for p in eligible if p["name"] != last_nudged] + target = (different or eligible)[0] + + name = target["name"] + conn.execute( + "UPDATE chores SET assigned_to = ?, assigned_at = ?, reminder_count = reminder_count + 1, " + "last_reminder_at = ? WHERE id = ?", + (name, _now(), _now(), chore["id"]), + ) + redirected = last_nudged is not None and last_nudged != name + _log_event(conn, chore["id"], "nudged", name) + redirect_note = f" (redirected from {last_nudged})" if redirected and last_nudged else "" + LOG.info("chores: nudged %s about %r chore #%d%s", name, chore["type"], chore["id"], redirect_note) + _notify( + name, + chore["type"], + redirected, + is_culprit=chore["type"] in _ATTRIBUTE_TO_RECENT_VIEWER, + reminder_style=target.get("chore_reminder_style"), + ) + + +def _default_message(name: str, chore_type: str, redirected: bool, is_culprit: bool) -> str: + if is_culprit: + return f"{name}, looks like something was left out — could you put it in the bin?" + if redirected: + return f"{name}, this one's still open — could you take care of: {chore_type}?" + return f"{name}, could you take care of: {chore_type}?" + + +def _compose_message(name: str, chore_type: str, redirected: bool, is_culprit: bool, reminder_style: str | None) -> str: + """Wording only — see module docstring's OLLAMA_TEXT_MODEL paragraph. Falls back + to _default_message() verbatim whenever there's no model configured, no style + set for this person, or the call fails/returns nothing — never blocks a nudge + from going out over an LLM hiccup. + """ + fallback = _default_message(name, chore_type, redirected, is_culprit) + if not OLLAMA_TEXT_MODEL or not reminder_style: + return fallback + + prompt = ( + f"Write ONE short household chore reminder (max 2 sentences) addressed to {name}. " + f"The chore is: {chore_type}. " + f"Follow {name}'s own stated preference for how they like to be reminded: \"{reminder_style}\". " + + ("Something was left out and may belong to them specifically — ask them to put it away, don't accuse them outright. " + if is_culprit else "") + + ("They were already asked about this once before and it's still not done. " if redirected else "") + + "Reply with ONLY the message text itself — no preamble, no quotation marks." + ) + try: + payload = {"model": OLLAMA_TEXT_MODEL, "prompt": prompt, "stream": False, "options": {"temperature": 0.4}} + req = urllib.request.Request( + f"{OLLAMA_HOST}/api/generate", data=json.dumps(payload).encode(), headers={"Content-Type": "application/json"} + ) + with urllib.request.urlopen(req, timeout=30) as resp: + result = json.loads(resp.read()) + text = (result.get("response") or "").strip() + return text if text else fallback + except Exception: + LOG.warning("chores: Ollama message-phrasing call failed, using plain template", exc_info=True) + return fallback + + +def _notify(name: str, chore_type: str, redirected: bool, is_culprit: bool, reminder_style: str | None = None) -> None: + if not NTFY_URL or not NTFY_TOPIC: + return + message = _compose_message(name, chore_type, redirected, is_culprit, reminder_style) + try: + req = urllib.request.Request(f"{NTFY_URL}/{NTFY_TOPIC}", data=message.encode("utf-8"), method="POST") + urllib.request.urlopen(req, timeout=10).close() + except Exception: + LOG.warning("chores: ntfy notification failed", exc_info=True) + + +def print_tally(conn) -> None: + """Read-only reporting, per the module docstring — never feeds back into + nudge_open_chores()'s choice of who to nudge next. Logged, not served + anywhere yet — see README.md's "What's not built." + """ + rows = conn.execute( + "SELECT detail AS name, COUNT(*) AS n FROM chore_events " + "WHERE event = 'nudged' AND created_at >= ? GROUP BY detail ORDER BY n DESC", + ((datetime.now(timezone.utc) - timedelta(days=30)).isoformat(),), + ).fetchall() + if rows: + LOG.info("chores: 30-day nudge tally (for comparison only, not used to pick who's next): %s", + ", ".join(f"{r['name']}: {r['n']}" for r in rows)) + + +def main() -> int: + logging.basicConfig( + level=os.environ.get("LOG_LEVEL", "INFO").upper(), + format="%(asctime)s %(levelname)-7s %(name)s: %(message)s", + ) + + conn = _db() + try: + init_db(conn) + check_trash_day(conn) + check_cameras(conn) + conn.commit() + nudge_open_chores(conn) + conn.commit() + print_tally(conn) + finally: + conn.close() + + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/chores/chores.env.example b/chores/chores.env.example new file mode 100644 index 0000000..7e1d9bf --- /dev/null +++ b/chores/chores.env.example @@ -0,0 +1,81 @@ +# chores configuration template. +# +# Copy this to the container host as (for example) +# /opt/smart-home/chores/chores.env, fill in real values, and chmod 600 it. +# Works with NOTHING filled in below except the identity/ntfy bits — the camera +# checks, calendar busy-check, and trash-day-eve check each individually no-op if +# left unconfigured. + +# --------------------------------------------------------------------------- +# identity — required for nudging (who's home, who's near which room, who was +# recently seen by a camera). Same token identity itself uses. +# --------------------------------------------------------------------------- +IDENTITY_URL=http://127.0.0.1:8097 +IDENTITY_TOKEN= + +# --------------------------------------------------------------------------- +# Trash-day-eve — same feed trash-calendar reads, this script only ever reads it +# too (never writes to the calendar). Leave blank to skip this check entirely. +# --------------------------------------------------------------------------- +WASTE_ICS_URL= + +# --------------------------------------------------------------------------- +# Household calendar busy-check (optional) — SAME CALDAV_* credentials as +# digest-engine's ingest/caldav.py and trash-calendar, reused for a third, +# independent READ (never a write here either). If set, a currently-active +# calendar event whose summary contains one of CALDAV_QUIET_KEYWORDS pauses +# nudging for that run — see README.md's honest limitation (this is household- +# wide, not per-person; it can't tell only one person is busy). +# --------------------------------------------------------------------------- +CALDAV_URL= +CALDAV_USERNAME= +CALDAV_PASSWORD= +CALDAV_VERIFY_TLS=true +CALDAV_QUIET_KEYWORDS=busy,meeting,call,movie,sleep + +# --------------------------------------------------------------------------- +# Camera checks — OFF until both FRIGATE_URL and CAMERA_WATCHPOINTS are set. No +# camera hardware has been chosen yet (docs/project-plan.md §1.18) — leave this +# blank until Tapo pan/tilt cameras are actually deployed as Frigate camera +# sources. +# +# CAMERA_WATCHPOINTS format: "type:frigate_camera_name[:ptz_preset_name],...". +# type must be "bin_full", "dishes", or "litter" ("has someone left trash out +# somewhere it doesn't belong" — gets told to whoever the camera most recently +# recognized nearby, not just whoever's home in general, see README.md). preset +# is optional — omit it if the camera doesn't need to move. Example: +# CAMERA_WATCHPOINTS=bin_full:driveway_cam:trash_preset,dishes:kitchen_cam,litter:hallway_cam +# --------------------------------------------------------------------------- +FRIGATE_URL= +CAMERA_WATCHPOINTS= + +# Same LLM host as digest-engine/pantry-vision. OLLAMA_VISION_MODEL must be a +# vision-capable model (see pantry-vision/README.md's identical caveat — plain text +# models cannot see images at all). +# +# OLLAMA_TEXT_MODEL is OPTIONAL and does something narrower than it sounds: it only +# PHRASES a reminder in a person's chore_reminder_style (set via identity's +# POST /people//chore-settings, see identity/README.md) — it never decides who +# gets nudged or when, that stays presence/calendar-driven, see check.py's module +# docstring. Leave blank (default) and every reminder just uses the plain template, +# same behavior as before this existed. +OLLAMA_HOST=http://llm-host:11434 +OLLAMA_VISION_MODEL=llava +OLLAMA_TEXT_MODEL= + +# --------------------------------------------------------------------------- +# Nudging — ntfy, already in the stack (ENABLE_NTFY in setup-container-host.sh). +# A single shared topic, not per-person — the notification text names who it's +# for, but everyone subscribed to this topic sees every nudge. Set up per-person +# topics/subscriptions yourself if that's not granular enough for your household. +# +# NEGLECT_THRESHOLD_HOURS: how long an open chore sits before the system tries +# redirecting the nudge to someone else who's around ("the next person that walks +# by") instead of re-nagging whoever was last nudged. +# --------------------------------------------------------------------------- +NTFY_URL=http://127.0.0.1:8090 +NTFY_TOPIC=chores +NEGLECT_THRESHOLD_HOURS=4 + +CHORES_DB_PATH=/data/chores.db +LOG_LEVEL=INFO diff --git a/chores/requirements.txt b/chores/requirements.txt new file mode 100644 index 0000000..b96f17b --- /dev/null +++ b/chores/requirements.txt @@ -0,0 +1,6 @@ +# icalendar: parsing the shared WASTE_ICS_URL feed (this script only reads it, +# trash-calendar owns writing to the calendar). caldav: the household-busy check +# (_household_currently_busy(), also read-only) — same two packages, same +# reasoning, as digest-engine/ingest/caldav.py and trash-calendar. +icalendar>=5.0 +caldav>=2.0 diff --git a/docs/network-integration.md b/docs/network-integration.md new file mode 100644 index 0000000..3460c3a --- /dev/null +++ b/docs/network-integration.md @@ -0,0 +1,142 @@ +# Network integration — OPNsense, VLANs, and whether to port-forward anything + +**Short answer: don't port-forward any of this to the WAN. Nothing in this repo +needs to be reachable from the public internet, and forwarding it would trade a +huge amount of security for very little.** If you want to reach the household +stack while away from home, the answer is a VPN back into your own network, not +opening ports on the firewall. The rest of this document explains why, and how to +segment things internally with VLANs so a compromised IoT device (cameras +especially — see Phase 5/20's "zero WAN egress" guardrail) can't reach the rest of +your network either. + +This is guidance, not automation — **nothing under this repo touches your OPNsense +config**, same convention as this project's HA-integration catalog entries +(`docs/project-plan.md` §2): you apply this by hand in the OPNsense web UI. + +## 1. Why no port forward + +Every custom service this repo builds (`identity`, `pantry-vision`, `transit`, +`admin-canvas`, `digest-web`/`admin-web`/`pantry-web`/`identity-web`, `chores`, +`trash-calendar`) is a small stdlib-`http.server`/nginx process, bearer-token +gated where it needs to be, but **none of it was built or hardened with "reachable +from the raw internet" as a threat model** — no rate limiting, no WAF, no +DDoS/abuse handling, no security audit. Home Assistant, Grocy, Frigate, Node-RED, +Portainer, Netdata, and every other off-the-shelf piece of the stack are the same +story: capable, actively maintained software, but not written or configured here +with public exposure in mind. Port-forwarding any of them turns "a bug in one of +these" into "a bug reachable by the entire internet, scanning for it constantly." +None of them need to be reachable from the internet in the first place — every +real use case (checking the dashboard from work, registering a guest while +out, planning a trip) is solved just as well by a VPN. + +## 2. Remote access: WireGuard, not forwarded HTTP ports + +OPNsense ships a WireGuard implementation (Instances/Peers under VPN → WireGuard, +built in since 22.1 — no separate package needed on current OPNsense). Set up one +instance on the firewall and a peer per device you want remote access from (your +phone, a laptop). The only thing that ever needs forwarding is **one UDP port** +for WireGuard itself (commonly 51820, but pick anything free) — and that's a +categorically different exposure than forwarding this stack's HTTP services +directly: WireGuard silently drops any packet that isn't from an already-paired, +cryptographically-authenticated peer, so an internet-wide scanner sees nothing to +attack at all, versus a live HTTP endpoint answering every request that reaches it. +Once connected, your phone/laptop is (virtually) on your LAN/VLAN and reaches +`identity`/`pantry-vision`/HA/etc. at their normal LAN IPs — no per-service +forwarding, no bearer tokens exposed to the WAN, nothing else to configure network- +side per new service this repo adds later. + +If you don't need remote access at all, skip this section entirely — every service +in this stack works purely on the LAN with zero WAN configuration. + +## 3. VLAN segmentation — not a DMZ, a blast-radius boundary + +A traditional DMZ exists to host something the WAN needs to reach. Nothing here +needs that (§1), so "do I need a DMZ" isn't really the right question — the useful +question is **whether an IoT device on this network should be able to reach your +laptop, NAS, or anything else you actually care about if it's ever compromised.** +Zigbee/RuView nodes, ESPHome BLE proxies, and especially the Tapo cameras (Phase +20 — IP cameras have a genuinely bad industry-wide security track record) are the +class of device this matters most for. The fix is a dedicated VLAN, not a DMZ. + +A reasonable split for this project's device inventory: + +| VLAN | What goes on it | Internet access | Reaches trusted LAN? | +|---|---|---|---| +| **Trusted LAN** (existing) | Your own laptops/desktops, phones (when not on the VPN) | Yes | — | +| **Smart-home VLAN** | The container host, kiosks/thin-clients/touch panels, RuView nodes, ESPHome BLE proxies, Zigbee coordinator | Yes (needed: Ollama model pulls if not fully pre-cached, container image pulls, NTP, and any digest-engine ingestion sources — mail/Telegram/Discord/news feeds/FRED/Stooq all reach out to real external APIs, see `digest-engine/README.md`) | No, by default — see below | +| **Camera VLAN** | Tapo pan/tilt cameras (Phase 20), the existing peephole cam (Phase 5) | **No — block entirely** | No | + +Rules to actually write in OPNsense (Firewall → Rules → \): + +1. **Inter-VLAN default-deny.** OPNsense's default "allow all outbound, block + inbound from other interfaces" behavior already gets you most of the way — + just don't add a blanket "Smart-home VLAN → Trusted LAN, any/any" rule. Add + narrow allow rules only for what's actually needed (e.g. if you want to browse + to Home Assistant from a trusted-LAN laptop without a VPN, that's a Trusted + LAN → Smart-home VLAN rule scoped to port 8123, not the reverse direction). +2. **Camera VLAN: block outbound to the WAN entirely**, and block Camera VLAN → + every other VLAN except the one narrow path Frigate needs (RTSP/API from the + container host's own interface, since Frigate is what actually pulls camera + streams — the camera itself never needs to *initiate* anything toward the + container host, only *receive* the stream pull, so even that path can often be + a single allow rule scoped to the container host's IP and the camera's RTSP + port). This is the same invariant this project's own testing checklist already + states for the peephole cam (`docs/project-plan.md`'s Phase 5 guardrail, "are + cameras verified to have zero WAN egress?") — it applies identically to every + Tapo camera added in Phase 20. +3. **Smart-home VLAN → Trusted LAN: deny by default**, add narrow exceptions only + if you have a real reason (e.g. Nextcloud/CalDAV living on a trusted-LAN + machine rather than inside this stack — then just that one host:port, not the + whole VLAN). +4. **WireGuard peers land wherever you configure the tunnel's allowed-IPs to + route** — typically you'd route a WireGuard peer into the Smart-home VLAN + (so your phone, once connected, can reach `identity`/`pantry-vision`/HA the + same way it would sitting on that VLAN at home), not into Trusted LAN. + +None of this is automated by anything in this repo — the container host's own +network interface/VLAN tagging, and every rule above, is configured by hand in +OPNsense, same "nothing under this repo builds the HA/network side" convention as +the rest of this project's integration catalog entries. + +## 4. Full port inventory (LAN-only — none of these get forwarded, see §1) + +Every port `setup-container-host.sh` publishes, current as of Phase 20. `ENABLE_*` +column shows which are opt-in vs. always-on with the base stack. + +| Port | Service | Always on? | Auth | +|---|---|---|---| +| 8123 | Home Assistant | Yes | HA's own login | +| 1883 | Mosquitto (MQTT) | Yes | MQTT username/password (`mosquitto/config`) | +| 8080 | zigbee2mqtt frontend | Yes | none by default — VERIFY you've set `frontend.auth` if this VLAN isn't fully trusted | +| 1880 | Node-RED | Yes | Node-RED's own login (if enabled) | +| 3000 | Homepage dashboard | `ENABLE_HOMEPAGE` (default on) | none | +| 8090 | ntfy | `ENABLE_NTFY` (default on) | ntfy's own auth (if configured) — off by default, treat the topic name as the only barrier until you set one | +| 9000 | Portainer | `ENABLE_PORTAINER` (default on) | Portainer's own login | +| 445 | Gallery SMB share | `ENABLE_GALLERY_SMB` (default off) | SMB username/password you set (`GALLERY_SMB_USERNAME`/`PASSWORD`) | +| 9925 | Mealie | `ENABLE_MEALIE` (default off) | Mealie's own login | +| 5000, 8554, 8555 | Frigate (web UI, RTSP, WebRTC) | Yes (Phase 5+) | Frigate's own login for the web UI; RTSP/WebRTC unauthenticated on the LAN by Frigate's own design — this is exactly why the camera VLAN's isolation (§3) matters, not the port itself | +| 9283 | Grocy | Yes (Phase 1/7) | Grocy's own login | +| 8091 | digest-web | `ENABLE_DIGEST_ENGINE` | none — read-only rendered output, no credentials in it by design | +| 8094 | admin-web | `ENABLE_ADMIN_CANVAS` | none — same reasoning as digest-web | +| 8095 | pantry-vision | `ENABLE_PANTRY_VISION` | bearer token (`PANTRY_VISION_TOKEN`) | +| 8096 | pantry-web | `ENABLE_PANTRY_VISION` | none | +| 8097 | identity | `ENABLE_IDENTITY` | bearer token (`IDENTITY_TOKEN`) | +| 8098 | identity-web | `ENABLE_IDENTITY` | none | +| 8099 | transit | `ENABLE_TRANSIT` | bearer token (`TRANSIT_TOKEN`) | +| 8100 | OpenTripPlanner (OTP) | `ENABLE_TRIP_PLANNING` | none — OTP has no built-in auth; this is why it's only ever called server-side by `transit`, never exposed to a kiosk browser directly | +| ~8095 (VERIFY) | Music Assistant | `ENABLE_MUSIC_ASSISTANT` | Music Assistant's own auth (if configured) — **also flagged as an assumed/unverified port that collides with `PANTRY_VISION_PORT`, see `docs/project-plan.md` open decision #31** — resolve the actual port before relying on this table for it | + +`admin-canvas` (port 8092) is deliberately **not** in this table — it has no +`ports:` mapping in the generated compose file at all, reachable only from other +containers on the compose network (i.e. Home Assistant), by design. + +## 5. What's still unverified here + +This entire document was written against `setup-container-host.sh`'s current +`ports:` mappings, not tested against a real OPNsense instance — the VLAN/firewall +rule guidance in §3 is standard practice for this class of network, not something +run against your specific hardware/interface names. Confirm your OPNsense version +actually ships WireGuard where you expect (VPN menu) before planning around it, +and treat the port table in §4 as something to re-check against a live +`docker compose ps` after deployment, not a permanent guarantee — a new +`ENABLE_*` service added to the script later needs a new row here too. diff --git a/docs/project-plan.md b/docs/project-plan.md index 8d11852..0583ca0 100644 --- a/docs/project-plan.md +++ b/docs/project-plan.md @@ -48,7 +48,8 @@ Already covered — using your existing Haozee CC2652P USB dongle. No coordinato ### 1.6 RuView presence mesh | Item | Est. Price/unit | Suggested qty | Notes | |---|---|---|---| -| ESP32-S3 dev board | €8–12 | 1 per room/zone needing CSI presence | Runs RuView firmware — separate boards from the Bermuda BLE proxies above (one chip = one firmware) | +| ESP32-S3 dev board | ~€9 | 1 per room/zone needing CSI presence | RuView's primary/confirmed target board (`firmware/ruview/README.md`) — separate boards from the Bermuda BLE proxies above (one chip = one firmware). **ESP32-C3 and the original ESP32 are explicitly unsupported** ("single-core, insufficient for CSI DSP," per RuView's own README) — this is now a confirmed hardware requirement, not just a recommendation | +| ESP32-C6 dev board (optional) | €6–10 | Research/WiFi-6 variant | Not required for basic per-room presence — see `firmware/ruview/README.md` §1 | ### 1.7 Door-spy / peephole camera | Item | Est. Price | Notes | @@ -129,6 +130,20 @@ camera-based; a registration photo is a nice-to-have via the same webcam pool as hardware — `identity` is a container on the existing Phase 1 host, same as `pantry-vision`.)* +### 1.17 TP-Link Tapo pan/tilt security cameras (Phase 20) +| Item | Est. Price (EUR) | Notes | +|---|---|---| +| TP-Link Tapo C500/C520WS (or similar pan/tilt "look around" model) | €40–70 each | Number of units and placement (trash-bin sightlines, general household areas) is your call, not this plan's — start with 1–2 and expand once Frigate integration is confirmed working | + +*(No new container-host hardware — these are IP cameras feeding into the existing +Phase 5 Frigate NVR as additional camera sources, same as the peephole cam (§1.7). +**Not RTSP-onvif-native out of the box** — Tapo cameras generally need either +"Advanced Settings -> Camera Account" enabled for a direct RTSP stream, or go2rtc +(bundled with recent Frigate) as a protocol bridge; VERIFY the exact stream URL and +whether your specific model's firmware exposes RTSP at all before buying more than +one — see `identity/README.md`'s and `chores/README.md`'s own "not verified against +real hardware" callouts for everything downstream of this.)* + --- ## 2. Software (all open source / self-hosted) @@ -142,7 +157,7 @@ hardware — `identity` is a container on the existing Phase 1 host, same as | LLM runtime | **Ollama** | Serves Qwen2.5-14B-Instruct (GPU tier) or Qwen2.5-7B/3B (CPU tier) | | Conversation agent | **HA Ollama conversation integration** | Ties LLM into Assist + AI Task | | Voice STT/TTS | **Wyoming faster-whisper** + **Piper** | Local speech pipeline | -| Presence (CSI) | **RuView** (ESP32-S3 firmware + server) | Anonymous room-level presence, feeds gating automations | +| Presence (CSI) | **RuView** ([github.com/ruvnet/ruview](https://github.com/ruvnet/ruview), Rust/ESP-IDF firmware, integrated not forked — `firmware/ruview/`) | Room-level presence via WiFi CSI, **not anonymous** — also publishes 10 inferred semantic states (sleep, vitals-adjacent, bathroom occupancy, fall risk, etc.) over MQTT auto-discovery; see `firmware/ruview/README.md` §2's privacy callout before wiring automations against anything beyond plain presence | | Presence (webcam) | Your custom daemon (dotfiles repo) | Desktop-specific, not tied to HA sensors directly | | Camera NVR / face recognition | **Frigate** (0.16+) | Native face recognition, object detection, peephole cam ingest | | Identity store | Small dict/table (SQLite or JSON) — name, face label, associated MACs, confidence | Built up conversationally via LLM tool calls (`propose_person_link`, `confirm_person`, `rename_person`); merges require confirmation, never silent | @@ -206,6 +221,37 @@ hardware — `identity` is a container on the existing Phase 1 host, same as | Door-panel OS build | **live-build** (custom config, `hosts/door-panel/live-build/`) | Reuses the thin client's build tool/convention, structurally `hosts/kitchen-display/`'s twin — see Phase 18 | | Door-panel scripted control | **door-panel-agent** (custom) | HA MQTT-discovery entity for **Show home/registration** only — identical security shape to every other host's agent | | Door-panel voice | **wyoming-satellite** + **openWakeWord** | Same components as the thin client's Phase 11.8 rooms and `hosts/kitchen-display/`'s opt-in mic, but **on by default** here — voice registration is this device's actual purpose | +| Trash-day sync | **trash-calendar** (custom Python, stdlib) | Reads a personal collection-date ICS feed (Kennelbach, AT), writes matching events onto the shared household CalDAV calendar. Read-only against the source feed, write-only (create/update, never delete anyone else's events) against Nextcloud — see `trash-calendar/README.md` | +| Public transit — schedules | **GTFS** (static feed, Vorarlberg/VAO) + `transit/sync_gtfs.py` | Parsed with stdlib `csv`/`zipfile`, no external deps; `GET /departures` answers "when's the next bus/train from X" for voice | +| Public transit — trip planning | **OpenTripPlanner (OTP)** (self-hosted, official image) | Multi-modal journey planning ("get me from A to B"), proxied via GraphQL through `transit/server.py`'s `/plan` — this repo does not build or manage the OTP graph itself, see `transit/README.md`'s "Route planning scope" (Austria-wide is a moderate commitment, global is a real infrastructure decision) | +| Public transit backend | **transit** (custom Python, stdlib `http.server`) | `GET /departures` + `GET /plan`, published (unlike admin-canvas) since HA's `rest_command` needs to reach it and the `homeassistant` container's `network_mode: host` means it can't resolve container DNS names | +| Household chore distribution | **chores** (custom Python, stdlib) | Presence/calendar-driven nudge-and-redirect system (see Phase 20) — reads `identity`'s `/presence`, the same trash-day ICS feed as `trash-calendar`, and (optionally) a CalDAV busy-check and Frigate camera checks. Runs as a systemd-timed oneshot, not a long-lived service | +| Music library/multi-room audio | **Music Assistant** (own container, official image) | Unifies Spotify Connect + any local/streaming sources behind one HA-native multi-room player abstraction. **Not an HA add-on** — this project runs HA as a plain Container install (§2, above), which has no add-on store, so Music Assistant runs as its own `music-assistant` compose service (`ENABLE_MUSIC_ASSISTANT`, `setup-container-host.sh`, `network_mode: host` for player-discovery mDNS) with HA's own Music Assistant integration pointed at it. **Optional and additive** — does not replace any host's existing per-room spotifyd/librespot/Spotify-client setup (Phase 11.6/15/16), those keep working standalone either way. VERIFY its default port (assumed 8095) against `PANTRY_VISION_PORT` before enabling both — see the setup script's own callout | + +### HA integrations catalog — nothing under this repo builds these; they're HA-side installs/configs against systems this repo doesn't touch + +| Integration | Purpose | Notes | +|---|---|---| +| **UniFi Network** (HA core) | Presence/device tracking off your UniFi access points, network health sensors | A second, corroborating presence signal alongside Bermuda/RuView/Frigate face recognition — same "OR-ed in, never authoritative for registration" principle as every other presence source in this plan | +| **CalDAV** (HA core) | Read/write bridge to Nextcloud Calendar | Already load-bearing in this plan (Phase 8, `trash-calendar`, `chores`'s busy-check) — listed here for completeness as an explicit HA-side integration too, not just this repo's own direct CalDAV clients | +| **Matter** (HA core, via the Thread/Matter add-on) | Native support for Matter-certified smart-home devices, if any get added later | Not currently required by anything in this plan's hardware list (§1) — listed as available groundwork, not a current dependency | +| **1-Wire** (HA core) | Temperature/humidity sensors on a 1-Wire bus (e.g. DS18B20), if wired in later | Same "available groundwork, not a current dependency" status as Matter above | +| **Proxmox VE** (HA core) | VM/container/node status sensors, if any of this stack ends up virtualized on a Proxmox host | Purely a monitoring integration — does not change where any container in this plan actually runs | +| **Steam** (HA core) | Friend/game-status sensors from a Steam account | Unrelated to Phase 16's Steam Link game-streaming client — this is presence/activity data, not the streaming path itself | +| **Discord** (HA core, notify platform) | Send HA notifications to a Discord channel/webhook | A second notification channel alongside `ntfy` — not a replacement, `chores`/`identity`/everything else keeps using ntfy by default | +| **HP iLO** (HA core, via `hpilo` sensor or IPMI) | Server health/power sensors, if any host in this stack is HP server hardware with iLO | Purely a monitoring integration, same class as Proxmox above | +| **GTFS** (HA core `gtfs` sensor) | A second, simpler departure-board sensor directly in HA, alongside this repo's own `transit`/`GET /departures` | Redundant with `transit` by design, not a replacement for it — HA's own sensor is single-stop/single-route per entity, `transit`'s voice path is the more general "ask about any stop" interface | + +### Self-check / hardware monitoring integrations catalog — same "nothing under this repo builds these" status + +| Integration | Purpose | Notes | +|---|---|---| +| **System Monitor** (HA core) | CPU/RAM/disk/network sensors for the machine HA itself runs on | The baseline "is the container host healthy" check, zero extra software | +| **SNMP** (HA core) | Polls SNMP-capable network gear (managed switches, the OPNsense box itself, NAS units) | Complements UniFi Network above for anything not UniFi-branded | +| **Network UPS Tools (NUT)** (HA core) | Battery/load/runtime sensors from a UPS, and a clean-shutdown trigger on low battery | Recommended if a UPS protects the container host — an unplanned power loss is a real risk to `identity`/`chores`/every other SQLite-backed service's on-disk state | +| **Glances** (HA core) | A richer alternative/companion to System Monitor — per-process detail, more sensor granularity | Optional; System Monitor alone covers the basics with no extra service to run | +| **Uptime Kuma** (self-hosted, HACS integration or its own HA `webhook`) | External uptime/latency checks for this repo's own published services (`identity`, `pantry-vision`, `transit`, `digest-web`/`admin-web`) | The one entry here this repo's *services* are actually the target of, not the network they run on — worth pointing at every bearer-token-gated published port in this plan once deployed | +| **Netdata** | Already in this plan (§2, Phase 9) — per-container/per-host real-time resource monitoring | Listed again here only to make clear it's the same "self-check" category as the rest of this table, not a separate concern | --- @@ -222,8 +268,8 @@ hardware — `identity` is a container on the existing Phase 1 host, same as 3. Confirm entities populate correctly in HA. ### Phase 2 — Presence: RuView + Bermuda, in parallel -1. Deploy RuView ESP32-S3 nodes per room (dedicated CSI firmware). -2. Separately, flash plain ESP32 boards with ESPHome (`bluetooth_proxy`), install Bermuda via HACS, configure Private BLE Device for phone IRK resolution, and/or distribute fixed-MAC BLE tags per person. +1. Deploy RuView ESP32-S3 nodes per room (dedicated CSI firmware) — `firmware/ruview/` now documents the real upstream project ([github.com/ruvnet/ruview](https://github.com/ruvnet/ruview)), a per-room provisioning wrapper (`provision-room.sh`), and a real privacy callout: RuView publishes 10 inferred semantic states (sleep, vitals-adjacent, bathroom occupancy, fall risk, etc.) over MQTT, not just anonymous occupancy — read that README's §2 before wiring automations beyond plain presence. +2. Separately, flash plain ESP32 boards with ESPHome (`bluetooth_proxy` — `firmware/esphome-ble-proxy/`, a stock ESPHome component, safe to build directly unlike RuView's own DSP firmware), install Bermuda via HACS, configure Private BLE Device for phone IRK resolution, and/or distribute fixed-MAC BLE tags per person. 3. Build plain HA automations: presence (RuView) on → light on at neutral default; off (with delay) → light off. **Validate this works with the LLM host powered off — this is your safety-net baseline.** ### Phase 3 — LLM host + conversation agent @@ -586,6 +632,101 @@ entirely a client of Phase 6's `identity` (and, for one dashboard section, Phase pieces of any single interaction in this project. See the itemized list in `hosts/door-panel/README.md`. +### Phase 19 — Trash-day calendar sync + public transit (Kennelbach, AT) + +New backend directories `trash-calendar/` and `transit/`. No new hardware, no new +host — both are container-host services with no kiosk-facing UI of their own; +`transit`'s voice path goes through HA Assist like every other voice interaction +in this plan. + +1. **Trash-day sync (`trash-calendar/`)**: a scheduled (systemd timer, daily 06:15) + oneshot that reads Kennelbach's personal collection-date ICS feed and writes + matching "Trash: " events onto the shared household CalDAV calendar — + read-only against the source feed, and write-only (create/update its own + events, never touch anyone else's) against Nextcloud. `chores/check.py`'s own + trash-day-eve check (Phase 20) reads this exact same feed independently, rather + than depending on this service's calendar writes — see "duplicated, not + shared" in `chores/README.md`. +2. **Public transit — departures + trip planning (`transit/`)**: `GET /departures` + answers "when's the next bus/train from X" from a static GTFS feed (Vorarlberg + VAO), refreshed weekly (systemd timer, Monday 04:00 — schedules are published a + season at a time, not daily). `GET /plan` answers "get me from A to B" by + proxying OpenTripPlanner's GraphQL API — **this repo does not build or manage + the OTP graph itself**, that's a manual, one-time-per-OSM/GTFS-update step; see + `transit/README.md`'s "Route planning scope" section for why "Austria, possibly + global" is a real infrastructure sizing decision (Geofabrik extract size, OTP + memory requirements), not a config flag. Both endpoints are voice-usable + through HA Assist, same "custom sentence → intent script → this service's API" + shape as `identity`'s voice registration. +3. **Slow walking speed assumed throughout** — `WALK_SPEED_MPS=0.9` (about half of + OTP's ~1.4 m/s default), configurable, applied to every `/plan` call. +4. Nothing here has been run against a live GTFS feed or a real OTP instance — + both `trash-calendar`'s ICS parsing and `transit`'s GTFS/OTP integration are + written from documented formats/APIs, not a verified live source; see each + service's own README for the exact assumptions flagged. + +### Phase 20 — Tapo pan/tilt cameras (presence + trash-bin/litter checks) + household chore distribution + +New hardware: §1.17. New backend directory `chores/`. No new host — this phase is +entirely container-host services plus additional Frigate camera sources. + +1. **Tapo cameras feed into Frigate as additional camera sources** (§1.17), same + integration point as the existing peephole cam (Phase 5) — this repo does not + add a separate camera-control layer, Frigate (and, for pan/tilt aiming, its PTZ + preset API) is the one interface everything downstream talks to. +2. **Presence integration is `identity`'s job, not a new one** — Phase 6's + `identity` already gained Frigate face-recognition as a second, corroborating + presence signal (never a registration signal, see `identity/README.md`'s + "Camera face recognition" section) specifically so that Tapo cameras plug into + the exact same anti-spoofing-respecting presence pipeline as everything else, + rather than this phase inventing its own. +3. **Trash-bin fullness / dishes / litter checks are `chores/`'s job**: every ~2 + hours (systemd timer, `RandomizedDelaySec=1800` — the "+/-30 min in case + something else is running" jitter, systemd's own built-in feature, not custom + code), `chores/check.py` grabs a Frigate snapshot per configured watch point + (optionally moving a PTZ camera to a preset first) and asks an Ollama vision + model a one-word question. A "needs attention" result opens a chore; a "clear" + result auto-closes one. +4. **Household chore distribution — presence/calendar-driven nudging, not LLM + assignment.** The governing principle, stated in `chores/check.py`'s own module + docstring: *"I don't care who does it, as long as it gets done."* This is a + nudge-and-redirect system, not a fair-assignment algorithm — the first run + after a chore opens nudges whoever `identity` reports home right now; if it's + still open after `NEGLECT_THRESHOLD_HOURS` (and the household calendar isn't + showing a busy window, via a household-wide — not per-person — CalDAV + busy-check that fails open), the nudge redirects to someone different who's + available, "the next person that walks by." A rolling 30-day tally of who got + nudged about what is kept and logged **purely for fairness comparison** — it + never feeds back into who gets nudged next, a hard rule carried over from an + explicit correction during this phase's design (an earlier LLM-picks-a-fair- + assignee design was scrapped in favor of this one). +5. **Anyone leaving trash out gets told to bin it — this is the one case that + ignores chore-exemption status.** `identity`'s `chore_exempt` flag (§ below) + takes a household member out of the general nudge rotation (e.g. a frequent + guest who isn't a household member and doesn't owe chores), but litter is + special-cased in both directions: it's attributed to whoever the camera most + recently recognized nearby (a best-effort "who left this" guess, not a + certainty) rather than just whoever's home in general, and chore-exempt status + is deliberately ignored for it — putting away trash you personally left out + isn't "doing a chore," it's cleaning up after yourself. +6. **Per-person chore settings live in `identity`, not `chores`** — `chore_exempt` + (bool) and `chore_reminder_style` (free text, e.g. "be assertive" / "be gentle, + give me a few minutes of grace") are set via `identity`'s + `POST /people//chore-settings` and read by `chores` off `GET /presence`, + the same "identity owns who someone is" principle as the profile-photo and + floor-plan groundwork already living there. `chore_reminder_style` is passed to + an optional LLM call that **phrases** the reminder message in that tone — it + never decides who or when, only how the words come out, keeping the same + deterministic-fallback discipline as every other LLM-in-the-loop feature in + this project (an un-styled plain template if no model is configured, the style + is empty, or the call fails). +7. Nothing here has been run against real hardware or a real Frigate PTZ + integration — no Tapo camera model has been chosen, the PTZ move-to-preset API + shape is assumed from Frigate's general feature set, and vision-model accuracy + for "is this bin full" / "is this counter dirty" / "is there litter out" is + completely unmeasured. See `chores/README.md`'s "Manual verification still + outstanding" for the full list. + ### Testing checklist before calling any phase "done" - Does the reactive path (presence → light on) work with the LLM host powered off? (It must.) - Does a bad/slow LLM response ever block a light switch? (It must not.) @@ -629,10 +770,17 @@ entirely a client of Phase 6's `identity` (and, for one dashboard section, Phase - Does `/presence` ever report a device-less person (no identifiers, no manual override set) as `home: false`? (It must report `null`/unknown — defaulting to "away" would be actively wrong the moment they're actually home, not just imprecise.) - Can the LLM reach the door panel through any path other than HA service call → MQTT → `door-panel-agent`, for *which screen is showing*? (It must not — registration and presence/weather/groceries reads are separate, intentionally-published paths through `identity`/`pantry-vision` themselves, not a violation of this rule.) - Does the door panel's voice registration path ever bypass HA's Assist pipeline (i.e. the kiosk device talking to `identity` on its own initiative from a wake word, with no HA intent script in between)? (It must not — voice is HA Assist → a custom intent script → `identity`'s API, same "HA mediates" shape as every other voice/tool-call path in this project.) +- Does `trash-calendar` ever delete or modify an event on the shared calendar that it didn't itself create? (It must not — write-only in the sense of create/update its own events, never touch anyone else's.) +- Does `transit`'s `/plan` ever silently fall back to guessed directions if `OTP_URL` is unset or OTP is unreachable? (It must not — it returns an explicit "not configured"/error, never a fabricated route.) +- Does Frigate face recognition (the Tapo-camera presence signal) ever create or claim a new `identity` person on its own? (It must not — it is only ever an OR-ed-in corroborating signal for an already-registered person, same hard rule as every other camera-adjacent presence source in this plan.) +- Does `chores`' nudge logic ever read its own fairness tally to decide who gets nudged next? (It must not — the tally is passive reporting only, per this phase's explicit design correction.) +- Does a `chore_exempt` person ever get nudged about a non-litter chore? (It must not.) Does a `chore_exempt` person ever get skipped for a litter chore? (It must not — litter ignores exemption status in both directions.) +- Does `chores`' optional LLM message-phrasing (`chore_reminder_style`) ever change *who* gets nudged or *when*, rather than only the wording of the notification? (It must not — and a failed/empty LLM call must fall back to the plain template, never block the nudge from going out.) +- Does `chores`' household calendar busy-check, if unreachable or misconfigured, ever become the reason nudges stop going out entirely? (It must not — it fails open, treating an error as "not busy.") --- -## 4. Open decisions (Phases 6, 11–18) +## 4. Open decisions (Phases 6, 11–20) These need a decision before their respective implementation steps can be built — everything above is written to accommodate any answer, but nothing should be built against an unresolved item. @@ -658,3 +806,14 @@ These need a decision before their respective implementation steps can be built 20. **`identity`'s `TRUSTED_ENTITY_PREFIXES` default is a guess, and it's the single highest-risk unknown in Phase 6** (new) — the whole anti-spoofing design rests on this allowlist actually matching real Private BLE Device / fixed-tag entity IDs; until it's checked against Developer Tools -> States on a real HA instance, registration will most likely just report "no candidate" for everything. Same open dependency as §1.5's original Bermuda/Private BLE Device setup, which itself has never been built (see the top-level README status checklist). 21. **Identity's HA-side voice wiring (custom sentence + intent script + `rest_command`) is written from HA's documented shape, not tested** (new, Phase 6) — `identity/README.md` has the worked example; nothing under this repo builds or verifies it, same convention as admin-canvas's/digest-engine's own HA-side integration points. 22. **The floor-plan UI itself doesn't exist** (new, Phase 6) — `identity`'s `/presence` reports a best-effort `room` per person as groundwork, but there is no floor-plan image, room↔coordinate mapping, or rendering anywhere in this repo, and `AREA_ATTRIBUTE`'s default is an unconfirmed guess at what Bermuda actually publishes. Needs a real floor plan and room list before there's anything to design a coordinate format against — deliberately deferred rather than built against a guess. +23. **No Tapo camera model or count has been chosen, and Frigate's PTZ move-to-preset API shape is assumed** (new, Phase 20) — §1.17 lists a placeholder model/price only; whether a specific Tapo model even exposes RTSP without go2rtc as a bridge is unverified, and `chores/check.py`'s `_frigate_snapshot()`'s `POST /api//ptz/move/` is assumed from Frigate's general PTZ feature set, not a real deployment. This is the single highest-risk unknown in Phase 20, same class of risk as open decision #18's vision-model pick for Phase 17. +24. **Vision-model accuracy for bin-fullness/dishes/litter checks is completely unmeasured** (new, Phase 20) — same caveat as open decision #18, applied to a different prompt; a wrong FULL/DIRTY/YES answer just means a chore opens or stays open incorrectly, never a hard failure, but nobody has checked how often that actually happens. +25. **`chores`' household calendar busy-check is household-wide, not per-person** (new, Phase 20) — `_household_currently_busy()` can't tell that only one person is in a flagged-busy calendar event and nudge someone else who's free; everyone's nudges pause together. A real per-person availability model would need per-person calendars, which this project doesn't have. Documented as a known limitation, not a bug, in `chores/README.md`. +26. **No frontend exists yet for setting `chore_exempt`/`chore_reminder_style`** (new, Phase 20) — set via a direct `POST /people//chore-settings` call to `identity` (HA script/automation, or `curl`) until a UI is built into `register.html`/`dashboard.html`. +27. **No way to manually mark a chore done exists yet** (new, Phase 20) — the only way a `chores` chore currently closes is a camera re-check finding it clear, or direct SQLite surgery; a real deployment probably wants an HA button or voice phrase for "mark the trash as done," deliberately left out of this pass rather than guessed at. +28. **Which UniFi/CalDAV/Matter/1-Wire/Proxmox/Steam/Discord/HP-iLO/GTFS HA integrations actually get installed is unresolved** (new) — all nine are catalogued in §2's "HA integrations catalog" as available options with their purpose/notes, but none has been installed, configured, or verified against real hardware/accounts; several (Matter, 1-Wire, Proxmox, HP iLO) also depend on hardware/infrastructure decisions this plan hasn't made yet (whether anything in the household actually uses those platforms at all). +29. **Music Assistant has not been installed or configured** (new) — catalogued in §2 as an optional, additive HA add-on; whether it's worth adding on top of the existing per-room spotifyd/librespot/Spotify-client setup (which keeps working standalone regardless) is a real usage-pattern question, not answerable until the existing per-room setups (Phase 11.6/15/16) are actually running. +30. **The self-check/hardware-monitoring integrations catalog (§2) is a menu, not a deployment plan** (new) — System Monitor, SNMP, NUT, Glances, and Uptime Kuma are all listed with their purpose, but which ones are actually worth installing depends on hardware decisions not yet made (is there a UPS? managed switches? which of this repo's published services matter enough to alert on?). +31. **Music Assistant's default port is a guess, and it collides with `PANTRY_VISION_PORT` in this exact stack** (new) — assumed 8095 from Music Assistant's own docs, not confirmed against a running instance; `PANTRY_VISION_PORT` is also 8095. Because Music Assistant runs with `network_mode: host` (needed for player-discovery mDNS), Docker Compose's own port-collision checking doesn't catch this the way a normal `ports:` mapping would — `setup-container-host.sh` warns if both `ENABLE_MUSIC_ASSISTANT` and `ENABLE_PANTRY_VISION` are set, but resolving the actual clash (changing Music Assistant's configured listen port) is a manual step, not automated. +32. ~~RuView's semantic-state MQTT entities have no opt-out or visibility restriction beyond this network's normal trust boundary~~ — **household decision made**: real automations are now built on this data (sleep → dim lights, possible-distress → whole-household alert, concurrent elevated heart rate → colored lighting, bathroom occupancy → an external door indicator — see `firmware/ruview/README.md` §5 and `firmware/ruview/automations.yaml.example`). **Still genuinely open**: there is no technical opt-out for a specific person/room and no access restriction on these MQTT topics beyond this network's normal trust boundary — worth revisiting if anyone not on board with being sensed this way ever stays over. Every automation's `entity_id` is also still an unconfirmed placeholder (see #33), and rule 3 (concurrent two-person heart rate) rests on an unconfirmed assumption that a single RuView node can report two people's heart rates at once — multi-target vital-sign separation from WiFi CSI is a genuinely hard, unconfirmed capability, not something to trust until checked against real entities. +33. **RuView's build/flash commands and `provision.py`'s exact flags beyond `--port`/`--ssid`/`--password`/`--mqtt` are transcribed from its README, not independently run** (new, Phase 2) — see `firmware/ruview/README.md`'s own "Manual verification still outstanding," same category of risk as every other "written from documentation, not a live instance" open decision in this list (#19, #21). diff --git a/firmware/esphome-ble-proxy/README.md b/firmware/esphome-ble-proxy/README.md new file mode 100644 index 0000000..40f156a --- /dev/null +++ b/firmware/esphome-ble-proxy/README.md @@ -0,0 +1,70 @@ +# ESPHome BLE proxy — Bermuda's radios (Phase 2) + +Plain ESPHome config wrapping ESPHome's own first-party `bluetooth_proxy` +component — this is the hardware Bermuda (HACS) and HA's Private BLE Device use to +actually see BLE advertisements and resolve them into room-level presence. It is +**not** RuView (`firmware/ruview/`) — separate hardware, separate job, see that +directory's own README for why CSI presence sensing is a fundamentally different +(and much higher-risk-to-fabricate) kind of firmware than this one. + +## Why this one is safe to just write, unlike RuView + +`bluetooth_proxy` is a stock, actively-maintained ESPHome component with a +documented, stable config schema — there is no custom signal-processing, no +hardware-specific register-level bring-up, no invented protocol here. The only +real per-unit decision is `room` (`ble-proxy.yaml`'s substitutions). This is the +same category of "safe to build directly" as this project's other plain ESPHome +usage (the BLE proxy pattern is explicitly the "separate hardware from RuView" +half of `docs/project-plan.md` §1.5), unlike RuView's CSI DSP work. + +## Flash it + +1. Install [ESPHome](https://esphome.io/) (CLI or the dashboard add-on/container — + this repo doesn't bundle a specific install method, use whichever you already + use for other ESPHome devices). +2. `cp secrets.yaml.example secrets.yaml` in this directory, fill in your Wi-Fi + credentials and generate an `api_encryption_key` (command in the file). +3. Edit `ble-proxy.yaml`'s `substitutions.room`/`friendly_name` for this specific + unit — every proxy needs a distinct name so HA (and Bermuda's triangulation, + which deliberately uses ALL proxies' RSSI readings together, not just the + nearest one) can tell them apart. +4. `esphome run ble-proxy.yaml` over USB for the first flash; OTA (the `ota:` + block) works for every flash after that. +5. Repeat steps 3–4 per room needing BLE presence coverage — §1.5 suggests one + per room, same density as RuView's own per-room CSI nodes. + +## What you still have to do in Home Assistant — this repo doesn't build the HA side + +Same convention as every other "nothing under this repo builds the HA side" entry +in `docs/project-plan.md` §2 (UniFi, CalDAV, Matter, ...): + +1. **Bermuda** (HACS integration) — install it, it auto-discovers ESPHome + Bluetooth proxies on the network via HA's native `esphome` integration once + this firmware is flashed and online; no manual proxy registration needed + beyond that. +2. **Private BLE Device** (HA core integration) — set up per phone, using each + phone's actual IRK (iOS: requires pairing the phone as a bookmark in the + Apple/Google "Find My"-style private-address rotation scheme via HA's own + guided flow; Android: similar, see HA's own Private BLE Device docs). This is + what turns a rotating randomized MAC into a stable `person.*`-trackable + device — without it, Bermuda sees BLE adverts but can't attribute them to a + specific phone reliably. +3. Once both are set up, `identity`'s `TRUSTED_ENTITY_PREFIXES` env var (see + `identity/README.md`) needs to match whatever entity ID pattern Private BLE + Device / your fixed-MAC BLE tags actually create — **this is flagged as the + single highest-risk unverified assumption in Phase 6** (`docs/project-plan.md` + open decision #20) precisely because steps 1–2 above have never been run + against a real HA instance from within this project. + +## Manual verification still outstanding + +1. `esp32dev` as the board id is a generic fallback for "D1 Mini32 or similar" + (§1.5) — if your actual board has its own more specific ESPHome board id, use + that instead; a wrong-but-pin-compatible generic id usually still works, but + isn't guaranteed to for every board variant. +2. `esp-idf` vs. Arduino framework for `bluetooth_proxy`/`esp32_ble_tracker` — see + `ble-proxy.yaml`'s own VERIFY comment; check your ESPHome version's release + notes/docs if you'd rather use Arduino. +3. Never flashed to real hardware or run against a real Bermuda install — this + config passes ESPHome's own config validator (`esphome config ble-proxy.yaml`) + but nothing further. diff --git a/firmware/esphome-ble-proxy/ble-proxy.yaml b/firmware/esphome-ble-proxy/ble-proxy.yaml new file mode 100644 index 0000000..55ce5da --- /dev/null +++ b/firmware/esphome-ble-proxy/ble-proxy.yaml @@ -0,0 +1,77 @@ +# Plain ESPHome Bluetooth proxy — feeds Bermuda + HA's Private BLE Device (Phase 2, +# docs/project-plan.md §1.5). One of these per room, separate hardware from the +# RuView CSI-presence nodes (firmware/ruview/) — one radio chip, one job each, +# per §1.5's own "separate hardware from RuView nodes" note. +# +# This is a thin ESPHome config around a stock, first-party ESPHome component +# (`bluetooth_proxy`) — unlike firmware/ruview/, there is no custom signal- +# processing or hardware-specific bring-up here to get wrong; the only real +# per-unit decision is `room` below. See README.md before flashing. + +substitutions: + # REQUIRED per physical unit — every proxy needs a distinct hostname/name so HA + # can tell them apart; Bermuda then uses ALL of them together (RSSI-triangulates + # across whichever proxies see a given device) rather than picking just one. + room: "living-room" # lowercase, hyphens only — becomes part of the hostname + friendly_name: "BLE proxy (Living room)" + +esphome: + name: ble-proxy-${room} + friendly_name: ${friendly_name} + +# esp-idf (not Arduino) for the BLE proxy — VERIFY: current ESPHome versions have +# closed most of the gap between the two frameworks for esp32_ble_tracker/ +# bluetooth_proxy, but esp-idf remains the framework ESPHome's own docs describe +# BLE proxy support against; check your ESPHome version's release notes if you'd +# rather use Arduino for a smaller/simpler build. +esp32: + board: esp32dev # generic ESP32 dev board pinout — matches a Wemos D1 Mini32 or + # similar per docs/project-plan.md §1.5's "D1 Mini32 or similar" + # hardware pick. If your specific board has a distinct ESPHome + # board id (check https://esphome.io/components/esp32.html), + # use that instead — esp32dev is the safe generic fallback. + framework: + type: esp-idf + +wifi: + ssid: !secret wifi_ssid + password: !secret wifi_password + ap: + ssid: "${friendly_name} Fallback" + password: !secret ap_password + +captive_portal: + +logger: + level: WARN # esp32_ble_tracker is chatty at INFO — see README.md if debugging + +api: + encryption: + key: !secret api_encryption_key + +ota: + - platform: esphome + password: !secret ota_password + +# --- The actual point of this device ----------------------------------------- +esp32_ble_tracker: + scan_parameters: + interval: 1100ms + window: 1100ms + active: true + +bluetooth_proxy: + active: true # active-mode (not just passive) — required for Private BLE Device + # to resolve rotating IRK-based MACs, not just report raw adverts + +# Nice-to-haves that cost nothing extra and make "is this proxy alive" a one-glance +# check from HA, same reasoning as thinclient-agent's own MQTT sensors elsewhere +# in this project — not load-bearing for presence itself. +sensor: + - platform: wifi_signal + name: "${friendly_name} WiFi signal" + update_interval: 60s + +button: + - platform: restart + name: "${friendly_name} restart" diff --git a/firmware/esphome-ble-proxy/secrets.yaml.example b/firmware/esphome-ble-proxy/secrets.yaml.example new file mode 100644 index 0000000..8b5a085 --- /dev/null +++ b/firmware/esphome-ble-proxy/secrets.yaml.example @@ -0,0 +1,24 @@ +# ble-proxy.yaml configuration template. +# +# Copy this to firmware/esphome-ble-proxy/secrets.yaml (same directory as +# ble-proxy.yaml — that's where ESPHome's !secret lookup expects it) and fill in +# real values. Same never-commit handling as digest-engine.env / voice-display's +# own secrets.yaml elsewhere in this repo: the real secrets.yaml is git-ignored, +# only this .example is tracked. +# +# ONE SECRETS.YAML PER PHYSICAL UNIT is fine to skip here, UNLIKE the voice +# display's per-room media_player_entity_id — every BLE proxy's secrets are +# identical (same Wi-Fi, same API key can even be shared safely since these +# devices report nothing person-identifying themselves, just raw BLE adverts for +# Bermuda to resolve). Only `room` in ble-proxy.yaml's substitutions needs to +# change per unit; a single secrets.yaml can be reused for every proxy you flash. + +wifi_ssid: "your-wifi-ssid" +wifi_password: "your-wifi-password" +ap_password: "changeme-fallback-ap-password" + +# Generate with the ESPHome dashboard/CLI, or by hand: +# python3 -c "import base64, os; print(base64.b64encode(os.urandom(32)).decode())" +api_encryption_key: "" + +ota_password: "changeme-ota-password" diff --git a/firmware/ruview/README.md b/firmware/ruview/README.md new file mode 100644 index 0000000..cf58a63 --- /dev/null +++ b/firmware/ruview/README.md @@ -0,0 +1,179 @@ +# RuView — WiFi CSI spatial-intelligence nodes (Phase 2) + +Real upstream project: **[github.com/ruvnet/ruview](https://github.com/ruvnet/ruview)** +— a WiFi Channel-State-Information (CSI) platform that detects presence, tracks +movement/pose, and infers semantic room states (see §2 below) through walls, +without a camera. **This repo does not reimplement any of RuView's CSI +firmware or signal processing** — that's genuine DSP research code (Rust + +ESP-IDF, running on-device), and fabricating it from scratch rather than +integrating the real thing would be exactly the kind of guess this project's own +conventions exist to avoid. What lives in this directory is a thin, honest wrapper +around RuView's own tooling, matching how this repo already treats Frigate/ +Zigbee2MQTT/Grocy — integrated, not forked. + +This is a different device from `firmware/esphome-ble-proxy/` — separate +hardware, separate job, per `docs/project-plan.md` §1.5's "separate hardware from +RuView nodes" note. BLE proxies resolve *who* (via Bermuda/Private BLE Device); +RuView nodes sense *room activity/vitals*, camera-free. + +## 1. Hardware + +Per RuView's own README (confirmed via the link above, not guessed): + +| Board | Price | Role | +|---|---|---| +| **ESP32-S3** | ~€9 | Primary target — CSI capture + edge processing | +| ESP32-C6 | €6–10 | Research variant, WiFi 6 + dual-band | +| Intel 5300 / Atheros AR9580 (research NICs) | €50–100 | Higher-fidelity research hardware, not a household pick | +| Cognitum Seed (optional) | ~€140 | Persistent storage / vector search / witness-chain, not required for basic presence | + +**ESP32-C3 and the original ESP32 are explicitly NOT supported** (RuView's own +README: "single-core, insufficient for CSI DSP") — this updates +`docs/project-plan.md` §1.6's original "ESP32-S3 dev board" pick from a general +recommendation to a confirmed hard requirement. + +## 2. What it publishes — and why that's more sensitive than plain presence + +RuView ships **21 MQTT entities per node** via its own `HA-DISCO` MQTT +auto-discovery publisher: 11 raw signals plus 10 inferred semantic states — +`someone-sleeping`, `possible-distress`, `room-active`, `elderly-inactivity- +anomaly`, `meeting-in-progress`, `bathroom-occupied`, `fall-risk-elevated`, +`bed-exit`, `no-movement`, `multi-room-transition`. It can also estimate +breathing/heart rate. + +**This is a meaningfully bigger privacy surface than the plain occupancy sensor +`docs/project-plan.md`'s original Phase 2 sketch assumed.** Bathroom-occupancy, +sleep state, and vitals-adjacent inference are the kind of data this project has +otherwise been careful about (see `identity/README.md`'s "Anti-spoofing" and +"People without a device" sections for the same care applied to identity/consent). +**Nothing in this repo currently gives household members a way to opt out of a +specific room's RuView sensing, or restricts who these MQTT topics are visible +to beyond the same trust boundary as every other entity on this network.** Treat +this as an explicit open decision (`docs/project-plan.md` §4) before wiring +automations against the more sensitive semantic states, not something silently +safe by default just because the network is local. + +## 3. Build + flash — RuView's own tooling, not this repo's + +This repo does not vendor RuView's source. Clone and build it yourself per its own +README: + +```sh +git clone https://github.com/ruvnet/ruview ~/ruview +cd ~/ruview +# Rust 1.85+ and the ESP-IDF toolchain are prerequisites — see RuView's own docs +idf.py set-target esp32s3 # or esp32c6 for that variant +idf.py build +python -m esptool --chip esp32s3 --port /dev/ttyUSB0 --baud 460800 write_flash ... +``` + +**VERIFY the exact build/flash command against whatever version you actually +clone** — the commands above are transcribed from RuView's own README as fetched +during this session, not independently run against real hardware from within this +project; upstream commands/flags can and do change between releases. + +## 4. Provisioning per room — `provision-room.sh` + +Once a node is flashed, it needs Wi-Fi + this stack's MQTT broker credentials. +RuView ships its own `firmware/esp32-csi-node/provision.py` for this; +`provision-room.sh` in this directory is a **thin wrapper** that fills in this +project's own settings (same Mosquitto broker every other service already uses, +Phase 1) so you don't retype them per node: + +```sh +cp rooms.env.example rooms.env # fill in RUVIEW_REPO_DIR, Wi-Fi, MQTT_BROKER_HOST +chmod 600 rooms.env +./provision-room.sh living-room /dev/ttyUSB0 +``` + +Only `--port`/`--ssid`/`--password`/`--mqtt` are passed by the wrapper — those +four flags are the ones confirmed against RuView's own README during this +session. **Anything else `provision.py` might need in your specific installed +version (MQTT auth flags, a node-naming flag, etc.) is NOT guessed here** — pass +it as an extra argument to `provision-room.sh` after checking +`python3 ~/ruview/firmware/esp32-csi-node/provision.py --help` against your real +checkout first. + +## 5. Automations built on RuView's semantic states + +You've confirmed you want these wired up despite §2's privacy note — the concrete +rules requested, all in `automations.yaml.example` (paste whichever ones you want +into your own HA config, same "nothing under this repo merges into your HA config +for you" convention as identity's own worked `rest_command` example): + +1. **Someone asleep → dim that room's lights.** Straightforward state trigger → + `light.turn_on` at low brightness/warm color temp. +2. **Possible distress → alert everyone home.** Mapped to RuView's own + `possible-distress` semantic state — **this is RuView's closest documented + inference to "a heart attack," there is no literal heart-attack-detection + entity, and this repo does not build its own medical-anomaly detection on top + of RuView's raw signals.** The automation bridges that state to an urgent + `ntfy` push (via a `rest_command:`, same pattern as every other ntfy-posting + service in this stack) plus a whole-house TTS announcement across every + `media_player` you list, plus flashing every light red as a visual backup. + Whatever RuView's real accuracy turns out to be is entirely upstream's, not + something this bridge changes — **do not treat this as a substitute for an + actual medical alert device/service.** +3. **Two people's heart rate elevated concurrently in the same room → red/ + magenta/violet lighting.** Approximated as "both above an elevated threshold + at the same time" rather than a true rate-of-change trigger (deliberately — + derivative triggers on noisy CSI-derived vitals are far more false-trigger- + prone than a plain threshold). **This is the single most uncertain rule in + this file**: RuView's fetched README doesn't confirm whether one node can + actually distinguish and report two distinct people's heart rates + concurrently at all — multi-target vital-sign separation from WiFi CSI is a + genuinely hard, actively-researched problem. Confirm you have two real + per-person heart-rate entities for a room before trusting this can fire as + described; if there's only one aggregate entity per node, this rule needs a + different design once you know that, not a guess made now. +4. **Bathroom occupied → an external indicator outside the door.** You mentioned + you'll likely add a dedicated light/actor for this — the automation targets a + placeholder `switch.bathroom_door_indicator`; swap it for whatever you end up + choosing (worth adding to `components.md`'s Zigbee list once picked, same as + every other Zigbee actuator in this project). + +**Every `entity_id` in `automations.yaml.example` is a placeholder** — RuView's +README documents semantic-state *names*, not their exact post-MQTT-discovery HA +`entity_id`s (domain, underscore-vs-hyphen, per-node naming). Confirm every one +against Developer Tools → States on a real HA instance after provisioning a real +node before relying on any of this. + +## 6. Home Assistant integration — likely nothing to build, but unverified + +RuView's `HA-DISCO` publisher uses MQTT auto-discovery against the **same +Mosquitto broker** every other service in this stack already connects to (Phase 1) +— if that's accurate, entities should appear in HA automatically once a node is +provisioned and online, no manual entity/automation setup required to just *see* +the data. RuView's own docs reportedly include `docs/integrations/home-assistant.md` +and three starter HA Blueprints (per its README) for turning the semantic states +into actual automations — **none of that has been read in detail or verified from +within this project**, only the top-level README summary this section is built +from. Confirm the real doc before wiring anything beyond "the entities show up." + +Same "nothing under this repo builds the HA side" convention as every entry in +`docs/project-plan.md`'s HA-integrations catalog (§2) — automations against +RuView's semantic states are a decision for you to make deliberately, especially +given §2's privacy note above, not something this repo pre-wires. + +## Manual verification still outstanding + +1. Nothing here has been run against real hardware, a real RuView checkout, or a + real MQTT broker from within this project — §3's build commands and §4's + `provision.py` flags are transcribed from RuView's own README, not + independently confirmed. +2. `docs/integrations/home-assistant.md` / RuView's HA Blueprints (§6) — described + secondhand from the top-level README, not read directly. +3. The privacy/consent gap in §2 — the household-level decision has been made + (you want this data and the automations built on it), but there is still no + technical opt-out for a specific person/room, and no access restriction on + these MQTT topics beyond this network's normal trust boundary. Worth revisiting + if anyone who isn't fully on board with being sensed this way ever stays over. +4. `docs/project-plan.md` §1.6/§2's RuView entries — updated this session to + match §1 above (ESP32-C3/original-ESP32 unsupported, real hardware tiers), but + the phased implementation plan (Phase 2) itself still describes RuView only at + the "deploy nodes per room" level of detail from before this integration pass. +5. **`automations.yaml.example`'s entity_ids are ALL placeholders** — none of + §5's four automations have been run against real RuView entities; in + particular rule 3 (concurrent two-person heart rate) rests on an unconfirmed + assumption that a single node can even report two people's heart rates at + once, see that automation's own prominent warning comment. diff --git a/firmware/ruview/automations.yaml.example b/firmware/ruview/automations.yaml.example new file mode 100644 index 0000000..341ca20 --- /dev/null +++ b/firmware/ruview/automations.yaml.example @@ -0,0 +1,167 @@ +# Home Assistant automations built on RuView's semantic-state MQTT entities. +# +# See README.md's "Automations built on RuView's semantic states" section before +# using this — in particular: every entity_id below is a PLACEHOLDER, not a +# confirmed real one. RuView's own README documents the semantic-state NAMES +# (someone-sleeping, possible-distress, bathroom-occupied, ...) but not their +# exact post-MQTT-discovery Home Assistant entity_ids (domain, underscore vs. +# hyphen, per-node naming). Check Developer Tools -> States on your real HA +# instance after provisioning a node and correct every entity_id below before +# relying on any of this — same "VERIFY against a live instance" discipline as +# TRUSTED_ENTITY_PREFIXES/AREA_ATTRIBUTE elsewhere in this project. +# +# Paste whichever rules you want into your own configuration.yaml / automations.yaml +# (or the individual automation.*.yaml files HA's UI editor creates) — this repo +# does not merge this file into your HA config for you, same "nothing under this +# repo builds the HA side" convention as everywhere else automations appear. +# +# rest_command: block at the bottom is shared by more than one automation below — +# add it once, not once per automation. + +automation: + # --- 1. Someone asleep -> dim that room's lights --------------------------- + # One instance per room with a RuView node. Dims rather than turns off, on the + # assumption a fully dark room is a worse experience if someone gets up in the + # night than a low, warm glow — change brightness_pct/color_temp_kelvin to taste. + - alias: "RuView: someone sleeping in bedroom -> dim lights" + trigger: + - platform: state + entity_id: binary_sensor.ruview_bedroom_someone_sleeping # VERIFY real entity_id + to: "on" + action: + - service: light.turn_on + target: + area_id: bedroom # your real HA area — must match the room the light + # physically sits in, not the RuView node's naming + data: + brightness_pct: 10 + color_temp_kelvin: 2200 + + # --- 2. Possible distress -> alert everyone home ---------------------------- + # "Possible distress" is RuView's own closest documented inference to what you + # described as "a heart attack" — see README.md's honesty note: this repo does + # not build its own medical-anomaly detection on top of RuView's raw signals, + # it bridges RuView's OWN inferred state to a household-wide alert. Whatever + # RuView's real false-positive/false-negative rate turns out to be is entirely + # upstream's accuracy, not something this automation can improve on. + # + # One instance per room with a RuView node — duplicate this block per room, + # changing the trigger entity_id and the {{ }} room name in the message. + - alias: "RuView: possible distress (living room) -> alert everyone home" + trigger: + - platform: state + entity_id: binary_sensor.ruview_livingroom_possible_distress # VERIFY real entity_id + to: "on" + action: + # Urgent ntfy push — same ntfy instance/topic every other service in this + # stack already uses (see rest_command: block below). + - service: rest_command.ruview_alert_ntfy + data: + message: "RuView detected possible distress in the living room. Check on them now." + # Whole-house TTS announcement — list every media_player you actually have + # (thin clients, headless audio endpoints, the touch panel) so this reaches + # people even if their phone is elsewhere in the house. Silently no-ops on + # any media_player that's off/unavailable at the moment — HA does not error + # the whole action out over one unreachable target. + - service: tts.speak + target: + entity_id: + - media_player.living_room_thinclient # VERIFY / replace with your real media_player entities + - media_player.kitchen_audio_endpoint + - media_player.touch_panel + data: + message: "Possible medical emergency detected in the living room. Please check on the household." + # A strong, unmissable visual signal too — useful if whoever's near a light + # can't hear the announcement, or is the person in distress themselves. + - service: light.turn_on + target: + entity_id: all + data: + rgb_color: [255, 0, 0] + brightness_pct: 100 + flash: long + + # --- 3. Two people's heart rate elevated at the same time, same room ------- + # "Rises concurrently" is approximated here as "both people's heart rate is + # SIMULTANEOUSLY above an elevated threshold" rather than a true rate-of- + # change/derivative trigger — derivative triggers on noisy CSI-derived vitals + # are much more prone to false triggers than a plain threshold, so this trades + # a little precision for a lot more reliability. Adjust `above:` to a real + # resting-heart-rate-plus-margin for your household if the flat 100 bpm + # default doesn't fit. + # + # HIGH-RISK UNVERIFIED ASSUMPTION, more so than anything else in this file: + # RuView's fetched README does not confirm whether a single node actually + # distinguishes and reports MULTIPLE people's heart rates concurrently within + # one room at all — multi-target vital-sign separation from WiFi CSI is a + # genuinely hard, actively-researched problem, and RuView may only report one + # aggregate/first-detected person's heart rate per node. Confirm you actually + # have two distinct per-person heart-rate entities for a room (Developer Tools + # -> States, after provisioning) before trusting this rule can ever fire as + # described — if there's only one heart-rate entity per node, "two people's + # pulse rising together" isn't something this hardware can currently tell you, + # and this rule needs a different design (or dropping) once you know that. + - alias: "RuView: two people's heart rate elevated concurrently (living room) -> alert lighting" + trigger: + - platform: numeric_state + entity_id: sensor.ruview_livingroom_heart_rate_person1 # VERIFY this entity exists at all, see note above + above: 100 + - platform: numeric_state + entity_id: sensor.ruview_livingroom_heart_rate_person2 # VERIFY this entity exists at all, see note above + above: 100 + condition: + - condition: numeric_state + entity_id: sensor.ruview_livingroom_heart_rate_person1 + above: 100 + - condition: numeric_state + entity_id: sensor.ruview_livingroom_heart_rate_person2 + above: 100 + action: + - service: light.turn_on + target: + area_id: living_room + data: + rgb_color: [178, 0, 255] # violet — swap for magenta [255, 0, 144] or + # red [255, 0, 0] to taste, or cycle between + # them with a light.turn_on + delay + repeat + brightness_pct: 100 + + # --- 4. Bathroom occupied -> external door indicator ------------------------ + # You mentioned you'll likely add a dedicated light/actor outside the door for + # this — swap switch.bathroom_door_indicator below for whatever you end up + # picking (add it to components.md's Zigbee list once chosen, same as every + # other Zigbee actuator in this project). + - alias: "RuView: bathroom occupied -> door indicator on" + trigger: + - platform: state + entity_id: binary_sensor.ruview_bathroom_occupied # VERIFY real entity_id + to: "on" + action: + - service: switch.turn_on + target: + entity_id: switch.bathroom_door_indicator # VERIFY / replace once hardware is chosen + - alias: "RuView: bathroom vacated -> door indicator off" + trigger: + - platform: state + entity_id: binary_sensor.ruview_bathroom_occupied # VERIFY real entity_id + to: "off" + action: + - service: switch.turn_off + target: + entity_id: switch.bathroom_door_indicator # VERIFY / replace once hardware is chosen + +# Shared by automation #2 above — same ntfy instance/topic pattern chores/ +# trash-calendar/etc. all already POST to directly from Python; this is the HA- +# side equivalent for automations that need to reach it. Fill in your real ntfy +# host/topic (same NTFY_URL/NTFY_TOPIC style as chores.env.example) and put the +# actual URL in HA's own secrets.yaml, same pattern as identity's rest_command +# example in identity/README.md. +rest_command: + ruview_alert_ntfy: + url: "http://:8090/smarthome-alerts" # <-- your real ntfy host:port/topic + method: POST + headers: + Title: "Possible medical emergency" + Priority: "urgent" + payload: "{{ message }}" + content_type: "text/plain" diff --git a/firmware/ruview/provision-room.sh b/firmware/ruview/provision-room.sh new file mode 100755 index 0000000..4c65213 --- /dev/null +++ b/firmware/ruview/provision-room.sh @@ -0,0 +1,52 @@ +#!/usr/bin/env bash +# Thin per-room wrapper around RuView's OWN provision.py +# (https://github.com/ruvnet/ruview, firmware/esp32-csi-node/provision.py) — this +# repo does not reimplement RuView's provisioning, it just fills in this stack's +# own Wi-Fi/MQTT settings so you don't have to retype them per room. See +# README.md for why RuView's actual CSI firmware is integrated, not forked. +# +# Usage: ./provision-room.sh [extra provision.py args] +# Example: ./provision-room.sh living-room /dev/ttyUSB0 + +set -euo pipefail + +ROOM="${1:?Usage: ./provision-room.sh [extra args]}" +PORT="${2:?Usage: ./provision-room.sh [extra args]}" +shift 2 + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +ENV_FILE="$SCRIPT_DIR/rooms.env" +if [[ ! -f "$ENV_FILE" ]]; then + echo "Missing $ENV_FILE — cp rooms.env.example rooms.env and fill it in first." >&2 + exit 1 +fi +# shellcheck disable=SC1090 +source "$ENV_FILE" + +: "${RUVIEW_REPO_DIR:?rooms.env is missing RUVIEW_REPO_DIR}" +: "${WIFI_SSID:?rooms.env is missing WIFI_SSID}" +: "${WIFI_PASSWORD:?rooms.env is missing WIFI_PASSWORD}" +: "${MQTT_BROKER_HOST:?rooms.env is missing MQTT_BROKER_HOST}" + +PROVISION_PY="$RUVIEW_REPO_DIR/firmware/esp32-csi-node/provision.py" +if [[ ! -f "$PROVISION_PY" ]]; then + echo "Error: $PROVISION_PY not found." >&2 + echo " RUVIEW_REPO_DIR ($RUVIEW_REPO_DIR) doesn't look like a real RuView checkout —" >&2 + echo " clone https://github.com/ruvnet/ruview there first, per its own README." >&2 + exit 1 +fi + +echo "--- Provisioning RuView node for room '$ROOM' on $PORT ---" +echo "(Only --port/--ssid/--password/--mqtt below are confirmed against RuView's own" +echo " README — if you need MQTT auth or a specific node-name flag, check" +echo " 'python3 $PROVISION_PY --help' against YOUR checked-out version and pass it as" +echo " an extra argument to this script; nothing here guesses flag names it hasn't" +echo " confirmed. See README.md's 'Manual verification still outstanding'.)" +echo + +python3 "$PROVISION_PY" \ + --port "$PORT" \ + --ssid "$WIFI_SSID" \ + --password "$WIFI_PASSWORD" \ + --mqtt "$MQTT_BROKER_HOST" \ + "$@" diff --git a/firmware/ruview/rooms.env.example b/firmware/ruview/rooms.env.example new file mode 100644 index 0000000..ccdd31e --- /dev/null +++ b/firmware/ruview/rooms.env.example @@ -0,0 +1,25 @@ +# provision-room.sh configuration template. +# +# Copy this to firmware/ruview/rooms.env (same directory as provision-room.sh) and +# fill in real values. Same never-commit handling as every other *.env.example in +# this repo: the real rooms.env is git-ignored, only this .example is tracked. +# +# Shared across every room you provision — unlike the voice-display satellite's +# per-unit secrets.yaml, RuView nodes don't need distinct Wi-Fi/MQTT credentials +# per room, only a distinct node/room name, which provision-room.sh takes as its +# own first argument instead of living in here. + +# Where you've cloned https://github.com/ruvnet/ruview locally. This repo does +# NOT vendor RuView's source — same "integrate, don't fork" treatment as Frigate/ +# Zigbee2MQTT/Grocy elsewhere in this stack — you clone/build it yourself per its +# own README (Rust + ESP-IDF toolchain required). +RUVIEW_REPO_DIR="$HOME/ruview" + +WIFI_SSID="your-wifi-ssid" +WIFI_PASSWORD="your-wifi-password" + +# Same Mosquitto broker every other service in this stack uses (Phase 1) — LAN IP +# of the container host, not a hostname (these nodes aren't on the compose +# network and can't resolve container DNS names, same situation documented for +# Node-RED/identity/transit elsewhere in this repo). +MQTT_BROKER_HOST="192.168.1.10" diff --git a/hosts/container-host/scripts/setup-container-host.sh b/hosts/container-host/scripts/setup-container-host.sh index 7ffcaee..3860a4b 100755 --- a/hosts/container-host/scripts/setup-container-host.sh +++ b/hosts/container-host/scripts/setup-container-host.sh @@ -37,6 +37,16 @@ # hosts/door-panel's and hosts/kitchen-display's dashboards — optional, off by # default, needs identity/ from this repo checked out on this host, see # IDENTITY_SRC below and identity/README.md) +# - trash-calendar (Phase 19, optional, off by default — syncs Kennelbach's +# personal collection-date ICS feed into the household's own Nextcloud +# calendar, daily via systemd timer, needs trash-calendar/ from this repo +# checked out on this host, see TRASH_CALENDAR_SRC below and +# trash-calendar/README.md) +# - transit + optional otp (Phase 19, optional, off by default — "when's the +# next bus" voice lookups from a weekly-refreshed GTFS feed, plus optional +# on-demand route planning via a self-hosted OpenTripPlanner; needs transit/ +# from this repo checked out on this host, see TRANSIT_SRC below and +# transit/README.md) # # Run as: sudo ./setup-container-host.sh # @@ -127,6 +137,66 @@ IDENTITY_SRC="/opt/smart-home/src/identity" IDENTITY_PORT="8097" # LAN-facing — kiosk browsers call this directly IDENTITY_WEB_PORT="8098" # LAN-facing read-only static serving (register.html/dashboard.html) +# --- Trash collection date sync (Phase 19) — off by default until WASTE_ICS_URL and +# --- CALDAV_TARGET_CALENDAR are provisioned. See trash-calendar/README.md. A oneshot, +# --- like digest-engine, not a listener — no port, nothing to publish. +ENABLE_TRASH_CALENDAR="false" +# Where this repo's trash-calendar/ directory lives on THIS host (build context). +TRASH_CALENDAR_SRC="/opt/smart-home/src/trash-calendar" + +# --- Public transit "when's the next bus" voice lookup (Phase 19) — off by default +# --- until TRANSIT_TOKEN and GTFS_FEED_URL are provisioned. See transit/README.md. +# --- Published like pantry-vision/identity — homeassistant's network_mode: host +# --- can't resolve container DNS names, so its rest_command needs a real port. +ENABLE_TRANSIT="false" +# Where this repo's transit/ directory lives on THIS host (build context). +TRANSIT_SRC="/opt/smart-home/src/transit" +TRANSIT_PORT="8099" # reachable by HA's rest_command (voice lookups) + +# --- On-demand route planning (Phase 19) — a SEPARATE opt-in from ENABLE_TRANSIT +# --- above on purpose: a real OpenTripPlanner graph (OSM + GTFS) is a meaningfully +# --- bigger data/hardware commitment than the small filtered GTFS DB /departures +# --- uses. See transit/README.md's "Route planning scope" before turning this on — +# --- "Austria-wide" is a moderate commitment, "global" is a real infrastructure +# --- decision, not a flag. This script does NOT build the OTP graph for you — that's +# --- a manual, one-time (per OSM/GTFS update) step; see OpenTripPlanner's own docs. +ENABLE_TRIP_PLANNING="false" +OTP_GRAPHS_DIR="/opt/smart-home/otp-graphs" # you populate this by hand, see above +# Host-side published port for OTP's own web/GraphQL API. NOT 8080 — zigbee2mqtt's +# frontend (always-on, below) already publishes 8080:8080; OTP's own container- +# internal port stays 8080 regardless (transit.env.example's OTP_URL correctly +# reaches it via container DNS as http://otp:8080), only the host-side mapping +# needed to move to avoid the two colliding on the same host. +OTP_PORT="8100" + +# --- Household chore distribution + reminders + camera verification (Phase 20) — +# --- off by default. Works with just ENABLE_IDENTITY on (assignment) and ntfy +# --- (reminders); the camera-check and trash-day-eve steps each individually +# --- no-op until their own env vars are set — see chores/README.md. +ENABLE_CHORES="false" +# Where this repo's chores/ directory lives on THIS host (build context). +CHORES_SRC="/opt/smart-home/src/chores" + +# --- Music Assistant (optional, additive — docs/project-plan.md §2) --------- +# Unifies Spotify Connect + other sources behind one HA-native multi-room player. +# Runs as its OWN container (official image, not an HA add-on — this stack is a +# plain HA Container install, which has no add-on store) — configure HA's own +# "Music Assistant" integration afterwards to point at it. Does not replace or +# depend on any host's existing per-room spotifyd/librespot/Spotify-client setup +# (Phase 11.6/15/16); those keep working standalone either way. network_mode: host +# because player discovery (Chromecast/AirPlay/Sonos-style mDNS) needs it — same +# reasoning as the homeassistant service itself using host networking. +# +# VERIFY: Music Assistant's own default web UI/API port is ASSUMED to be 8095, +# not confirmed against a running instance — and 8095 is already this stack's +# PANTRY_VISION_PORT (below). Because network_mode: host bypasses Compose's own +# port-mapping/collision-checking entirely, enabling both together with an actual +# port clash would just mean one of them fails to bind, not an obvious error. +# Check Music Assistant's own docs/config for how to change its listen port +# BEFORE flipping this on if ENABLE_PANTRY_VISION is also true — the sanity check +# below only warns, it does not change either port for you. +ENABLE_MUSIC_ASSISTANT="false" + # --------------------------------------------------------------------------- # Sanity checks # --------------------------------------------------------------------------- @@ -175,6 +245,40 @@ if [[ "$ENABLE_IDENTITY" == "true" && ! -d "$IDENTITY_SRC" ]]; then echo " context for the identity image — then re-run." fi +if [[ "$ENABLE_TRASH_CALENDAR" == "true" && ! -d "$TRASH_CALENDAR_SRC" ]]; then + echo "Warning: ENABLE_TRASH_CALENDAR=true but $TRASH_CALENDAR_SRC does not exist." + echo " Copy or clone this repo's trash-calendar/ directory there — it is the" + echo " build context for the trash-calendar image — then re-run." +fi + +if [[ "$ENABLE_TRANSIT" == "true" && ! -d "$TRANSIT_SRC" ]]; then + echo "Warning: ENABLE_TRANSIT=true but $TRANSIT_SRC does not exist." + echo " Copy or clone this repo's transit/ directory there — it is the build" + echo " context for the transit image — then re-run." +fi + +if [[ "$ENABLE_TRIP_PLANNING" == "true" && ! -d "$OTP_GRAPHS_DIR" ]]; then + echo "Warning: ENABLE_TRIP_PLANNING=true but $OTP_GRAPHS_DIR does not exist or is" + echo " empty. OpenTripPlanner needs a built graph there before it can serve" + echo " anything — see transit/README.md's 'Route planning scope' section. The" + echo " otp container will start but /plan will fail until a graph exists." +fi + +if [[ "$ENABLE_CHORES" == "true" && ! -d "$CHORES_SRC" ]]; then + echo "Warning: ENABLE_CHORES=true but $CHORES_SRC does not exist." + echo " Copy or clone this repo's chores/ directory there — it is the build" + echo " context for the chores image — then re-run." +fi + +if [[ "$ENABLE_MUSIC_ASSISTANT" == "true" && "$ENABLE_PANTRY_VISION" == "true" ]]; then + echo "Warning: ENABLE_MUSIC_ASSISTANT=true and ENABLE_PANTRY_VISION=true together." + echo " Music Assistant's own default port is ASSUMED to be $PANTRY_VISION_PORT (unverified," + echo " see ENABLE_MUSIC_ASSISTANT's own comment above) — the SAME as PANTRY_VISION_PORT." + echo " Because Music Assistant runs with network_mode: host, a real clash here is NOT" + echo " caught by Docker Compose the way a normal port mapping would be. Check Music" + echo " Assistant's own config for its listen port before starting the stack." +fi + if [[ "$ENABLE_WHATSAPP_INGEST" == "true" ]]; then echo "WARNING: WhatsApp ingestion is enabled." echo " There is no officially sanctioned way to read WhatsApp programmatically." @@ -320,6 +424,41 @@ if [[ "$ENABLE_IDENTITY" == "true" ]]; then echo " Access Tokens), and TRUSTED_ENTITY_PREFIXES (Developer Tools -> States)." fi fi +if [[ "$ENABLE_TRASH_CALENDAR" == "true" ]]; then + mkdir -p "$BASE_DIR"/trash-calendar + if [[ ! -f "$BASE_DIR/trash-calendar/trash-calendar.env" ]]; then + cp "$TRASH_CALENDAR_SRC/trash-calendar.env.example" "$BASE_DIR/trash-calendar/trash-calendar.env" + chmod 600 "$BASE_DIR/trash-calendar/trash-calendar.env" + echo " Seeded $BASE_DIR/trash-calendar/trash-calendar.env from the template — fill in" + echo " a real WASTE_ICS_URL and CALDAV_TARGET_CALENDAR (same CALDAV_* credentials as" + echo " digest-engine's own ingest/caldav.py, reused — see trash-calendar/README.md)." + fi +fi +if [[ "$ENABLE_TRANSIT" == "true" ]]; then + mkdir -p "$BASE_DIR"/transit/data + if [[ ! -f "$BASE_DIR/transit/transit.env" ]]; then + cp "$TRANSIT_SRC/transit.env.example" "$BASE_DIR/transit/transit.env" + chmod 600 "$BASE_DIR/transit/transit.env" + echo " Seeded $BASE_DIR/transit/transit.env from the template — fill in a real" + echo " TRANSIT_TOKEN and GTFS_FEED_URL (see transit/README.md for where to get it)." + fi +fi +if [[ "$ENABLE_TRIP_PLANNING" == "true" ]]; then + mkdir -p "$OTP_GRAPHS_DIR" +fi +if [[ "$ENABLE_CHORES" == "true" ]]; then + mkdir -p "$BASE_DIR"/chores/data + if [[ ! -f "$BASE_DIR/chores/chores.env" ]]; then + cp "$CHORES_SRC/chores.env.example" "$BASE_DIR/chores/chores.env" + chmod 600 "$BASE_DIR/chores/chores.env" + echo " Seeded $BASE_DIR/chores/chores.env from the template — fill in IDENTITY_TOKEN" + echo " and ntfy settings at minimum; camera checks and trash-day-eve are each" + echo " optional, see chores/README.md." + fi +fi +if [[ "$ENABLE_MUSIC_ASSISTANT" == "true" ]]; then + mkdir -p "$BASE_DIR"/music-assistant/data +fi # --------------------------------------------------------------------------- # 4. Mosquitto config @@ -779,6 +918,127 @@ if [[ "$ENABLE_IDENTITY" == "true" ]]; then " fi +# trash-calendar (Phase 19) — oneshot, driven by smart-home-trash-calendar.timer +# below, same profiles:[oneshot] shape as digest-engine above. +TRASH_CALENDAR_BLOCK="" +if [[ "$ENABLE_TRASH_CALENDAR" == "true" ]]; then + TRASH_CALENDAR_BLOCK=" + trash-calendar: + build: ${TRASH_CALENDAR_SRC} + image: smart-home/trash-calendar:local + container_name: trash-calendar + profiles: + - oneshot + restart: \"no\" + env_file: + - ${BASE_DIR}/trash-calendar/trash-calendar.env + environment: + - TZ=${TIMEZONE} +" +fi + +# transit (Phase 19) — /departures is always-on (the server container below); +# /transit-sync (weekly GTFS refresh) is a oneshot, same profiles:[oneshot] shape as +# digest-engine/trash-calendar above, sharing the server's own /data volume so the +# refreshed DB is immediately visible to it. +TRANSIT_BLOCK="" +TRANSIT_SYNC_BLOCK="" +OTP_BLOCK="" +if [[ "$ENABLE_TRANSIT" == "true" ]]; then + TRANSIT_BLOCK=" + transit: + build: ${TRANSIT_SRC} + image: smart-home/transit:local + container_name: transit + restart: unless-stopped + ports: + - \"${TRANSIT_PORT}:${TRANSIT_PORT}\" + env_file: + - ${BASE_DIR}/transit/transit.env + volumes: + - ${BASE_DIR}/transit/data:/data + environment: + - TRANSIT_PORT=${TRANSIT_PORT} + - TZ=${TIMEZONE} +" + + TRANSIT_SYNC_BLOCK=" + transit-sync: + build: ${TRANSIT_SRC} + image: smart-home/transit:local + container_name: transit-sync + profiles: + - oneshot + restart: \"no\" + command: [\"python\", \"sync_gtfs.py\"] + env_file: + - ${BASE_DIR}/transit/transit.env + volumes: + - ${BASE_DIR}/transit/data:/data + environment: + - TZ=${TIMEZONE} +" +fi +if [[ "$ENABLE_TRIP_PLANNING" == "true" ]]; then + # Official OpenTripPlanner image. VERIFY the exact CLI flags/serve command against + # whichever OTP version you deploy — this repo does not build or manage the graph + # itself, see transit/README.md's 'Route planning scope'. + OTP_BLOCK=" + otp: + image: opentripplanner/opentripplanner:latest + container_name: otp + restart: unless-stopped + ports: + - \"${OTP_PORT}:8080\" + volumes: + - ${OTP_GRAPHS_DIR}:/var/opentripplanner + command: [\"--load\", \"--serve\"] + environment: + - TZ=${TIMEZONE} +" +fi + +# chores (Phase 20) — oneshot, driven by smart-home-chores.timer below, same +# profiles:[oneshot] shape as trash-calendar/transit-sync above. Reads from +# identity/Frigate/ntfy over the network, not container DNS — no shared volume +# needed with any of them. +CHORES_BLOCK="" +if [[ "$ENABLE_CHORES" == "true" ]]; then + CHORES_BLOCK=" + chores: + build: ${CHORES_SRC} + image: smart-home/chores:local + container_name: chores + profiles: + - oneshot + restart: \"no\" + env_file: + - ${BASE_DIR}/chores/chores.env + volumes: + - ${BASE_DIR}/chores/data:/data + environment: + - TZ=${TIMEZONE} +" +fi + +# Music Assistant — own container, host networking (see ENABLE_MUSIC_ASSISTANT's +# module-level comment above for why). No published-port line needed since host +# networking exposes its own default port (8095) directly. +MUSIC_ASSISTANT_BLOCK="" +if [[ "$ENABLE_MUSIC_ASSISTANT" == "true" ]]; then + MUSIC_ASSISTANT_BLOCK=" + music-assistant: + image: ghcr.io/music-assistant/server:stable + container_name: music-assistant + restart: unless-stopped + network_mode: host + volumes: + - ${BASE_DIR}/music-assistant/data:/data + environment: + - TZ=${TIMEZONE} +" +fi + if [[ "$ENABLE_DIGEST_ENGINE" == "true" && "$ENABLE_WHATSAPP_INGEST" == "true" ]]; then # Long-lived, unlike digest-engine: holds the logged-in WhatsApp Web session # open and appends to /data/messages.jsonl, which digest-engine drains each @@ -875,7 +1135,7 @@ ${FRIGATE_DEVICES} - PUID=1000 - PGID=1000 - TZ=${TIMEZONE} -${MEALIE_BLOCK}${NODERED_BLOCK}${NETDATA_BLOCK}${HOMEPAGE_BLOCK}${NTFY_BLOCK}${PORTAINER_BLOCK}${GALLERY_SMB_BLOCK}${DIGEST_ENGINE_BLOCK}${DIGEST_WEB_BLOCK}${WHATSAPP_BRIDGE_BLOCK}${ADMIN_CANVAS_BLOCK}${ADMIN_WEB_BLOCK}${PANTRY_VISION_BLOCK}${PANTRY_WEB_BLOCK}${IDENTITY_BLOCK}${IDENTITY_WEB_BLOCK} +${MEALIE_BLOCK}${NODERED_BLOCK}${NETDATA_BLOCK}${HOMEPAGE_BLOCK}${NTFY_BLOCK}${PORTAINER_BLOCK}${GALLERY_SMB_BLOCK}${DIGEST_ENGINE_BLOCK}${DIGEST_WEB_BLOCK}${WHATSAPP_BRIDGE_BLOCK}${ADMIN_CANVAS_BLOCK}${ADMIN_WEB_BLOCK}${PANTRY_VISION_BLOCK}${PANTRY_WEB_BLOCK}${IDENTITY_BLOCK}${IDENTITY_WEB_BLOCK}${TRASH_CALENDAR_BLOCK}${TRANSIT_BLOCK}${TRANSIT_SYNC_BLOCK}${OTP_BLOCK}${CHORES_BLOCK}${MUSIC_ASSISTANT_BLOCK} EOF # --------------------------------------------------------------------------- @@ -995,6 +1255,118 @@ EOF echo " Run it manually any time with: cd $BASE_DIR && docker compose run --rm digest-engine" fi +# --------------------------------------------------------------------------- +# 9c. Scheduled trash-calendar sync (Phase 19) — optional, mirrors the digest timer +# --------------------------------------------------------------------------- +if [[ "$ENABLE_TRASH_CALENDAR" == "true" ]]; then + echo "--- Setting up the daily trash-calendar sync timer ---" + + cat > /etc/systemd/system/smart-home-trash-calendar.service < /etc/systemd/system/smart-home-trash-calendar.timer < /etc/systemd/system/smart-home-transit-sync.service < /etc/systemd/system/smart-home-transit-sync.timer < /etc/systemd/system/smart-home-chores.service < /etc/systemd/system/smart-home-chores.timer < States) before rely on `room` being populated at all; it degrades to `null` if missing, never breaks the response. +## Chore-system settings — owned here, used by `chores/` + +Two per-person fields, set via `POST /people//chore-settings`. **No frontend +for this exists yet** — neither `register.html` nor `dashboard.html` expose a way +to set them — call the endpoint directly (an HA script/automation, or `curl`) until +one is built. Read by `chores/` off `GET /presence`: + +- **`chore_exempt`** — a household member who's tracked for presence/identity like + anyone else but never nudged about chores in general (the "cousin visits often + but doesn't owe me chores" case). **Litter is the deliberate exception** — + `chores/check.py`'s `_EXEMPTIONS_DONT_APPLY` still nudges an exempt person about + putting trash they left out into the bin, because that responsibility isn't + "doing a chore," it's cleaning up after yourself. +- **`chore_reminder_style`** — free text describing how a person wants to be + reminded ("be assertive, don't let up" / "be gentle, give me a few minutes of + grace"). `chores/` passes this to an LLM that **phrases** the reminder message in + that style — it never decides *who* or *when* to nudge, only *how the words come + out*, per that system's own hard rule that presence/schedule drives every + assignment decision (see `chores/README.md`). Empty/unset falls back to a plain, + un-styled template with no LLM call at all. + +Both fields live on `people` (not a separate table) because they're household- +standing facts about a person, same category as their name or photo — `identity` is +already this project's source of truth for who someone is, so this is where "how do +I relate to this specific household member" facts belong, not duplicated into +`chores/`'s own database. + +## Camera face recognition — a second presence signal, never a registration one + +If Tapo pan/tilt cameras are wired into Frigate as additional camera sources +(`docs/project-plan.md` Phase 20) and their faces are enrolled in Frigate's own +0.16+ face recognition, `identity` subscribes to `FRIGATE_EVENTS_TOPIC` +(`frigate/events` by default) and treats a recognized name matching a registered +person (case-insensitive) as a **corroborating** presence signal — `home` becomes +`true` if either their BLE identifier reports present *or* their face was seen in +the last `FACE_PRESENCE_WINDOW_SECONDS`. This is genuinely useful for the +device-less case too (a grandmother with no phone can now show as home the moment a +camera recognizes her, not just via the manual toggle). + +**It is never a registration signal** — `/register` never reads +`_last_face_seen`, and there's no path from "camera saw a face" to "a new person +got created." That stays BLE/IRK-only and human-confirmed, per this file's +"Anti-spoofing" section above; the camera can corroborate an existing person's +presence, never mint a new identity. + +`sub_label = ["name", confidence]` is Frigate's documented shape for object +sub-labels generally (used for both face and license-plate recognition plugins); +whether Frigate 0.16+'s specific face-recognition feature publishes into that exact +field on `frigate/events` is **not verified against a real deployment** — a wrong +topic or field name just means this signal never fires, degrading silently back to +BLE/manual presence only. + ## Voice: single-utterance, not multi-turn The whole flow is designed around one spoken sentence: **"register me as ``"** @@ -194,7 +246,8 @@ not network placement. | `DELETE /people//identifiers/` | revoke a mistaken or compromised identifier | | `DELETE /people/` | remove a person entirely (their identifiers go with them) — mainly for cleaning up stale Guest records | | `POST /presence/manual` | `{"person_id", "home"}` — hand-operated Home/Away for anyone with no identifiers | -| `GET /presence` | `{"people": [{"id", "name", "home", "room", "has_device", "has_photo"}], "generated_at"}` — `home` is `true`/`false`/`null` (unknown), `room` is best-effort floor-plan groundwork (see below) | +| `POST /people//chore-settings` | `{"chore_exempt"?, "chore_reminder_style"?}` — see below; either field omitted/`null` leaves it unchanged | +| `GET /presence` | `{"people": [{"id", "name", "home", "room", "has_device", "has_photo", "chore_exempt", "chore_reminder_style"}], "generated_at"}` — `home` is `true`/`false`/`null` (unknown), `room` is best-effort floor-plan groundwork (see below) | | `GET /weather` | proxies `smarthome/weather/current`, same JSON shape (`temperature`/`condition`/`location`) `hosts/thin-client`'s weather overlay already uses | **Every person gets a profile picture, automatically** — whichever registration photo diff --git a/identity/identity.env.example b/identity/identity.env.example index 1305d2e..425a7ed 100644 --- a/identity/identity.env.example +++ b/identity/identity.env.example @@ -40,13 +40,20 @@ HA_TOKEN= TRUSTED_ENTITY_PREFIXES=device_tracker.pble_,device_tracker.bletag_ # --------------------------------------------------------------------------- -# MQTT — for the /weather proxy only (smarthome/weather/current, the same -# household-wide topic hosts/thin-client's idle-gallery overlay already reads). +# MQTT — the /weather proxy (smarthome/weather/current, the same household-wide +# topic hosts/thin-client's idle-gallery overlay already reads), and Frigate face +# recognition as a SECOND, corroborating presence signal (Phase 20, Tapo pan/tilt +# cameras) — never a registration signal, only ever OR-ed into /presence's "home" +# result. FRIGATE_EVENTS_TOPIC's payload shape (sub_label = ["name", confidence]) +# is unverified against a real Frigate 0.16+ face-recognition deployment — see +# README.md. # --------------------------------------------------------------------------- MQTT_BROKER_HOST=mosquitto MQTT_BROKER_PORT=1883 MQTT_USERNAME= MQTT_PASSWORD= +FRIGATE_EVENTS_TOPIC=frigate/events +FACE_PRESENCE_WINDOW_SECONDS=600 # --------------------------------------------------------------------------- # Run behaviour diff --git a/identity/server.py b/identity/server.py index b8a0573..4cc26ca 100755 --- a/identity/server.py +++ b/identity/server.py @@ -116,6 +116,18 @@ _db_lock = threading.Lock() _weather_lock = threading.Lock() _last_weather: dict = {"available": False} +# Frigate face-recognition as a SECOND, corroborating presence signal (docs/ +# project-plan.md Phase 20 — Tapo pan/tilt cameras) — never a registration signal +# (see the module docstring: the camera is audit-only for /register, always). +# `frigate/events`' recognized-name field is written from Frigate's documented +# `sub_label` shape (`["name", confidence]`), NOT verified against a real Frigate +# 0.16+ face-recognition deployment — VERIFY before trusting this. Degrades to +# simply never firing (BLE/manual presence still work) if the topic/shape is wrong. +FRIGATE_EVENTS_TOPIC = os.environ.get("FRIGATE_EVENTS_TOPIC", "frigate/events") +FACE_PRESENCE_WINDOW_SECONDS = int(os.environ.get("FACE_PRESENCE_WINDOW_SECONDS", "600")) +_face_lock = threading.Lock() +_last_face_seen: dict[str, float] = {} # lowercased name -> unix timestamp + def _now() -> str: return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z") @@ -137,7 +149,21 @@ def init_db() -> None: id INTEGER PRIMARY KEY, name TEXT NOT NULL UNIQUE COLLATE NOCASE, created_at TEXT NOT NULL, - photo_path TEXT + photo_path TEXT, + -- Chore-system settings (see chores/README.md). Owned here, not in + -- chores/ itself, because a person's identity/household-standing is + -- this service's job — chores just reads /people and /presence. + -- chore_exempt: household member who's still tracked (a frequent + -- guest, e.g. a cousin) but never nudged about chores in general. + -- Litter is the deliberate exception (see chores/check.py's + -- _EXEMPTIONS_DONT_APPLY) — everyone is responsible for putting + -- trash they left in the bin, exempt or not. + chore_exempt INTEGER NOT NULL DEFAULT 0, + -- chore_reminder_style: free text, e.g. "be assertive, don't let up" + -- or "be gentle, give me a few minutes of grace" — passed to the + -- LLM that phrases (never decides) chore reminders. NULL/empty means + -- the plain, un-styled template. + chore_reminder_style TEXT ); CREATE TABLE IF NOT EXISTS identifiers ( id INTEGER PRIMARY KEY, @@ -356,7 +382,8 @@ def _log_event(conn, person_id, device_id, photo_path, outcome) -> None: def list_people() -> list[dict]: with _db_lock, _db() as conn: people = conn.execute( - "SELECT id, name, created_at, photo_path FROM people ORDER BY name COLLATE NOCASE" + "SELECT id, name, created_at, photo_path, chore_exempt, chore_reminder_style " + "FROM people ORDER BY name COLLATE NOCASE" ).fetchall() result = [] for person in people: @@ -374,12 +401,32 @@ def list_people() -> list[dict]: # filesystem detail — just whether GET /people//photo has # anything to serve. "has_photo": person["photo_path"] is not None, + "chore_exempt": bool(person["chore_exempt"]), + "chore_reminder_style": person["chore_reminder_style"], "identifiers": [dict(i) for i in identifiers], } ) return result +def set_chore_settings(person_id: int, chore_exempt: bool | None, reminder_style: str | None) -> bool: + """Either field left as None leaves that column untouched — lets a caller update + just one of the two without needing to know the other's current value first. + """ + with _db_lock, _db() as conn: + exists = conn.execute("SELECT 1 FROM people WHERE id = ?", (person_id,)).fetchone() + if not exists: + return False + if chore_exempt is not None: + conn.execute("UPDATE people SET chore_exempt = ? WHERE id = ?", (int(chore_exempt), person_id)) + if reminder_style is not None: + conn.execute( + "UPDATE people SET chore_reminder_style = ? WHERE id = ?", + (reminder_style.strip() or None, person_id), + ) + return True + + def get_person_photo(person_id: int) -> bytes | None: with _db_lock, _db() as conn: row = conn.execute("SELECT photo_path FROM people WHERE id = ?", (person_id,)).fetchone() @@ -486,12 +533,21 @@ def presence() -> dict: for row in conn.execute("SELECT person_id, home FROM manual_presence") } + with _face_lock: + seen_recently = { + name for name, ts in _last_face_seen.items() if time.time() - ts <= FACE_PRESENCE_WINDOW_SECONDS + } + result = [] for person in people: room = None + face_seen = person["name"].strip().lower() in seen_recently if person["identifiers"] and ha_ok: entity_states = [states[i["ha_entity_id"]] for i in person["identifiers"] if i["ha_entity_id"] in states] - home = any(s.get("state") in PRESENT_STATES for s in entity_states) + # A second, corroborating signal — see FRIGATE_EVENTS_TOPIC's + # module-level comment. Camera face recognition is never a registration + # signal (see the module docstring), only ever an OR-ed-in presence one. + home = face_seen or any(s.get("state") in PRESENT_STATES for s in entity_states) # Floor-plan groundwork — see AREA_ATTRIBUTE's module-level comment. # First identifier that actually reports one wins; a person with two # phones in two different rooms is a real but rare edge case not worth @@ -502,9 +558,14 @@ def presence() -> dict: room = area break elif person["identifiers"]: - home = None # HA unreachable + home = True if face_seen else None # HA unreachable, but a camera sighting still counts else: - home = manual.get(person["id"]) # None if never manually set either + # Device-less (grandmother, a guest): face recognition can confirm + # "home" even though nothing here can ever confirm "away" from a + # camera alone (not seen recently just means not seen, not proven + # absent) — so a recent sighting overrides a stale/never-set manual + # flag, but a manual "away" is never second-guessed by a camera miss. + home = True if face_seen else manual.get(person["id"]) result.append( { @@ -514,6 +575,11 @@ def presence() -> dict: "room": room, "has_device": bool(person["identifiers"]), "has_photo": person["has_photo"], + "face_seen_recently": face_seen, + # Chore-system settings, straight passthrough — see chores/README.md + # for how these are used (never anything presence-related itself). + "chore_exempt": person["chore_exempt"], + "chore_reminder_style": person["chore_reminder_style"], } ) @@ -523,7 +589,7 @@ def presence() -> dict: return payload -def _on_mqtt_message(_client, _userdata, message) -> None: +def _on_weather_message(_client, _userdata, message) -> None: try: payload = json.loads(message.payload.decode("utf-8", "replace")) except ValueError: @@ -533,6 +599,31 @@ def _on_mqtt_message(_client, _userdata, message) -> None: _last_weather = {**payload, "available": True} +def _on_frigate_event(_client, _userdata, message) -> None: + """Best-effort only — see FRIGATE_EVENTS_TOPIC's module-level comment. A + malformed or unexpected payload just means this particular event contributes no + presence signal, never an error that could take the weather subscription (or + anything else) down with it. + """ + try: + payload = json.loads(message.payload.decode("utf-8", "replace")) + sub_label = ((payload.get("after") or {}).get("sub_label")) + name = sub_label[0] if isinstance(sub_label, list) and sub_label else None + if not name: + return + with _face_lock: + _last_face_seen[str(name).strip().lower()] = time.time() + except Exception: + pass + + +def _on_mqtt_message(client, userdata, message) -> None: + if message.topic == "smarthome/weather/current": + _on_weather_message(client, userdata, message) + elif message.topic == FRIGATE_EVENTS_TOPIC: + _on_frigate_event(client, userdata, message) + + def start_mqtt(broker_host: str, broker_port: int, username: str, password: str) -> None: if not broker_host: LOG.warning("identity: MQTT_BROKER_HOST not set — /weather will always report unavailable") @@ -546,6 +637,7 @@ def start_mqtt(broker_host: str, broker_port: int, username: str, password: str) def on_connect(c, _userdata, _flags, rc): if rc == 0: c.subscribe("smarthome/weather/current", qos=1) + c.subscribe(FRIGATE_EVENTS_TOPIC, qos=0) client.on_connect = on_connect client.on_message = _on_mqtt_message @@ -627,6 +719,7 @@ class Handler(BaseHTTPRequestHandler): self._respond(HTTPStatus.UNAUTHORIZED, {"error": "missing or invalid bearer token"}) return path = urlsplit(self.path).path + chore_settings_match = re.match(r"^/people/(\d+)/chore-settings$", path) if path == "/register/photo": self._handle_register_photo() elif path == "/register": @@ -635,6 +728,8 @@ class Handler(BaseHTTPRequestHandler): self._handle_register_guest() elif path == "/presence/manual": self._handle_presence_manual() + elif chore_settings_match: + self._handle_chore_settings(int(chore_settings_match.group(1))) else: self._respond(HTTPStatus.NOT_FOUND, {"error": "no such endpoint"}) @@ -751,6 +846,25 @@ class Handler(BaseHTTPRequestHandler): ok = set_manual_presence(person_id, home) self._respond(HTTPStatus.OK if ok else HTTPStatus.NOT_FOUND, {"ok": ok}) + def _handle_chore_settings(self, person_id: int) -> None: + try: + raw = self._read_body(MAX_JSON_BYTES) + payload = json.loads(raw or b"{}") + except (ValueError, json.JSONDecodeError) as exc: + self._respond(HTTPStatus.BAD_REQUEST, {"error": f"bad request body: {exc}"}) + return + + chore_exempt = payload.get("chore_exempt") + if chore_exempt is not None: + chore_exempt = bool(chore_exempt) + reminder_style = payload.get("chore_reminder_style") + if reminder_style is not None and not isinstance(reminder_style, str): + self._respond(HTTPStatus.BAD_REQUEST, {"error": "'chore_reminder_style' must be a string"}) + return + + ok = set_chore_settings(person_id, chore_exempt, reminder_style) + self._respond(HTTPStatus.OK if ok else HTTPStatus.NOT_FOUND, {"ok": ok}) + def main() -> int: logging.basicConfig( diff --git a/transit/Dockerfile b/transit/Dockerfile new file mode 100644 index 0000000..6460155 --- /dev/null +++ b/transit/Dockerfile @@ -0,0 +1,17 @@ +# transit — one image, two entrypoints: the always-on departures API (server.py, +# the default CMD) and the weekly GTFS refresh (sync_gtfs.py, run via +# `docker compose run --rm transit-sync` from a systemd timer, same two-entrypoint- +# one-image shape as nothing else in this repo needs, but the straightforward one). +FROM python:3.11-slim + +ENV PYTHONUNBUFFERED=1 \ + PYTHONDONTWRITEBYTECODE=1 + +WORKDIR /app + +# stdlib only (csv, zipfile, sqlite3, http.server, urllib) — no requirements.txt. +COPY server.py sync_gtfs.py ./ + +RUN mkdir -p /data + +CMD ["python", "server.py"] diff --git a/transit/README.md b/transit/README.md new file mode 100644 index 0000000..3dc28ff --- /dev/null +++ b/transit/README.md @@ -0,0 +1,139 @@ +# transit + +"When's the next bus?" via voice, from [Phase 19 of the project plan](../docs/project-plan.md). + +Not a trip planner — no transfers, no routing. Just: for one of the household's own +configured stops (Kennelbach's own stop(s), by default), what's leaving next. Data +comes from Vorarlberg's own published GTFS static schedule (VVV/VMOBIL), sourced via +Austria's national aggregator at mobilitaetsdaten.gv.at — real, existing +infrastructure this repo reads from, not something it generates. + +- **`sync_gtfs.py`** — oneshot, weekly timer (`ENABLE_TRANSIT` in + `setup-container-host.sh`). Downloads the GTFS zip, keeps only the + stops/routes/trips near the household's configured stop(s) + (`GTFS_STOP_NAMES`), writes a small local SQLite DB. A full regional GTFS feed is + large and mostly irrelevant to one household; filtering at sync time is what + keeps lookups fast. +- **`server.py`** — always-on, bearer-token gated, `GET /departures?stop=` + returns the next few departures computed from `calendar.txt`/`calendar_dates.txt` + (which services run today) joined against `stop_times.txt` for the matched + stop(s). **Published**, like `identity`/`pantry-vision` — `homeassistant` runs + `network_mode: host` in this stack and can't resolve plain container DNS names, + so a `rest_command` needs a real published port to reach this on, same reasoning + as `identity/identity.env.example`'s note about `HA_URL`. + +## Route planning scope — "Austria, possibly global" has a real cost + +`/plan` doesn't do any journey-planning itself — it proxies straight through to a +self-hosted **[OpenTripPlanner](https://www.opentripplanner.org/)** (OTP) instance, +because real A-to-B routing (transfers, walking legs, multi-modal itineraries) is a +mature, well-studied problem with good open-source engines already solving it, and +hand-rolling one here would be a bad trade. + +**What OTP actually needs to answer a query is data loaded into its "graph": an OSM +map extract for the road/path network, plus every GTFS feed for the transit systems +you want it to route through.** That's the real scoping decision, not a code +feature: + +- **Austria-wide** — one Austria OSM extract (a few hundred MB, e.g. from + [Geofabrik](https://download.geofabrik.de/europe/austria.html)) plus the GTFS + feeds for however many of Austria's regional operators you want covered + (`mobilitaetsdaten.gv.at`'s catalog lists them all, not just VVV) — a realistic, + moderate self-hosting commitment (single-digit GB of graph data, a build that + takes minutes, RAM in the low single-digit GB range for OTP itself). +- **Actually global** — every country's OSM data (the full planet extract is + **~80+ GB** compressed) plus GTFS for transit systems worldwide that publish one + at all (many don't, or only via a paid/restricted feed) — this is not a + "flip a flag" step up from Austria-wide, it's a full self-hosted mapping + infrastructure project with real disk/RAM/build-time cost, and results would still + be missing any transit system that has no public GTFS feed. **Start with Austria + (or Austria + neighboring countries if cross-border trips matter), and treat + "global" as a real future infrastructure decision, not a config value** — see the + open decision in `docs/project-plan.md` §4. + +`sync_gtfs.py`'s own filtered SQLite DB (for `/departures`) and OTP's graph (for +`/plan`) are two **completely separate data pipelines**, on purpose — one small and +household-specific, one general-purpose and much larger. Nothing here builds or +manages OTP's graph-build step; that's a one-time (re-run when you update the +OSM/GTFS inputs) manual operation — see OTP's own docs for the current build command +for whichever version you deploy. + +## Voice: "when's the next bus" + +**Nothing under this repo builds the HA-side custom-sentence/intent-script/ +`rest_command` wiring** — same convention as `identity`'s voice registration and +every other HA integration point in this project. + +```yaml +# configuration.yaml (excerpt) — worked example, unverified against a real instance. +intent_script: + NextDeparture: + speech: + text: > + {% if reg_result.content.departures %} + Next from {{ reg_result.content.stop }}: {{ reg_result.content.departures[0].route }} + at {{ reg_result.content.departures[0].time }}. + {% else %} + I couldn't find a departure for {{ stop | default('your stop') }}. + {% endif %} + action: + - service: rest_command.transit_departures + data: + stop: "{{ stop | default('') }}" + response_variable: reg_result + +rest_command: + transit_departures: + url: "http://127.0.0.1:8099/departures?stop={{ stop | urlencode }}" + method: GET + headers: + Authorization: "Bearer !secret transit_token" +``` + +Plus a custom sentence (`"when's the next bus"` / `"when's the next bus from +{stop}"`) mapping to `NextDeparture` — see +[HA's custom sentences docs](https://www.home-assistant.io/voice_control/custom_sentences/). +`stop` is optional in the sentence; `TRANSIT_DEFAULT_STOP` covers the bare "when's +the next bus" case. + +## Configure + +```sh +cp transit/transit.env.example /opt/smart-home/transit/transit.env +openssl rand -hex 32 # TRANSIT_TOKEN +chmod 600 /opt/smart-home/transit/transit.env +$EDITOR /opt/smart-home/transit/transit.env +``` + +`GTFS_FEED_URL` is the one thing you have to go get by hand — see the template's own +comment for where. Then run the first sync: + +```sh +cd /opt/smart-home && docker compose run --rm transit-sync +``` + +Check the log for which stops matched `GTFS_STOP_NAMES` — if it says none matched, +the feed's own `stop_name` spelling differs from what you guessed; `GET /stops` +(once any sync has run, even a differently-filtered one) lists what's actually in +the feed near a broader search term. + +## Manual verification still outstanding + +1. **The exact `GTFS_FEED_URL` for VVV/VMOBIL** — confirmed the provider and the + national aggregator exist (`docs/project-plan.md` §1.17's research note), not the + literal `.zip` download URL, which the household has to get from the aggregator's + own catalog page. +2. **GTFS_STOP_NAMES="Kennelbach"** — a reasonable guess at how the feed names the + local stop(s), not confirmed against the real `stops.txt`. +3. Post-midnight trips: GTFS allows `departure_time` past `24:00:00` for a service + day's late-night trips (e.g. `25:30:00` for 1:30 AM the next calendar day). + `server.py`'s string comparison against the current `HH:MM:SS` handles same-day + departures correctly but does **not** special-case this — a departure logged as + `25:30:00` will never match a same-day query after midnight has actually passed. + Rare for a village bus stop, not fixed here. +4. Whether VVV's feed includes GTFS-realtime (delays/cancellations) — this only + reads static schedule data; if a realtime feed exists it isn't consumed, so + answers are "scheduled," not "actual." +5. Whether `mobilitaetsdaten.gv.at`'s feed structure matches plain GTFS exactly (all + the standard files, no unusual encoding) — parsed against the documented GTFS + spec, not a downloaded copy of this specific feed. diff --git a/transit/server.py b/transit/server.py new file mode 100644 index 0000000..3a43e60 --- /dev/null +++ b/transit/server.py @@ -0,0 +1,294 @@ +"""transit — "when's the next bus/train" and on-demand route planning for voice +conversation, from docs/project-plan.md Phase 19. + +Two different jobs, kept as two different data paths on purpose: + - /departures — next-departure lookups from the small SQLite DB `sync_gtfs.py` + refreshes weekly (this repo's own code, see that file). + - /plan — real A-to-B route planning, proxied straight through to a self-hosted + **OpenTripPlanner** (OTP) instance, NOT reimplemented here. Journey planning + (transfers, walking legs, multi-modal routing) is a genuinely hard, well-studied + problem with mature open-source engines already solving it; hand-rolling one + would be a bad trade against just running OTP. See "Route planning scope" in + README.md for what "Austria, possibly global" actually costs to run. + +Published (like pantry-vision/identity), not compose-network-only (like +admin-canvas) — reachable at a fixed host port so Home Assistant's `rest_command` +can reach it regardless of HA's own networking mode (this stack's `homeassistant` +container runs `network_mode: host`, which cannot resolve plain container DNS names +— see identity/identity.env.example's identical note). Bearer-token gated the same +way every other published service in this project is. +""" + +from __future__ import annotations + +import json +import logging +import os +import sqlite3 +import sys +import urllib.error +import urllib.request +from datetime import datetime +from http import HTTPStatus +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +from pathlib import Path +from urllib.parse import parse_qs, urlsplit + +LOG = logging.getLogger("transit") + +TOKEN = os.environ.get("TRANSIT_TOKEN", "") +DB_PATH = Path(os.environ.get("TRANSIT_DB_PATH", "/data/transit.db")) +DEFAULT_STOP = os.environ.get("TRANSIT_DEFAULT_STOP", "") + +# OpenTripPlanner — optional, separate opt-in from the /departures path above (see +# ENABLE_TRIP_PLANNING in transit.env.example and README.md's "Route planning +# scope" section for why: a real OTP graph is a meaningfully bigger data/hardware +# commitment than the small filtered GTFS DB /departures uses). GraphQL path is +# OTP2's documented default — VERIFY against whichever OTP version you actually run; +# OTP1 uses a different REST shape entirely. +OTP_URL = os.environ.get("OTP_URL", "").rstrip("/") +OTP_GRAPHQL_PATH = os.environ.get("OTP_GRAPHQL_PATH", "/otp/gtfs/v1") + + +def _db() -> sqlite3.Connection: + conn = sqlite3.connect(DB_PATH) + conn.row_factory = sqlite3.Row + return conn + + +def _active_service_ids(conn: sqlite3.Connection, today: datetime) -> set[str]: + weekday_col = ["monday", "tuesday", "wednesday", "thursday", "friday", "saturday", "sunday"][today.weekday()] + date_str = today.strftime("%Y%m%d") + rows = conn.execute( + f"SELECT service_id FROM calendar WHERE {weekday_col} = 1 AND start_date <= ? AND end_date >= ?", + (date_str, date_str), + ).fetchall() + active = {r["service_id"] for r in rows} + + for row in conn.execute("SELECT service_id, exception_type FROM calendar_dates WHERE date = ?", (date_str,)): + if row["exception_type"] == 1: + active.add(row["service_id"]) + elif row["exception_type"] == 2: + active.discard(row["service_id"]) + return active + + +def departures(stop_query: str, limit: int) -> dict: + if not DB_PATH.exists(): + return {"error": "no GTFS data synced yet — run sync_gtfs.py first"} + + conn = _db() + try: + matched = conn.execute( + "SELECT stop_id, stop_name FROM stops WHERE lower(stop_name) LIKE ?", + (f"%{stop_query.lower()}%",), + ).fetchall() + if not matched: + return {"error": f"no known stop matches {stop_query!r}", "stop": stop_query} + + stop_ids = [m["stop_id"] for m in matched] + now = datetime.now() + active_services = _active_service_ids(conn, now) + if not active_services: + return {"stop": matched[0]["stop_name"], "departures": []} + + now_str = now.strftime("%H:%M:%S") + placeholders_stops = ",".join("?" * len(stop_ids)) + placeholders_services = ",".join("?" * len(active_services)) + rows = conn.execute( + f""" + SELECT st.departure_time, r.short_name, r.long_name, t.headsign + FROM stop_times st + JOIN trips t ON t.trip_id = st.trip_id + JOIN routes r ON r.route_id = t.route_id + WHERE st.stop_id IN ({placeholders_stops}) + AND t.service_id IN ({placeholders_services}) + AND st.departure_time >= ? + ORDER BY st.departure_time + LIMIT ? + """, + (*stop_ids, *active_services, now_str, limit), + ).fetchall() + + return { + "stop": matched[0]["stop_name"], + "departures": [ + { + "time": r["departure_time"], + "route": r["short_name"] or r["long_name"], + "headsign": r["headsign"], + } + for r in rows + ], + } + finally: + conn.close() + + +def list_stops() -> dict: + if not DB_PATH.exists(): + return {"stops": []} + conn = _db() + try: + rows = conn.execute("SELECT DISTINCT stop_name FROM stops ORDER BY stop_name").fetchall() + return {"stops": [r["stop_name"] for r in rows]} + finally: + conn.close() + + +# A minimal, documented-shape GraphQL query for OTP2's /otp/gtfs/v1 endpoint — asks +# for the first itinerary only (a voice answer wants "the way there," not five +# alternatives) with each leg's mode, route, and duration. VERIFY: written against +# OTP2's published schema, never run against a real instance — see README.md. +_OTP_PLAN_QUERY = """ +query Plan($from: String!, $to: String!, $walkSpeed: Float!) { + plan( + fromPlace: $from + toPlace: $to + numItineraries: 1 + walkSpeed: $walkSpeed + ) { + itineraries { + duration + legs { + mode + route { shortName } + from { name } + to { name } + duration + } + } + } +} +""" + +# OTP's own default is ~1.33 m/s (~4.8 km/h, a brisk adult pace) — deliberately +# slower here (~3.2 km/h) so walking-leg durations and connection feasibility (does +# a transfer's walking leg actually make the next departure?) reflect an unhurried +# real walking pace rather than a fit commuter's, per household preference. +# Override with a real per-person value if this still doesn't match reality. +WALK_SPEED_MPS = float(os.environ.get("WALK_SPEED_MPS", "0.9")) + + +def plan_trip(from_place: str, to_place: str) -> dict: + """`from_place`/`to_place` are OTP's own "lat,lon" or geocoded-name format — + passed through as given, not resolved/geocoded here. Degrades to a clear error + (never a stack trace) if OTP isn't configured or unreachable, same "degrade, + don't blank" rule as every other renderer in this project. + """ + if not OTP_URL: + return {"error": "trip planning is not configured (OTP_URL unset) — see README.md"} + + payload = json.dumps( + { + "query": _OTP_PLAN_QUERY, + "variables": {"from": from_place, "to": to_place, "walkSpeed": WALK_SPEED_MPS}, + } + ).encode("utf-8") + req = urllib.request.Request( + f"{OTP_URL}{OTP_GRAPHQL_PATH}", data=payload, method="POST", + headers={"Content-Type": "application/json"}, + ) + try: + with urllib.request.urlopen(req, timeout=30) as resp: + result = json.loads(resp.read()) + except (urllib.error.URLError, urllib.error.HTTPError, TimeoutError): + LOG.warning("transit: OTP request failed", exc_info=True) + return {"error": f"could not reach the trip planner at {OTP_URL}"} + except ValueError: + LOG.warning("transit: OTP returned non-JSON", exc_info=True) + return {"error": "trip planner returned an unexpected response"} + + itineraries = (result.get("data") or {}).get("plan", {}).get("itineraries") or [] + if not itineraries: + return {"from": from_place, "to": to_place, "found": False} + + best = itineraries[0] + return { + "from": from_place, + "to": to_place, + "found": True, + "duration_minutes": round(best.get("duration", 0) / 60), + "legs": [ + { + "mode": leg.get("mode"), + "route": (leg.get("route") or {}).get("shortName"), + "from": (leg.get("from") or {}).get("name"), + "to": (leg.get("to") or {}).get("name"), + "duration_minutes": round((leg.get("duration") or 0) / 60), + } + for leg in best.get("legs", []) + ], + } + + +class Handler(BaseHTTPRequestHandler): + server_version = "transit/1" + + def log_message(self, format, *args): # noqa: A002 + LOG.info("%s - %s", self.address_string(), format % args) + + def _authorized(self) -> bool: + if not TOKEN: + return False + return self.headers.get("Authorization", "") == f"Bearer {TOKEN}" + + def _respond(self, status: HTTPStatus, payload) -> None: + body = json.dumps(payload).encode("utf-8") + self.send_response(status) + self.send_header("Content-Type", "application/json") + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + + def do_GET(self): # noqa: N802 + if not self._authorized(): + self._respond(HTTPStatus.UNAUTHORIZED, {"error": "missing or invalid bearer token"}) + return + + split = urlsplit(self.path) + params = parse_qs(split.query) + + if split.path == "/departures": + stop = (params.get("stop", [DEFAULT_STOP])[0] or "").strip() + if not stop: + self._respond(HTTPStatus.BAD_REQUEST, {"error": "'stop' is required (or set TRANSIT_DEFAULT_STOP)"}) + return + try: + limit = min(int(params.get("limit", ["5"])[0]), 20) + except ValueError: + limit = 5 + self._respond(HTTPStatus.OK, departures(stop, limit)) + elif split.path == "/stops": + self._respond(HTTPStatus.OK, list_stops()) + elif split.path == "/plan": + from_place = (params.get("from", [""])[0] or "").strip() + to_place = (params.get("to", [""])[0] or "").strip() + if not from_place or not to_place: + self._respond(HTTPStatus.BAD_REQUEST, {"error": "'from' and 'to' are both required"}) + return + self._respond(HTTPStatus.OK, plan_trip(from_place, to_place)) + else: + self._respond(HTTPStatus.NOT_FOUND, {"error": "no such endpoint"}) + + +def main() -> int: + logging.basicConfig( + level=os.environ.get("LOG_LEVEL", "INFO").upper(), + format="%(asctime)s %(levelname)-7s %(name)s: %(message)s", + ) + if not TOKEN: + LOG.error("TRANSIT_TOKEN is not set — every request will be rejected until it is.") + + port = int(os.environ.get("TRANSIT_PORT", "8099")) + server = ThreadingHTTPServer(("0.0.0.0", port), Handler) + LOG.info("transit listening on :%d (db: %s)", port, DB_PATH) + try: + server.serve_forever() + except KeyboardInterrupt: + pass + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/transit/sync_gtfs.py b/transit/sync_gtfs.py new file mode 100644 index 0000000..478c564 --- /dev/null +++ b/transit/sync_gtfs.py @@ -0,0 +1,183 @@ +"""transit's GTFS static-schedule refresh, from docs/project-plan.md Phase 19. + +Downloads Vorarlberg's own published GTFS static feed (VVV/VMOBIL, sourced via +Austria's national aggregator at mobilitaetsdaten.gv.at — real, existing +infrastructure, not something this repo generates) and keeps only the +stops/routes/trips relevant to the household's configured stop(s), in a small local +SQLite DB `server.py` queries. A full national/regional GTFS zip is large and mostly +irrelevant to one household; filtering at sync time is what keeps `server.py`'s +lookups fast and the dataset small, not a general-purpose transit database. + +Oneshot + systemd timer (weekly — GTFS schedules are published a season/year at a +time, not daily), same shape as digest-engine/trash-calendar. +""" + +from __future__ import annotations + +import csv +import io +import logging +import os +import sqlite3 +import sys +import urllib.request +import zipfile +from pathlib import Path + +LOG = logging.getLogger("transit-sync") + +DB_PATH = Path(os.environ.get("TRANSIT_DB_PATH", "/data/transit.db")) + + +def _stop_names() -> list[str]: + return [s.strip().lower() for s in os.environ.get("GTFS_STOP_NAMES", "").split(",") if s.strip()] + + +def _read_csv(zf: zipfile.ZipFile, name: str) -> list[dict]: + try: + with zf.open(name) as fh: + text = io.TextIOWrapper(fh, encoding="utf-8-sig") + return list(csv.DictReader(text)) + except KeyError: + LOG.warning("transit-sync: %s not present in the GTFS feed, skipping", name) + return [] + + +def init_db(conn: sqlite3.Connection) -> None: + conn.executescript( + """ + DROP TABLE IF EXISTS stops; + DROP TABLE IF EXISTS routes; + DROP TABLE IF EXISTS trips; + DROP TABLE IF EXISTS stop_times; + DROP TABLE IF EXISTS calendar; + DROP TABLE IF EXISTS calendar_dates; + CREATE TABLE stops (stop_id TEXT PRIMARY KEY, stop_name TEXT); + CREATE TABLE routes (route_id TEXT PRIMARY KEY, short_name TEXT, long_name TEXT); + CREATE TABLE trips (trip_id TEXT PRIMARY KEY, route_id TEXT, service_id TEXT, headsign TEXT); + CREATE TABLE stop_times (trip_id TEXT, stop_id TEXT, departure_time TEXT); + CREATE TABLE calendar (service_id TEXT PRIMARY KEY, monday INT, tuesday INT, wednesday INT, + thursday INT, friday INT, saturday INT, sunday INT, start_date TEXT, end_date TEXT); + CREATE TABLE calendar_dates (service_id TEXT, date TEXT, exception_type INT); + CREATE INDEX idx_stop_times_stop ON stop_times(stop_id); + CREATE INDEX idx_trips_service ON trips(service_id); + """ + ) + + +def sync() -> int: + feed_url = os.environ.get("GTFS_FEED_URL", "").strip() + if not feed_url: + LOG.error("transit-sync: GTFS_FEED_URL is not set — see transit.env.example and README.md") + return 1 + + wanted_names = _stop_names() + if not wanted_names: + LOG.error("transit-sync: GTFS_STOP_NAMES is not set — nothing to filter down to") + return 1 + + LOG.info("transit-sync: downloading %s", feed_url) + try: + req = urllib.request.Request(feed_url, headers={"User-Agent": "smartesthome-transit/1"}) + with urllib.request.urlopen(req, timeout=120) as resp: + data = resp.read() + except Exception: + LOG.error("transit-sync: could not download GTFS_FEED_URL", exc_info=True) + return 1 + + try: + zf = zipfile.ZipFile(io.BytesIO(data)) + except zipfile.BadZipFile: + LOG.error("transit-sync: downloaded file is not a valid GTFS zip") + return 1 + + all_stops = _read_csv(zf, "stops.txt") + matched_stops = [ + s for s in all_stops + if any(name in (s.get("stop_name") or "").lower() for name in wanted_names) + ] + if not matched_stops: + LOG.error( + "transit-sync: none of GTFS_STOP_NAMES (%s) matched any stop_name in the feed — " + "check spelling against the feed's own stops.txt", + ", ".join(wanted_names), + ) + return 1 + matched_stop_ids = {s["stop_id"] for s in matched_stops} + LOG.info("transit-sync: matched %d stop(s): %s", len(matched_stops), [s["stop_name"] for s in matched_stops]) + + all_stop_times = _read_csv(zf, "stop_times.txt") + relevant_stop_times = [st for st in all_stop_times if st.get("stop_id") in matched_stop_ids] + relevant_trip_ids = {st["trip_id"] for st in relevant_stop_times} + + all_trips = _read_csv(zf, "trips.txt") + relevant_trips = [t for t in all_trips if t.get("trip_id") in relevant_trip_ids] + relevant_route_ids = {t["route_id"] for t in relevant_trips} + relevant_service_ids = {t["service_id"] for t in relevant_trips} + + all_routes = _read_csv(zf, "routes.txt") + relevant_routes = [r for r in all_routes if r.get("route_id") in relevant_route_ids] + + all_calendar = _read_csv(zf, "calendar.txt") + relevant_calendar = [c for c in all_calendar if c.get("service_id") in relevant_service_ids] + + all_calendar_dates = _read_csv(zf, "calendar_dates.txt") + relevant_calendar_dates = [c for c in all_calendar_dates if c.get("service_id") in relevant_service_ids] + + DB_PATH.parent.mkdir(parents=True, exist_ok=True) + conn = sqlite3.connect(DB_PATH) + try: + init_db(conn) + conn.executemany( + "INSERT OR REPLACE INTO stops VALUES (?, ?)", + [(s["stop_id"], s.get("stop_name", "")) for s in matched_stops], + ) + conn.executemany( + "INSERT OR REPLACE INTO routes VALUES (?, ?, ?)", + [(r["route_id"], r.get("route_short_name", ""), r.get("route_long_name", "")) for r in relevant_routes], + ) + conn.executemany( + "INSERT OR REPLACE INTO trips VALUES (?, ?, ?, ?)", + [(t["trip_id"], t.get("route_id", ""), t.get("service_id", ""), t.get("trip_headsign", "")) for t in relevant_trips], + ) + conn.executemany( + "INSERT INTO stop_times VALUES (?, ?, ?)", + [(st["trip_id"], st["stop_id"], st.get("departure_time", "")) for st in relevant_stop_times], + ) + conn.executemany( + "INSERT OR REPLACE INTO calendar VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)", + [ + ( + c["service_id"], int(c.get("monday", 0) or 0), int(c.get("tuesday", 0) or 0), + int(c.get("wednesday", 0) or 0), int(c.get("thursday", 0) or 0), + int(c.get("friday", 0) or 0), int(c.get("saturday", 0) or 0), + int(c.get("sunday", 0) or 0), c.get("start_date", ""), c.get("end_date", ""), + ) + for c in relevant_calendar + ], + ) + conn.executemany( + "INSERT INTO calendar_dates VALUES (?, ?, ?)", + [(c["service_id"], c.get("date", ""), int(c.get("exception_type", 0) or 0)) for c in relevant_calendar_dates], + ) + conn.commit() + finally: + conn.close() + + LOG.info( + "transit-sync: stored %d stop_times across %d trips for %d stop(s)", + len(relevant_stop_times), len(relevant_trips), len(matched_stops), + ) + return 0 + + +def main() -> int: + logging.basicConfig( + level=os.environ.get("LOG_LEVEL", "INFO").upper(), + format="%(asctime)s %(levelname)-7s %(name)s: %(message)s", + ) + return sync() + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/transit/transit.env.example b/transit/transit.env.example new file mode 100644 index 0000000..c1e58d2 --- /dev/null +++ b/transit/transit.env.example @@ -0,0 +1,56 @@ +# transit configuration template. +# +# Copy this to the container host as (for example) +# /opt/smart-home/transit/transit.env, fill in real values, and chmod 600 it. + +# --------------------------------------------------------------------------- +# Auth — required. Fails closed while empty, same as every other published +# service in this project. Also goes into HA's rest_command config — see +# README.md's worked voice-intent example. +# --------------------------------------------------------------------------- +TRANSIT_TOKEN= + +# --------------------------------------------------------------------------- +# GTFS source — Vorarlberg's own published static schedule (VVV/VMOBIL), via +# Austria's national aggregator. Get the real feed URL from: +# https://mobilitaetsdaten.gv.at/en/daten/soll-fahrplandaten-gtfs +# https://mobilitaetsdaten.gv.at/en/node/308 (the VVV/VMOBIL provider page) +# Not a fixed URL this repo can hardcode — the catalog links to per-provider +# downloads that can change. Paste the real one here once you have it. +# --------------------------------------------------------------------------- +GTFS_FEED_URL= + +# Comma-separated, case-insensitive SUBSTRINGS matched against the feed's own +# stop_name field — e.g. "Kennelbach" matches every stop with that in its name +# (there may be more than one, different directions/platforms). sync_gtfs.py logs +# exactly which stops matched on every run; check that log the first time. +GTFS_STOP_NAMES=Kennelbach + +# --------------------------------------------------------------------------- +# Route planning (optional, separate from the GTFS departures lookup above) — a +# self-hosted OpenTripPlanner instance this service proxies /plan requests to. +# Leave OTP_URL blank to skip trip planning entirely; /departures works either way. +# See README.md's "Route planning scope" section before turning this on — a real +# OTP graph (OSM + GTFS) is a meaningfully bigger data/hardware commitment than the +# small filtered GTFS DB /departures uses, and "how much of the world" is a real +# storage/RAM tradeoff, not a config toggle. +# --------------------------------------------------------------------------- +OTP_URL=http://otp:8080 +# OTP2's documented default GraphQL path — VERIFY against whichever OTP version you +# actually deploy; OTP1 uses a different REST API shape entirely. +OTP_GRAPHQL_PATH=/otp/gtfs/v1 +# Meters/second. OTP's own default is ~1.33 (a brisk ~4.8 km/h adult pace); +# deliberately slower here (~3.2 km/h) by household preference — override with a +# real per-person value if this still doesn't match reality. +WALK_SPEED_MPS=0.9 + +# --------------------------------------------------------------------------- +# Run behaviour +# --------------------------------------------------------------------------- +TRANSIT_PORT=8099 +TRANSIT_DB_PATH=/data/transit.db +# Used when a voice query doesn't name a stop at all ("when's the next bus" with no +# stop specified) — leave blank to require the stop be named every time. +TRANSIT_DEFAULT_STOP=Kennelbach + +LOG_LEVEL=INFO diff --git a/trash-calendar/Dockerfile b/trash-calendar/Dockerfile new file mode 100644 index 0000000..72ecf79 --- /dev/null +++ b/trash-calendar/Dockerfile @@ -0,0 +1,15 @@ +# trash-calendar — oneshot, driven by a systemd timer (daily), same shape as +# digest-engine's own Dockerfile. +FROM python:3.11-slim + +ENV PYTHONUNBUFFERED=1 \ + PYTHONDONTWRITEBYTECODE=1 + +WORKDIR /app + +COPY requirements.txt ./ +RUN pip install --no-cache-dir -r requirements.txt + +COPY sync.py ./ + +CMD ["python", "sync.py"] diff --git a/trash-calendar/README.md b/trash-calendar/README.md new file mode 100644 index 0000000..d282199 --- /dev/null +++ b/trash-calendar/README.md @@ -0,0 +1,67 @@ +# trash-calendar + +Mirrors Kennelbach's trash/recycling collection dates into the household's own +Nextcloud calendar, from [Phase 19 of the project plan](../docs/project-plan.md). + +**Not a new trash-schedule data source** — Kennelbach's own site (kennelbach.at) and +Vorarlberg's Umweltverband (umweltv.at) already publish a personal, address-specific +ICS feed of your collection dates; this repo doesn't scrape or re-derive that. What's +missing without this tool is getting those dates into the **same** calendar the +household already looks at (the one `digest-engine/ingest/caldav.py` already reads +from) instead of a second app/URL nobody checks. + +## How it works + +1. Download the household's real personal ICS feed URL (see + `trash-calendar.env.example` for where to get it — it's address-specific, this + repo can't hardcode it). +2. `sync.py` downloads and parses that feed, keeps only events in the next + `WASTE_LOOKAHEAD_DAYS` (default 60). +3. For each one, it checks whether an event with a deterministic UID + (`smartesthome-trash-`) already exists in + `CALDAV_TARGET_CALENDAR`; if not, it creates it. **It never updates or deletes + anything** — a re-run that finds nothing new touches nothing at all, and it never + reads or writes any event outside its own `smartesthome-trash-` prefix. + +Oneshot + systemd timer (daily via `ENABLE_TRASH_CALENDAR` in +`setup-container-host.sh`), not an always-on service — same shape as +`digest-engine`'s own scheduling. + +## Why this isn't part of digest-engine + +`digest-engine/ingest/caldav.py` has one documented invariant: it never mutates the +calendar it reads from (see that file's own docstring, "READ-ONLY INVARIANT"). Adding +a write path into the same component would either break that invariant or need a +confusing second mode. A separate, smaller tool that reuses the exact same +`CALDAV_URL`/`CALDAV_USERNAME`/`CALDAV_PASSWORD`/`CALDAV_VERIFY_TLS` credentials (one +Nextcloud app password, not two) keeps the "never mutates" guarantee real for the +digest while still solving the actual problem. + +## Configure + +```sh +cp trash-calendar/trash-calendar.env.example /opt/smart-home/trash-calendar/trash-calendar.env +chmod 600 /opt/smart-home/trash-calendar/trash-calendar.env +$EDITOR /opt/smart-home/trash-calendar/trash-calendar.env +``` + +`WASTE_ICS_URL` and `CALDAV_TARGET_CALENDAR` are both required and both +household-specific — see the template's own comments for where to get each. + +## Manual verification still outstanding + +1. **The exact "download/subscribe" UI on kennelbach.at's/umweltv.at's Abfallkalender + page** — confirmed these pages exist and serve a personal calendar (see + `docs/project-plan.md` §1.17's research note), but the precise click-path to an + ICS URL wasn't captured here; get it once, it should be a fixed URL after that. +2. Whether `calendar.event_by_uid()` on a real Nextcloud instance behaves the way + the `caldav` library's docs describe (raises when not found — caught here as + "doesn't exist yet, create it") — this exact call is already used read-side by + nothing in this repo, only written fresh here; unverified against a live server. +3. Whether Nextcloud's CalDAV endpoint accepts a bare `save_event()` with the + minimal VEVENT built here (no `DTEND`, all-day via `DTSTART;VALUE=DATE`) without + complaint — a deliberately minimal event, not tested against a real server. +4. Timezone/date-boundary edge cases right around midnight — `date.today()` uses the + container's own clock/timezone; confirm the container host's `TZ` is set + correctly (`setup-container-host.sh`'s own `TIMEZONE` variable already threads + through to every other container, this one included). diff --git a/trash-calendar/requirements.txt b/trash-calendar/requirements.txt new file mode 100644 index 0000000..ceecc1b --- /dev/null +++ b/trash-calendar/requirements.txt @@ -0,0 +1,4 @@ +# Same two packages, same versions-of-record, as digest-engine/requirements.txt — +# see digest-engine/ingest/caldav.py's docstring for why these aren't hand-rolled. +caldav>=2.0 +icalendar>=5.0 diff --git a/trash-calendar/sync.py b/trash-calendar/sync.py new file mode 100644 index 0000000..d5cc828 --- /dev/null +++ b/trash-calendar/sync.py @@ -0,0 +1,181 @@ +"""trash-calendar — mirrors the household's trash/recycling collection dates into +the same Nextcloud calendar everything else already uses, from +docs/project-plan.md Phase 19. + +THE PROBLEM: Kennelbach's own Abfallkalender (kennelbach.at) and Vorarlberg's +Umweltverband (umweltv.at) both publish collection dates as a **personal ICS feed** +keyed to your street/house number — real, live infrastructure, not something this +repo re-implements. What's missing is getting those dates into the SAME calendar the +household already looks at (Nextcloud, via `digest-engine/ingest/caldav.py`'s +existing read path) instead of a second place nobody checks. + +Oneshot script + systemd timer (daily), not an always-on service — same shape as +`digest-engine/run.py`, and deliberately not folded into digest-engine itself: this +writes to the calendar, digest-engine's own `ingest/caldav.py` is READ-ONLY by +explicit invariant (see that file's docstring), and mixing a write path into a +component whose one documented guarantee is "never mutates" is exactly the kind of +scope creep worth a separate, smaller tool instead. + +CREDENTIALS: reuses digest-engine's own CALDAV_URL/CALDAV_USERNAME/CALDAV_PASSWORD/ +CALDAV_VERIFY_TLS names and the same "Nextcloud app password, not the account +password" reasoning (see `digest-engine/ingest/caldav.py`'s docstring) — one +Nextcloud app password, shared by both the read and write paths, not two separate +credentials to manage. `CALDAV_TARGET_CALENDAR` is new: which calendar (by display +name) this writes into — unlike the read path, which can read several, this writes +to exactly one, on purpose (never guess which of several calendars a light +household chore belongs in). + +OWNERSHIP INVARIANT: every event this script creates gets a UID prefixed +`smartesthome-trash-`, deterministic from the source feed's own event content. It +only ever creates events under that prefix and only ever checks for their existence +before creating — it never reads, modifies, or deletes anything else in the target +calendar. A no-op re-run (nothing new in the source feed) touches nothing. +""" + +from __future__ import annotations + +import hashlib +import logging +import os +import sys +import urllib.request +from datetime import date, datetime, timedelta, timezone + +LOG = logging.getLogger("trash-calendar") + +UID_PREFIX = "smartesthome-trash-" +DEFAULT_LOOKAHEAD_DAYS = 60 + + +def _stable_uid(summary: str, start: date) -> str: + digest = hashlib.sha1(f"{summary}|{start.isoformat()}".encode("utf-8")).hexdigest()[:16] + return f"{UID_PREFIX}{digest}@smartesthome" + + +def fetch_source_events(ics_url: str, lookahead_days: int) -> list[dict]: + """Downloads and parses the household's personal collection-date ICS feed. + Degrades to an empty list (never raises past this point) if the feed is + unreachable or malformed — a missed sync run just means tomorrow's run tries + again, same "degrade, don't blank" rule as every renderer in this project. + """ + from icalendar import Calendar as ICalendar # see module docstring: caldav.py's identical import-inside-function reasoning + + try: + req = urllib.request.Request(ics_url, headers={"User-Agent": "smartesthome-trash-calendar/1"}) + with urllib.request.urlopen(req, timeout=30) as resp: + raw = resp.read() + except Exception: + LOG.warning("trash-calendar: could not fetch WASTE_ICS_URL", exc_info=True) + return [] + + today = date.today() + horizon = today + timedelta(days=lookahead_days) + + events = [] + try: + for component in ICalendar.from_ical(raw).walk("VEVENT"): + dtstart = component.get("dtstart") + if dtstart is None: + continue + start = dtstart.dt + start_date = start.date() if isinstance(start, datetime) else start + if not (today <= start_date <= horizon): + continue + summary = str(component.get("summary") or "Müllabfuhr").strip() + events.append({"summary": summary, "start": start_date}) + except Exception: + LOG.warning("trash-calendar: could not parse WASTE_ICS_URL as iCalendar", exc_info=True) + return [] + + LOG.info("trash-calendar: %d collection date(s) in the next %d days", len(events), lookahead_days) + return events + + +def sync_to_caldav(events: list[dict]) -> None: + url = os.environ.get("CALDAV_URL", "").strip() + username = os.environ.get("CALDAV_USERNAME", "").strip() + password = os.environ.get("CALDAV_PASSWORD", "") + target_name = os.environ.get("CALDAV_TARGET_CALENDAR", "").strip() + + if not (url and username and password and target_name): + LOG.error( + "trash-calendar: CALDAV_URL/CALDAV_USERNAME/CALDAV_PASSWORD/CALDAV_TARGET_CALENDAR " + "must all be set — see trash-calendar.env.example" + ) + return + + verify = os.environ.get("CALDAV_VERIFY_TLS", "true").strip().lower() == "true" + if not verify: + LOG.warning("trash-calendar: CALDAV_VERIFY_TLS is false, the app password is sent over an unverified session") + + import caldav as caldav_lib + + with caldav_lib.DAVClient(url=url, username=username, password=password, ssl_verify_cert=verify) as client: + target = None + for calendar in client.principal().calendars(): + try: + name = str(calendar.name or "") + except Exception: + continue + if name == target_name: + target = calendar + break + + if target is None: + LOG.error("trash-calendar: no calendar named %r found on this Nextcloud account", target_name) + return + + created = 0 + for event in events: + uid = _stable_uid(event["summary"], event["start"]) + try: + target.event_by_uid(uid) + continue # already exists — this script never updates, only creates (see module docstring) + except Exception: + pass # not found — fall through and create it + + ics = ( + "BEGIN:VCALENDAR\r\n" + "VERSION:2.0\r\n" + "PRODID:-//SmartestHome//trash-calendar//EN\r\n" + "BEGIN:VEVENT\r\n" + f"UID:{uid}\r\n" + f"DTSTAMP:{datetime.now(timezone.utc).strftime('%Y%m%dT%H%M%SZ')}\r\n" + f"DTSTART;VALUE=DATE:{event['start'].strftime('%Y%m%d')}\r\n" + f"SUMMARY:{event['summary']}\r\n" + "END:VEVENT\r\n" + "END:VCALENDAR\r\n" + ) + try: + target.save_event(ics) + created += 1 + except Exception: + LOG.warning("trash-calendar: could not create event for %s on %s", event["summary"], event["start"], exc_info=True) + + LOG.info("trash-calendar: created %d new event(s) in %r (existing ones left untouched)", created, target_name) + + +def main() -> int: + logging.basicConfig( + level=os.environ.get("LOG_LEVEL", "INFO").upper(), + format="%(asctime)s %(levelname)-7s %(name)s: %(message)s", + ) + + ics_url = os.environ.get("WASTE_ICS_URL", "").strip() + if not ics_url: + LOG.error("trash-calendar: WASTE_ICS_URL is not set — see trash-calendar.env.example and README.md") + return 1 + + lookahead_days = int(os.environ.get("WASTE_LOOKAHEAD_DAYS") or DEFAULT_LOOKAHEAD_DAYS) + + events = fetch_source_events(ics_url, lookahead_days) + if not events: + LOG.info("trash-calendar: nothing to sync this run") + return 0 + + sync_to_caldav(events) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/trash-calendar/trash-calendar.env.example b/trash-calendar/trash-calendar.env.example new file mode 100644 index 0000000..bda23c9 --- /dev/null +++ b/trash-calendar/trash-calendar.env.example @@ -0,0 +1,40 @@ +# trash-calendar configuration template. +# +# Copy this to the container host as (for example) +# /opt/smart-home/trash-calendar/trash-calendar.env, fill in real values, and +# chmod 600 it. Same never-commit handling as digest-engine.env / identity.env. + +# --------------------------------------------------------------------------- +# Your PERSONAL collection-date ICS feed — not a generic Kennelbach-wide URL, +# because collection days differ by street/collection zone. Get yours from: +# - Kennelbach's own Abfallkalender: https://www.kennelbach.at (Service -> +# Aktuelles -> Abfallkalender) — select your street/house number, look for a +# "download"/iCal/subscribe option on the resulting calendar view, or +# - Vorarlberg's Umweltverband: https://www.umweltv.at/Service_Info/Abfallkalender +# — same idea, region-wide, also county for Kennelbach +# Neither of the above is a fixed URL this repo can hardcode — it's generated +# per-address. Paste the real one here once you have it. +# --------------------------------------------------------------------------- +WASTE_ICS_URL= + +# How far into the future to sync. Collection calendars are usually published a +# year at a time; 60 days is enough to always have the next pickup or two visible +# without re-importing the entire year on every run. +WASTE_LOOKAHEAD_DAYS=60 + +# --------------------------------------------------------------------------- +# Nextcloud CalDAV — SAME credentials as digest-engine's own ingest/caldav.py +# (CALDAV_URL/CALDAV_USERNAME/CALDAV_PASSWORD/CALDAV_VERIFY_TLS), reused +# identically so there's one Nextcloud app password to manage, not two. See that +# module's docstring for the "app password, not your account password" reasoning. +# CALDAV_TARGET_CALENDAR is new here: the exact display name of the ONE calendar +# this writes into — deliberately singular, never guessed, never "whichever +# calendar looked right." +# --------------------------------------------------------------------------- +CALDAV_URL= +CALDAV_USERNAME= +CALDAV_PASSWORD= +CALDAV_VERIFY_TLS=true +CALDAV_TARGET_CALENDAR= + +LOG_LEVEL=INFO