SmartestHome/workshop/README.md

# workshop

The project notebook behind the workshop/office assistant — **step 1 of
[`docs/workshop-assistant.md`](../docs/workshop-assistant.md), and deliberately only
step 1.**

That note ranks four capabilities (conversational planning, spec research, guide
lookup, camera identification) and says to build this one first: it is the only one
with no perception problem in it, and it is where all the others land. An identified
mainboard is worth nothing until there is a project to attach it to.

## Two databases and a share

| | Holds | Lifetime |
|---|---|---|
| **`workshop.db`** | projects, notes, decisions — a record of **work** | scoped to a project; over when it's over |
| **`workshop-knowledge.db`** | workflow instructions, facts, project learnings, **hardware inventory** — a record of **what is true** | **forever. Nothing here is ever pruned** |
| **`/workspace`** (SMB) | datasheets, generated diagrams, photos, long-form notes | whatever you leave there |

Two files rather than four more tables because they have different lifetimes and
different blast radius: rebuilding the project store must not take the standing
instruction about how you solder with it.

**Nothing in the knowledge store expires.** Every other store in this stack has a
retention window — `doorway.py` keeps sightings 30 days, `digest-engine`'s archive 400
— because stale observations are worse than none. This one is the opposite kind of
data: the entire point is that you tell it once. A retention policy would be a policy
of forgetting the thing the feature exists to remember, so there is no cutoff and no
cleanup pass anywhere in `knowledge.py`. Corrections are per-row `DELETE`s — no
expiry does not mean nothing is ever wrong.

### The knowledge tables

- **`workflow_notes`** — retrieved by **activity**. "When I'm soldering, flux first."
  Activity `*` means always. Re-adding the same instruction *updates* rather than
  duplicating, since telling it twice is the exact problem the table solves.
- **`facts`** — retrieved by **keyword**. Device specs, URLs, part numbers. Re-adding
  the same subject overwrites: a spec you looked up again is usually one you had wrong.
- **`project_knowledge`** — retrieved by **project**, and deliberately not a foreign
  key into `workshop.db`. What you learned building the NAS stays true after the NAS
  is gone.
- **`hardware`** — what you own, where it is, and what it's promised to.

**`GET /context?activity=&q=&project=`** is what makes these a memory rather than four
lists: one call returns the standing instructions, the relevant facts, the project's
learnings and its hardware. An unknown activity still returns the always-notes —
"I don't know what you're doing" is no reason to forget how you like to work.

### There is exactly one inventory

An earlier cut had a `parts` table in the project store. It's gone. Two inventories is
the failure this repo's own docs keep warning about: the one you didn't update becomes
a lie, and you find out by buying a part you already own. What you *own* is knowledge;
what a project *needs* is a claim on it, which is `hardware.project_slug`.

**Four statuses, and the middle two are the point:**

| | Means | Can I use it right now? |
|---|---|---|
| `available` | on the shelf, unpromised | yes |
| `assigned` | **reserved** for a project, still on the shelf | yes, if you're willing to change your mind |
| `in_use` | **installed and working** somewhere | not without taking something apart |
| `retired` | dead, sold, given away | no — kept so "didn't I have one?" has an answer |

Naming a project *is* the assignment: `{"project_slug": "nas-build"}` moves an item to
`assigned` by itself, and clearing it releases to `available` — an item assigned to a
project while still reading `available` is how an inventory starts lying. **The one
exception: `in_use` is never silently downgraded by a re-assignment.** Saying which
project something is in must not turn a thing you'd have to unscrew into a thing the
list says you can just take.

### Why a share and not blobs in the database

Mixing them gets the worst of both: a database you can't browse and files nothing can
query. Keeping artefacts as files means **every one of them outlives this service** — a datasheet in the
share is a PDF you can open from a laptop with no API, no export step, and no
dependence on this project still existing in two years.

Consequently this service **never serves file contents**. `GET /projects/<slug>/files`
is a directory listing; opening the file is the share's job, and a second copy of the
bytes over HTTP would just be a second place for them to go stale.

## The workspace is the one writable share in this stack

The photo gallery share (`gallery-smb`) is `read only = yes` on purpose. This one
can't be — the assistant writes into it — and that difference is the whole security
story:

- **Its own volume**, not a subdirectory of anything else, so the blast radius of a
  bad path is "the workshop workspace" and is recoverable.
- **Its own SMB account.** Guest off, same as the gallery.
- **Nothing on the host ever executes anything found in it.** The moment it becomes a
  place for scripts, it is a different security question than this component answers.
- **Slugs are validated, never sanitised** (`SLUG_RE` in `server.py`). A project slug
  is a directory name on that share, which makes it the one input here with a
  filesystem consequence. Silently rewriting `../../etc` to `etc` would let two
  different requests write to one directory with nobody finding out, so a bad slug is
  a 400.

`_ensure_project_dir()` is best-effort: an unmounted or read-only share does **not**
stop a project being created, because the notebook is in SQLite and that is the part
that matters. Every response that touches the workspace carries `available`, so the
API says plainly whether the share is working instead of implying it by silence.

## Specs carry their source, or they are marked as not having one

`specs` always sits next to `source_url`, in both `hardware` and `facts`. This is the schema half of the
rule in the feasibility note:

> **Every spec must be quoted from a fetched document, with its URL, and never
> generated from the model's memory.**

A hallucinated TDP wastes an afternoon; a hallucinated pinout destroys hardware. A
spec you read off the part in your hand is a perfectly good source and is accepted
without a URL — it is just recorded as unsourced, so the UI can mark it and a later
research pass knows which specs still need a document behind them. **Never populate
`specs` from an LLM's memory.** That is not a style preference here, it is the reason
the column pair exists.

## Why decisions are their own table

"What did I decide last Tuesday, and why" is the question a project notebook exists to
answer, and a pile of notes answers it worst. A decision is a question, an answer, and
a *because* — the last one being the column the table is for, because an answer
without its reasoning is something you will re-litigate in three weeks.

## API

All endpoints are bearer-token gated; an unset `WORKSHOP_TOKEN` rejects everything.

| Endpoint | What it does |
|---|---|
| `GET /projects[?room=<area_id>]` | every project, or the ones belonging to a room |
| `POST /projects` | `{"name", "slug"?, "room"?, "summary"?}` → creates it and its workspace directories |
| `GET /projects/<slug>` | the project with its notes, decisions, durable knowledge and claimed hardware |
| `POST /projects/<slug>` | `{"name"?, "status"?, "summary"?, "room"?}` — status is `active`/`parked`/`done` |
| `GET /projects/<slug>/files` | directory listing of the workspace, never contents |
| `POST /projects/<slug>/notes` | `{"body", "author"?}` — `author` distinguishes what you wrote from what the assistant wrote |
| `POST /projects/<slug>/decisions` | `{"question", "answer", "because"?}` |
| `POST /projects/<slug>/knowledge` | `{"body", "keywords"?}` — a durable learning, not a log entry |
| `POST /projects/<slug>/repo` | `{"name"?, "description"?, "private"?}` → creates a Gitea repo, applies branch protection, sets up `code/` and records the URL |
| `GET /projects/<slug>/git` | branch, dirty files, last five commits |
| `POST /projects/<slug>/commit` | `{"message", "push"?}` — commit and push. Reported separately: a failed push is not a failed commit |
| `POST /projects/<slug>/branch` | `{"name"}` — create and switch. There is no delete counterpart, on purpose |
| `POST /projects/<slug>/scrub-request` | `{"path"?, "note"?}` → the manual commands to remove something from history. **Executes nothing** |
| `GET /hardware[?q=&status=&project=]` | the inventory |
| `POST /hardware` | `{"designation", "kind"?, "quantity"?, "storage_location"?, "status"?, "project_slug"?, "specs"?, "source_url"?, "notes"?}` |
| `POST /hardware/<id>` | edit anything, including assignment |
| `GET /knowledge/workflow[?activity=]` · `POST /knowledge/workflow` | standing instructions |
| `GET /knowledge/facts[?q=]` · `POST /knowledge/facts` | keyword-retrieved specifics |
| `GET /knowledge/projects[?project=&q=]` | durable project learnings |
| `GET /context[?activity=&q=&project=]` | **everything that applies right now, in one call** |
| `GET /health` | CheckMK + every firewall: current state, how long it's held, the problems |
| `GET /cameras` | network camera names + go2rtc stream ids |
| `GET /fleet` | every slot, its versions, and what each endpoint reported |
| `GET /fleet/script/<platform>` | the **published** script for a platform (what endpoints fetch) |
| `POST /fleet/upload` | `{"platform", "body", "note"?}` → a new **draft** version |
| `POST /fleet/publish` | `{"platform", "version"}` → make it the one endpoints run |
| `POST /fleet/report` | an endpoint saying what it ran and whether it worked |
| `DELETE /hardware/<id>`, `DELETE /knowledge/{workflow,facts,projects}/<id>` | corrections |

`parked` is not `done`: a shelved project whose parts are still allocated to it is
exactly what you want to find *before* buying those parts again.

`room` is an HA `area_id`, the same vocabulary as everywhere else in this project —
see [`docs/rooms-and-endpoints.md`](../docs/rooms-and-endpoints.md). It is what lets a
room-scoped assistant answer "what am I working on" with the projects of the room it
was asked in.

## Configure

```sh
cp workshop/workshop.env.example /opt/smart-home/workshop/workshop.env
openssl rand -hex 32   # WORKSHOP_TOKEN
chmod 600 /opt/smart-home/workshop/workshop.env
```

## The web interface

`frontend/` — a static page for **editing the hardware inventory by hand**: search,
filter by status or project, inline edit, add, delete. Same shape as
`pantry-vision/frontend` (vanilla JS, no build step, `?api=&token=` from the URL).

It edits the inventory and nothing else, on purpose. Every other table records
something that was *said or decided*, and those are only wrong if someone recorded
them wrong. The inventory describes **physical reality**, which drifts without telling
anybody — so it's the one table that needs somewhere a person can sit down and fix it.

Deliberately **not** the purple/magenta holo theme from the feasibility note: that's
for the canvas surfaces the assistant draws on, where the look is the point. This is a
form you correct data in, and a glow behind a text field is a cost with no benefit.

## Gitea

`POST /projects/<slug>/repo` creates a repository and records its URL on the project.
Needs `GITEA_URL` + `GITEA_TOKEN` (Settings → Applications → Generate New Token, scope
`write:repository`); refused with a clear message while unset.

### The assistant writes history. It never rewrites it.

**Allowed, unattended:** commit, push, branch, tag, merge — everything that *adds*.
The worst case is a bad commit you revert.

**Never, by any path:** force-push, rebase, amend, `reset --hard`, `filter-branch`,
`filter-repo`, branch or tag deletion, reflog expiry, `gc --prune`. Rewriting has no
undo, and the whole point is that a rollback is always feasible.

That refusal lives in `git_ops.py` as a deny-list checked on every git invocation —
but **client-side refusal is a policy, not a control.** The enforcement is Gitea
**branch protection**, applied automatically at repo creation (`enable_force_push:
false`, `enable_delete: false`), which rejects the push regardless of what asked for
it, including a human who typed `--force` out of habit. Check
`repo.protection.applied` in the create response; never assume it's on because the
repo exists.

**When history genuinely has to be scrubbed** — a committed API token —
`POST /projects/<slug>/scrub-request` returns the commands **for you to run by hand**
from the repo on the SMB share. It executes nothing. The manual step *is* the safety
mechanism: the person running `git filter-repo` can read it first and take a copy;
a service doing it on a timer cannot. The instructions lead with *rotate the secret
first*, because the commit was already pushed and a scrub that makes you feel finished
without rotating is worse than no scrub.

The working tree is **`code/`** inside the project workspace, never the workspace root
— so no git operation of any kind has a path to the datasheets and photos beside it.

Repository deletion isn't implemented at all, even though the token permits it.

Use a **dedicated Gitea user** ("workshop-bot") scoped to one organisation via
`GITEA_OWNER`, not a token on your own account. The blast radius of a leaked env file
is then one org of generated repos.

An existing repo of the same name is a **success, not a conflict** — asking twice means
you wanted it to exist, and erroring there would make every retry look like a failure
while leaving the caller with no URL to record.

## Infrastructure health

`health.py` polls **CheckMK** and **every OPNsense firewall** every few minutes and
keeps ~30 days of samples in `infra_status`.

**Why here and not in digest-engine**, which already reads one firewall: the digest
runs four times a day, and "is the NAS disk failing right now" is not a question with a
six-hour answer. This service is always on, so it polls; `digest-engine`'s
`ingest/infra_health.py` reads *this* over HTTP. One poller, two consumers — and the
digest gets **"critical since Tuesday"** instead of a snapshot it can't compare to
anything.

Three states, not two: `ok`, `problem`, and **`unreachable`**. "I could not ask" is a
different fact from "I asked and it is broken", and collapsing them is how a display
shows green for a machine that has been off all day. Every failure is written as a row
rather than raised — silence has to be visible.

**Multiple firewalls are the point.** `OPNSENSE_JSON` is a list and every row carries
its firewall's `name`, all the way to the display and the digest prompt. A household
with two firewalls must never be told "the IDS is running".

Read-only, and not by promise: the CheckMK user should hold the **Guest** role, which
cannot acknowledge, downtime or reschedule. From OPNsense this reads exactly one
endpoint (`GET /api/ids/service/status`); the alert query stays in digest-engine, which
already does the paging properly — two implementations of one read would eventually
give two different answers about one log file.

Unlike the rest of the knowledge store, these samples **do** expire (30 days). A
service state from three weeks ago is an observation, not a fact.

## Cameras

`GET /cameras` returns names and go2rtc stream ids; the frontend's Cameras tab embeds
go2rtc's own player per stream. **This service never proxies video** — a Python HTTP
server in the path of an H.264 stream is how a working camera starts stuttering.

Network cameras only. Every USB camera in this project is aimed at one fixed thing at
an angle useless for anything else.

## Fleet scripts — the monitoring-agent admin surface

The **Fleet scripts** tab holds one script per kind of machine, uploaded through the
browser and fetched by each endpoint's `fleet-bootstrap` timer
(`tools/fleet-bootstrap.sh`). It is how every machine in the house gets a CheckMK
agent without SSHing into eleven hosts and pasting the same installer.

| Slot | Covers | Fetched by a device? |
|---|---|---|
| `debian-x86` | thin clients, touch panel, kitchen display, door panel, workshop | yes |
| `raspbian-arm` | arm64 audio endpoints, any future Pi | yes |
| `container-host` | the Docker host — agent *plus* container and disk checks | yes |
| `llm-host` | the GPU machine — different packages, and the only host where VRAM is worth checking | yes |
| `esphome` | voice satellites, BLE proxies, RuView nodes | **no** |
| `network-appliance` | OPNsense, switches, cameras | **no** |

The last two **cannot run a script** — a microcontroller has no shell. Their slots hold
the CheckMK-*server* side (SNMP config, a special agent that polls them), and the UI
says so. They exist because "are the ESPs monitored?" deserves a visible answer rather
than being quietly out of scope.

Slots are a **fixed set**, not a free-form file list: each endpoint knows which slot is
its own without being told, and empty slots are *listed* — the thing someone setting
this up needs to see is which platforms are still uncovered.

### The rules that make this safe enough to have

This is remote code execution by design; there is no version of "distribute scripts to
the endpoints" that isn't. So the honesty is in the constraints:

- **Upload is not deploy.** An upload is a *draft*; endpoints only ever fetch the
  *published* version, and publishing is a second, deliberate click. Same
  propose-then-confirm rule as pantry-vision's stock writes — and it matters more here,
  because what you are confirming runs as root on every machine in the house.
- **This service never executes anything.** It stores text and serves text. There is no
  "deploy now" button, because a button that runs a script on eleven machines at once
  is a button that breaks eleven machines at once.
- **Scripts live in SQLite, not on the SMB share** — the one place the "artefacts go on
  the share" rule is wrong. The share is writable by anyone with the SMB password, and
  a file executed as root on every endpoint must not be writable by that path: it would
  make a share credential a whole-fleet code-execution credential with no history.
- **Every version is kept**, so rolling back is picking an older row — the same
  append-only reasoning `git_ops.py` uses.
- **The endpoint verifies the sha256** before running, runs a version once, and
  **reports back pass or fail**. A script that was *served* is not a script that
  *succeeded*, and only the endpoint knows which. The admin page flags a host running
  an older version than the one now published.
- **Opt-in per host**: no `/etc/fleet-bootstrap.conf`, no execution.

Worth saying in your own words before enabling it: **anyone who can publish here can
run code as root on every machine in the house.** `WORKSHOP_TOKEN` is the most powerful
credential in this project.

## Not built yet

Everything past step 1 of the feasibility note, and it is worth being explicit because
this component is the foundation the rest attaches to:

1. **The canvas `svg` window kind and its server-side renderers** (Graphviz first) —
   the diagram half of `docs/workshop-assistant.md`. The frontend here is an inventory
   editor plus cameras plus health; it is not yet the assistant's display surface.
2. **Spec research, guide lookup, camera identification.** Steps 2–6 of that doc. All
   write into this schema; none exist.
3. **Nothing has run against a real deployment.** Covered by smoke tests against
   temporary databases (projects, knowledge, hardware assignment including the
   `in_use` exception, git commit/branch/deny-list, an unreachable health target), and
   that is the whole of the testing. In particular: **the CheckMK API shape and the
   Gitea branch-protection payload are written from documentation**, and both have
   version-sensitive field names.