SmartestHome/docs/workshop-assistant.md

# A workshop/office assistant: what's worth building

Feasibility note. The ask: identify laptops and computers by camera, identify
components (mainboards especially), pull up disassembly guides, research part specs
automatically, and hold a conversation about planning a project — all scoped per room.

**The one finding that reorganises the whole thing: stop trying to recognise the
object, and read the label on it.** Everything below follows from that.

## Why "identify it by shape" is the wrong problem

Ask a local 7–13B vision model what laptop it is looking at and you will get a
confident, wrong answer, because:

- **Laptops are visually identical by design.** A closed ThinkPad T480 and a T490 are
  the same black rectangle. So are half of Dell's Latitude line. There is no visual
  feature to learn, and a model that claims otherwise is pattern-matching on the
  wallpaper.
- **Mainboards are worse, not better.** Board layout varies more than a laptop shell
  does, but the discriminating detail — where the fourth M.2 slot is, which VRM
  configuration — is exactly what a photo at workbench distance and a general-purpose
  VLM cannot resolve.
- **Getting it wrong is expensive here**, unlike in the pantry. A wrong yoghurt costs a
  tap. A disassembly guide for the wrong laptop revision costs a broken clip or a
  ribbon cable, and a wrong pinout costs the board.

Meanwhile, every one of these objects **carries its own identity in printed text**:

| Object | Where its identity actually is |
|---|---|
| Laptop | Service tag / serial sticker on the base — and on Dell/Lenovo/HP that tag resolves to the *exact factory configuration* |
| Mainboard | Model silkscreened on the PCB (`PRIME B450M-A`), usually near the RAM slots or the PCIe bracket |
| GPU / PSU / drives | Model and part number on the label |
| ICs | Top-marking, when it hasn't been sanded |

So the pipeline is **OCR-first**: photo → detect text regions → read them → match
against a parts database. The VLM's job is the small, tractable one it is good at —
"what kind of thing is this, and where on it is the label" — and the identification
comes from characters, not from vibes. This also degrades honestly: an unreadable
label produces "I can't read it, hold it closer" rather than a plausible wrong model.

**What stays hard, permanently:** an unmarked part. A capacitor, a sanded IC, an
anonymous barrel connector. Nothing here will identify those, and the assistant should
say so rather than guess.

## The four capabilities, ranked by whether they're worth it

### 1. Conversational project planning — build this first

The cheapest and the most useful, and the only one with no perception problem in it.
It is a text LLM with a notebook: what am I building, what have I got, what's the next
step, what did I decide last Tuesday and why. The value is entirely in the *notebook*
being persistent and per-project, not in the model being clever.

This is also the piece that makes the others worth having: an identified mainboard is
only useful if there is somewhere to put it ("this is for the NAS build").

Shape: a small service in the pattern this repo already uses four times over
(`identity`, `chores`, `pantry-vision`) — SQLite, an HTTP API, bearer token, a static
frontend on a display. Projects, notes, parts, photos, decisions.

### 2. Part-spec research — build it, with one hard rule

**Every spec must be quoted from a fetched document, with its URL, and never generated
from the model's memory.** This is the same rule `digest-engine`'s political prompt
enforces on claims about the world and `pantry-vision` enforces on stock writes, and
here it is a safety property rather than a quality one: a hallucinated TDP wastes an
afternoon, a hallucinated pinout destroys hardware.

Practically: fetch the vendor page or datasheet PDF, cache it locally, extract, and
show the extract next to its source. If nothing was fetched, the answer is "I couldn't
find a datasheet", not a paragraph of plausible numbers.

**This is the first component in this project that deliberately reaches the open
internet for content**, which is worth stating plainly given the local-first doctrine
everything else follows. It is outbound-only, it does not require anything to be
exposed, and it should run against an explicit allowlist of sources (vendor domains,
iFixit, datasheet archives) with everything it fetches cached on disk — so the same
board is looked up once, not once per question.

### 3. Disassembly guides — mostly a link, and that's fine

iFixit is the corpus, and it has a public API. The honest scope is: resolve the
identified model to a guide, show the steps and images on the workshop display, cache
what you fetch. Do not rewrite the guide through an LLM — the steps are the thing, and
an LLM paraphrase of "disconnect the battery before removing the board" is strictly
worse than the sentence itself.

Vendor service manuals (Lenovo's HMM PDFs, Dell's service manuals) cover what iFixit
doesn't, and are the reason the cache is worth having: they are large PDFs you want
locally, once.

### 4. Camera identification — build it last, and OCR-first

By the time the three above exist, this is a convenience: it saves typing a model
number. Useful convenience — hands are usually dirty or full — but it is the piece
with the perception risk, and it should not be the first thing attempted.

The interaction that actually works: hold the label to the camera, get the text back
with a confidence, **confirm before anything acts on it** — the identical
propose-never-auto-commit rule `pantry-vision` runs on. Scanning a barcode or QR code
where one exists (most service tags have one) is strictly better than OCR and should
be tried first.

## Per-room scoping, which the plumbing now supports

"Make this a per-room system" splits into two questions, and the boring one is now
answered:

**Which devices are in which room** — done. Every kiosk and audio endpoint declares a
`room` (an HA area_id) in `CoreSystemConfig.json`; it is baked into the agent and
published as `suggested_area`, so HA files each device in the right area by itself. See
`docs/rooms-and-endpoints.md`. The workshop is a room like any other: give it a display
and it declares `room: workshop`.

**Which assistant is active in which room** — the design question, and the answer that
fits this project is: *the room selects the toolset, not the personality.* A voice
request in the workshop can reach the parts database and the project notebook; the same
words in the kitchen reach `pantry-vision` and the recipes. That is a routing table
from area_id to capability set, sitting in front of HA Assist, and it is small.

Two reasons to do it that way rather than one assistant with everything:

- **Ambiguity collapses.** "What have I got?" means stock in the kitchen and parts in
  the workshop, and no amount of prompt engineering makes one agent reliably guess
  which room the words came from — but `identity`'s `/speaker` already answers "who
  spoke in this area", and the area is right there in the request.
- **The dangerous tools stay where they belong.** Nothing in the kitchen should be able
  to open a disassembly guide, and nothing in the workshop needs to book out groceries.

## The workspace, and why it is an SMB share

The assistant needs somewhere to put things: fetched datasheets, extracted specs,
generated diagrams, photos of the board, project notes, a scratch directory per
project. Making that a **plain directory on the container host, exported over SMB**, is
right for a reason worth stating: it means every artefact is a file you can open from a
laptop with no API, no export step, and no dependence on this project still existing in
two years. The assistant's output should outlive the assistant.

```
/opt/smart-home/workshop/
  projects/<slug>/
    notes/          markdown the assistant and you both write
    datasheets/     everything fetched, cached, named by part
    diagrams/       generated SVGs (see the next section)
    photos/         what the camera captured
    scratch/        the assistant's working directory
```

**This project already runs Samba** — `gallery-smb`, from `setup-container-host.sh`.
Two things follow, and the second is a real trap:

- **Reuse the container, add a share.** Samba serves many shares on one port; a second
  Samba container would fight the first for **445, which is already claimed**
  (`ports.gallery_smb`). That is the same class of collision already recorded as open
  decision #31 for Music Assistant, and it is avoidable here by construction: one
  `smb` container, a `gallery` share and a `workshop` share, separate accounts.
- **The gallery share is `read only = yes`. This one cannot be**, and that is the
  entire security difference between them. A writable share is fine; a writable share
  that an LLM writes into deserves three limits, none of which are exotic:
  - **Its own volume**, not a subdirectory of anything else. The blast radius of a bad
    path is then "the workshop workspace", which is recoverable.
  - **Its own account**, not the gallery's. Guest access off, same as the gallery.
  - **No execution.** Nothing in the workspace is ever run by anything on the host —
    it is a place for documents, and the moment it becomes a place for scripts it is a
    different security question than this note answers.

Keep the assistant's *state* — projects, parts, decisions — in SQLite as with every
other service here, and the *artefacts* on the share. The database is the index; the
share is the filing cabinet. Mixing them (blobs in SQLite, or state in files) gets you
the worst of both: a database you cannot browse and files nothing can query.

## Technical display widgets, and the one decision that makes them cheap

The thin-client canvas SDK today has `stat`, `chart`, `image`, `video`, and plain
markdown-ish text. The wanted additions — schematics, board plans, code-flow and
data-structure diagrams, breadboard layouts, 3D — look like six new widgets. They are
mostly one:

> **Render to SVG on the container host; show it in the canvas.** One new `svg` window
> kind, plus renderers server-side.

That keeps the SDK's dependency-free, no-build-step property (the thing that makes it
maintainable), puts heavy tooling on the machine that already has heavy tooling, and
means a diagram is a *file in the workspace share* as well as something on a screen.
Client-side rendering libraries would put a megabyte of JavaScript on a kiosk to
produce a picture the server could have produced once and cached.

| Widget | How | Honest difficulty |
|---|---|---|
| **Circuit diagrams** | `netlistsvg` (or KiCad's own SVG export) from a netlist the assistant produces | Medium — see the notation note below, which is the whole job |
| **Board plans** | KiCad `.kicad_pcb`/Gerber → SVG, server-side, pre-rendered | Medium, and only for boards you have files for. There is no path from a *photo* of a board to a layout drawing |
| **Code-flow diagrams** | Graphviz `dot` → SVG | **Easy.** Graphviz is small, deterministic, and an LLM writes `dot` reliably |
| **Data-structure diagrams** | Graphviz (records/HTML-like labels) → SVG | Easy, same pipeline |
| **Assembly guides** | The guide's own step images in sequence — see the iFixit section | Easy in 2D |
| **Breadboard diagrams** | Fritzing-style, and Fritzing's value *is* its part library, which is not callable | **Hard, and least worth it** — a photo of the actual breadboard is usually better and always more honest |
| **3D models** | glTF + a vendored three.js on the kiosk | The only one that genuinely needs client-side code — see below |

### EU/IEC notation is a symbol-library decision, not a rendering one

Rectangular resistors, not zigzags. This matters more than it sounds: **most schematic
tooling defaults to ANSI/US symbols**, and a renderer that cannot be given a symbol set
will quietly produce American schematics forever.

So the selection criterion for the schematic renderer is *"can I supply my own symbol
library?"* before anything about output quality. `netlistsvg` takes custom skins, which
is the concrete lever — an IEC skin, written once, vendored in the repo like every
other asset here. KiCad can be configured the same way. Anything that hardcodes its
symbols is disqualified regardless of how good its output looks, because the first
zigzag resistor is the last time anyone trusts the diagram.

The same applies downstream: state the convention in the prompt that generates the
netlist, and check it on the way out. A diagram in the wrong notation is not a style
complaint — it is a diagram that reads wrong to the person holding the soldering iron.

### The holographic look is a variable override, not a new stylesheet

The canvas SDK is already a holo aesthetic — glow text, glowing panel edges, a pulse
animation, a dark blue base — and, importantly, it is already parameterised. Every
colour in `glow.css` comes from custom properties on `:root`
(`--admin-accent`, `--admin-glow-color`, `--admin-edge`, `--admin-panel`, `--admin-bg`).

So "make the workshop look holographic in purple/magenta" is **one small override file
loaded after the SDK**, not a fork of it:

```css
/* workshop-theme.css — loaded after glow.css. Overrides only. */
:root {
  --admin-bg:     #0a0510;             /* near-black, violet-biased */
  --admin-panel:  rgba(28, 12, 44, 0.82);
  --admin-edge:   rgba(200, 120, 255, 0.32);
  --admin-accent: #c084fc;             /* the purple everything glows in */
  --admin-text:   #eadcff;
  --admin-muted:  #a98fc4;
  --admin-glow-color: #ff3ec8;         /* magenta glow against a purple accent */
}
```

Two magenta-on-purple choices worth making deliberately: the **accent** (borders,
labels, chrome) reads better as the lighter purple, and the **glow** as the magenta —
glow bleeds and saturates, so the more aggressive colour belongs to the effect rather
than to the thing being read. Keep `--admin-good`/`--admin-warn` as they are; green
and amber mean something, and recolouring them into the theme would cost that meaning.

**Do not tint the technical drawings.** This is the one rule that keeps the look from
becoming a liability: a schematic, a board plan or a pinout table needs contrast and
neutral colour far more than it needs to match the frame — trace colours mean things,
and a magenta glow over a resistor value is how you misread it. So:

- Holo treatment on the **chrome**: window frames, titles, edges, the pulse, the
  background.
- The **drawing surface stays neutral** — a light panel for schematics (they are drawn
  for paper and read best that way), or high-contrast line art on near-black. The
  frame around it can glow all it likes.
- Anything conveying state by colour — a red LED in a diagram, an error, a
  live measurement — keeps its own colour. The theme owns the furniture, not the data.

The same override trick works if a future room wants its own palette; the SDK never
needed to know about themes for this to be possible, which is why it costs one file.

### 3D: worth doing, but not first, and not with a live viewer at first

3D is the one item that cannot be an SVG. Two paths:

- **Pre-rendered orbit frames** — render N views server-side, ship them as an image
  sequence, scrub with a drag. Cheap, works on every surface including the ones with
  no GPU, and covers "let me see what this connector looks like from behind", which is
  most of the actual need.
- **A real viewer** — vendored three.js plus glTF. It is roughly a megabyte of
  JavaScript on a kiosk that currently ships none, which is a real break with the
  SDK's "vendored, dependency-free" rule and should be a deliberate decision rather
  than a drift into one.

Start with the frames. Move to a viewer if and only if manipulating the model turns
out to be what people do, rather than glancing at it.

**3D assembly guides** — exploded views with per-step visibility — are the most
speculative item in this entire note. They need a model *with assembly structure*,
which almost nothing you download has, and authoring one per project is more work than
the repair. Park it.

## Hardware

The workshop needs what the kitchen already has: a screen you can touch with one
knuckle, a camera, a microphone. `hosts/kitchen-display/` is exactly this shape
already — a Sway kiosk, a webcam, an agent, one workspace — so a workshop display is
that image with a different frontend, not a new class of device.

The camera wants to be better than the kitchen's, though, and differently: reading a
service tag or a PCB silkscreen is a **macro** problem. Autofocus at 10–20 cm and
enough resolution to resolve 2 mm text matter far more than field of view. A cheap
USB microscope/inspection camera is a better fit than a webcam, and is worth pricing
before assuming the C920 answer carries over.

## The compute problem, again

One RTX 3060, already carrying Assist, four digests a day, `pantry-vision` and
`chores`. Adding OCR is fine (it is small, and CPU OCR is viable). Adding a second
large model that has to be resident is not — and "the workshop assistant felt slow
because someone asked for a digest" is exactly the contention already flagged as open
decision #4. Design for one model swapped predictably, not two resident.

## Recommended order

1. **The project notebook + conversational planning, and the workspace share.** No
   camera, no internet, no new hardware. Useful on day one, and it is where everything
   else lands — including every artefact the later steps produce.
2. **Spec research with mandatory citation**, against an allowlist, cached on disk.
3. **The `svg` window kind + a Graphviz renderer.** One kind and one small tool buys
   code-flow and data-structure diagrams immediately, and is the pipeline the schematic
   and board-plan renderers then plug into.
4. **Guide lookup**, by typed model number.
5. **Schematics with an IEC symbol skin** — the notation work is the work.
6. **Camera identification**, barcode first and OCR second, proposing rather than
   acting.
7. **3D**, as pre-rendered orbit frames. A live viewer only if manipulation turns out
   to be what people actually do.

The purple/magenta holo theme is not in that order because it is not a step: it is one
override file and can land whenever.

Steps 1–3 need no new hardware at all — a browser on any existing thin client reaches
them. That matters: it means the whole idea can be proved before buying a workshop
display, and if the conversation-and-notebook half turns out to be the only part
anyone uses, that is a complete and useful outcome rather than a failure.

## What is unverified here

Everything past step 1. Specifically: whether iFixit's API terms permit caching what
this would cache; whether a local VLM can read a PCB silkscreen at all under workshop
lighting (testable today with any webcam and a spare board — do this before buying an
inspection camera); whether service-tag lookups work without a vendor account; and
whether the OCR-first pipeline holds up on the labels that are scratched, curved, or
under a warranty sticker, which in a workshop is most of them.

On the display side specifically: whether `netlistsvg`'s skin mechanism really can
express a full IEC symbol set (the notation requirement rests entirely on that, and it
is worth a two-hour spike before committing to the tool); whether an LLM produces
netlists that are *correct* rather than merely well-formed, which is a different and
much harder property; and how big a vendored three.js actually is against the kiosk's
budget before deciding 3D is affordable.