174 lines
8.9 KiB
Markdown
174 lines
8.9 KiB
Markdown
# Two shared endpoint surfaces: the now-playing visualiser and the 3D floorplan
|
||
|
||
Design notes for two things that must look the same on every screen in the house — a
|
||
thin client on a TV, the Loggia all-in-one, the kitchen panel, the door panel — and
|
||
which therefore belong in a **shared, vendored SDK** rather than being written four
|
||
times.
|
||
|
||
> **Status: both are built** — `render/media-visualiser/` and `render/floorplan-3d/`.
|
||
> One thing changed in the building, and it is called out in section 2: the 3D view is
|
||
> **canvas 2D, not three.js**. The scene turned out to be prisms on a plane, which an
|
||
> isometric projection draws in ~200 lines without the megabyte, without a build step,
|
||
> and with predictable performance on the small panels. Everything else below was
|
||
> implemented as specified.
|
||
|
||
---
|
||
|
||
## 1. The now-playing visualiser
|
||
|
||
Wanted: a circular CAVA-style spectrum behind every playing-status screen, coloured
|
||
from the album art, with lyrics scrolling under the cover when they exist.
|
||
|
||
### The audio problem, which is the whole problem
|
||
|
||
CAVA reads an audio stream. The endpoints do not all have one:
|
||
|
||
| Endpoint | Is the audio local? | Can it get a spectrum? |
|
||
|---|---|---|
|
||
| Thin client running mpv/spotifyd | **yes** | Yes — real FFT of the actual output |
|
||
| Audio endpoint (Spotify Connect box) | yes, but it is headless | Yes, but there is no screen to draw on |
|
||
| Kitchen / door panel showing *what the living room is playing* | **no** | **No. There is no audio here at all** |
|
||
|
||
So a design that requires real audio analysis works on one endpoint and silently
|
||
degrades to a dead circle on the others — the worst outcome, because the screen looks
|
||
broken rather than looking deliberate.
|
||
|
||
**Two-tier, declared up front:**
|
||
|
||
- **Reactive tier** — where audio is local, CAVA (or a WebAudio `AnalyserNode` when the
|
||
player is in the browser) drives the bars. `cava` has a `raw` output mode writing
|
||
plain numbers to a FIFO, which a small agent can publish over MQTT; that is the least
|
||
fragile path on a machine already running an agent.
|
||
- **Synthetic tier** — everywhere else, the ring is animated from **track position and
|
||
tempo**, not from silence. It breathes with the beat rather than pretending to
|
||
analyse it. Nobody watching a kitchen panel from across the room can tell, and it
|
||
never looks broken.
|
||
|
||
Do not let the synthetic tier claim to be the reactive one anywhere in the UI. It is a
|
||
mood light, and the moment somebody believes it is a spectrum they will report it as a
|
||
bug every time it does not match a bass drop.
|
||
|
||
### Colours from the album art
|
||
|
||
Pull 3–5 dominant colours from the cover, client-side, at load:
|
||
|
||
1. Draw the cover into a small offscreen canvas (64×64 is plenty — this is a palette,
|
||
not a photograph).
|
||
2. Bucket pixels in RGB space, take the top buckets by population.
|
||
3. **Reject near-greys and near-blacks** before ranking. Album art is full of them, and
|
||
a palette extracted naively from a dark cover gives you four indistinguishable dark
|
||
greys and a visualiser that looks switched off.
|
||
4. **Force a minimum contrast against the background.** Lift the chosen colours in
|
||
lightness until they clear the panel behind them; a visualiser you cannot see is
|
||
the same as no visualiser, and this is the step that gets skipped.
|
||
|
||
No library needed — that is about forty lines of canvas work, and it keeps the
|
||
"vendored, dependency-free" property the canvas SDKs already have.
|
||
|
||
### Lyrics
|
||
|
||
- **Source**: whatever the player exposes. Music Assistant and MPD both surface
|
||
synced-lyrics fields when the provider has them; `.lrc` sidecar files are the other
|
||
common case for local libraries.
|
||
- **Synced (LRC) vs plain**: with timestamps, highlight the current line and scroll it
|
||
to centre. Without them, do not fake it — scroll slowly, or just show the text. A
|
||
plain-lyrics block auto-scrolled at a guessed rate is wrong within ten seconds and
|
||
stays wrong.
|
||
- **Absent is the normal case.** Most tracks in most libraries have no lyrics. The
|
||
layout must be designed for "no lyrics" as the default state, with lyrics as the
|
||
addition — not a gap where they would go.
|
||
- Never fetch lyrics from the internet at render time. If lyrics are worth having,
|
||
they are worth caching where the track is.
|
||
|
||
### Where it lives
|
||
|
||
A new `render/media-visualiser/` in the shared SDK style: one JS file, one CSS file,
|
||
vendored into each host that needs it, exactly as `digest-canvas-sdk` and
|
||
`canvas-sdk` already are. Inputs are a normalised now-playing object
|
||
(`{title, artist, album, art_url, position_ms, duration_ms, tempo?, lyrics?}`) and an
|
||
optional spectrum feed. Every endpoint already receives now-playing over MQTT; nothing
|
||
new has to be plumbed for the synthetic tier.
|
||
|
||
---
|
||
|
||
## 2. The 3D floorplan
|
||
|
||
Wanted: the Pebble app's presence view, in 3D, on any endpoint — rooms, and who is in
|
||
them, with profile pictures.
|
||
|
||
### It is the same data, and that is the point
|
||
|
||
`GET /floorplan/presence` already returns everything: room polygons normalised 0–1,
|
||
each room's occupants, each occupant's `color`, `initial` and `has_photo`, plus
|
||
`unplaced` for people who are home but not locatable. The Pebble app and this render
|
||
the same JSON at different fidelities — which is the reason to build the second one at
|
||
all. If they diverge, one of them is lying.
|
||
|
||
### Extrusion, not modelling
|
||
|
||
Do not author a 3D model. Take the existing 2D polygons and **extrude them to a wall
|
||
height**, which is one `THREE.ExtrudeGeometry` per room and needs no new data beyond a
|
||
single `wall_height_m` per level. A hand-authored model would be prettier, immediately
|
||
stale the first time a room is redrawn, and unmaintainable by the person who drew the
|
||
plan in a 2D editor.
|
||
|
||
Camera: fixed isometric by default, drag to orbit, no free-fly. An orbit camera on a
|
||
wall panel is something people knock out of alignment and cannot get back.
|
||
|
||
### The occupant markers
|
||
|
||
This is where 3D earns itself, because the Pebble's constraint — 18×18 px, no room for
|
||
a face — is gone:
|
||
|
||
- A **billboarded disc** above each occupied room, always facing the camera (never a
|
||
flat sprite lying on the floor, which is unreadable at a glance).
|
||
- **Profile picture inside the disc**, ringed in the person's colour, falling back to
|
||
the initial on the colour when `has_photo` is false. The ring matters even with a
|
||
photo: it is what ties this marker to the same person's marker on the watch and in
|
||
the admin panel.
|
||
- **Occupied rooms lit, empty rooms dark**, the same rule the watchapp uses, and for
|
||
the same reason: state should read before detail. In 3D that is an emissive floor
|
||
material rather than a fill colour, and the third state — *drawn but never reported
|
||
by HA* — stays visually distinct, because rendering "no data" as "empty" is a quiet
|
||
lie on any screen.
|
||
- **`unplaced` people get a shelf**, not a hidden list: a strip along the bottom
|
||
showing everyone who is home but unlocatable. They are the people you are most often
|
||
looking for.
|
||
|
||
### The honest constraint — and how it was resolved
|
||
|
||
The plan was to vendor three.js and call it a deliberate break with the
|
||
dependency-free rule. Building it made the cheaper answer obvious and it was taken
|
||
instead: **canvas 2D with an isometric projection and painter's-algorithm sorting.**
|
||
|
||
The scene is prisms standing on a plane — no lighting model worth the name, no
|
||
textures, no physics, no camera motion beyond an orbit. That is ~200 lines of canvas
|
||
2D, it redraws in well under a millisecond on the small panels (where a WebGL context
|
||
on integrated graphics is a much less predictable proposition), and the frontend stays
|
||
at zero dependencies with nothing to keep patched.
|
||
|
||
This is not a compromise version of the three.js plan; it is the smaller correct tool
|
||
for this specific scene. If the view ever grows real lighting or an imported model,
|
||
three.js becomes right and `floorplan3d.js` becomes the fallback for the weak panels.
|
||
|
||
And the same caveat as the watchapp, which no amount of rendering fixes: **this is only
|
||
as right as room-level presence is**, which has never been measured in this house. A
|
||
beautiful 3D house with everybody sitting in the `unplaced` shelf is a beautiful 3D
|
||
house that tells you nothing. Test that first — the admin panel's floorplan tab with
|
||
**Live** ticked answers it for free.
|
||
|
||
---
|
||
|
||
## Build order
|
||
|
||
1. **The visualiser's synthetic tier**, on one endpoint. It needs no new data and no
|
||
audio plumbing, and it is what tells you whether the effect is worth the reactive
|
||
tier's complexity.
|
||
2. **Album-art palette extraction**, which is where the look actually comes from.
|
||
3. **Lyrics**, synced-only at first.
|
||
4. **The reactive tier**, on the thin clients that have local audio.
|
||
5. **The 3D floorplan** — built. It turned out not to be a dependency addition at all
|
||
(see above), so the sequencing argument that put it last no longer applies. The
|
||
*other* reason still does, in full: **it is only as right as room-level presence,
|
||
which has never been measured here.** Test that before believing the picture.
|