Active · 24 entries since 2026-10 · updated 2026-10-06 · RSS
Computation that lives in the room instead of behind glass. The question behind this stream is how much of everyday computing can leave the screen and still feel ordinary: displays, voice, macropads, paper, and hands, spread around a room, instead of one device that wants all the attention.
This stream continues Project Manus, which began in 2023 with a custom macropad and moved to a ceiling-mounted projector and camera in 2025. In Rome, the work starts from the room itself: Studio Humane, a digital twin of our studio with Home Assistant underneath and screens, panels, keys, and voice around it.
When the carts are assembled, the Mega Alex is the biggest free surface in the studio: 118 × 118 cm of birch. With fiducial tags on it, the projector and camera overhead could find the surface, know exactly where it is, and use it as an ambient screen while I work there: a quick lookup, the steps of an instruction guide, whatever the work needs, without bringing a phone or a laptop to the table.
The candidate is AprilTag 3 with the tagStandard41h12 family, which its README calls “the correct choice” for the vast majority of applications. Tags at the four corners would give the surface’s position even when hands or objects cover some of them. They would also tell the twin that the carts are assembled, so the layout switch could update itself.
Open: how large the tags need to be for a camera on the ceiling, whether projected light washes them out, and how to fix them to the birch top without glare.
An idea to test: a projector and a camera on a motorized gimbal, mounted on the ceiling, that can point anywhere in the room. The twin already knows every surface, so for any target, whether a shelf, a drawer front, or a stretch of wall, it could work out exactly which surface the projector will hit, pre-warp the image for that angle and distance, and project data onto it.
The first use is finding books. The Study IVAR is mostly books, and a search could end with light on the right spine instead of a highlight on a screen. It’s the sharper version of brightening the shelf spots when an answer lands on a shelf.
What I don’t know yet: how precisely an affordable gimbal can aim, whether a small projector stays bright and sharp at that distance and angle, and how to calibrate the projector’s position and lens against the model, so that what the twin computes lands where it should.
Two research systems to read first. IBM’s Everywhere Displays projector (2001) steered a projector’s light onto different surfaces of a room with a rotating mirror, and already ran into brightness, oblique distortion, focus, and obstruction. Microsoft Research’s Beamatron (2012) put a projector and a Kinect on a pan-and-tilt platform and reasoned about the room in 3D from live depth data.
Voice is the surface I haven’t designed yet. The idea so far: Home Assistant’s Assist on the panels, with catalog search as one of its tools, so “where is the soldering iron?” gets a spoken answer while the wall screen points at the drawer. Nothing beyond that is decided.
A Stream Deck + XL will run the same scenes and checks as the panels. It goes on the small surface between Ludo’s desk and mine, so we both have equal access to it. I’m right-handed and Ludo is left-handed, so it sits on the right of my desk and on the left of hers, on each of our dominant sides.
The left five columns of keys carry the scenes, the lights, the doors and both ACs, the resin printer, and a key that sends the wall screen back to its overview. Leave studio asks for a second press when a door is open. Six push-dials set brightness and white for the Lab and the Study, then each AC, and the touch strip shows their values.
The deck in the mockup. The right-hand block of keys stays free for later.
Elgato doesn’t publish the key grid, so the mockup draws it 9 × 4. A small plugin calling the studio brain would keep the deck in step with everything else. Since the deck is shared, it shouldn’t depend on either of our laptops: on the Mac mini, through Elgato’s Network Dock, it works without one.
The phone gets a small web app from the studio brain, added to the home screen, with no App Store. Find searches the catalog. In the mockup’s example catalog, “solder” turns up three things in cart drawer C2·3, and Show on wall screen flies the wall screen into that drawer.
The wall screen answering the phone: drawer C2·3.
The same answer could reach the room itself: when a search lands on the IVAR wall, the two shelf spots above it could brighten. The phone’s other tab, Capture, is where cataloging by photo happens. Both are steps toward finding the soldering iron for real.
The first surface is an always-on display showing the model in cutaway, slowly turning. Every lamp is a spotlight aimed at what it lights, in the white or RGB color that Home Assistant reports. When a door sensor opens, the door swings open in the model. Around it sit the time and daylight, the rooms, the doors, the machines, recent activity, and a banner for anything that needs attention.
Simulated: the main door opens, the evening scenes come on, the doorbell rings.
It’s read-only and shows no camera feeds. When the studio is empty or the sun is down, it dims to ink. Which display, where it hangs, and what drives it are still open.
The studio system splits into layers. Home Assistant is the device layer: it talks to the Hue bridge, the Aqara hub, and later the IR blasters and printers, and it keeps the automations that must work when everything else is off.
Beside it runs a small service, the studio brain. It holds what Home Assistant doesn’t: the 3D model and its 73 places, the catalog and its photos, Claude’s readings and who confirmed them, the position of every Home Assistant entity in the model, and an event log. It mirrors Home Assistant’s state and serves every screen in the studio: a wall screen, touch panels, phones, a Stream Deck, and a management site.
How it fits together, from the studio-system mockup. Open the image for full size.
The brain runs on the Mac mini for now and moves to the NAS later. Keeping it apart from Home Assistant means an update to the brain never takes the lights down.
Two of the studio’s scenes come from my notes on color-managed printing. Grading, in the Study, asks for a dim, steady, neutral surround near D50: about 32 lux, never above 64, and no light falling on the screens. Print viewing, in the Lab, puts the editing wall at D50 and full brightness, aiming for about 2,000 lux for critical viewing and 800 for ordinary viewing. The everyday scenes are simpler: Work at 4000 K and Evening at 2700 K.
The brightness percentages behind them are first guesses. One light-meter reading at my desk and one at the editing wall will turn them into the lux targets.
Each air conditioner gets a Broadlink RM4 mini placed where it can see the unit, controlled locally through Home Assistant’s Broadlink integration, with SmartIR for a thermostat card. If the codes aren’t in SmartIR’s library, the RM4 learns them from the original remote.
The Study needs a climate sensor of its own before its AC can follow a temperature. Still to find: the brand of the units, on a sticker on the indoor unit or the remote.
The most ambient computing a studio already has is its smart home. Lamps, door and window sensors, and climate sensors sit in the walls and on the ceiling, quietly doing their job. I’ll use Home Assistant integrations to talk to all of them, in two directions:
Actions, such as turning on lights or running a scene.
Reports, such as warnings about open doors and windows, and room temperatures.
The Hue Bridge Pro, with 16 lamps and 2 controls, and the Aqara Hub M200, with contact sensors on both doors and the WC window and a temperature and humidity sensor in the Lab, are installed in the Darkroom. Home Assistant itself will run on a Home Assistant Green.
The touch panel in the studio-system mockup: scenes, every Lab lamp as a key, the room plan, the air conditioner, and doors and windows. Simulated; nothing is connected yet.
Both desks in the Study, Ludo’s by the window and mine at the back, adjust from 70 to 120 cm, so their height is part of the studio’s state. The viewer gives each one a fader, and the sliding door gets one too. Moving a fader moves the desk in the model.
The Study view with both desks raised from 73 cm and lowered again.
If the desks or the door ever report their position, the same faders become read-outs.
The viewer lights the model with the real sun: SunCalc for its position, the building’s approximate orientation from OpenStreetMap, and Rome time. Scrubbing through a day shows when the sun faces the street windows and when it moves behind the frontage.
A day from 07:30 to 19:30, with the room lights switched on at dusk.
That matters beyond looks. The frontage faces east, which puts cooling the studio early on hot mornings on the list of first automations. Nearby buildings and trees aren’t in the model yet.
The viewer is rebuilt around search. Type an address in any of its spellings and the camera flies to it, the slot lights up in the model, and a card shows the unit with that slot filled in. Number keys pick a view, and the slash key jumps to search.
C2·3, a cart drawer in the Lab, then DA·b3, the resin printer’s bay in the Darkroom.
Every card still says “Not catalogued yet,” because the places exist and the things don’t. Filling them in is the next layer.
Four white ALEX drawer carts on castors, C1–C4, usually live apart: two parked in the bottom bays of the Lab IVAR wall, two against the Lab’s back wall. Assembled, they form a pinwheel in the middle of the Lab under a 118 × 118 cm birch top: the Mega Alex.
So the studio has two layouts, and the twin needs both. The model switches between them with one key, and every cart drawer keeps its address when the carts move: C2·3 can be found whether the carts are parked or assembled.
The same plan with the carts parked, then assembled. The state line under the switches follows the layout.
Switching is also a small workflow worth visualizing: which cart goes where, in what order, and where the top comes from. That last part is still open. The top’s spot when the carts are parked isn’t in the model yet, so it’s hidden.
Every QR code on a label encodes a Home Assistant tag URL. Once Home Assistant runs, scanning a label with its companion app fires a tag event that carries the address, so a scan can open that slot’s page. An NFC sticker behind the label could do the same with a tap.
Part of sheet REV.A: 48 × 16 mm strips sized for the front edge of an IVAR shelf.
A first print run, REV.A, was rendered and all 94 of its QR codes decoded correctly. Then printing went on hold with the rest of the label question.
Addresses only help if they’re on the furniture. I compared five label systems:
One label per slot, with a small diagram of the unit and the slot filled in.
One tall directory per bay on the upright, with colored dots on the shelf edges.
Labels that lead with what lives there, colored by kind of thing.
Rulers along each shelf edge, with Gridfinity coordinates inside drawers.
A 50 cm floor grid taken from the 3D model, plus a height band.
Whatever wins needs white labels, room colors, real coordinates instead of “13 of 25” style counts, a small diagram, and a date, all printable on an inkjet. The leading combination so far: per-slot labels for fixed places, Gridfinity coordinates inside the cart drawers, and a handwritten “checked” date on box labels. Printing is on hold until I decide.
The catalog of things gets filled from photographs:
Snap the shelf, drawer, or open box with its label in frame.
Claude reads the label for the address and lists the items, counts, and brands.
It compares that with what the catalog expects there: new, still here, not seen, or unsure.
I confirm with one tap or fix a line.
The catalog updates, and the photo stays as evidence of when the place was last seen.
Claude proposes, a person confirms. Anything a photo misses is marked as not seen since that date, never deleted.
The first version needs no new infrastructure: photos go into an inbox folder, Claude Code updates plain files, and I review the change as a git diff. Later, an iOS Shortcut could post one photo to a small service.
Where is it, and do we have it? Both answers come from the same catalog of places and things. Everything else stays out of scope: who has something, whether it’s in use, when it left the shelf.
Each room has a color, and a room color never appears without its letter. Device states follow the same rule: every state color comes with its own shape, so the twin still reads in grayscale and for anyone who sees color differently. Text on a color is white or ink, whichever reads better on it.
We have a lot of drawers and shelves, and I move between two or three cities and more than one studio. I regularly forget what I have and where it is.
What I want is to ask the studio “where is the soldering iron?” and get an exact answer: the place in the virtual studio, so I can walk over and pick it up in the real one. Just as often the question is whether I own something at all, before I buy it twice.
The studio model comes from three sources: the original CAD drawing, twelve reference photographs, and a walkthrough video recorded before the renovation. They disagree in places, so each one got a job. The drawing controls the room footprints and the widths of openings. The photos and the walkthrough control everything you can see: frames, finishes, the thickness of the wall above the opening between rooms.
The plan view: the Lab on the left, the Study on the right, the Darkroom, Passage, and WC behind them.
Wherever a dimension is a guess, the guess is recorded on the object itself, so the model says what it doesn’t know. The measurements still to take: a shelf unit’s height in the Study, the Darkroom ceiling, a desk’s exact position, the WC window frame, and the mounting heights of the cameras and sensors.
I’ve started the practical side of this stream: a studio environment system for Project Humane, built in and for our studio in Rome. The project is called Studio Humane.
The first step is a virtual copy of the studio: a 3D model accurate enough to point at a single drawer. With it I want to find out how useful such a twin really is, and how much of the studio I can run through displays, voice, macropads, and small tools spread around the room, so the studio becomes a more pleasant and more useful place to work.
The twin in its browser viewer: the whole studio in cutaway, then the Lab, the Study, and the Darkroom.
A Blender script generates the model, which is exported as a GLB and shown in a browser viewer that runs offline from a single file.
