Hand the arm to a human, from a gamepad or a browser #1

Merged
cristhian merged 10 commits from feat/gamepad-and-web-teleop into main 2026-09-15 21:27:52 +00:00
Owner

Summary

A new node that lets a person drive the Littlehand arm: pick up a gamepad, or open a
browser, with one arbiter deciding which of the two is driving.

It is the piece in the middle and nothing else. The arm is dora-ulite6, reached through
that node's ordinary jog and request ports; the browser is carried by
dora-websocket-transceiver. There is no web server and no robot driver here, and
AGENTS.md says not to add either — a teleop node that grew its own HTTP server is exactly
the monolith dora-ulite6 was rewritten to escape.

Depends on liria/dora-websocket-transceiver#1, which adds the three things this node
needs from the bridge: request_id across the socket, the clients presence output, and
static file serving. Merge that first.

Related Issues

None — the repository was created for this work and has no tracker entries yet.

Changes

Six commits, one per module, each with its tests passing at that point.

The gamepad (gamepad.py) reads one pad on a thread of its own. Nothing outside that
file touches pygame; everything else sees a plain immutable Sample. Sample.present is
the load-bearing field — an unplugged pad reads as absent, not as centred.

The mapping (mapping.py) is pure functions: deadzone, expo, axis to velocity. Analog
and blended, so speed follows how far you push and several axes move at once. That is the
whole reason a pad beats the console's four-sector dial, which asks for one axis at a time
at one fixed speed. The deadzone rescales rather than clips, so motion starts from zero
just outside the band instead of jumping to a fraction of full speed.

The lease (arbiter.py) is ported from the monolithic repo's RobotService rather
than replaced with last-writer-wins, because the console is already built on it — it gates
its jog pad on t.owner === "web" and draws an owner chip. The gamepad takes the arm
implicitly on the first nudge of a stick and gives it back after sitting untouched; the
web takes and releases explicitly and may preempt.

The console translation (ui.py) reconciles two vocabularies that do not line up: the
arm reports connecting|ready|moving|recovering|unavailable, the console draws
idle|moving|jogging|freedrive|estop|stopped|disconnected. The telemetry keys are a
contract — app.js reads them by name — so they are reproduced rather than improved.

The loop and the deadman (main.py) is the part worth reading closely. See below.

Manifest, dataflow, CI, docs. dataflow.yml runs the whole stack from checkouts.
web/ holds a placeholder page, not the real console — enough to read telemetry and
take/release/stop, so the node can be checked end to end before the Control section is
ported across. The STATIC_DIR the bridge now serves has to point somewhere.

The deadman, which is the part to review hardest

An arm under a velocity command keeps moving until told otherwise, so the question is
never whether to start but whether anything has happened that means it must stop.

  1. Re-send every tick (50 ms) while a source is driving.
  2. JOG_TTL_S (0.4 s) when the owner goes quiet — and immediately, without waiting out
    the timer
    , when the pad is unplugged or the last browser disconnects.
  3. Behind both, dora-ulite6's own VELOCITY_COMMAND_TTL_S (0.5 s), enforced by the
    controller whether or not this process is alive.

Ours must stay shorter than the controller's, or the backstop fires before the thing it is
backing up. There is a test asserting exactly that.

Two invariants that are easy to break later, both stated in AGENTS.md:

  • The stop is sent once, not every tick. A stream of zeroes would keep dora-ulite6
    believing a jog is live and hold robot_state at moving forever.
  • E-STOP is never gated on the lease. Somebody reaching for the stop is the one moment
    when asking who owns the arm is the wrong question.

home is deliberately not bound to a gamepad button. It is a large unattended motion and
belongs behind a deliberate click, not a thumb resting on a face button.

Validation

  • 66 tests pass — no gamepad, arm, browser or daemon needed
  • ruff check . and ruff format --check . clean
  • markdownlint-cli2 "*.md" clean
  • uv lock --check clean
  • The gamepad reader exercised against real pygame with no pad attached: degrades
    to absent, retries, and the thread joins cleanly on shutdown
  • Confirmed pygame.init() had to become display.init() + joystick.init() — the
    full init opens audio and fills a headless robot's stderr with ALSA errors. The
    display subsystem is needed despite there being no window, because SDL refreshes
    joystick state during the event pump

Not verified — no hardware, no dora CLI, no browser

  • Nothing has run with a gamepad actually plugged in. The axis and button indices are
    SDL's documented layout, not observed ones. First thing to check: that the sticks move
    the axes this README claims, and that Back is where E-STOP landed.
  • dora validate dataflow.yml has not run — there is no dora CLI on the machine this
    was written on, and dataflow.yml wires three nodes across two sibling checkouts. CI is
    its first real test.
  • Nothing has run against a real arm, so the deadman is verified by its tests and by
    reading, not by watching an arm stop.
  • The placeholder page has never been opened in a browser.

A limitation worth stating plainly

The lease binds only what goes through this node. dora-ulite6 has no concept of an owner,
so another node publishing request will preempt an operator mid-motion and nothing here
can prevent it. README.md and AGENTS.md both say so rather than implying otherwise.

jog also has no feedback channel at all — the arm node logs and drops a malformed one,
with no response and no error output. What this node sends, it learns nothing back about
except through robot_state and telemetry.

# Summary A new node that lets a person drive the Littlehand arm: pick up a gamepad, or open a browser, with one arbiter deciding which of the two is driving. It is the piece in the middle and nothing else. The arm is `dora-ulite6`, reached through that node's ordinary `jog` and `request` ports; the browser is carried by `dora-websocket-transceiver`. There is no web server and no robot driver here, and `AGENTS.md` says not to add either — a teleop node that grew its own HTTP server is exactly the monolith `dora-ulite6` was rewritten to escape. **Depends on liria/dora-websocket-transceiver#1**, which adds the three things this node needs from the bridge: `request_id` across the socket, the `clients` presence output, and static file serving. Merge that first. ## Related Issues None — the repository was created for this work and has no tracker entries yet. ## Changes Six commits, one per module, each with its tests passing at that point. **The gamepad** (`gamepad.py`) reads one pad on a thread of its own. Nothing outside that file touches pygame; everything else sees a plain immutable `Sample`. `Sample.present` is the load-bearing field — an unplugged pad reads as *absent*, not as centred. **The mapping** (`mapping.py`) is pure functions: deadzone, expo, axis to velocity. Analog and blended, so speed follows how far you push and several axes move at once. That is the whole reason a pad beats the console's four-sector dial, which asks for one axis at a time at one fixed speed. The deadzone rescales rather than clips, so motion starts from zero just outside the band instead of jumping to a fraction of full speed. **The lease** (`arbiter.py`) is ported from the monolithic repo's `RobotService` rather than replaced with last-writer-wins, because the console is already built on it — it gates its jog pad on `t.owner === "web"` and draws an owner chip. The gamepad takes the arm implicitly on the first nudge of a stick and gives it back after sitting untouched; the web takes and releases explicitly and may preempt. **The console translation** (`ui.py`) reconciles two vocabularies that do not line up: the arm reports `connecting|ready|moving|recovering|unavailable`, the console draws `idle|moving|jogging|freedrive|estop|stopped|disconnected`. The telemetry keys are a contract — `app.js` reads them by name — so they are reproduced rather than improved. **The loop and the deadman** (`main.py`) is the part worth reading closely. See below. **Manifest, dataflow, CI, docs.** `dataflow.yml` runs the whole stack from checkouts. `web/` holds a **placeholder page, not the real console** — enough to read telemetry and take/release/stop, so the node can be checked end to end before the Control section is ported across. The `STATIC_DIR` the bridge now serves has to point somewhere. ## The deadman, which is the part to review hardest An arm under a velocity command keeps moving until told otherwise, so the question is never whether to start but whether anything has happened that means it must stop. 1. Re-send every tick (50 ms) while a source is driving. 2. `JOG_TTL_S` (0.4 s) when the owner goes quiet — and **immediately, without waiting out the timer**, when the pad is unplugged or the last browser disconnects. 3. Behind both, `dora-ulite6`'s own `VELOCITY_COMMAND_TTL_S` (0.5 s), enforced by the controller whether or not this process is alive. Ours must stay shorter than the controller's, or the backstop fires before the thing it is backing up. There is a test asserting exactly that. Two invariants that are easy to break later, both stated in `AGENTS.md`: - **The stop is sent once, not every tick.** A stream of zeroes would keep `dora-ulite6` believing a jog is live and hold `robot_state` at `moving` forever. - **E-STOP is never gated on the lease.** Somebody reaching for the stop is the one moment when asking who owns the arm is the wrong question. `home` is deliberately not bound to a gamepad button. It is a large unattended motion and belongs behind a deliberate click, not a thumb resting on a face button. ## Validation - [x] 66 tests pass — no gamepad, arm, browser or daemon needed - [x] `ruff check .` and `ruff format --check .` clean - [x] `markdownlint-cli2 "*.md"` clean - [x] `uv lock --check` clean - [x] The gamepad reader exercised against real pygame with **no pad attached**: degrades to absent, retries, and the thread joins cleanly on shutdown - [x] Confirmed `pygame.init()` had to become `display.init()` + `joystick.init()` — the full init opens audio and fills a headless robot's stderr with ALSA errors. The display subsystem is needed despite there being no window, because SDL refreshes joystick state during the event pump ### Not verified — no hardware, no dora CLI, no browser - **Nothing has run with a gamepad actually plugged in.** The axis and button indices are SDL's documented layout, not observed ones. First thing to check: that the sticks move the axes this README claims, and that Back is where E-STOP landed. - **`dora validate dataflow.yml`** has not run — there is no dora CLI on the machine this was written on, and `dataflow.yml` wires three nodes across two sibling checkouts. CI is its first real test. - **Nothing has run against a real arm**, so the deadman is verified by its tests and by reading, not by watching an arm stop. - The placeholder page has never been opened in a browser. ## A limitation worth stating plainly The lease binds only what goes through this node. `dora-ulite6` has no concept of an owner, so another node publishing `request` will preempt an operator mid-motion and nothing here can prevent it. `README.md` and `AGENTS.md` both say so rather than implying otherwise. `jog` also has no feedback channel at all — the arm node logs and drops a malformed one, with no response and no error output. What this node sends, it learns nothing back about except through `robot_state` and telemetry.
cristhian added 6 commits 2026-09-15 19:44:57 +00:00
The reader owns the pad on a thread of its own, and nothing outside that file
may touch pygame: everything else sees a plain immutable Sample. That is what
lets the rest of the node be tested with no hardware, and what keeps SDL's
quirks out of the arm logic.

Sample.present is the field that matters. A pad that has been unplugged reads as
absent rather than as centred, so "the operator let go" and "the pad fell off
the desk" stay distinguishable — they call for the same stop, but only one of
them is worth logging.

Only the display and joystick subsystems are initialised. pygame.init() also
opens audio, which on a headless robot fills stderr with ALSA errors for a
device this node never uses; the display is needed despite there being no
window, because SDL refreshes joystick state during the event pump.

Settings are parsed once at startup so a deadzone that would silently make the
pad inert stops the node before it joins the dataflow.
Pure functions, no state and no hardware, because this is the part that has to
be right and the cheapest to check.

Analog and blended: how fast the arm moves depends on how far the stick is
pushed, and several axes move at once. That is the whole reason a pad beats the
web console's four-sector dial pad, which can only ask for one axis at a time at
one fixed speed.

The deadzone rescales what is left of the travel rather than just clipping it,
so the arm starts from zero just outside the band instead of jumping to a
fraction of full speed the instant the stick leaves it. Expo bends the response
because fine positioning happens near centre, where a linear stick gives almost
no resolution.

Up on the stick sends the arm along +X, matching the console's dial, so the two
agree about which way is which.
Two sources want one arm and the arm cannot take both. Putting the decision in
one place means there is exactly one answer to "why did it not move" rather than
one answer per input path.

The model is ported from the monolithic repo's RobotService lease rather than
replaced with last-writer-wins, because the console is already built on it: its
jog pad is gated on owning the arm and it draws an owner chip. A simpler scheme
would mean rewriting the page to match the node instead of the other way round.

The gamepad takes the arm implicitly, on the first nudge of a stick, so nobody
has to reach for a browser to use the thing in their hands — and gives it back
after sitting untouched, so a pad left on the bench cannot lock the browser out
forever. The web takes and releases explicitly, and may preempt: an operator who
can see the arm is better placed than this code to decide they should have it.
The console's telemetry keys are a contract: app.js reads arm_state, owner,
estop, vacuum, freedrive, position and joints by name, so renaming one breaks
the page silently. They are reproduced rather than improved for that reason,
vacuum included, which is the console's name for whatever tool is bolted on.

The two vocabularies do not line up and this is where they are reconciled. The
arm node reports a driver's lifecycle; the console draws an operator's screen.
The ordering in arm_state is the ordering of what an operator most needs to
know: a latched e-stop outranks everything, because the arm may well read ready
and saying so would invite somebody to push it.

A message this node does not recognise is dropped rather than refused. The
console and the node version independently, and a page from a newer build
saying something we have not learned yet must not take the arm down.
The loop owns the one decision this node exists to make: whether a velocity
reaches the arm this tick, and from whom.

An arm under a velocity command keeps moving until told otherwise, so the
question is never whether to start but whether anything has happened that means
it must stop. Every path through _wanted_velocity that is not an owner actively
asking for motion returns a stop, so a case nobody thought of fails by halting
rather than by carrying on. The default has to be still.

Three ways a source goes away, all ending in the same place. It goes quiet, and
the watchdog fires after JOG_TTL_S. It goes away outright — the pad unplugged,
the last browser gone — and the arm stops at once, without waiting out the
timer, because an unplugged pad is not a centred pad. Or the process itself
goes away, and one all-zero velocity is sent before it does: a motion lives in
the controller, not here.

The stop is sent once, not every tick. A stream of zeroes would keep the arm
node believing a jog is live and hold robot_state at moving forever.

E-STOP is not gated on the lease, from either source. Somebody reaching for the
stop is the one moment when asking who owns the arm is the wrong question.
add the node manifest, dataflow, CI and docs
validate / validate (pull_request) Failing after 58m48s
validate / test (pull_request) Successful in 58m48s
validate / markdownlint (pull_request) Successful in 58m43s
7250396a1f
dora-node.yml is the typed contract a consumer wires against, and the only one
they get: no source package is shared between node repositories. The jog,
request and ui ports stay untyped, because dora treats a port carrying
request_id as pattern-correlated and several Arrow schemas legitimately flow
over each — one per service, one per console message.

dataflow.yml runs the whole operator stack from checkouts: this node, the
WebSocket bridge and the arm.

web/ holds a placeholder page, not the real console. It exists so the dataflow
has something to serve and so the node can be checked end to end before the
Control section is ported across from the monolithic repo. It reads telemetry
and can take, release and stop; nothing else.
cristhian added 1 commit 2026-09-15 19:49:01 +00:00
add a probe that shows what the gamepad is doing
validate / validate (pull_request) Failing after 58m47s
validate / test (pull_request) Successful in 58m48s
validate / markdownlint (pull_request) Successful in 58m43s
8a7bfb02e0
The axis and button indices in mapping.py are SDL's documented layout rather
than an observed one, and until now the only way to find a pad that numbers
things differently was to watch a real arm move the wrong way.

`dora-teleop-probe` prints the raw axes, the buttons it sees and the velocity
that would be sent, live, with no dataflow and no arm — it cannot move anything.
It goes through the same Gamepad and velocity() the node runs, so what it prints
is what the arm would get; that is the whole reason it is worth having, and the
reason it must not grow its own copy of the mapping.

Every axis is listed, including ones the layout does not use, because an axis
marked unused that moves when a stick is pushed is exactly the finding this is
for. The centred bar makes a worn stick that does not return to zero obvious.
cristhian added 1 commit 2026-09-15 20:52:55 +00:00
fix the node manifest and make the example dataflow self-contained
validate / validate (pull_request) Successful in 58m46s
validate / test (pull_request) Successful in 58m47s
validate / markdownlint (pull_request) Successful in 58m43s
e9fa4bf36b
Two problems, both found by the validate job once it could finally run.

categories named `teleop` and `input`, neither of which dora defines. The set is
fixed — sensor, actuator, robot, transform, filter, ml-inference, llm, speech,
communication, recorder, visualization, simulator, debug — and an unknown
variant makes the whole manifest unparseable, so every type annotation in it was
being ignored.

dataflow.yml spanned three repositories through sibling paths, which meant dora
could not resolve the other nodes' manifests and reported their typed outputs as
unannotated. It also could not be validated in CI, which clones one repo. The
house pattern is a self-contained dataflow, and this now follows it: teleop plus
a bench stand-in for the arm and the browser, both from this checkout. The real
three-node wiring moves to README.md, where a consumer would look for it.

The bench answers every service, publishes plausible telemetry, and prints the
velocities it is sent, so the node can be driven from a real gamepad with no
robot in the room. It is not a simulator and must not become one: it never
integrates a velocity or moves a pose.
Author
Owner

Updated after CI. Two fixes in e9fa4bf, and most of the "Not verified" list above is now verified.

CI caught a mistake I had invented. categories: [robot, teleop, input] names two variants dora does not define — the set is fixed at sensor, actuator, robot, transform, filter, ml-inference, llm, speech, communication, recorder, visualization, simulator, debug. An unknown variant makes the whole manifest unparseable, so every type annotation in it was being ignored. Now [robot].

And a deeper design mistake of mine. dataflow.yml spanned three repositories through sibling paths (../dora-ulite6), so dora could not resolve the other nodes' manifests and reported their typed outputs as unannotated. It also could not be validated in CI at all, which clones one repo. The house pattern — ulite6, plot, the transceiver — is a self-contained dataflow, and I had broken it without noticing.

It now follows the pattern: teleop plus examples/bench.py, a stand-in for the arm and the browser, both built from this checkout. The real three-node wiring moved to README.md, where a consumer would look for it.

The bench is deliberately not a simulator and AGENTS.md says it must not become one — it never integrates a velocity or moves a pose. It closes the loop so this node's behaviour can be watched, which is the answer to "can I test this with just a joystick": yes, dora run dataflow.yml --uv with a pad and no robot.

I also installed the dora CLI locally, so this is the first change here verified before being pushed rather than after.

Now verified:

  • dora validate dataflow.yml → All type annotations OK. — both manifests resolved, 8 contract types injected
  • The dataflow actually runs. Both nodes spawn under a real daemon, run, and shut down clean — zero errors, zero warnings in the log. The console state goes disconnected → idle, which means bench's robot_state reached teleop, teleop's snapshot reached bench, and ui.arm_state() translated correctly. The whole request/response/telemetry loop works end to end.
  • 76 tests, ruff, ruff format, markdownlint, uv lock --check all clean

Still not verified, and unchanged:

  • No gamepad has ever been plugged in. The axis and button indices are SDL's documented layout, not observed ones. uv run dora-teleop-probe is there to check that in about ten seconds, and if they are wrong it is a one-line change to Layout in mapping.py.
  • Nothing has run against a real arm. The deadman is verified by its tests, by reading, and now by watching the node's own behaviour on the bench — but not by watching an arm stop.
  • The placeholder page in web/ has still never been opened in a browser.

Note liria/dora-websocket-transceiver#2 fixes the same invalid-category mistake there; main on that repo is red until it lands.

Updated after CI. Two fixes in `e9fa4bf`, and most of the "Not verified" list above is now verified. **CI caught a mistake I had invented.** `categories: [robot, teleop, input]` names two variants dora does not define — the set is fixed at `sensor`, `actuator`, `robot`, `transform`, `filter`, `ml-inference`, `llm`, `speech`, `communication`, `recorder`, `visualization`, `simulator`, `debug`. An unknown variant makes the whole manifest unparseable, so every type annotation in it was being ignored. Now `[robot]`. **And a deeper design mistake of mine.** `dataflow.yml` spanned three repositories through sibling paths (`../dora-ulite6`), so dora could not resolve the other nodes' manifests and reported their typed outputs as unannotated. It also could not be validated in CI at all, which clones one repo. The house pattern — ulite6, plot, the transceiver — is a **self-contained** dataflow, and I had broken it without noticing. It now follows the pattern: teleop plus `examples/bench.py`, a stand-in for the arm *and* the browser, both built from this checkout. The real three-node wiring moved to `README.md`, where a consumer would look for it. The bench is deliberately **not a simulator** and `AGENTS.md` says it must not become one — it never integrates a velocity or moves a pose. It closes the loop so this node's behaviour can be watched, which is the answer to "can I test this with just a joystick": yes, `dora run dataflow.yml --uv` with a pad and no robot. **I also installed the dora CLI locally**, so this is the first change here verified before being pushed rather than after. Now verified: - `dora validate dataflow.yml` → `All type annotations OK.` — both manifests resolved, 8 contract types injected - **The dataflow actually runs.** Both nodes spawn under a real daemon, run, and shut down clean — zero errors, zero warnings in the log. The console state goes `disconnected` → `idle`, which means bench's `robot_state` reached teleop, teleop's snapshot reached bench, and `ui.arm_state()` translated correctly. The whole request/response/telemetry loop works end to end. - 76 tests, ruff, ruff format, markdownlint, `uv lock --check` all clean Still not verified, and unchanged: - **No gamepad has ever been plugged in.** The axis and button indices are SDL's documented layout, not observed ones. `uv run dora-teleop-probe` is there to check that in about ten seconds, and if they are wrong it is a one-line change to `Layout` in `mapping.py`. - **Nothing has run against a real arm.** The deadman is verified by its tests, by reading, and now by watching the node's own behaviour on the bench — but not by watching an arm stop. - The placeholder page in `web/` has still never been opened in a browser. Note `liria/dora-websocket-transceiver#2` fixes the same invalid-category mistake there; `main` on that repo is red until it lands.
cristhian added 1 commit 2026-09-15 21:09:57 +00:00
enable debug inspection on the example dataflow
validate / validate (pull_request) Successful in 58m48s
validate / test (pull_request) Successful in 58m47s
validate / markdownlint (pull_request) Successful in 58m43s
fb14545850
`dora topic echo` and `dora topic hz` only read the wire when the dataflow asks
for it. Turning it on here means the velocities this node sends can be watched
live, and their cadence measured, without the bench printing anything and
without a robot — which is most of what testing this node with only a gamepad
needs.

Measured with it: the get_state poll runs at 1.88 Hz against a configured
STATE_POLL_S of 0.5.
cristhian added 1 commit 2026-09-15 21:23:15 +00:00
docs: explain what debug inspection is for
validate / validate (pull_request) Successful in 58m43s
validate / test (pull_request) Successful in 58m50s
validate / markdownlint (pull_request) Successful in 58m42s
ac2540629e
The dataflow turned on `enable_debug_inspection` without the README saying why
anyone would want it, which made it a line of configuration nobody could act on.

Documents the three commands it buys — topic list, topic echo, topic hz — and
that they need a coordinator rather than `dora run`. hz is the one that checks
the timings this node promises, so it names the measurement to expect.
cristhian merged commit 885b7a569f into main 2026-09-15 21:27:52 +00:00
Sign in to join this conversation.
No Reviewers
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: liria/dora-teleop#1