add depth and camera intrinsics, and adopt the typed Arrow image #4

Merged
cristhian merged 14 commits from depth-panel into main 2026-09-15 09:36:17 +00:00
2 changed files with 11 additions and 4 deletions
Showing only changes of commit 2b70d07868 - Show all commits
+1 -1
View File
@@ -34,7 +34,7 @@ CI=true SOURCE_FRAMES=30 dora run dataflow.yml --uv # the same, headless
- **`dora_image_view/payload.py`** — `as_float32()`: reinterprets a `uint8` payload rather than converting it, because a C++ node sending a raw byte slice produces a `UInt8` arrow array despite declaring `Float32`. Shared by `depth` and `intrinsics`, which both meet it.
- **`dora_image_view/intrinsics.py`** — `Intrinsics`, `parse()` for the `camera_info` payload and `from_metadata()` for the copy senders attach to each frame. Both exist because dora-zed publishes the same numbers twice, and the per-frame copy is the one that survives a viewer starting after the camera did — `camera_info` goes out **once per session**. A nine-element row-major K is accepted alongside the compact four values, since that is how ROS spells the same model. The node needs none of this to draw; it is held for whatever turns depth into 3D, and drawn so a wrong camera model is visible rather than merely plausible.
- **`dora_image_view/bbox.py`** — Parses the `bbox` struct into xyxy corners. `xywh` is centre-based (x, y are the box centre). `_flat()` unwraps the single-element list that `pa.array([{...}])` produces around each field, and reads a field stored flat as-is.
- **`examples/source.py`** — The synthetic source node: an animated frame, a depth map of the same scene (a flat wall with the square in front of it), the box around the square, and a caption. It exists so the example dataflow exercises all three inputs without a camera or a detector, and so CI can run the dataflow rather than only validate it. `SOURCE_FRAMES` bounds the run, which is what lets CI wait for a clean exit instead of killing it on a timeout.
- **`examples/source.py`** — The synthetic source node: an animated frame, a depth map of the same scene (a flat wall with the square in front of it), the box around the square, and a caption. It is both the demo and the CI gate, and the second role has been shaping it: it publishes depth and camera_info because those inputs are *typed*, and `dora validate --strict` fails on a typed input the dataflow does not wire. Before adding an output here, know which of the two reasons you are adding it for. `SOURCE_FRAMES` bounds the run, which is what lets CI wait for a clean exit instead of killing it on a timeout.
- **`dora-node.yml`** — Node manifest (`apiVersion: 1`). `dora validate` injects these types into any dataflow that reaches the node by `path:`; a `git:`-sourced node does **not** pick it up, which is why the example builds from the checkout. `image` and `text` carry type URNs naming the element type because each payload is a raw buffer described by its metadata, not an Arrow struct. `bbox` is deliberately **untyped**: senders disagree on whether its numeric fields are 32- or 64-bit, and the node reads either.
- **`dataflow.yml`** — The example and the CI gate, with `strict_types: true`. The viewer is a `path:` node, so its manifest is injected; the source is typed inline with `output_types:` and the two are compared against each other. `bbox` is annotated on neither side, matching the manifest.
+10 -3
View File
@@ -38,9 +38,10 @@ pixels, and the caption on the composite.
### Runnable example
[`dataflow.yml`](dataflow.yml) is self-contained. [`examples/source.py`](examples/source.py)
is a synthetic source node that emits an animated frame, the box around the square
moving in it, and a caption — the three inputs this node draws — so the example needs
no camera, no detector and no network:
is a synthetic source node that emits an animated frame, a depth map of the same scene,
the camera intrinsics, the box around the square moving in the frame, and a caption —
every input this node draws — so the example needs no camera, no detector and no
network:
```bash
dora run dataflow.yml --uv
@@ -52,6 +53,12 @@ to quit.
It builds the viewer from this checkout (`path:` + `build:`) rather than from a tag,
which is what lets `dora validate` pick up the manifest described below.
This example is also the end-to-end gate: CI runs the same dataflow headless on every
push, which is what keeps it from rotting. It earns its place twice over — it is the
only worked example of the *producing* side of these contracts, which is the fiddly
half of wiring this node. The realistic wiring, against a real camera, is the block at
the top of this file.
Set `SOURCE_FRAMES` to stop the source after a fixed number of frames instead of running
until you quit; that plus `CI=true` is how the same dataflow runs headless as a test: