docs: shorten the README and correct how the tick is described
validate / validate (push) Successful in 58m47s
validate / test (push) Successful in 58m52s
validate / test (pull_request) Successful in 58m49s
validate / validate (pull_request) Successful in 58m48s

The README carried explanations that belong in AGENTS.md; it now follows
the shape of the sibling node repositories and is a quarter shorter.

The tick description was also wrong: it does not trigger a grab. Capture
is free-running at ZED_FPS and a tick forwards the newest frame there is,
so the tick sets the publish rate, and frames captured between two ticks
are dropped. The usage snippet listed four of the five outputs, which
read as an omission rather than the choice it is.
This commit is contained in:
2026-09-15 10:52:12 -05:00
parent c1a7d70ba6
commit a4cd493330
2 changed files with 166 additions and 201 deletions
+19 -9
View File
@@ -4,10 +4,18 @@ This file provides guidance to AI coding assistants when working with code in th
## What this is
A dora-rs C++ node that interfaces with ZED stereo cameras. On each tick it grabs a frame and publishes it over the dora dataflow: a BGR image and a depth map every frame, camera intrinsics once per camera session, a camera state on every transition, and an XYZRGBA point cloud when `ZED_POINT_CLOUD` is on. Camera management lives in `src/zed/` and the dora event loop in `src/main.cc`.
A dora-rs C++ node that interfaces with ZED stereo cameras. Capture is free-running on its own thread at `ZED_FPS`; on each tick the event loop publishes the newest frame it has produced, or nothing if none arrived since the last tick. What goes out: a BGR image and a depth map every frame, camera intrinsics once per camera session, a camera state on every transition, and an XYZRGBA point cloud when `ZED_POINT_CLOUD` is on. Every payload is a typed Arrow array — the two frames as `std/media/v1/Image`, the rest as `std/core/v1/Float32` and `.../String` — which is what lets the node feed [dora-image-view](https://gitea.liria.cl/liria/dora-image-view) and anything else built against those types. Camera management lives in `src/zed/`, payload packing in `src/arrow/`, and the dora event loop in `src/main.cc`.
The node is a camera source only. It ran inference in-process through `v0.1.0`, under `src/engines/`; that was removed so the node has no inference dependency and builds on any platform the ZED SDK supports. The removed code remains reachable at tag `v0.1.0`.
## Test
```bash
tests/run.sh
```
Needs `g++` and `uv`; no camera, no ZED SDK, no CUDA.
## Build
```bash
@@ -28,8 +36,8 @@ For a build without installing the SDK stack on the host, see the `Dockerfile` (
```bash
dora up
dora start dataflow.yml # camera alone
dora start dataflow_plot.yml # camera plus the opencv-plot viewer
dora start dataflow.yml # camera alone
dora start dataflow_view.yml --uv # camera plus dora-image-view
```
The two dataflows both declare the camera node, so run one at a time.
@@ -42,20 +50,22 @@ Camera behavior is configured entirely through environment variables (see README
- **`src/zed/camera.h`** — `zed::Config`, `zed::Calibration`, and `zed::Camera` class. Camera wraps the ZED SDK lifecycle: `open()` (one attempt, returns success), warmup, grab (image + depth + point cloud), calibration, device timestamp, blank-frame check, `close()` and `reset()` (close plus `sl::Camera::reboot()` where the model allows it). Exposes `handle()` for engine access.
- **`src/zed/capture.h/cc`** — `zed::Capture` and `zed::Frame`. Owns the camera on a dedicated thread: open, warmup, grab, blank/stall detection, backoff and escalation all live here, along with the packing of every payload into owned buffers. The newest frame is handed to the reader by swapping under a mutex (`take_frame`), which also recycles the previous frame's buffers so steady-state capture does not reallocate. **Nothing outside this file may call the ZED SDK** — it blocks unboundedly on a disconnected camera, which is what froze the node when the loop drove the camera directly.
- **`src/zed/camera.cc`** — Implementation. SDK enum parsers live in an anonymous namespace. `parse_depth_mode` runs in `Config::from_env()`, not in the constructor, so an unrecognised `ZED_DEPTH_MODE` exits before the node joins the dataflow rather than falling back to a default.
- **`src/main.cc`** — Dora event loop and output publishing only. On any input event (intended to be driven by a tick timer; the id is not inspected): calls `camera.grab()`, then sends `image_bgr` and `depth`, plus `point_cloud` when `ZED_POINT_CLOUD` is on and `camera_info` on the first frame of a camera session (`camera_info_pending`, re-armed on reconnection). `pack_depth` copies the depth map into a tightly packed `[H, W]` float buffer (sl::Mat rows can be padded) and clears `inf`/`nan` to `0`. `set_intrinsics` and `set_frame_stamps` stamp every output of a frame with the same intrinsics, `frame_id` and the two timestamps (`device_timestamp_ns`, `host_timestamp_ns`; both Unix-epoch nanoseconds). Point cloud output varies by mode: `FULL` sends the raw `sl::Mat` pointer (no copy); `HALF` downsamples by 2 in each dimension. The loop never touches the SDK: it drains `take_state_change` and `take_frame` from the capture thread and publishes. On exit it calls `capture.stop()` and waits 2 s for the thread; if the thread is still inside a blocking SDK call it gives up and `_Exit(0)`s rather than let dora time out and kill the node.
- **`src/arrow/array.h/cc`** — Builds the Arrow arrays the node publishes, against the Arrow C data interface: the two ABI structs are defined in the header rather than pulled from Arrow C++, which would be a heavy dependency for a producer that exports one array per output and never reads one. `image()` packs an `std/media/v1/Image` row, `float32()` and `string()` the primitive payloads. The payload bytes are **borrowed**, not copied — they only have to outlive the send, since dora IPC-encodes the array into shared memory inside `send_arrow_output` and then calls the release callbacks. Each node's release frees its children before itself, so releasing the root frees the tree.
- **`src/main.cc`** — Dora event loop and output publishing only. On any input event (intended to be driven by a tick timer; the id is not inspected) it drains `take_state_change` and `take_frame` from the capture thread and publishes `image` and `depth`, plus `point_cloud` when `ZED_POINT_CLOUD` is on and `camera_info` on the first frame of a camera session (`camera_info_pending`, re-armed on reconnection). `send()` hands one `arrow_c::Exported` to `send_arrow_output`. `set_intrinsics` and `set_frame_stamps` stamp every output of a frame with the same intrinsics, `frame_id` and the two timestamps (`device_timestamp_ns`, `host_timestamp_ns`; both Unix-epoch nanoseconds); nothing about a frame's shape or encoding travels as metadata any more, since the Image struct carries it. The loop never touches the SDK. On exit it calls `capture.stop()` and waits 2 s for the thread; if the thread is still inside a blocking SDK call it gives up and `_Exit(0)`s rather than let dora time out and kill the node.
- **`DoraTargets.cmake`** — Handles building or fetching dora-rs and generating the CXX bridge files. Two paths: local (`DORA_ROOT_DIR`) or git clone. Produces `dora_node_api_cxx` library and the `node_bridge.cc` glue.
- **`scripts/build.sh`** — Configure/compile/install in one shell-free-callable step. Referenced by both dataflows, the `Dockerfile` and the README.
- **`dora-node.yml`** — Node manifest (`apiVersion: 1`): typed input/output contract, entrypoint, build command. `dora validate` injects these types into any dataflow that reaches the node by `path:`; a `git:`-sourced node does **not** pick it up, which is why both dataflows build from the checkout. Type URNs name the element type (`std/core/v1/UInt8`, `.../Float32`, `.../String`) because every payload is a raw buffer described by its metadata, not an Arrow struct.
- **`dora-node.yml`** — Node manifest (`apiVersion: 1`): typed input/output contract, entrypoint, build command. `dora validate` injects these types into any dataflow that reaches the node by `path:`; a `git:`-sourced node does **not** pick it up, which is why both dataflows build from the checkout. `image` and `depth` are `std/media/v1/Image`; dora's compatibility graph has no edge from `std/core/v1/UInt8` to it, so a consumer declaring the struct rejects a producer declaring the buffer whatever it actually sends — the manifest has to move with the payload.
- **`dataflow.yml`** — Example dora dataflow: the camera alone, no viewer and no Python dependency. Built from this checkout with `path:` + `build:` and `strict_types: true`. A consumer wires the node from a tag instead (see README) — dora 1.0 pins a source with the mutually exclusive `tag:`, `rev:` or `branch:`.
- **`dataflow_plot.yml`** — The same camera node plus an `opencv-plot` viewer on `image_bgr`. Split out because the viewer pulls dora-hub over `pip` and needs a display. dora 1.0 has no cross-dataflow wiring (an input's `source` must resolve to a node in the same descriptor), so the camera node is duplicated here rather than referenced, and the two files are run one at a time.
- **`.gitea/workflows/validate.yml`** — CI gate running `dora validate` on both dataflows on push and pull request. Needs no camera and no ZED SDK.
- **`dataflow_view.yml`** — The same camera node plus [dora-image-view](https://gitea.liria.cl/liria/dora-image-view) on `image`, `depth` and `camera_info`, fetched from a tag. Run it with `--uv`, which gives the viewer its own managed environment. The viewer's manifest is not read at validate time (only a `path:` node's is), so the file repeats its contract in `input_types:` to get the two checked against each other. dora 1.0 has no cross-dataflow wiring (an input's `source` must resolve to a node in the same descriptor), so the camera node is duplicated here rather than referenced, and the two files are run one at a time.
- **`tests/run.sh`** — Builds and runs the payload tests: `tests/arrow_test.cc` under `-fsanitize=address,undefined` (the release callbacks have to free the whole tree), and `tests/test_arrow.py`, which imports the exported arrays through pyarrow's C data interface and checks the values. `tests/arrow_export.cc` is the C shim the Python side calls. Covers `src/arrow` only, so it needs no camera, no ZED SDK and no dora.
- **`.gitea/workflows/validate.yml`** — CI gates on push and pull request: `dora validate` on both dataflows, and `tests/run.sh`. Neither needs a camera, the ZED SDK or CUDA.
- **`Dockerfile`** — Ubuntu 24.04 x86-64 build environment (CUDA 12.8, ZED SDK 5.2, Rust). No ARM64/Jetson image; build natively there. No camera needed; run the compile with `--gpus all`. Sets `NVIDIA_DRIVER_CAPABILITIES=compute,utility,video` — the `video` capability is what makes the container runtime mount NVDEC/NVENC, which `libsl_zed.so` links against. Rust lives at `/usr/local/{rustup,cargo}` rather than `/root` so the build can run as the invoking user. `docker build -t dora-zed-build . && docker run --rm --gpus all --user "$(id -u):$(id -g)" -v "$PWD:/src" -v dora-zed-cargo:/usr/local/cargo/registry dora-zed-build`.
## Key APIs
- **dora C++ API** (`<dora-node-api.h>`): `init_dora_node()`, `dora_node.events->next()`, `event_type()`, `send_output_with_metadata()`, `new_metadata()`.
- **dora C++ API** (`<dora-node-api.h>`): `init_dora_node()`, `dora_node.events->next()`, `event_type()`, `send_arrow_output()`, `new_metadata()`.
- **ZED SDK** (`<sl/Camera.hpp>`): `sl::Camera`, `sl::InitParameters`, `sl::RuntimeParameters`, `sl::Mat`.
- Output data is sent as raw byte slices (`rust::Slice<const uint8_t>`) with typed metadata (shape, dtype, encoding).
- Output data is sent as Arrow arrays: `send_arrow_output` takes an `ArrowArray`/`ArrowSchema` pair by pointer and moves both out, so a payload is built once with `src/arrow` and handed over.
## Tools