dora-zed/AGENTS.md

AGENTS.md

This file provides guidance to AI coding assistants when working with code in this repository.

What this is

A dora-rs C++ node that interfaces with ZED stereo cameras. Capture is free-running on its own thread at ZED_FPS; on each tick the event loop publishes the newest frame it has produced, or nothing if none arrived since the last tick. What goes out: a BGR image and a depth map every frame, camera intrinsics once per camera session, a camera state on every transition, and an XYZRGBA point cloud when ZED_POINT_CLOUD is on. Every payload is a typed Arrow array — the two frames as std/media/v1/Image, the rest as std/core/v1/Float32 and .../String — which is what lets the node feed dora-image-view and anything else built against those types. Camera management lives in src/zed/, payload packing in src/arrow/, and the dora event loop in src/main.cc.

The node is a camera source only. It ran inference in-process through v0.1.0, under src/engines/; that was removed so the node has no inference dependency and builds on any platform the ZED SDK supports. The removed code remains reachable at tag v0.1.0.

Test

tests/run.sh

Needs g++ and uv; no camera, no ZED SDK, no CUDA.

Build

scripts/build.sh

The binary is installed to bin/dora_zed relative to the repo root. The script is also the dataflow's build: command: dora 1.0 executes build commands directly rather than through a shell, so cd, && and command substitution do not work there.

Requirements: dora 1.0+ (verified with dora-cli 1.0.1), CMake 3.21+, ZED SDK 5.x (at /usr/local/zed), CUDA Toolkit, Rust stable toolchain (for dora-rs C++ bindings).

The first build clones dora-rs from GitHub and compiles dora-node-api-cxx via Cargo. This takes a while. Subsequent builds are incremental. If DORA_ROOT_DIR is set to a local dora checkout, it builds against that instead.

For a build without installing the SDK stack on the host, see the Dockerfile (Ubuntu 24.04 — the newest release with a ZED SDK; there is no Stereolabs build for 26.04). Run the compile with --gpus all.

Run

dora up
dora start dataflow.yml            # camera alone
dora start dataflow_view.yml --uv  # camera plus dora-image-view

The two dataflows both declare the camera node, so run one at a time.

Camera behavior is configured entirely through environment variables (see README.md for the full list: ZED_RESOLUTION, ZED_FPS, ZED_DEPTH_MODE, etc.).

Architecture

• src/util/env.h — Shared inline env-variable parsing helpers (get_env_int, get_env_str, get_env_bool, get_env_float).
• src/zed/camera.h — zed::Config, zed::Calibration, and zed::Camera class. Camera wraps the ZED SDK lifecycle: open() (one attempt, returns success), warmup, grab (image + depth + point cloud), calibration, device timestamp, blank-frame check, close() and reset() (close plus sl::Camera::reboot() where the model allows it). Exposes handle() for engine access.
• src/zed/capture.h/cc — zed::Capture and zed::Frame. Owns the camera on a dedicated thread: open, warmup, grab, blank/stall detection, backoff and escalation all live here, along with the packing of every payload into owned buffers. The newest frame is handed to the reader by swapping under a mutex (take_frame), which also recycles the previous frame's buffers so steady-state capture does not reallocate. Nothing outside this file may call the ZED SDK — it blocks unboundedly on a disconnected camera, which is what froze the node when the loop drove the camera directly.
• src/zed/camera.cc — Implementation. SDK enum parsers live in an anonymous namespace. parse_depth_mode runs in Config::from_env(), not in the constructor, so an unrecognised ZED_DEPTH_MODE exits before the node joins the dataflow rather than falling back to a default.
• src/arrow/array.h/cc — Builds the Arrow arrays the node publishes, against the Arrow C data interface: the two ABI structs are defined in the header rather than pulled from Arrow C++, which would be a heavy dependency for a producer that exports one array per output and never reads one. image() packs an std/media/v1/Image row, float32() and string() the primitive payloads. The payload bytes are borrowed, not copied — they only have to outlive the send, since dora IPC-encodes the array into shared memory inside send_arrow_output and then calls the release callbacks. Each node's release frees its children before itself, so releasing the root frees the tree.
• src/main.cc — Dora event loop and output publishing only. On any input event (intended to be driven by a tick timer; the id is not inspected) it drains take_state_change and take_frame from the capture thread and publishes image and depth, plus point_cloud when ZED_POINT_CLOUD is on and camera_info on the first frame of a camera session (camera_info_pending, re-armed on reconnection). send() hands one arrow_c::Exported to send_arrow_output. set_intrinsics and set_frame_stamps stamp every output of a frame with the same intrinsics, frame_id and the two timestamps (device_timestamp_ns, host_timestamp_ns; both Unix-epoch nanoseconds); nothing about a frame's shape or encoding travels as metadata any more, since the Image struct carries it. The loop never touches the SDK. On exit it calls capture.stop() and waits 2 s for the thread; if the thread is still inside a blocking SDK call it gives up and _Exit(0)s rather than let dora time out and kill the node.
• DoraTargets.cmake — Handles building or fetching dora-rs and generating the CXX bridge files. Two paths: local (DORA_ROOT_DIR) or git clone. Produces dora_node_api_cxx library and the node_bridge.cc glue.
• scripts/build.sh — Configure/compile/install in one shell-free-callable step. Referenced by both dataflows, the Dockerfile and the README.
• dora-node.yml — Node manifest (apiVersion: 1): typed input/output contract, entrypoint, build command. dora validate injects these types into any dataflow that reaches the node by path:; a git:-sourced node does not pick it up, which is why both dataflows build from the checkout. image and depth are std/media/v1/Image; dora's compatibility graph has no edge from std/core/v1/UInt8 to it, so a consumer declaring the struct rejects a producer declaring the buffer whatever it actually sends — the manifest has to move with the payload.
• dataflow.yml — Example dora dataflow: the camera alone, no viewer and no Python dependency. Built from this checkout with path: + build: and strict_types: true. A consumer wires the node from a tag instead (see README) — dora 1.0 pins a source with the mutually exclusive tag:, rev: or branch:.
• dataflow_view.yml — The same camera node plus dora-image-view on image, depth and camera_info, fetched from a tag. Run it with --uv, which gives the viewer its own managed environment. The viewer's manifest is not read at validate time (only a path: node's is), so the file repeats its contract in input_types: to get the two checked against each other. dora 1.0 has no cross-dataflow wiring (an input's source must resolve to a node in the same descriptor), so the camera node is duplicated here rather than referenced, and the two files are run one at a time.
• tests/run.sh — Builds and runs the payload tests: tests/arrow_test.cc under -fsanitize=address,undefined (the release callbacks have to free the whole tree), and tests/test_arrow.py, which imports the exported arrays through pyarrow's C data interface and checks the values. tests/arrow_export.cc is the C shim the Python side calls. Covers src/arrow only, so it needs no camera, no ZED SDK and no dora.
• .gitea/workflows/validate.yml — CI gates on push and pull request: dora validate on both dataflows, and tests/run.sh. Neither needs a camera, the ZED SDK or CUDA.
• Dockerfile — Ubuntu 24.04 x86-64 build environment (CUDA 12.8, ZED SDK 5.2, Rust). No ARM64/Jetson image; build natively there. No camera needed; run the compile with --gpus all. Sets NVIDIA_DRIVER_CAPABILITIES=compute,utility,video — the video capability is what makes the container runtime mount NVDEC/NVENC, which libsl_zed.so links against. Rust lives at /usr/local/{rustup,cargo} rather than /root so the build can run as the invoking user. docker build -t dora-zed-build . && docker run --rm --gpus all --user "$(id -u):$(id -g)" -v "$PWD:/src" -v dora-zed-cargo:/usr/local/cargo/registry dora-zed-build.

Key APIs

• dora C++ API (<dora-node-api.h>): init_dora_node(), dora_node.events->next(), event_type(), send_arrow_output(), new_metadata().
• ZED SDK (<sl/Camera.hpp>): sl::Camera, sl::InitParameters, sl::RuntimeParameters, sl::Mat.
• Output data is sent as Arrow arrays: send_arrow_output takes an ArrowArray/ArrowSchema pair by pointer and moves both out, so a payload is built once with src/arrow and handed over.

Tools

• Issue tracker: Gitea. Use the Gitea MCP tools (mcp__gitea__*) to read and manage issues, pull requests, and other repository operations.

Rules

• Do not mention AI tools in commit messages or documentation.