dora-zed/AGENTS.md

AGENTS.md

This file provides guidance to AI coding assistants when working with code in this repository.

What this is

A dora-rs C++ node that interfaces with ZED stereo cameras. On each tick it grabs a frame and publishes it over the dora dataflow: a BGR image and a depth map every frame, camera intrinsics once per camera session, a camera state on every transition, and an XYZRGBA point cloud when ZED_POINT_CLOUD is on. Camera management lives in src/zed/ and the dora event loop in src/main.cc.

The node is a camera source only. It ran inference in-process through v0.1.0, under src/engines/; that was removed so the node has no inference dependency and builds on any platform the ZED SDK supports. The removed code remains reachable at tag v0.1.0.

Build

scripts/build.sh

The binary is installed to bin/dora_zed relative to the repo root. The script is also the dataflow's build: command: dora 1.0 executes build commands directly rather than through a shell, so cd, && and command substitution do not work there.

Requirements: dora 1.0+ (verified with dora-cli 1.0.1), CMake 3.21+, ZED SDK 5.x (at /usr/local/zed), CUDA Toolkit, Rust stable toolchain (for dora-rs C++ bindings).

The first build clones dora-rs from GitHub and compiles dora-node-api-cxx via Cargo. This takes a while. Subsequent builds are incremental. If DORA_ROOT_DIR is set to a local dora checkout, it builds against that instead.

For a build without installing the SDK stack on the host, see the Dockerfile (Ubuntu 24.04 — the newest release with a ZED SDK; there is no Stereolabs build for 26.04). Run the compile with --gpus all.

Run

dora up
dora start dataflow.yml       # camera alone
dora start dataflow_plot.yml  # camera plus the opencv-plot viewer

The two dataflows both declare the camera node, so run one at a time.

Camera behavior is configured entirely through environment variables (see README.md for the full list: ZED_RESOLUTION, ZED_FPS, ZED_DEPTH_MODE, etc.).

Architecture

• src/util/env.h — Shared inline env-variable parsing helpers (get_env_int, get_env_str, get_env_bool, get_env_float).
• src/zed/camera.h — zed::Config, zed::Calibration, and zed::Camera class. Camera wraps the ZED SDK lifecycle: open() (one attempt, returns success), warmup, grab (image + depth + point cloud), calibration, device timestamp, blank-frame check, close() and reset() (close plus sl::Camera::reboot() where the model allows it). Exposes handle() for engine access.
• src/zed/capture.h/cc — zed::Capture and zed::Frame. Owns the camera on a dedicated thread: open, warmup, grab, blank/stall detection, backoff and escalation all live here, along with the packing of every payload into owned buffers. The newest frame is handed to the reader by swapping under a mutex (take_frame), which also recycles the previous frame's buffers so steady-state capture does not reallocate. Nothing outside this file may call the ZED SDK — it blocks unboundedly on a disconnected camera, which is what froze the node when the loop drove the camera directly.
• src/zed/camera.cc — Implementation. SDK enum parsers live in an anonymous namespace. parse_depth_mode runs in Config::from_env(), not in the constructor, so an unrecognised ZED_DEPTH_MODE exits before the node joins the dataflow rather than falling back to a default.
• src/main.cc — Dora event loop and output publishing only. On any input event (intended to be driven by a tick timer; the id is not inspected): calls camera.grab(), then sends image_bgr and depth, plus point_cloud when ZED_POINT_CLOUD is on and camera_info on the first frame of a camera session (camera_info_pending, re-armed on reconnection). pack_depth copies the depth map into a tightly packed [H, W] float buffer (sl::Mat rows can be padded) and clears inf/nan to 0. set_intrinsics and set_frame_stamps stamp every output of a frame with the same intrinsics, frame_id and the two timestamps (device_timestamp_ns, host_timestamp_ns; both Unix-epoch nanoseconds). Point cloud output varies by mode: FULL sends the raw sl::Mat pointer (no copy); HALF downsamples by 2 in each dimension. The loop never touches the SDK: it drains take_state_change and take_frame from the capture thread and publishes. On exit it calls capture.stop() and waits 2 s for the thread; if the thread is still inside a blocking SDK call it gives up and _Exit(0)s rather than let dora time out and kill the node.
• DoraTargets.cmake — Handles building or fetching dora-rs and generating the CXX bridge files. Two paths: local (DORA_ROOT_DIR) or git clone. Produces dora_node_api_cxx library and the node_bridge.cc glue.
• scripts/build.sh — Configure/compile/install in one shell-free-callable step. Referenced by both dataflows, the Dockerfile and the README.
• dora-node.yml — Node manifest (apiVersion: 1): typed input/output contract, entrypoint, build command. dora validate injects these types into any dataflow that reaches the node by path:; a git:-sourced node does not pick it up, which is why both dataflows build from the checkout. Type URNs name the element type (std/core/v1/UInt8, .../Float32, .../String) because every payload is a raw buffer described by its metadata, not an Arrow struct.
• dataflow.yml — Example dora dataflow: the camera alone, no viewer and no Python dependency. Built from this checkout with path: + build: and strict_types: true. A consumer wires the node from a tag instead (see README) — dora 1.0 pins a source with the mutually exclusive tag:, rev: or branch:.
• dataflow_plot.yml — The same camera node plus an opencv-plot viewer on image_bgr. Split out because the viewer pulls dora-hub over pip and needs a display. dora 1.0 has no cross-dataflow wiring (an input's source must resolve to a node in the same descriptor), so the camera node is duplicated here rather than referenced, and the two files are run one at a time.
• .gitea/workflows/validate.yml — CI gate running dora validate on both dataflows on push and pull request. Needs no camera and no ZED SDK.
• Dockerfile — Ubuntu 24.04 x86-64 build environment (CUDA 12.8, ZED SDK 5.2, Rust). No ARM64/Jetson image; build natively there. No camera needed; run the compile with --gpus all. Sets NVIDIA_DRIVER_CAPABILITIES=compute,utility,video — the video capability is what makes the container runtime mount NVDEC/NVENC, which libsl_zed.so links against. Rust lives at /usr/local/{rustup,cargo} rather than /root so the build can run as the invoking user. docker build -t dora-zed-build . && docker run --rm --gpus all --user "$(id -u):$(id -g)" -v "$PWD:/src" -v dora-zed-cargo:/usr/local/cargo/registry dora-zed-build.

Key APIs

• dora C++ API (<dora-node-api.h>): init_dora_node(), dora_node.events->next(), event_type(), send_output_with_metadata(), new_metadata().
• ZED SDK (<sl/Camera.hpp>): sl::Camera, sl::InitParameters, sl::RuntimeParameters, sl::Mat.
• Output data is sent as raw byte slices (rust::Slice<const uint8_t>) with typed metadata (shape, dtype, encoding).

Tools

• Issue tracker: Gitea. Use the Gitea MCP tools (mcp__gitea__*) to read and manage issues, pull requests, and other repository operations.

Rules

• Do not mention AI tools in commit messages or documentation.