Files

T

cesarandClaude Opus 5 d8956f269d fix camera recovery blocking the dora event loop

Camera::open() retried forever inside a fixed sleep, so a camera that did not
come back left the node stuck there: it stopped reading dora events and no
longer honoured Stop or AllInputsClosed.

Recovery is now a state machine over ticks. open() makes one attempt and
returns; the loop decides when to try again from a 1/2/5/10 s backoff deadline,
and nothing sleeps, so events keep being read while the camera is down. The
first attempt closes the handle; past that the device is wedged and the node
escalates to sl::Camera::reboot(), guarded to the ZED 2 and newer since it does
nothing on the ZED and ZED Mini. Reopening pins the serial so recovery gets the
same physical camera back.

Set async_grab_camera_recovery so a link hiccup returns CAMERA_REBOOTING
immediately instead of blocking the grab for ~17 s, and enable_image_validity_
check so the SDK flags frames a stalled camera hands back looking valid. Those
transient errors no longer drop the handle, since the SDK is already
reconnecting it.

An all-black frame now counts as a failed grab rather than being published, and
a camera delivering nothing usable for 5 s (30 s while the SDK reports it is
reconnecting) triggers recovery. A new camera_state output publishes
opening/streaming/recovering so a downstream node can tell "no frame yet" from
"camera lost".

ZED_RETRY_INTERVAL_SECONDS is replaced by the backoff.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7g2cBJ5e3r15ZbJ9WZn8c

2026-09-08 09:26:17 -03:00

5.5 KiB

Raw Blame History

AGENTS.md

This file provides guidance to AI coding assistants when working with code in this repository.

What this is

A dora-rs C++ node that interfaces with ZED stereo cameras. On each tick it grabs a frame and publishes it over the dora dataflow: a BGR image, an XYZRGBA point cloud and camera intrinsics. Camera management lives in src/zed/ and the dora event loop in src/main.cc.

The node is a camera source only. It ran inference in-process through v0.1.0, under src/engines/; that was removed so the node has no inference dependency and builds on any platform the ZED SDK supports. The removed code remains reachable at tag v0.1.0.

Build

scripts/build.sh

The binary is installed to bin/dora_zed relative to the repo root. The script is also the dataflow's build: command: dora 1.0 executes build commands directly rather than through a shell, so cd, && and command substitution do not work there.

Requirements: dora 1.0+ (verified with dora-cli 1.0.1), CMake 3.21+, ZED SDK 5.x (at /usr/local/zed), CUDA Toolkit, Rust stable toolchain (for dora-rs C++ bindings).

The first build clones dora-rs from GitHub and compiles dora-node-api-cxx via Cargo. This takes a while. Subsequent builds are incremental. If DORA_ROOT_DIR is set to a local dora checkout, it builds against that instead.

For a build without installing the SDK stack on the host, see the Dockerfile (Ubuntu 24.04 — the newest release with a ZED SDK; there is no Stereolabs build for 26.04). Run the compile with --gpus all.

Run

dora up
dora start dataflow.yml

Camera behavior is configured entirely through environment variables (see README.md for the full list: ZED_RESOLUTION, ZED_FPS, ZED_DEPTH_MODE, etc.).

Architecture

src/util/env.h — Shared inline env-variable parsing helpers (get_env_int, get_env_str, get_env_bool, get_env_float).
src/zed/camera.h — zed::Config, zed::Calibration, and zed::Camera class. Camera wraps the ZED SDK lifecycle: open() (one attempt, returns success), warmup, grab (image + depth + point cloud), calibration, device timestamp, blank-frame check, close() and reset() (close plus sl::Camera::reboot() where the model allows it). Exposes handle() for engine access.
src/zed/camera.cc — Implementation. SDK enum parsers live in an anonymous namespace. parse_depth_mode runs in Config::from_env(), not in the constructor, so an unrecognised ZED_DEPTH_MODE exits before the node joins the dataflow rather than falling back to a default.
src/main.cc — Dora event loop. Parses ZED_PC_MODE (FULL/HALF) once before the loop. On any input event (intended to be driven by a tick timer; the id is not inspected): calls camera.grab(), then sends image_bgr and depth, plus point_cloud when ZED_POINT_CLOUD is on and camera_info on the first frame of a camera session (camera_info_pending, re-armed on reconnection). pack_depth copies the depth map into a tightly packed [H, W] float buffer (sl::Mat rows can be padded) and clears inf/nan to 0. set_intrinsics and set_frame_stamps stamp every output of a frame with the same intrinsics, frame_id and the two timestamps (device_timestamp_ns, host_timestamp_ns; both Unix-epoch nanoseconds). Point cloud output varies by mode: FULL sends the raw sl::Mat pointer (no copy); HALF downsamples by 2 in each dimension. Recovery is a state machine over ticks rather than a blocking retry: the camera is opened from inside the loop, begin_recovery closes (first attempt) or resets (later ones) and sets a backoff deadline, and camera_state is published on each transition. Nothing sleeps, so Stop is honoured while the camera is down.
DoraTargets.cmake — Handles building or fetching dora-rs and generating the CXX bridge files. Two paths: local (DORA_ROOT_DIR) or git clone. Produces dora_node_api_cxx library and the node_bridge.cc glue.
scripts/build.sh — Configure/compile/install in one shell-free-callable step. Referenced by dataflow.yml, the Dockerfile and the README.
dataflow.yml — Example dora dataflow definition using the git: source format, pinned with tag: — dora 1.0 pins a source with the mutually exclusive tag:, rev: or branch:.
Dockerfile — Ubuntu 24.04 x86-64 build environment (CUDA 12.8, ZED SDK 5.2, Rust). No ARM64/Jetson image; build natively there. No camera needed; run the compile with --gpus all. Sets NVIDIA_DRIVER_CAPABILITIES=compute,utility,video — the video capability is what makes the container runtime mount NVDEC/NVENC, which libsl_zed.so links against. Rust lives at /usr/local/{rustup,cargo} rather than /root so the build can run as the invoking user. docker build -t dora-zed-build . && docker run --rm --gpus all --user "$(id -u):$(id -g)" -v "$PWD:/src" -v dora-zed-cargo:/usr/local/cargo/registry dora-zed-build.

Key APIs

dora C++ API (<dora-node-api.h>): init_dora_node(), dora_node.events->next(), event_type(), send_output_with_metadata(), new_metadata().
ZED SDK (<sl/Camera.hpp>): sl::Camera, sl::InitParameters, sl::RuntimeParameters, sl::Mat.
Output data is sent as raw byte slices (rust::Slice<const uint8_t>) with typed metadata (shape, dtype, encoding).

Tools

Issue tracker: Gitea. Use the Gitea MCP tools (mcp__gitea__*) to read and manage issues, pull requests, and other repository operations.

Rules

Do not mention AI tools in commit messages or documentation.

5.5 KiB Raw Blame History