dora-image-view takes std/media/v1/Image, and dora's compatibility graph
has no edge from std/core/v1/UInt8 to it, so wiring the two failed
`dora validate` whatever the node actually sent. Its flat-buffer fallback
would not have decoded these frames either: it reads width/height
metadata, and this node only published shape.
image and depth now go out as the Image struct, camera_info and
point_cloud as Float32 arrays and camera_state as a string, all built
against the Arrow C data interface in src/arrow so the node still links
no Arrow C++. Shape, dtype and encoding metadata are dropped where the
payload describes itself; intrinsics, frame_id and both timestamps are
unchanged. tests/run.sh covers the packing without a camera or the SDK,
round-tripping each array through pyarrow and rebuilding it under the
sanitizers.
Breaking: image_bgr is renamed to image and every payload changes shape.
dataflow_plot.yml is replaced by dataflow_view.yml, which wires the
viewer from a tag and repeats its contract in input_types so validate
checks both sides of the wiring.
Hardware testing showed the tick-driven recovery state machine did not work.
async_grab_camera_recovery does not make grab() prompt on a disconnected ZED-M
under SDK 5.3: grabs took ~3 s each while the link was down and then blocked
indefinitely. The event loop stopped turning, so Stop went unanswered, the
30 s patience timer and the backoff were never reached, and replugging the
camera did nothing. Measured: 122 s frozen, then dora stop needed 10 s and a
forced kill.
Move everything that can block onto a capture thread. zed::Capture owns the
camera and runs open, warmup, grab, blank and stall detection, backoff and
escalation; it packs each frame into owned buffers and hands the newest one to
the reader by swapping under a mutex. The dora event loop now only drains that
buffer and publishes, so it never calls into the SDK and stays responsive
however long the SDK blocks.
The newest frame wins and uncollected frames are dropped, which is what a
consumer wants from a camera source. Swapping rather than assigning returns the
previous frame's buffers to the capture thread, so steady state does not
reallocate. The point cloud is now copied out of the SDK buffer rather than
published in place, since the reader is another thread.
On exit the loop waits 2 s for the thread and then _Exit()s if it is still
inside the SDK, so Stop is always answered promptly.
This also removes the blocking warmup from the event loop: it stays a blocking
burst, which is harmless on the capture thread.
Verified on a ZED-M: unplug leaves the node responsive and logs the backoff
1, 2, 5 then 10 s; replug gives "camera recovered after 13 attempt(s)" and
publishing resumes without restarting the dataflow; dora stop with the camera
unplugged returns in 0.70 s with the thread joining gracefully.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7g2cBJ5e3r15ZbJ9WZn8c
dora-zed publishes camera data only. The two engines under src/engines/, the
objects output, the ZED_ENGINE/ZED_DETECTION_*/ZED_ONNX_* variables and the
ZED_PC_MODE=ROI crop are gone, along with the standalone engine test and the
model export script.
This drops the unconditional ONNX Runtime requirement from CMakeLists, the
Dockerfile and the install scripts, so the node now configures on a machine
with no ONNX Runtime — which is what blocked the Jetson build, since there is
no official ARM64 GPU binary.
zed_native is dropped rather than kept: it is bound to the live camera handle
and cannot run in a node that receives images over a dataflow. The removed
code remains reachable at tag v0.1.0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7g2cBJ5e3r15ZbJ9WZn8c
dora 1.0 runs a node's build command directly rather than through a shell, so
the mkdir/cd/&& sequence in dataflow.yml could not work. Move it to
scripts/build.sh and invoke that instead.
Default CMAKE_BUILD_TYPE to Release so DoraTargets.cmake picks the release
cargo profile and the binary does not link debug dora bindings.
Pin the dataflow source with tag: — 1.0 replaced git_checkout with the
mutually exclusive tag/rev/branch.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7g2cBJ5e3r15ZbJ9WZn8c
Deletes the yoloworld_onnx engine, the CLIP BPE tokenizer, the standalone
test binary and both export scripts. Drops the update_vocabulary input and
the yoloworld_onnx/open_vocab dispatch from main.cc, along with the now
unused Engine::set_labels. Documentation follows.
Split the monolithic open_vocab engine into two focused engines:
yolo_onnx (closed-vocabulary, any single-input YOLO ONNX) and
yoloworld_onnx (open-vocabulary YOLO-World with CLIP encoder).
Shared NMS/letterbox/fuse helpers extracted to yolo_helpers.h.
Add export_yolo.py script and separate test binaries for each engine.
- Add OpenVocab engine: YOLO-World ONNX detection with runtime
vocabulary via CLIP text embeddings (two-input model support)
- Add header-only C++ CLIP BPE tokenizer (no external deps)
- Add detect(cv::Mat) overload to OpenVocab for camera-free testing
- Add standalone test binary (test_open_vocab) using test/assets images
- Add export scripts for YOLO-World and CLIP text encoder (uv/PEP 723)
- Add installation scripts for Linux x64, ARM (Jetson), and Windows
- Update CMakeLists to use find_package(onnxruntime) instead of FetchContent
- Update README and AGENTS with env vars, setup workflow, and test usage
Add engines::ZedNative class wrapping ZED SDK's TensorRT-optimized
object detection. On every frame grab, detected objects (label,
confidence, 3D position, 2D/3D bounding boxes) are serialized as a
flat float32 array and sent as a new "objects" dora output with
frame_id correlation to image_bgr.
Extract shared env-var helpers into src/util/env.h and expose
sl::Camera& via Camera::handle() for engine access.
Closes#3
Extract camera lifecycle (open, warmup, grab, close) and
environment-based configuration into src/zed/camera.{h,cc}.
Thin main.cc now only handles the dora event loop.
Closes#2
Implements a dora-rs node that captures frames from ZED cameras
and publishes BGR images, XYZRGBA point clouds, and camera
intrinsics. Includes CMake build system with automatic dora-rs
C++ bindings, dataflow definition, and documentation.
cristhian
