dora-zed publishes camera data only. The two engines under src/engines/, the
objects output, the ZED_ENGINE/ZED_DETECTION_*/ZED_ONNX_* variables and the
ZED_PC_MODE=ROI crop are gone, along with the standalone engine test and the
model export script.
This drops the unconditional ONNX Runtime requirement from CMakeLists, the
Dockerfile and the install scripts, so the node now configures on a machine
with no ONNX Runtime — which is what blocked the Jetson build, since there is
no official ARM64 GPU binary.
zed_native is dropped rather than kept: it is bound to the live camera handle
and cannot run in a node that receives images over a dataflow. The removed
code remains reachable at tag v0.1.0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7g2cBJ5e3r15ZbJ9WZn8c
dora 1.0 runs a node's build command directly rather than through a shell, so
the mkdir/cd/&& sequence in dataflow.yml could not work. Move it to
scripts/build.sh and invoke that instead.
Default CMAKE_BUILD_TYPE to Release so DoraTargets.cmake picks the release
cargo profile and the binary does not link debug dora bindings.
Pin the dataflow source with tag: — 1.0 replaced git_checkout with the
mutually exclusive tag/rev/branch.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7g2cBJ5e3r15ZbJ9WZn8c
Dockerfile provisions CUDA 12.8, ZED SDK 5.2, ONNX Runtime and Rust on
Ubuntu 24.04, the newest release Stereolabs ships a ZED SDK for, so the
node can be compiled and the build verified without installing the SDK
stack on the host.
The compile needs --gpus all: libsl_zed.so links against the CUDA driver
API, NVDEC and NVENC, which ship with the display driver rather than the
CUDA toolkit and so exist in no image. The container runtime mounts them
from the host, and only includes NVDEC/NVENC when the video capability
is requested.
Runs as the invoking user so build/ and bin/ are not left root-owned,
which needs the toolchain outside /root and the ZED SDK directory
readable to an arbitrary uid.
ONNX Runtime is a configure-time dependency for every build, not only
for the yolo_onnx engine. The event loop no longer inspects the input
id, so it grabs on any input rather than specifically on tick.
Deletes the yoloworld_onnx engine, the CLIP BPE tokenizer, the standalone
test binary and both export scripts. Drops the update_vocabulary input and
the yoloworld_onnx/open_vocab dispatch from main.cc, along with the now
unused Engine::set_labels. Documentation follows.
Split the monolithic open_vocab engine into two focused engines:
yolo_onnx (closed-vocabulary, any single-input YOLO ONNX) and
yoloworld_onnx (open-vocabulary YOLO-World with CLIP encoder).
Shared NMS/letterbox/fuse helpers extracted to yolo_helpers.h.
Add export_yolo.py script and separate test binaries for each engine.
Three modes: FULL (full HxW grid, zero-copy), HALF (stride-2
downsample, ~4x reduction), ROI (valid points within detected
bounding boxes, flat [N,4] array). Parsed once before the event loop.
The open_vocab engine now accepts an update_vocabulary input event
carrying a UTF-8 comma-separated label string. Labels are re-encoded
immediately via the CLIP text encoder; the next tick uses the new
vocabulary. ZedNative ignores the input via a no-op base class method.
- Add OpenVocab engine: YOLO-World ONNX detection with runtime
vocabulary via CLIP text embeddings (two-input model support)
- Add header-only C++ CLIP BPE tokenizer (no external deps)
- Add detect(cv::Mat) overload to OpenVocab for camera-free testing
- Add standalone test binary (test_open_vocab) using test/assets images
- Add export scripts for YOLO-World and CLIP text encoder (uv/PEP 723)
- Add installation scripts for Linux x64, ARM (Jetson), and Windows
- Update CMakeLists to use find_package(onnxruntime) instead of FetchContent
- Update README and AGENTS with env vars, setup workflow, and test usage
Add engines::ZedNative class wrapping ZED SDK's TensorRT-optimized
object detection. On every frame grab, detected objects (label,
confidence, 3D position, 2D/3D bounding boxes) are serialized as a
flat float32 array and sent as a new "objects" dora output with
frame_id correlation to image_bgr.
Extract shared env-var helpers into src/util/env.h and expose
sl::Camera& via Camera::handle() for engine access.
Closes#3
Implements a dora-rs node that captures frames from ZED cameras
and publishes BGR images, XYZRGBA point clouds, and camera
intrinsics. Includes CMake build system with automatic dora-rs
C++ bindings, dataflow definition, and documentation.
cristhian