Dockerfile provisions CUDA 12.8, ZED SDK 5.2, ONNX Runtime and Rust on
Ubuntu 24.04, the newest release Stereolabs ships a ZED SDK for, so the
node can be compiled and the build verified without installing the SDK
stack on the host.
The compile needs --gpus all: libsl_zed.so links against the CUDA driver
API, NVDEC and NVENC, which ship with the display driver rather than the
CUDA toolkit and so exist in no image. The container runtime mounts them
from the host, and only includes NVDEC/NVENC when the video capability
is requested.
Runs as the invoking user so build/ and bin/ are not left root-owned,
which needs the toolchain outside /root and the ZED SDK directory
readable to an arbitrary uid.
ONNX Runtime is a configure-time dependency for every build, not only
for the yolo_onnx engine. The event loop no longer inspects the input
id, so it grabs on any input rather than specifically on tick.
Deletes the yoloworld_onnx engine, the CLIP BPE tokenizer, the standalone
test binary and both export scripts. Drops the update_vocabulary input and
the yoloworld_onnx/open_vocab dispatch from main.cc, along with the now
unused Engine::set_labels. Documentation follows.
Split the monolithic open_vocab engine into two focused engines:
yolo_onnx (closed-vocabulary, any single-input YOLO ONNX) and
yoloworld_onnx (open-vocabulary YOLO-World with CLIP encoder).
Shared NMS/letterbox/fuse helpers extracted to yolo_helpers.h.
Add export_yolo.py script and separate test binaries for each engine.
Three modes: FULL (full HxW grid, zero-copy), HALF (stride-2
downsample, ~4x reduction), ROI (valid points within detected
bounding boxes, flat [N,4] array). Parsed once before the event loop.
The open_vocab engine now accepts an update_vocabulary input event
carrying a UTF-8 comma-separated label string. Labels are re-encoded
immediately via the CLIP text encoder; the next tick uses the new
vocabulary. ZedNative ignores the input via a no-op base class method.
- Add OpenVocab engine: YOLO-World ONNX detection with runtime
vocabulary via CLIP text embeddings (two-input model support)
- Add header-only C++ CLIP BPE tokenizer (no external deps)
- Add detect(cv::Mat) overload to OpenVocab for camera-free testing
- Add standalone test binary (test_open_vocab) using test/assets images
- Add export scripts for YOLO-World and CLIP text encoder (uv/PEP 723)
- Add installation scripts for Linux x64, ARM (Jetson), and Windows
- Update CMakeLists to use find_package(onnxruntime) instead of FetchContent
- Update README and AGENTS with env vars, setup workflow, and test usage
Add engines::ZedNative class wrapping ZED SDK's TensorRT-optimized
object detection. On every frame grab, detected objects (label,
confidence, 3D position, 2D/3D bounding boxes) are serialized as a
flat float32 array and sent as a new "objects" dora output with
frame_id correlation to image_bgr.
Extract shared env-var helpers into src/util/env.h and expose
sl::Camera& via Camera::handle() for engine access.
Closes#3
Implements a dora-rs node that captures frames from ZED cameras
and publishes BGR images, XYZRGBA point clouds, and camera
intrinsics. Includes CMake build system with automatic dora-rs
C++ bindings, dataflow definition, and documentation.
cristhian
cristhian aguilera