liria/dora-zed
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

remove detection from the node

Browse Source 
dora-zed publishes camera data only. The two engines under src/engines/, the
objects output, the ZED_ENGINE/ZED_DETECTION_*/ZED_ONNX_* variables and the
ZED_PC_MODE=ROI crop are gone, along with the standalone engine test and the
model export script.

This drops the unconditional ONNX Runtime requirement from CMakeLists, the
Dockerfile and the install scripts, so the node now configures on a machine
with no ONNX Runtime — which is what blocked the Jetson build, since there is
no official ARM64 GPU binary.

zed_native is dropped rather than kept: it is bound to the live camera handle
and cannot run in a node that receives images over a dataflow. The removed
code remains reachable at tag v0.1.0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7g2cBJ5e3r15ZbJ9WZn8c
This commit is contained in:
cesar
2026-09-08 09:12:40 -03:00
co-authored by Claude Opus 5
parent a734692b60
commit 126dbf4f32
17 changed files with 18 additions and 1242 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+4 -80
View File
@@ -1,6 +1,6 @@
# dora-zed
Dora node for ZED stereo cameras. Publishes BGR images, point clouds, camera intrinsics, and 3D object detections.
Dora node for ZED stereo cameras. Publishes BGR images, point clouds and camera intrinsics.
## Usage
@@ -19,7 +19,6 @@ nodes:
- image_bgr
- point_cloud
- camera_info
- objects
```
See [`dataflow.yml`](dataflow.yml) for a complete runnable example with a camera display node.
@@ -33,7 +32,6 @@ See [`dataflow.yml`](dataflow.yml) for a complete runnable example with a camera
| [ZED SDK](https://www.stereolabs.com/developers/release) | 5.x | Bundles OpenCV; default path `/usr/local/zed` |
| [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) | — | Required by ZED SDK |
| [Rust toolchain](https://rustup.rs) | stable | Builds dora-rs C++ bindings |
| [ONNX Runtime](https://github.com/microsoft/onnxruntime/releases) | 1.20+ | Required at configure time; used at runtime only by `yolo_onnx` |
### Installation scripts
@@ -43,9 +41,7 @@ See [`dataflow.yml`](dataflow.yml) for a complete runnable example with a camera
| NVIDIA Jetson (ARM64) | `sudo scripts/install_linux_arm.sh` |
| Windows x64 (PowerShell admin) | `scripts/install_windows.ps1` |
> **Jetson:** No official ONNX Runtime GPU binary for ARM64. It is still needed to configure the build; only the `yolo_onnx` engine uses it at runtime. Requires [building from source](https://onnxruntime.ai/docs/build/eps.html#tensorrt).
>
> **Windows:** Install ZED SDK manually from [stereolabs.com](https://www.stereolabs.com/developers/release), then pass the ONNX Runtime path: `cmake .. -DCMAKE_PREFIX_PATH=C:\onnxruntime`
> **Windows:** Install the ZED SDK manually from [stereolabs.com](https://www.stereolabs.com/developers/release).
## Local build
@@ -65,7 +61,7 @@ bindings.
### Containerized build
[`Dockerfile`](Dockerfile) provides the full toolchain — CUDA, ZED SDK, ONNX Runtime and Rust — on Ubuntu 24.04, the newest release Stereolabs ships a ZED SDK for. No camera is needed.
[`Dockerfile`](Dockerfile) provides the full toolchain — CUDA, ZED SDK and Rust — on Ubuntu 24.04, the newest release Stereolabs ships a ZED SDK for. No camera is needed.
```bash
docker build -t dora-zed-build .
@@ -106,75 +102,18 @@ All behavior is controlled via environment variables.
| `ZED_WARMUP_FRAMES` | `30` | Frames to discard on startup |
| `ZED_RETRY_INTERVAL_SECONDS` | `5` | Reconnection interval (s) |
### Detection engine
Set `ZED_ENGINE` to select the detection backend:
| `ZED_ENGINE` | Custom model? | Classes |
|---|---|---|
| `native` (default) | No — ZED SDK built-in TensorRT | Fixed: 7 categories |
| `yolo_onnx` (alias: `onnx`) | Yes — any single-input YOLO ONNX (v5/v8/v11) | Fixed labels from file or env var |
#### `native` engine
Uses ZED SDK's built-in TensorRT models — no model files to download or configure. `ZED_DETECTION_MODEL` selects the speed/accuracy variant, not a file path.
Detects 7 fixed categories: PERSON, VEHICLE, BAG, ANIMAL, ELECTRONICS, FRUIT_VEGETABLE, SPORT.
| Variable | Default | Description |
|---|---|---|
| `ZED_DETECTION_MODEL` | `MULTI_CLASS_BOX_MEDIUM` | `MULTI_CLASS_BOX_FAST`, `MULTI_CLASS_BOX_MEDIUM`, or `MULTI_CLASS_BOX_ACCURATE` |
| `ZED_DETECTION_CONFIDENCE` | `40` | Confidence threshold (0–100) |
| `ZED_DETECTION_TRACKING` | `false` | Enable object tracking |
#### `yolo_onnx` engine (closed-vocabulary)
For any single-input YOLO ONNX model with fixed class names (YOLOv8, YOLOv11, YOLOv5, custom-trained).
Export a YOLOv11 model once:
```bash
uv run scripts/export_yolo.py --model n --output-dir models/
```
Then run:
```bash
ZED_ENGINE=yolo_onnx \
ZED_ONNX_MODEL_PATH=models/yolo11n_i640.onnx \
ZED_ONNX_LABELS_FILE=models/coco80.txt \
dora start dataflow.yml
```
**Required:**
| Variable | Description |
|---|---|
| `ZED_ONNX_MODEL_PATH` | Path to YOLO ONNX model |
| `ZED_ONNX_LABELS` or `ZED_ONNX_LABELS_FILE` | Class names (inline CSV or one-per-line file) |
**Optional:**
| Variable | Default | Description |
|---|---|---|
| `ZED_ONNX_CONFIDENCE` | `0.25` | Confidence threshold (0.0–1.0) |
| `ZED_ONNX_NMS_THRESHOLD` | `0.45` | NMS IoU threshold |
| `ZED_ONNX_INPUT_SIZE` | `640` | Model input size (px) |
| `ZED_ONNX_OUTPUT_FORMAT` | `auto` | Output layout: `auto`, `nc_first`/`yolov8`/`yolov11`, `nc_last`, `nc_last_obj`/`yolov5`. `auto` detects from tensor shape. |
### Point cloud
| `ZED_PC_MODE` | Shape | Description |
|---|---|---|
| `FULL` (default) | `[H, W, 4]` | Full resolution grid (~14.7 MB/frame at 720p) |
| `HALF` | `[H/2, W/2, 4]` | Stride-2 downsample (~4× smaller) |
| `ROI` | `[N, 4]` | Valid points within detected bounding boxes only |
## Inputs
| Input | Description |
|---|---|
| `tick` | Triggers a camera grab and detection. Connect to `dora/timer/millis/N`. |
| `tick` | Triggers a camera grab. Connect to `dora/timer/millis/N`. |
## Outputs
@@ -183,18 +122,3 @@ dora start dataflow.yml
| `image_bgr` | `uint8 [H, W, 3]` | BGR image. Metadata: `shape`, `encoding=bgr8`, intrinsics, `timestamp_ns`. |
| `point_cloud` | `float32 [*, 4]` | XYZRGBA point cloud. Shape depends on `ZED_PC_MODE`. Metadata: `units=mm`, intrinsics. |
| `camera_info` | `float32 [4]` | `[fx, fy, cx, cy]`. Metadata: `distortion` coefficients. |
| `objects` | `float32 [N, 37]` | Detected objects. See below. |
### `objects` layout
Each of the N rows contains 37 floats:
| Index | Field | Description |
|---|---|---|
| 0 | `label_id` | Class index |
| 1 | `confidence` | Score (0–100) |
| 2–4 | `x, y, z` | 3D center (mm) |
| 5–12 | `bb2d` | 2D bounding box: 4 corners × (x, y) px |
| 13–36 | `bb3d` | 3D bounding box: 8 corners × (x, y, z) mm |
Metadata includes `engine`, `label_names`, `image_width`, `image_height`, and `timestamp_ns`. An empty array `[0, 37]` is sent when no objects are detected.