liria/dora-zed
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

remove the YOLO-World open-vocabulary engine

Browse Source 
Deletes the yoloworld_onnx engine, the CLIP BPE tokenizer, the standalone
test binary and both export scripts. Drops the update_vocabulary input and
the yoloworld_onnx/open_vocab dispatch from main.cc, along with the now
unused Engine::set_labels. Documentation follows.
This commit is contained in:
cristhian
2026-09-07 05:10:59 -03:00
parent 44eec29b83
commit f1934c9b4a
14 changed files with 8 additions and 1132 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+2 -47
View File
@@ -32,7 +32,7 @@ See [`dataflow.yml`](dataflow.yml) for a complete runnable example with a camera
| [ZED SDK](https://www.stereolabs.com/developers/release) | 5.x | Bundles OpenCV; default path `/usr/local/zed` |
| [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) | — | Required by ZED SDK |
| [Rust toolchain](https://rustup.rs) | stable | Builds dora-rs C++ bindings |
| [ONNX Runtime](https://github.com/microsoft/onnxruntime/releases) | 1.20+ | Required only for `open_vocab` engine |
| [ONNX Runtime](https://github.com/microsoft/onnxruntime/releases) | 1.20+ | Required only for the `yolo_onnx` engine |
### Installation scripts
@@ -42,7 +42,7 @@ See [`dataflow.yml`](dataflow.yml) for a complete runnable example with a camera
| NVIDIA Jetson (ARM64) | `sudo scripts/install_linux_arm.sh` |
| Windows x64 (PowerShell admin) | `scripts/install_windows.ps1` |
> **Jetson:** No official ONNX Runtime GPU binary for ARM64. The `native` engine works without it. The `open_vocab` engine requires [building from source](https://onnxruntime.ai/docs/build/eps.html#tensorrt).
> **Jetson:** No official ONNX Runtime GPU binary for ARM64. The `native` engine works without it. The `yolo_onnx` engine requires [building from source](https://onnxruntime.ai/docs/build/eps.html#tensorrt).
>
> **Windows:** Install ZED SDK manually from [stereolabs.com](https://www.stereolabs.com/developers/release), then pass the ONNX Runtime path: `cmake .. -DCMAKE_PREFIX_PATH=C:\onnxruntime`
@@ -82,7 +82,6 @@ Set `ZED_ENGINE` to select the detection backend:
|---|---|---|
| `native` (default) | No — ZED SDK built-in TensorRT | Fixed: 7 categories |
| `yolo_onnx` (alias: `onnx`) | Yes — any single-input YOLO ONNX (v5/v8/v11) | Fixed labels from file or env var |
| `yoloworld_onnx` (alias: `open_vocab`) | Yes — YOLO-World two-input ONNX | Defined at runtime via CLIP |
#### `native` engine
@@ -131,49 +130,6 @@ dora start dataflow.yml
| `ZED_ONNX_INPUT_SIZE` | `640` | Model input size (px) |
| `ZED_ONNX_OUTPUT_FORMAT` | `auto` | Output layout: `auto`, `nc_first`/`yolov8`/`yolov11`, `nc_last`, `nc_last_obj`/`yolov5`. `auto` detects from tensor shape. |
#### `yoloworld_onnx` engine (open-vocabulary)
For YOLO-World two-input ONNX models with a CLIP text encoder. Labels are defined at runtime — no recompilation required. Use `ZED_ENGINE=open_vocab` as a legacy alias.
Export models once:
```bash
uv run scripts/export_yoloworld.py --model s --num-classes 10 --output-dir models/
uv run scripts/export_clip_text_encoder.py --output-dir models/
```
Then run:
```bash
ZED_ENGINE=yoloworld_onnx \
ZED_ONNX_MODEL_PATH=models/yolov8s-worldv2_n10_i640.onnx \
ZED_ONNX_TEXT_ENCODER_PATH=models/clip_text_encoder.onnx \
ZED_ONNX_CLIP_VOCAB=models/clip_vocab.txt \
ZED_ONNX_CLIP_MERGES=models/clip_merges.txt \
ZED_ONNX_LABELS="person,car,chair" \
dora start dataflow.yml
```
**Required:**
| Variable | Description |
|---|---|
| `ZED_ONNX_MODEL_PATH` | Path to YOLO-World ONNX model |
| `ZED_ONNX_TEXT_ENCODER_PATH` | Path to `clip_text_encoder.onnx` |
| `ZED_ONNX_CLIP_VOCAB` | Path to `clip_vocab.txt` |
| `ZED_ONNX_CLIP_MERGES` | Path to `clip_merges.txt` |
| `ZED_ONNX_LABELS` | Comma-separated labels, e.g. `person,car,chair` |
**Optional:**
| Variable | Default | Description |
|---|---|---|
| `ZED_ONNX_CONFIDENCE` | `0.25` | Confidence threshold (0.0–1.0) |
| `ZED_ONNX_NMS_THRESHOLD` | `0.45` | NMS IoU threshold |
| `ZED_ONNX_INPUT_SIZE` | `640` | Model input size (px) |
Labels can be updated at runtime without restarting — send a new comma-separated string to the `update_vocabulary` input.
### Point cloud
| `ZED_PC_MODE` | Shape | Description |
@@ -187,7 +143,6 @@ Labels can be updated at runtime without restarting — send a new comma-separat
| Input | Description |
|---|---|
| `tick` | Triggers a camera grab and detection. Connect to `dora/timer/millis/N`. |
| `update_vocabulary` | (`open_vocab` only) UTF-8 comma-separated label string. Re-encodes CLIP embeddings immediately. |
## Outputs