liria/dora-zed
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

add native 3D object detection using ZED SDK optimized models

Browse Source 
Add engines::ZedNative class wrapping ZED SDK's TensorRT-optimized
object detection. On every frame grab, detected objects (label,
confidence, 3D position, 2D/3D bounding boxes) are serialized as a
flat float32 array and sent as a new "objects" dora output with
frame_id correlation to image_bgr.

Extract shared env-var helpers into src/util/env.h and expose
sl::Camera& via Camera::handle() for engine access.

Closes #3
This commit is contained in:
cristhian aguilera
2026-03-26 10:53:38 -03:00
parent 55a87d3a14
commit 6e6dc8ef27
10 changed files with 303 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+18 -1
View File
@@ -1,6 +1,6 @@
# dora-zed
Dora C++ node for ZED stereo cameras. Captures frames and publishes BGR images, XYZRGBA point clouds, and camera intrinsics.
Dora C++ node for ZED stereo cameras. Captures frames and publishes BGR images, XYZRGBA point clouds, camera intrinsics, and 3D object detections.
## Usage via `git:` in a dataflow
@@ -16,6 +16,7 @@ nodes:
- image_bgr
- point_cloud
- camera_info
- objects
```
Then run:
@@ -61,6 +62,9 @@ The binary is installed to `bin/dora_zed` relative to the repository root.
| `ZED_FLIP` | `ON` | Image flip: `ON` or `OFF` |
| `ZED_WARMUP_FRAMES` | `30` | Number of warmup frames to discard |
| `ZED_RETRY_INTERVAL_SECONDS` | `5` | Seconds between reconnection attempts |
| `ZED_DETECTION_MODEL` | `MULTI_CLASS_BOX_MEDIUM` | Detection model: `MULTI_CLASS_BOX_FAST`, `MULTI_CLASS_BOX_MEDIUM`, or `MULTI_CLASS_BOX_ACCURATE` |
| `ZED_DETECTION_CONFIDENCE` | `40` | Detection confidence threshold (0-100) |
| `ZED_DETECTION_TRACKING` | `false` | Enable object tracking across frames (enables positional tracking) |
## Outputs
@@ -72,3 +76,16 @@ XYZRGBA point cloud as float32 bytes. Metadata includes `shape` (HxWx4), `units`
### `camera_info`
Four float32 values: `[fx, fy, cx, cy]`. Metadata includes `distortion` coefficients.
### `objects`
Detected objects as a flat float32 array with shape `[num_objects, 37]`. Each row contains 37 floats:
| Index | Field | Description |
|---|---|---|
| 0 | `label_id` | Object class (0=PERSON, 1=VEHICLE, 2=BAG, 3=ANIMAL, 4=ELECTRONICS, 5=FRUIT_VEGETABLE, 6=SPORT) |
| 1 | `confidence` | Detection confidence (0-100) |
| 2-4 | `x, y, z` | 3D center position in mm |
| 5-12 | `bb2d_*` | 2D bounding box: 4 corners × 2 pixel coords |
| 13-36 | `bb3d_*` | 3D bounding box: 8 corners × 3 world coords (mm) |
Metadata includes `frame_id` and `timestamp_ns` matching the corresponding `image_bgr`, plus `image_width` and `image_height` for interpreting 2D bounding box pixel coordinates. When no objects are detected, an empty array with shape `[0, 37]` is sent.