# dora-zed

Dora node for ZED stereo cameras. Publishes BGR images, point clouds and camera intrinsics.

## Usage

Add to your dataflow:

```yaml
nodes:
  - id: zed-camera
    git: ssh://git@gitea.liria.cl:5001/liria/dora-zed.git
    tag: v0.2.0
    build: scripts/build.sh
    path: bin/dora_zed
    inputs:
      tick: dora/timer/millis/100
    outputs:
      - image_bgr
      - depth
      - point_cloud
      - camera_info
```

See [`dataflow.yml`](dataflow.yml) for a complete runnable example with a camera display node.

## Dependencies

| Dependency | Version | Notes |
|---|---|---|
| [dora-rs](https://github.com/dora-rs/dora) | 1.0+ | Verified against `dora-cli 1.0.1`; the C++ bindings are built from source at configure time |
| [CMake](https://cmake.org/download/) | 3.21+ | |
| [ZED SDK](https://www.stereolabs.com/developers/release) | 5.x | Bundles OpenCV; default path `/usr/local/zed` |
| [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) | — | Required by ZED SDK |
| [Rust toolchain](https://rustup.rs) | stable | Builds dora-rs C++ bindings |

### Installation scripts

| Platform | Script |
|---|---|
| Ubuntu 24.04 x86_64 | `sudo scripts/install_linux_x64.sh` |
| NVIDIA Jetson (ARM64) | `sudo scripts/install_linux_arm.sh` |
| Windows x64 (PowerShell admin) | `scripts/install_windows.ps1` |

> **Windows:** Install the ZED SDK manually from [stereolabs.com](https://www.stereolabs.com/developers/release).

## Local build

```bash
git clone ssh://git@gitea.liria.cl:5001/liria/dora-zed.git
cd dora-zed
scripts/build.sh
```

Binary installed to `bin/dora_zed`.

`scripts/build.sh` is also the `build:` command in the dataflow. dora 1.0 runs build
commands directly rather than through a shell, so `cd`, `&&` and command substitution are
not available there; keeping the sequence in one script is what makes the dataflow build
work. It configures a `Release` build, which is also what links the optimized dora
bindings.

### Containerized build

[`Dockerfile`](Dockerfile) provides the full toolchain — CUDA, ZED SDK and Rust — on Ubuntu 24.04, the newest release Stereolabs ships a ZED SDK for. No camera is needed.

```bash
docker build -t dora-zed-build .
docker run --rm --gpus all --user "$(id -u):$(id -g)" \
    -v "$PWD:/src" -v dora-zed-cargo:/usr/local/cargo/registry dora-zed-build
```

The container runs the same `scripts/build.sh`.

The first run clones dora-rs and compiles it with Cargo; the named volume keeps that cache across runs.
`--user` keeps `build/` and `bin/` owned by you rather than root.

x86-64 only. On Jetson, run [`scripts/install_linux_arm.sh`](scripts/install_linux_arm.sh) on the
device and build natively — there is no ARM64 image here.

Needs an NVIDIA driver and the
[NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
on the host, as the ZED SDK does anyway. The image sets
`NVIDIA_DRIVER_CAPABILITIES=compute,utility,video`: `libsl_zed.so` links against NVDEC
(`libnvcuvid`) and NVENC (`libnvidia-encode`), which the container runtime mounts from the host
only when `video` is in the capability list.

## Configuration

All behavior is controlled via environment variables.

### Camera

| Variable | Default | Description |
|---|---|---|
| `ZED_RESOLUTION` | `1080` | `720`, `1080`, or `2000` |
| `ZED_FPS` | `15` | Frame rate |
| `ZED_DEPTH_MODE` | `NEURAL` | `NEURAL`, `NEURAL_PLUS`, `NEURAL_LIGHT`, `PERFORMANCE`, `QUALITY`, or `NONE`. An unrecognised value is a startup error, not a fallback. |
| `ZED_DEPTH_MIN_MM` | `200` | Minimum depth (mm) |
| `ZED_DEPTH_MAX_MM` | `600` | Maximum depth (mm) |
| `ZED_CONFIDENCE_THRESHOLD` | `60` | Reject depth less certain than this (1–100, lower is stricter) |
| `ZED_TEXTURE_CONFIDENCE_THRESHOLD` | `100` | Reject depth on surfaces less textured than this (1–100) |
| `ZED_DEPTH_FILL` | `false` | Interpolate depth over holes. Overrides both confidence thresholds. |
| `ZED_FLIP` | `ON` | `ON` or `OFF` |
| `ZED_WARMUP_FRAMES` | `30` | Frames to discard on startup |
| `ZED_RETRY_INTERVAL_SECONDS` | `5` | Reconnection interval (s) |

The resolved depth mode, both confidence thresholds and the depth range are logged at
startup. The defaults match the reference camera driver the perception stack was tuned
against, so depth quality on the robot matches what it expects.

### Point cloud

| `ZED_PC_MODE` | Shape | Description |
|---|---|---|
| `FULL` (default) | `[H, W, 4]` | Full resolution grid (~14.7 MB/frame at 720p) |
| `HALF` | `[H/2, W/2, 4]` | Stride-2 downsample (~4× smaller) |

## Inputs

| Input | Description |
|---|---|
| `tick` | Triggers a camera grab. Connect to `dora/timer/millis/N`. |

## Outputs

| Output | Type | Description |
|---|---|---|
| `image_bgr` | `uint8 [H, W, 3]` | BGR image. Metadata: `shape`, `encoding=bgr8`, intrinsics, timestamps. |
| `depth` | `float32 [H, W]` | Per-pixel depth in millimetres, published on every frame `image_bgr` is. Metadata: `dtype`, `layout=HW`, `shape`, `units=mm`, intrinsics, timestamps. `H` and `W` match `image_bgr` for the same `frame_id`. Missing depth is `0` — the payload contains no `inf` and no `nan`. Holes are not interpolated unless `ZED_DEPTH_FILL` is set. |
| `point_cloud` | `float32 [*, 4]` | XYZRGBA point cloud. Shape depends on `ZED_PC_MODE`. Metadata: `units=mm`, intrinsics, timestamps. |
| `camera_info` | `float32 [4]` | `[fx, fy, cx, cy]`. Metadata: `distortion` coefficients, timestamps. |

Consumers that need 3D back-project `depth` with the published intrinsics rather than
consuming `point_cloud`; the point cloud is not a prerequisite for 3D.

### Timestamps

Every output carries both clocks, as dora `timestamp` metadata parameters:

| Key | Epoch | Unit | Source |
|---|---|---|---|
| `device_timestamp_ns` | Unix epoch (UTC) | nanoseconds | `sl::Camera::getTimestamp(TIME_REFERENCE::IMAGE)` — when the camera captured the frame |
| `host_timestamp_ns` | Unix epoch (UTC) | nanoseconds | `std::chrono::system_clock` on the host — when this node published the frame |

Both are needed. The device timestamp is the capture instant and is the value two
nodes reading the same frame agree on; the host timestamp is the one the rest of the
dataflow and the session recorder share. On a Jetson the wall clock can step when NTP
synchronises after boot, so the device timestamp is recorded alongside the host one
rather than in place of it.

The node calls `sl::setTimestampClock(SYSTEM_CLOCK)` before opening the camera, which
is what makes the device timestamp a Unix-epoch value rather than an SDK-default clock.

Every output for a frame also carries the same `frame_id`, an integer counter starting
at 0.