Merge pull request 'Publish typed Arrow payloads and wire dora-image-view' (#20) from typed-image-payload into main

Reviewed-on: #20

.gitea/workflows/validate.yml

@@ -1,5 +1,6 @@
-# Checks both dataflows against the typed contract in dora-node.yml. This runs
+# Two gates, neither of which needs a camera, the ZED SDK or CUDA: the dataflows
-# no node, so it needs neither a camera nor the ZED SDK.
+# are checked against the typed contract in dora-node.yml without running the
+# node, and the payload packing is compiled and tested on its own.
 name: validate
 
 on: [push, pull_request]
@@ -28,9 +29,22 @@ jobs:
 
       # strict_types: true in each dataflow turns a type warning into a
       # non-zero exit, so a mismatched annotation fails here rather than at
-      # runtime. The plot file is checked too: it wires the same outputs, and a
+      # runtime. The viewer file is checked too: it wires the same outputs, and
-      # rename that breaks only the viewer should still fail the build.
+      # a rename that breaks only the consumer should still fail the build.
       - name: Validate the dataflows
         run: |
           dora validate dataflow.yml
-          dora validate dataflow_plot.yml
+          dora validate dataflow_view.yml
+
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install uv
+        run: |
+          curl -LsSf https://astral.sh/uv/install.sh | sh
+          echo "$HOME/.local/bin" >> "$GITHUB_PATH"
+
+      - name: Test the payload packing
+        run: tests/run.sh

AGENTS.md

@@ -4,10 +4,18 @@ This file provides guidance to AI coding assistants when working with code in th
 
 ## What this is
 
-A dora-rs C++ node that interfaces with ZED stereo cameras. On each tick it grabs a frame and publishes it over the dora dataflow: a BGR image and a depth map every frame, camera intrinsics once per camera session, a camera state on every transition, and an XYZRGBA point cloud when `ZED_POINT_CLOUD` is on. Camera management lives in `src/zed/` and the dora event loop in `src/main.cc`.
+A dora-rs C++ node that interfaces with ZED stereo cameras. Capture is free-running on its own thread at `ZED_FPS`; on each tick the event loop publishes the newest frame it has produced, or nothing if none arrived since the last tick. What goes out: a BGR image and a depth map every frame, camera intrinsics once per camera session, a camera state on every transition, and an XYZRGBA point cloud when `ZED_POINT_CLOUD` is on. Every payload is a typed Arrow array — the two frames as `std/media/v1/Image`, the rest as `std/core/v1/Float32` and `.../String` — which is what lets the node feed [dora-image-view](https://gitea.liria.cl/liria/dora-image-view) and anything else built against those types. Camera management lives in `src/zed/`, payload packing in `src/arrow/`, and the dora event loop in `src/main.cc`.
 
 The node is a camera source only. It ran inference in-process through `v0.1.0`, under `src/engines/`; that was removed so the node has no inference dependency and builds on any platform the ZED SDK supports. The removed code remains reachable at tag `v0.1.0`.
 
+## Test
+
+```bash
+tests/run.sh
+```
+
+Needs `g++` and `uv`; no camera, no ZED SDK, no CUDA.
+
 ## Build
 
 ```bash
@@ -28,8 +36,8 @@ For a build without installing the SDK stack on the host, see the `Dockerfile` (
 
 ```bash
 dora up
-dora start dataflow.yml       # camera alone
+dora start dataflow.yml            # camera alone
-dora start dataflow_plot.yml  # camera plus the opencv-plot viewer
+dora start dataflow_view.yml --uv  # camera plus dora-image-view
 ```
 
 The two dataflows both declare the camera node, so run one at a time.
@@ -42,20 +50,22 @@ Camera behavior is configured entirely through environment variables (see README
 - **`src/zed/camera.h`** — `zed::Config`, `zed::Calibration`, and `zed::Camera` class. Camera wraps the ZED SDK lifecycle: `open()` (one attempt, returns success), warmup, grab (image + depth + point cloud), calibration, device timestamp, blank-frame check, `close()` and `reset()` (close plus `sl::Camera::reboot()` where the model allows it). Exposes `handle()` for engine access.
 - **`src/zed/capture.h/cc`** — `zed::Capture` and `zed::Frame`. Owns the camera on a dedicated thread: open, warmup, grab, blank/stall detection, backoff and escalation all live here, along with the packing of every payload into owned buffers. The newest frame is handed to the reader by swapping under a mutex (`take_frame`), which also recycles the previous frame's buffers so steady-state capture does not reallocate. **Nothing outside this file may call the ZED SDK** — it blocks unboundedly on a disconnected camera, which is what froze the node when the loop drove the camera directly.
 - **`src/zed/camera.cc`** — Implementation. SDK enum parsers live in an anonymous namespace. `parse_depth_mode` runs in `Config::from_env()`, not in the constructor, so an unrecognised `ZED_DEPTH_MODE` exits before the node joins the dataflow rather than falling back to a default.
-- **`src/main.cc`** — Dora event loop and output publishing only. On any input event (intended to be driven by a tick timer; the id is not inspected): calls `camera.grab()`, then sends `image_bgr` and `depth`, plus `point_cloud` when `ZED_POINT_CLOUD` is on and `camera_info` on the first frame of a camera session (`camera_info_pending`, re-armed on reconnection). `pack_depth` copies the depth map into a tightly packed `[H, W]` float buffer (sl::Mat rows can be padded) and clears `inf`/`nan` to `0`. `set_intrinsics` and `set_frame_stamps` stamp every output of a frame with the same intrinsics, `frame_id` and the two timestamps (`device_timestamp_ns`, `host_timestamp_ns`; both Unix-epoch nanoseconds). Point cloud output varies by mode: `FULL` sends the raw `sl::Mat` pointer (no copy); `HALF` downsamples by 2 in each dimension. The loop never touches the SDK: it drains `take_state_change` and `take_frame` from the capture thread and publishes. On exit it calls `capture.stop()` and waits 2 s for the thread; if the thread is still inside a blocking SDK call it gives up and `_Exit(0)`s rather than let dora time out and kill the node.
+- **`src/arrow/array.h/cc`** — Builds the Arrow arrays the node publishes, against the Arrow C data interface: the two ABI structs are defined in the header rather than pulled from Arrow C++, which would be a heavy dependency for a producer that exports one array per output and never reads one. `image()` packs an `std/media/v1/Image` row, `float32()` and `string()` the primitive payloads. The payload bytes are **borrowed**, not copied — they only have to outlive the send, since dora IPC-encodes the array into shared memory inside `send_arrow_output` and then calls the release callbacks. Each node's release frees its children before itself, so releasing the root frees the tree.
+- **`src/main.cc`** — Dora event loop and output publishing only. On any input event (intended to be driven by a tick timer; the id is not inspected) it drains `take_state_change` and `take_frame` from the capture thread and publishes `image` and `depth`, plus `point_cloud` when `ZED_POINT_CLOUD` is on and `camera_info` on the first frame of a camera session (`camera_info_pending`, re-armed on reconnection). `send()` hands one `arrow_c::Exported` to `send_arrow_output`. `set_intrinsics` and `set_frame_stamps` stamp every output of a frame with the same intrinsics, `frame_id` and the two timestamps (`device_timestamp_ns`, `host_timestamp_ns`; both Unix-epoch nanoseconds); nothing about a frame's shape or encoding travels as metadata any more, since the Image struct carries it. The loop never touches the SDK. On exit it calls `capture.stop()` and waits 2 s for the thread; if the thread is still inside a blocking SDK call it gives up and `_Exit(0)`s rather than let dora time out and kill the node.
 - **`DoraTargets.cmake`** — Handles building or fetching dora-rs and generating the CXX bridge files. Two paths: local (`DORA_ROOT_DIR`) or git clone. Produces `dora_node_api_cxx` library and the `node_bridge.cc` glue.
 - **`scripts/build.sh`** — Configure/compile/install in one shell-free-callable step. Referenced by both dataflows, the `Dockerfile` and the README.
-- **`dora-node.yml`** — Node manifest (`apiVersion: 1`): typed input/output contract, entrypoint, build command. `dora validate` injects these types into any dataflow that reaches the node by `path:`; a `git:`-sourced node does **not** pick it up, which is why both dataflows build from the checkout. Type URNs name the element type (`std/core/v1/UInt8`, `.../Float32`, `.../String`) because every payload is a raw buffer described by its metadata, not an Arrow struct.
+- **`dora-node.yml`** — Node manifest (`apiVersion: 1`): typed input/output contract, entrypoint, build command. `dora validate` injects these types into any dataflow that reaches the node by `path:`; a `git:`-sourced node does **not** pick it up, which is why both dataflows build from the checkout. `image` and `depth` are `std/media/v1/Image`; dora's compatibility graph has no edge from `std/core/v1/UInt8` to it, so a consumer declaring the struct rejects a producer declaring the buffer whatever it actually sends — the manifest has to move with the payload.
 - **`dataflow.yml`** — Example dora dataflow: the camera alone, no viewer and no Python dependency. Built from this checkout with `path:` + `build:` and `strict_types: true`. A consumer wires the node from a tag instead (see README) — dora 1.0 pins a source with the mutually exclusive `tag:`, `rev:` or `branch:`.
-- **`dataflow_plot.yml`** — The same camera node plus an `opencv-plot` viewer on `image_bgr`. Split out because the viewer pulls dora-hub over `pip` and needs a display. dora 1.0 has no cross-dataflow wiring (an input's `source` must resolve to a node in the same descriptor), so the camera node is duplicated here rather than referenced, and the two files are run one at a time.
+- **`dataflow_view.yml`** — The same camera node plus [dora-image-view](https://gitea.liria.cl/liria/dora-image-view) on `image`, `depth` and `camera_info`, fetched from a tag. Run it with `--uv`, which gives the viewer its own managed environment. The viewer's manifest is not read at validate time (only a `path:` node's is), so the file repeats its contract in `input_types:` to get the two checked against each other. dora 1.0 has no cross-dataflow wiring (an input's `source` must resolve to a node in the same descriptor), so the camera node is duplicated here rather than referenced, and the two files are run one at a time.
-- **`.gitea/workflows/validate.yml`** — CI gate running `dora validate` on both dataflows on push and pull request. Needs no camera and no ZED SDK.
+- **`tests/run.sh`** — Builds and runs the payload tests: `tests/arrow_test.cc` under `-fsanitize=address,undefined` (the release callbacks have to free the whole tree), and `tests/test_arrow.py`, which imports the exported arrays through pyarrow's C data interface and checks the values. `tests/arrow_export.cc` is the C shim the Python side calls. Covers `src/arrow` only, so it needs no camera, no ZED SDK and no dora.
+- **`.gitea/workflows/validate.yml`** — CI gates on push and pull request: `dora validate` on both dataflows, and `tests/run.sh`. Neither needs a camera, the ZED SDK or CUDA.
 - **`Dockerfile`** — Ubuntu 24.04 x86-64 build environment (CUDA 12.8, ZED SDK 5.2, Rust). No ARM64/Jetson image; build natively there. No camera needed; run the compile with `--gpus all`. Sets `NVIDIA_DRIVER_CAPABILITIES=compute,utility,video` — the `video` capability is what makes the container runtime mount NVDEC/NVENC, which `libsl_zed.so` links against. Rust lives at `/usr/local/{rustup,cargo}` rather than `/root` so the build can run as the invoking user. `docker build -t dora-zed-build . && docker run --rm --gpus all --user "$(id -u):$(id -g)" -v "$PWD:/src" -v dora-zed-cargo:/usr/local/cargo/registry dora-zed-build`.
 
 ## Key APIs
 
-- **dora C++ API** (`<dora-node-api.h>`): `init_dora_node()`, `dora_node.events->next()`, `event_type()`, `send_output_with_metadata()`, `new_metadata()`.
+- **dora C++ API** (`<dora-node-api.h>`): `init_dora_node()`, `dora_node.events->next()`, `event_type()`, `send_arrow_output()`, `new_metadata()`.
 - **ZED SDK** (`<sl/Camera.hpp>`): `sl::Camera`, `sl::InitParameters`, `sl::RuntimeParameters`, `sl::Mat`.
-- Output data is sent as raw byte slices (`rust::Slice<const uint8_t>`) with typed metadata (shape, dtype, encoding).
+- Output data is sent as Arrow arrays: `send_arrow_output` takes an `ArrowArray`/`ArrowSchema` pair by pointer and moves both out, so a payload is built once with `src/arrow` and handed over.
 
 ## Tools
 

CMakeLists.txt

@@ -1,5 +1,5 @@
 cmake_minimum_required(VERSION 3.21)
-project(dora_zed VERSION 0.2.0 LANGUAGES C CXX)
+project(dora_zed VERSION 0.3.0 LANGUAGES C CXX)
 
 set(CMAKE_CXX_STANDARD 17)
 set(CMAKE_CXX_FLAGS "-fPIC")
@@ -24,6 +24,7 @@ link_directories(${ZED_LIBRARY_DIR})
 
 add_executable(dora_zed
     src/main.cc
+    src/arrow/array.cc
     src/zed/camera.cc
     src/zed/capture.cc
     ${node_bridge}

README.md

@@ -1,6 +1,8 @@
 # dora-zed
 
-Dora node for ZED stereo cameras. Publishes a BGR image and a depth map on every frame, plus camera intrinsics, a camera state, and an optional point cloud.
+Dora node for ZED stereo cameras. On every tick it publishes a colour image and a depth
+map as `std/media/v1/Image`, plus camera intrinsics, a camera state, and an optional
+point cloud. A source for a vision pipeline: it takes a tick and produces frames.
 
 ## Usage
 
@@ -9,100 +11,169 @@ Add to your dataflow:
 ```yaml
 nodes:
   - id: zed-camera
-    git: ssh://git@gitea.liria.cl:5001/liria/dora-zed.git
+    git: https://gitea.liria.cl/liria/dora-zed.git
-    tag: v0.2.0
+    tag: v0.3.0
     build: scripts/build.sh
     path: bin/dora_zed
     inputs:
       tick: dora/timer/millis/100
     outputs:
-      - image_bgr
+      - image
       - depth
       - point_cloud
       - camera_info
       - camera_state
 ```
 
-### Runnable examples
+The node is C++: building it needs the ZED SDK and CUDA on the machine that runs the
+dataflow — see [Build](#build). An output left out of `outputs:` is simply not published,
+so a dataflow that never turns the point cloud on can drop that line.
 
-Two dataflows live here, and they are run one at a time — dora has no cross-dataflow
+### Runnable dataflows
-wiring, so each declares the camera node itself:
 
 | File | Contents | Run |
 |---|---|---|
-| [`dataflow.yml`](dataflow.yml) | The camera alone | `dora run dataflow.yml` |
+| [`dataflow.yml`](dataflow.yml) | The camera alone, no viewer and no Python dependency | `dora run dataflow.yml` |
-| [`dataflow_plot.yml`](dataflow_plot.yml) | The camera plus an `opencv-plot` viewer on `image_bgr` | `dora run dataflow_plot.yml` |
+| [`dataflow_view.yml`](dataflow_view.yml) | The camera plus [dora-image-view][view] | `dora run dataflow_view.yml --uv` |
 
-The viewer is a separate file because it pulls [dora-hub](https://github.com/dora-rs/dora-hub)
+[view]: https://gitea.liria.cl/liria/dora-image-view
-and installs it with `pip`, and it needs a display. Neither is a cost the camera should
-carry on a headless robot, so `dataflow.yml` has no Python dependency at all.
 
-Both build the node from the checkout (`path:` + `build:`) rather than from a tag, which
+`dataflow_view.yml` is the worked example of consuming this node: a window opens with
-is what lets `dora validate` pick up the manifest described below.
+the colour image and the depth map side by side, the camera model captioned across them.
+Press `q` to quit. It needs a display, and `--uv` so the viewer gets its own managed
+environment.
 
-### Node manifest
+dora has no cross-dataflow wiring, so both files declare the camera node themselves: run
+one at a time. Both build it from the checkout, which is what lets `dora validate` check
+them against [`dora-node.yml`](dora-node.yml) — the typed contract a consumer wires
+against, and what [CI](.gitea/workflows/validate.yml) validates on every push.
 
-[`dora-node.yml`](dora-node.yml) declares the node's inputs and outputs with type URNs.
+## Inputs
-No source package is shared between node repositories, so it is the only contract a
-consumer has, and it is what makes a wiring mistake fail at build time instead of at
-runtime:
 
-```
+| Input | Description |
-$ dora validate dataflow.yml
-  node "zed-camera": injected 6 contract type(s) from .../dora-node.yml
-All type annotations OK.
-```
-
-Types are `std/core/v1/UInt8` for `image_bgr`, `std/core/v1/Float32` for `depth`,
-`camera_info` and `point_cloud`, and `std/core/v1/String` for `camera_state`; every
-payload is a raw buffer whose shape and dtype are in the metadata, so the URN names the
-element type rather than a struct. The `tick` input is `std/core/v1/UInt64`, matching how
-dora types timer inputs.
-
-`strict_types: true` in both dataflows makes a type warning a non-zero exit.
-[`.gitea/workflows/validate.yml`](.gitea/workflows/validate.yml) validates both on every
-push and pull request; it needs no camera and no ZED SDK, since validation never runs the
-node.
-
-## Dependencies
-
-| Dependency | Version | Notes |
-|---|---|---|
-| [dora-rs](https://github.com/dora-rs/dora) | 1.0+ | Verified against `dora-cli 1.0.1`; the C++ bindings are built from source at configure time |
-| [CMake](https://cmake.org/download/) | 3.21+ | |
-| [ZED SDK](https://www.stereolabs.com/developers/release) | 5.x | Bundles OpenCV; default path `/usr/local/zed` |
-| [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) | — | Required by ZED SDK |
-| [Rust toolchain](https://rustup.rs) | stable | Builds dora-rs C++ bindings |
-
-### Installation scripts
-
-| Platform | Script |
 |---|---|
-| Ubuntu 24.04 x86_64 | `sudo scripts/install_linux_x64.sh` |
+| `tick` | Publishes the newest frame captured since the last tick, or nothing. Wire to `dora/timer/millis/N`. |
-| NVIDIA Jetson (ARM64) | `sudo scripts/install_linux_arm.sh` |
-| Windows x64 (PowerShell admin) | `scripts/install_windows.ps1` |
 
-> **Windows:** Install the ZED SDK manually from [stereolabs.com](https://www.stereolabs.com/developers/release).
+The tick does not drive the camera. Capture is free-running on its own thread, paced by
+the SDK to `ZED_FPS`, and each frame overwrites whatever the previous one left
+uncollected; a tick forwards the newest there is, and publishes nothing when none arrived
+since the last one. So the tick sets the publish rate, never the capture rate: below
+`ZED_FPS` it drops frames — the default 100 ms tick against the default 15 fps keeps two
+out of every three — and above it, the ticks that find nothing publish nothing.
+`dora/timer/millis/66` at 15 fps forwards every frame, each at most one capture period
+old and none of them twice.
 
-## Local build
+## Outputs
+
+| Output | Payload | Metadata |
+|---|---|---|
+| `image` | `std/media/v1/Image` struct, `bgr8` | intrinsics, `frame_id`, timestamps |
+| `depth` | `std/media/v1/Image` struct, `32FC1` | `units=mm`, intrinsics, `frame_id`, timestamps |
+| `camera_info` | `float32` `[fx, fy, cx, cy]` | `distortion`, `frame_id`, timestamps |
+| `point_cloud` | `float32` XYZRGBA, off by default | `shape`, `channels`, `units=mm`, intrinsics, timestamps |
+| `camera_state` | `string` | `host_timestamp_ns` |
+
+`image` and `depth` are both `std/media/v1/Image`: an Arrow struct of `width: UInt32`,
+`height: UInt32`, `encoding: Utf8` and `data: LargeBinary`. The payload describes itself,
+so nothing about its shape travels as message metadata. A depth map is an image whose
+pixels are distances, as ROS also treats it: `32FC1` is one float32 per pixel, `units`
+says what those floats measure, and `0` means missing depth — the payload contains no
+`inf` and no `nan`. `depth` is published on every frame `image` is, at the same size and
+`frame_id`.
+
+`camera_info` is published **once per camera session** — with the first frame after the
+camera opens, and again after a reconnection — since the intrinsics are fixed for a given
+camera and resolution. Every frame also carries `fx`, `fy`, `cx` and `cy` in its
+metadata, so a consumer that started after the camera did still ends up with them.
+
+Every output of a frame carries the same `frame_id` and both clocks as dora `timestamp`
+metadata: `device_timestamp_ns`, when the camera captured the frame, and
+`host_timestamp_ns`, when this node published it — both Unix-epoch nanoseconds. Two nodes
+reading the same frame agree on the device clock; the rest of the dataflow shares the
+host one.
+
+### Camera state
+
+`camera_state` lets a consumer tell a camera that has not delivered its first frame yet
+from one that has been lost:
+
+| State | Meaning |
+|---|---|
+| `opening` | The camera has not delivered a usable frame yet in this run |
+| `streaming` | Frames are being published |
+| `recovering` | The camera was streaming and has stopped; the node is bringing it back |
+
+The camera runs on its own thread, since the SDK can block indefinitely inside `grab()`
+and `open()` on a disconnected camera and would otherwise freeze the event loop. A camera
+that delivers no usable frame for 5 seconds is recovered, reopening on a 1, 2, 5 then 10
+second backoff and escalating to `sl::Camera::reboot()` past the first attempt; an
+all-black frame counts as a failed grab, since a camera whose link has dropped can keep
+reporting success while handing back a uniform one. Uncollected frames are dropped, so a
+consumer always gets the newest one.
+
+## Configuration
+
+Settings are environment variables, which a dataflow can carry per node, keeping them
+with the wiring rather than in whatever shell starts it:
+
+```yaml
+    env:
+      ZED_RESOLUTION: 720
+      ZED_FPS: 30
+```
+
+A node's `env:` wins over the environment the dataflow was started in; variables it does
+not set still pass through.
+
+| Variable | Default | Meaning |
+|---|---|---|
+| `ZED_RESOLUTION` | `1080` | `720`, `1080` or `2000` |
+| `ZED_FPS` | `15` | Frame rate |
+| `ZED_DEPTH_MODE` | `NEURAL` | `NEURAL`, `NEURAL_PLUS`, `NEURAL_LIGHT`, `PERFORMANCE`, `QUALITY` or `NONE`. An unrecognised value is a startup error, not a fallback. |
+| `ZED_DEPTH_MIN_MM` | `200` | Near end of the depth range, in millimetres |
+| `ZED_DEPTH_MAX_MM` | `600` | Far end of the depth range, in millimetres |
+| `ZED_CONFIDENCE_THRESHOLD` | `60` | Reject depth less certain than this (1–100, lower is stricter) |
+| `ZED_TEXTURE_CONFIDENCE_THRESHOLD` | `100` | Reject depth on surfaces less textured than this (1–100) |
+| `ZED_DEPTH_FILL` | `false` | Interpolate depth over holes. Overrides both confidence thresholds. |
+| `ZED_FLIP` | `ON` | `ON` or `OFF` |
+| `ZED_WARMUP_FRAMES` | `30` | Frames to discard on startup |
+| `ZED_POINT_CLOUD` | `false` | Retrieve and publish the point cloud |
+| `ZED_PC_MODE` | `FULL` | `FULL` is a `[H, W, 4]` grid; `HALF` is a stride-2 downsample. Only applies with `ZED_POINT_CLOUD` on. |
+
+The defaults match the reference camera driver the perception stack was tuned against, and
+the resolved depth settings are logged at startup.
+
+The point cloud is off because it is expensive: roughly 14.7 MB per frame at 1080p, paid
+in retrieval and in a copy whether or not anything consumes it. Consumers that need 3D
+back-project `depth` with the published intrinsics instead.
+
+## Build
 
 ```bash
-git clone ssh://git@gitea.liria.cl:5001/liria/dora-zed.git
-cd dora-zed
 scripts/build.sh
 ```
 
-Binary installed to `bin/dora_zed`.
+The binary is installed to `bin/dora_zed`. The script is also the dataflow's `build:`
+command, since dora 1.0 runs build commands directly rather than through a shell — `cd`
+and `&&` are not available there. The first build clones dora-rs and compiles its C++
+bindings with Cargo, which takes a while; later builds are incremental, and
+`DORA_ROOT_DIR` builds against a local dora checkout instead.
 
-`scripts/build.sh` is also the `build:` command in the dataflow. dora 1.0 runs build
+**Requirements:** dora 1.0+ (verified with `dora-cli 1.0.1`), CMake 3.21+, ZED SDK 5.x at
-commands directly rather than through a shell, so `cd`, `&&` and command substitution are
+`/usr/local/zed` (it bundles OpenCV), the CUDA Toolkit it needs, and a stable Rust
-not available there; keeping the sequence in one script is what makes the dataflow build
+toolchain for the dora bindings.
-work. It configures a `Release` build, which is also what links the optimized dora
-bindings.
 
-### Containerized build
+| Platform | Install the toolchain with |
+|---|---|
+| Ubuntu 24.04 x86_64 | `sudo scripts/install_linux_x64.sh` |
+| NVIDIA Jetson (ARM64) | `sudo scripts/install_linux_arm.sh` |
+| Windows x64 (PowerShell admin) | `scripts/install_windows.ps1`, plus the [ZED SDK][sdk] by hand |
 
-[`Dockerfile`](Dockerfile) provides the full toolchain — CUDA, ZED SDK and Rust — on Ubuntu 24.04, the newest release Stereolabs ships a ZED SDK for. No camera is needed.
+[sdk]: https://www.stereolabs.com/developers/release
+
+[`Dockerfile`](Dockerfile) carries the same toolchain on Ubuntu 24.04 — the newest release
+Stereolabs ships a ZED SDK for — so the host needs none of it, only an NVIDIA driver and
+the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html):
 
 ```bash
 docker build -t dora-zed-build .
@@ -110,134 +181,18 @@ docker run --rm --gpus all --user "$(id -u):$(id -g)" \
     -v "$PWD:/src" -v dora-zed-cargo:/usr/local/cargo/registry dora-zed-build
 ```
 
-The container runs the same `scripts/build.sh`.
+It runs the same `scripts/build.sh`. The named volume keeps the Cargo cache across runs
+and `--user` keeps `build/` and `bin/` owned by you. x86-64 only: on Jetson, run
+[`scripts/install_linux_arm.sh`](scripts/install_linux_arm.sh) and build natively.
 
-The first run clones dora-rs and compiles it with Cargo; the named volume keeps that cache across runs.
+## Test
-`--user` keeps `build/` and `bin/` owned by you rather than root.
 
-x86-64 only. On Jetson, run [`scripts/install_linux_arm.sh`](scripts/install_linux_arm.sh) on the
+```bash
-device and build natively — there is no ARM64 image here.
+tests/run.sh
+```
 
-Needs an NVIDIA driver and the
+Unit tests over `src/arrow`, which packs every payload this node sends: each exported
-[NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
+Arrow array is round-tripped through pyarrow to check what a consumer reads back, and
-on the host, as the ZED SDK does anyway. The image sets
+built again under the sanitizers to check that releasing one frees it. They need neither a
-`NVIDIA_DRIVER_CAPABILITIES=compute,utility,video`: `libsl_zed.so` links against NVDEC
+camera, the ZED SDK nor dora, which is what lets CI run them; everything past the packing
-(`libnvcuvid`) and NVENC (`libnvidia-encode`), which the container runtime mounts from the host
+is covered by running a dataflow against a real camera.
-only when `video` is in the capability list.
-
-## Configuration
-
-All behavior is controlled via environment variables.
-
-### Camera
-
-| Variable | Default | Description |
-|---|---|---|
-| `ZED_RESOLUTION` | `1080` | `720`, `1080`, or `2000` |
-| `ZED_FPS` | `15` | Frame rate |
-| `ZED_DEPTH_MODE` | `NEURAL` | `NEURAL`, `NEURAL_PLUS`, `NEURAL_LIGHT`, `PERFORMANCE`, `QUALITY`, or `NONE`. An unrecognised value is a startup error, not a fallback. |
-| `ZED_DEPTH_MIN_MM` | `200` | Minimum depth (mm) |
-| `ZED_DEPTH_MAX_MM` | `600` | Maximum depth (mm) |
-| `ZED_CONFIDENCE_THRESHOLD` | `60` | Reject depth less certain than this (1–100, lower is stricter) |
-| `ZED_TEXTURE_CONFIDENCE_THRESHOLD` | `100` | Reject depth on surfaces less textured than this (1–100) |
-| `ZED_DEPTH_FILL` | `false` | Interpolate depth over holes. Overrides both confidence thresholds. |
-| `ZED_FLIP` | `ON` | `ON` or `OFF` |
-| `ZED_WARMUP_FRAMES` | `30` | Frames to discard on startup |
-
-The resolved depth mode, both confidence thresholds and the depth range are logged at
-startup. The defaults match the reference camera driver the perception stack was tuned
-against, so depth quality on the robot matches what it expects.
-
-### Point cloud
-
-Off by default. With `ZED_POINT_CLOUD` unset the XYZRGBA measure is not retrieved at all
-and no `point_cloud` output is published — at 1080p a full cloud is roughly 14.7 MB per
-frame, and retrieving it costs capture time whether or not anything consumes it.
-Consumers that need 3D back-project `depth` with the published intrinsics instead.
-
-| Variable | Default | Description |
-|---|---|---|
-| `ZED_POINT_CLOUD` | `false` | Retrieve and publish the point cloud |
-| `ZED_PC_MODE` | `FULL` | Only applies when `ZED_POINT_CLOUD` is on |
-
-| `ZED_PC_MODE` | Shape | Description |
-|---|---|---|
-| `FULL` (default) | `[H, W, 4]` | Full resolution grid (~14.7 MB/frame at 1080p) |
-| `HALF` | `[H/2, W/2, 4]` | Stride-2 downsample (~4× smaller) |
-
-The cloud is copied out of the SDK buffer, since the capture thread and the event loop are
-different threads. That copy is the main reason to leave `ZED_POINT_CLOUD` off unless a
-consumer needs it.
-
-## Inputs
-
-| Input | Description |
-|---|---|
-| `tick` | Triggers a camera grab. Connect to `dora/timer/millis/N`. |
-
-## Outputs
-
-| Output | Type | Description |
-|---|---|---|
-| `image_bgr` | `uint8 [H, W, 3]` | BGR image. Metadata: `shape`, `encoding=bgr8`, intrinsics, timestamps. |
-| `depth` | `float32 [H, W]` | Per-pixel depth in millimetres, published on every frame `image_bgr` is. Metadata: `dtype`, `layout=HW`, `shape`, `units=mm`, intrinsics, timestamps. `H` and `W` match `image_bgr` for the same `frame_id`. Missing depth is `0` — the payload contains no `inf` and no `nan`. Holes are not interpolated unless `ZED_DEPTH_FILL` is set. |
-| `point_cloud` | `float32 [*, 4]` | XYZRGBA point cloud, only when `ZED_POINT_CLOUD` is on. Shape depends on `ZED_PC_MODE`. Metadata: `units=mm`, intrinsics, timestamps. |
-| `camera_info` | `float32 [4]` | `[fx, fy, cx, cy]`. Metadata: `distortion` coefficients, timestamps. Published **once per camera session** — with the first frame after the camera opens, and again after a reconnection — not on every frame, since the intrinsics are fixed for a given camera and resolution. |
-| `camera_state` | `string` | UTF-8 state name, published on each change. Metadata: `dtype=string`, `encoding=utf8`, `host_timestamp_ns`. |
-
-### Camera state and recovery
-
-`camera_state` lets a downstream node tell a camera that has not delivered its first
-frame yet from one that has been lost:
-
-| State | Meaning |
-|---|---|
-| `opening` | The camera has not delivered a usable frame yet in this run |
-| `streaming` | Frames are being published |
-| `recovering` | The camera was streaming and has stopped; the node is trying to bring it back |
-
-The camera runs on its own thread. The ZED SDK can block for an unbounded time inside
-`grab()` and `open()` when the camera is disconnected — measured blocking indefinitely on
-a ZED-M even with `async_grab_camera_recovery` set — so driving it from the dora event
-loop would freeze the node: it would stop reading events, leaving `Stop` unanswered and
-the recovery logic itself unreachable. The event loop only reads the newest frame the
-capture thread has already produced, and therefore never calls into the SDK.
-
-Reopening is paced by a backoff of 1, 2, 5 then 10 seconds, the last repeated. Frames that
-are not collected before the next one arrives are dropped: a consumer always gets the
-newest frame rather than a queue of stale ones.
-
-The node treats an all-black frame as a failed grab: a camera whose link has dropped does
-not necessarily fail its grabs, and can keep reporting success while handing back a
-uniform frame. A camera that delivers no usable frame for 5 seconds is recovered — 30
-seconds if the SDK reports it is reconnecting the link itself, since taking the handle
-away mid-reconnect aborts a recovery that was already under way.
-
-The first recovery attempt closes the handle. Past that the device itself is wedged, and
-the node escalates to `sl::Camera::reboot()`, which resets it over USB without unplugging
-it. That call reaches the ZED 2 and newer only; on the first-generation ZED and the ZED
-Mini the node closes the handle and lets the backoff give the endpoint time to clear.
-Every attempt and every escalation is logged.
-
-### Timestamps
-
-Every frame output — `image_bgr`, `depth`, `point_cloud` and `camera_info` — carries both
-clocks, as dora `timestamp` metadata parameters. `camera_state` carries
-`host_timestamp_ns` only, since it is not tied to a frame.
-
-| Key | Epoch | Unit | Source |
-|---|---|---|---|
-| `device_timestamp_ns` | Unix epoch (UTC) | nanoseconds | `sl::Camera::getTimestamp(TIME_REFERENCE::IMAGE)` — when the camera captured the frame |
-| `host_timestamp_ns` | Unix epoch (UTC) | nanoseconds | `std::chrono::system_clock` on the host — when this node published the frame |
-
-Both are needed. The device timestamp is the capture instant and is the value two
-nodes reading the same frame agree on; the host timestamp is the one the rest of the
-dataflow and the session recorder share. On a Jetson the wall clock can step when NTP
-synchronises after boot, so the device timestamp is recorded alongside the host one
-rather than in place of it.
-
-The node calls `sl::setTimestampClock(SYSTEM_CLOCK)` before opening the camera, which
-is what makes the device timestamp a Unix-epoch value rather than an SDK-default clock.
-
-Every output for a frame also carries the same `frame_id`, an integer counter starting
-at 0.

dataflow.yml

@@ -3,7 +3,7 @@
 # A consumer wires the node from a tag instead — see README.md.
 #
 # The camera alone, with no viewer and no Python dependency. For a run that
-# also displays the stream, use dataflow_plot.yml.
+# also displays the stream, use dataflow_view.yml.
 strict_types: true
 
 nodes:
@@ -13,7 +13,7 @@ nodes:
     inputs:
       tick: dora/timer/millis/100
     outputs:
-      - image_bgr
+      - image
       - depth
       - point_cloud
       - camera_info

dataflow_plot.yml

@@ -1,27 +0,0 @@
-# The camera plus an opencv-plot viewer on image_bgr.
-#
-# Separate from dataflow.yml because the viewer pulls dora-hub and installs it
-# with pip, and needs a display: the camera on its own should cost neither.
-# dora has no cross-dataflow wiring, so the camera node is declared here too
-# rather than referenced — run this file or dataflow.yml, not both.
-strict_types: true
-
-nodes:
-  - id: zed-camera
-    path: bin/dora_zed
-    build: scripts/build.sh
-    inputs:
-      tick: dora/timer/millis/100
-    outputs:
-      - image_bgr
-      - depth
-      - point_cloud
-      - camera_info
-      - camera_state
-
-  - id: plot
-    git: https://github.com/dora-rs/dora-hub.git
-    build: pip install ./node-hub/opencv-plot
-    path: opencv-plot
-    inputs:
-      image: zed-camera/image_bgr

dataflow_view.yml

@@ -0,0 +1,46 @@
+# The camera into dora-image-view, and the worked example of wiring this node to
+# a consumer.
+#
+# dora has no cross-dataflow wiring, so the camera node is declared here too
+# rather than referenced — run this file or dataflow.yml, not both.
+#
+# Run it with `--uv`, which gives the viewer its own managed environment:
+#
+#     dora run dataflow_view.yml --uv
+#
+# Fetching the viewer needs anonymous read access to its repository. dora-cli
+# 1.0.1 fetches through libgit2 with no credential callback, so ssh:// cannot
+# authenticate at all; on Gitea, marking the repository public is not enough
+# either, since a repository inside a private organization answers an anonymous
+# clone with 401.
+strict_types: true
+
+nodes:
+  - id: zed-camera
+    path: bin/dora_zed
+    build: scripts/build.sh
+    inputs:
+      tick: dora/timer/millis/100
+    outputs:
+      - image
+      - depth
+      - point_cloud
+      - camera_info
+      - camera_state
+
+  - id: view
+    git: https://gitea.liria.cl/liria/dora-image-view.git
+    tag: v0.1.0
+    build: uv pip install .
+    path: dora-image-view
+    inputs:
+      image: zed-camera/image
+      depth: zed-camera/depth
+      camera_info: zed-camera/camera_info
+    # The viewer's own manifest says this, but a `git:` node's manifest is not
+    # read at validate time — only at build. Repeating it here is what makes the
+    # two contracts checked against each other now rather than at runtime.
+    input_types:
+      image: std/media/v1/Image
+      depth: std/media/v1/Image
+      camera_info: std/core/v1/Float32

dora-node.yml

@@ -7,33 +7,60 @@
 apiVersion: 1
 name: dora-zed
 namespace: liria
-description: ZED stereo camera source publishing a BGR image, a depth map, intrinsics and an optional point cloud
+description: ZED stereo camera source publishing an image, a depth map, intrinsics and an optional point cloud
 categories: [sensor]
 keywords: [zed, stereolabs, camera, depth, stereo, jetson]
 
 runtime: cpp
 entrypoint: bin/dora_zed
 build: scripts/build.sh
+dora: ">=1.0"
 
 inputs:
   tick:
     type: std/core/v1/UInt64
     required: true
-    description: Each tick triggers one camera grab. Connect to dora/timer/millis/N.
+    description: >-
+      Publishes the newest frame the capture thread has grabbed, or nothing if
+      none arrived since the last tick. It does not drive the camera: capture is
+      free-running at ZED_FPS, so this sets the publish rate and anything
+      captured between two ticks is dropped. Connect to dora/timer/millis/N.
 
 outputs:
-  image_bgr:
+  image:
-    type: std/core/v1/UInt8
+    type: std/media/v1/Image
-    description: BGR image bytes, [H, W, 3]. Shape and encoding are in the metadata.
+    description: >-
+      Captured frame as an Arrow struct {width, height, encoding, data}. The
+      encoding is always bgr8, three bytes per pixel.
   depth:
-    type: std/core/v1/Float32
+    type: std/media/v1/Image
-    description: Per-pixel depth in millimetres, [H, W]. Missing depth is 0, never inf or nan.
+    description: >-
+      Depth map as an Arrow struct {width, height, encoding, data}, one channel
+      per pixel -- dora has no depth type, and a depth map is an image whose
+      pixels are distances, which is how ROS spells it too. The encoding is
+      32FC1 and the `units` metadata is mm. Missing depth is 0, never inf or
+      nan.
   camera_info:
     type: std/core/v1/Float32
     description: Intrinsics [fx, fy, cx, cy], distortion in the metadata. Published once per camera session.
   point_cloud:
     type: std/core/v1/Float32
-    description: XYZRGBA point cloud. Published only when ZED_POINT_CLOUD is on.
+    description: XYZRGBA point cloud, shape in the metadata. Published only when ZED_POINT_CLOUD is on.
   camera_state:
     type: std/core/v1/String
     description: One of opening, streaming, recovering. Published on every change.
+
+example: |
+  - id: zed-camera
+    git: https://gitea.liria.cl/liria/dora-zed.git
+    tag: v0.3.0
+    build: scripts/build.sh
+    path: bin/dora_zed
+    inputs:
+      tick: dora/timer/millis/100
+    outputs:
+      - image
+      - depth
+      - point_cloud
+      - camera_info
+      - camera_state

src/arrow/array.cc

@@ -0,0 +1,179 @@
+#include "arrow/array.h"
+
+#include <cstring>
+#include <string>
+#include <utility>
+#include <vector>
+
+namespace arrow_c {
+namespace {
+
+// Producer-owned storage for one schema node. The struct's `format`, `name` and
+// `children` pointers point in here, which is why it outlives the call that
+// built it and is freed by the release callback rather than by the builder.
+struct SchemaData {
+    std::string format;
+    std::string name;
+    std::vector<ArrowSchema*> children;
+};
+
+// Producer-owned storage for one array node. `owned` holds the bytes this
+// node's own buffers point into - offset arrays, and scalars that have no
+// storage elsewhere. The payload itself is borrowed from the caller.
+struct ArrayData {
+    std::vector<uint8_t> owned;
+    std::vector<const void*> buffers;
+    std::vector<ArrowArray*> children;
+};
+
+// A node's release callback releases its children before freeing itself, so a
+// consumer that only releases the root - which is all the interface requires -
+// still frees the whole tree.
+void release_schema(ArrowSchema* schema) {
+    auto* data = static_cast<SchemaData*>(schema->private_data);
+
+    for (ArrowSchema* child : data->children) {
+        if (child->release != nullptr) {
+            child->release(child);
+        }
+        delete child;
+    }
+
+    delete data;
+    schema->release = nullptr;
+}
+
+void release_array(ArrowArray* array) {
+    auto* data = static_cast<ArrayData*>(array->private_data);
+
+    for (ArrowArray* child : data->children) {
+        if (child->release != nullptr) {
+            child->release(child);
+        }
+        delete child;
+    }
+
+    delete data;
+    array->release = nullptr;
+}
+
+void set_schema(ArrowSchema& schema, const char* format, const char* name,
+                std::vector<ArrowSchema*> children) {
+    auto* data = new SchemaData{format, name, std::move(children)};
+
+    schema.format = data->format.c_str();
+    schema.name = data->name.c_str();
+    schema.metadata = nullptr;
+    // Nullable everywhere, which is what pyarrow's defaults produce: the same
+    // struct built by a Python node has to compare equal to this one.
+    schema.flags = ARROW_FLAG_NULLABLE;
+    schema.n_children = static_cast<int64_t>(data->children.size());
+    schema.children = data->children.empty() ? nullptr : data->children.data();
+    schema.dictionary = nullptr;
+    schema.release = release_schema;
+    schema.private_data = data;
+}
+
+// `buffers` may point into `owned`: moving a vector keeps its heap block, so
+// pointers taken before the move stay valid after it.
+void set_array(ArrowArray& array, int64_t length, std::vector<uint8_t> owned,
+               std::vector<const void*> buffers, std::vector<ArrowArray*> children) {
+    auto* data = new ArrayData{std::move(owned), std::move(buffers), std::move(children)};
+
+    array.length = length;
+    // Nothing this node sends is null, so the validity bitmap - the first
+    // buffer of every node below - is left out entirely.
+    array.null_count = 0;
+    array.offset = 0;
+    array.n_buffers = static_cast<int64_t>(data->buffers.size());
+    array.n_children = static_cast<int64_t>(data->children.size());
+    array.buffers = data->buffers.data();
+    array.children = data->children.empty() ? nullptr : data->children.data();
+    array.dictionary = nullptr;
+    array.release = release_array;
+    array.private_data = data;
+}
+
+// One field of a struct, owned by its parent.
+struct Child {
+    ArrowSchema* schema;
+    ArrowArray* array;
+};
+
+Child new_child() {
+    return {new ArrowSchema{}, new ArrowArray{}};
+}
+
+void fill_uint32(ArrowSchema& schema, ArrowArray& array, const char* name, uint32_t value) {
+    std::vector<uint8_t> owned(sizeof(value));
+    std::memcpy(owned.data(), &value, sizeof(value));
+    const void* values = owned.data();
+
+    set_schema(schema, "I", name, {});
+    set_array(array, 1, std::move(owned), {nullptr, values}, {});
+}
+
+// One string, offsets and characters in a single allocation.
+void fill_utf8(ArrowSchema& schema, ArrowArray& array, const char* name, const char* text,
+               size_t size) {
+    const int32_t offsets[2] = {0, static_cast<int32_t>(size)};
+
+    std::vector<uint8_t> owned(sizeof(offsets) + size);
+    std::memcpy(owned.data(), offsets, sizeof(offsets));
+    std::memcpy(owned.data() + sizeof(offsets), text, size);
+    const void* offsets_buffer = owned.data();
+    const void* values_buffer = owned.data() + sizeof(offsets);
+
+    set_schema(schema, "u", name, {});
+    set_array(array, 1, std::move(owned), {nullptr, offsets_buffer, values_buffer}, {});
+}
+
+// One byte string. Only the offsets are owned; the bytes are the caller's.
+void fill_large_binary(ArrowSchema& schema, ArrowArray& array, const char* name,
+                       const uint8_t* data, size_t size) {
+    const int64_t offsets[2] = {0, static_cast<int64_t>(size)};
+
+    std::vector<uint8_t> owned(sizeof(offsets));
+    std::memcpy(owned.data(), offsets, sizeof(offsets));
+    const void* offsets_buffer = owned.data();
+
+    set_schema(schema, "Z", name, {});
+    set_array(array, 1, std::move(owned), {nullptr, offsets_buffer, data}, {});
+}
+
+} // namespace
+
+Exported image(uint32_t width, uint32_t height, const char* encoding, const uint8_t* data,
+               size_t size) {
+    Child w = new_child();
+    fill_uint32(*w.schema, *w.array, "width", width);
+
+    Child h = new_child();
+    fill_uint32(*h.schema, *h.array, "height", height);
+
+    Child e = new_child();
+    fill_utf8(*e.schema, *e.array, "encoding", encoding, std::strlen(encoding));
+
+    Child d = new_child();
+    fill_large_binary(*d.schema, *d.array, "data", data, size);
+
+    Exported out;
+    set_schema(out.schema, "+s", "", {w.schema, h.schema, e.schema, d.schema});
+    set_array(out.array, 1, {}, {nullptr}, {w.array, h.array, e.array, d.array});
+    return out;
+}
+
+Exported float32(const float* values, size_t count) {
+    Exported out;
+    set_schema(out.schema, "f", "", {});
+    set_array(out.array, static_cast<int64_t>(count), {}, {nullptr, values}, {});
+    return out;
+}
+
+Exported string(const char* text, size_t size) {
+    Exported out;
+    fill_utf8(out.schema, out.array, "", text, size);
+    return out;
+}
+
+} // namespace arrow_c

src/arrow/array.h

@@ -0,0 +1,75 @@
+#pragma once
+
+#include <cstddef>
+#include <cstdint>
+
+// The Arrow C data interface, verbatim from the Arrow specification.
+//
+// These two structs are the whole ABI dora's send_arrow_output takes, so
+// defining them here is what lets this node publish typed Arrow arrays without
+// linking Arrow C++ - a heavy dependency for a producer that exports one array
+// per output and never reads one.
+#ifndef ARROW_C_DATA_INTERFACE
+#define ARROW_C_DATA_INTERFACE
+
+#define ARROW_FLAG_DICTIONARY_ORDERED 1
+#define ARROW_FLAG_NULLABLE 2
+#define ARROW_FLAG_MAP_KEYS_SORTED 4
+
+struct ArrowSchema {
+    const char* format;
+    const char* name;
+    const char* metadata;
+    int64_t flags;
+    int64_t n_children;
+    struct ArrowSchema** children;
+    struct ArrowSchema* dictionary;
+
+    void (*release)(struct ArrowSchema*);
+    void* private_data;
+};
+
+struct ArrowArray {
+    int64_t length;
+    int64_t null_count;
+    int64_t offset;
+    int64_t n_buffers;
+    int64_t n_children;
+    const void** buffers;
+    struct ArrowArray** children;
+    struct ArrowArray* dictionary;
+
+    void (*release)(struct ArrowArray*);
+    void* private_data;
+};
+
+#endif // ARROW_C_DATA_INTERFACE
+
+namespace arrow_c {
+
+// One array with the schema describing it.
+//
+// dora moves both structs out of here and calls their release callbacks once it
+// has copied the payload into shared memory, so an Exported is built, sent
+// once, and never touched again.
+struct Exported {
+    ArrowArray array{};
+    ArrowSchema schema{};
+};
+
+// std/media/v1/Image: one row of {width, height, encoding, data}.
+//
+// The payload is borrowed rather than copied, so it has to stay valid until the
+// send returns. A capture-thread frame buffer does: the event loop holds the
+// frame across the call.
+Exported image(uint32_t width, uint32_t height, const char* encoding, const uint8_t* data,
+               size_t size);
+
+// std/core/v1/Float32: `count` values, borrowed on the same terms as image().
+Exported float32(const float* values, size_t count);
+
+// std/core/v1/String: one UTF-8 string. Copied, since every string this node
+// sends is a short state name.
+Exported string(const char* text, size_t size);
+
+} // namespace arrow_c

src/main.cc

@@ -1,12 +1,15 @@
+#include "arrow/array.h"
 #include "zed/capture.h"
 
 #include <dora-node-api.h>
 
 #include <chrono>
+#include <cstdint>
 #include <cstdlib>
 #include <initializer_list>
 #include <iostream>
 #include <string>
+#include <utility>
 #include <vector>
 
 namespace {
@@ -47,9 +50,18 @@ void set_frame_stamps(Metadata& metadata, const zed::Frame& frame) {
     metadata.set_timestamp("host_timestamp_ns", frame.host_ns);
 }
 
-void report(const char* output, const DoraResult& result) {
+// Publish one Arrow array. dora moves both structs out and releases them once
+// the payload is in shared memory, so the buffers they borrow only have to
+// outlive this call - which the frame the caller is holding does.
+void send(DoraNode& dora_node, const char* id, arrow_c::Exported payload,
+          rust::Box<Metadata> metadata) {
+    auto result = send_arrow_output(dora_node.send_output, id,
+                                    reinterpret_cast<uint8_t*>(&payload.array),
+                                    reinterpret_cast<uint8_t*>(&payload.schema),
+                                    std::move(metadata));
+
     if (!std::string(result.error).empty()) {
-        std::cerr << "Error sending " << output << ": " << std::string(result.error) << std::endl;
+        std::cerr << "Error sending " << id << ": " << std::string(result.error) << std::endl;
     }
 }
 
@@ -90,17 +102,11 @@ int main() {
 
         std::string state;
         if (capture.take_state_change(state)) {
-            rust::Slice<const uint8_t> state_slice{
-                reinterpret_cast<const uint8_t*>(state.data()), state.size()};
-
             auto state_metadata = new_metadata();
-            state_metadata->set_string("dtype", "string");
-            state_metadata->set_string("encoding", "utf8");
             state_metadata->set_timestamp("host_timestamp_ns", host_timestamp_ns());
 
-            report("camera_state",
+            send(dora_node, "camera_state", arrow_c::string(state.data(), state.size()),
-                   send_output_with_metadata(dora_node.send_output, "camera_state", state_slice,
+                 std::move(state_metadata));
-                                             std::move(state_metadata)));
         }
 
         // No new frame since the last tick: the camera is slower than the timer,
@@ -109,44 +115,41 @@ int main() {
             continue;
         }
 
-        // --- image_bgr ---
+        auto width = static_cast<uint32_t>(frame.width);
-        rust::Slice<const uint8_t> image_slice{frame.image_bgr.data(), frame.image_bgr.size()};
+        auto height = static_cast<uint32_t>(frame.height);
 
+        // --- image ---
+        // The payload carries its own width, height and encoding, so none of
+        // that travels as metadata: what is left is what describes the frame
+        // rather than the picture.
         auto image_metadata = new_metadata();
-        image_metadata->set_string("dtype", "uint8");
-        image_metadata->set_string("encoding", "bgr8");
-        image_metadata->set_string("layout", "HWC");
-        image_metadata->set_list_int("shape", make_vector_int({frame.height, frame.width, 3}));
         set_intrinsics(*image_metadata, frame.calib);
         set_frame_stamps(*image_metadata, frame);
 
-        report("image_bgr", send_output_with_metadata(dora_node.send_output, "image_bgr",
+        send(dora_node, "image",
-                                                      image_slice, std::move(image_metadata)));
+             arrow_c::image(width, height, "bgr8", frame.image_bgr.data(), frame.image_bgr.size()),
+             std::move(image_metadata));
 
         // --- depth ---
-        rust::Slice<const uint8_t> depth_slice{
+        // A depth map is an image whose pixels are distances, which is how ROS
-            reinterpret_cast<const uint8_t*>(frame.depth.data()),
+        // spells it too: 32FC1 is one float32 per pixel, and `units` says what
-            frame.depth.size() * sizeof(float)};
+        // those floats measure.
-
         auto depth_metadata = new_metadata();
-        depth_metadata->set_string("dtype", "float32");
-        depth_metadata->set_string("layout", "HW");
         depth_metadata->set_string("units", "mm");
-        depth_metadata->set_list_int("shape", make_vector_int({frame.height, frame.width}));
         set_intrinsics(*depth_metadata, frame.calib);
         set_frame_stamps(*depth_metadata, frame);
 
-        report("depth", send_output_with_metadata(dora_node.send_output, "depth", depth_slice,
+        send(dora_node, "depth",
-                                                  std::move(depth_metadata)));
+             arrow_c::image(width, height, "32FC1",
+                            reinterpret_cast<const uint8_t*>(frame.depth.data()),
+                            frame.depth.size() * sizeof(float)),
+             std::move(depth_metadata));
 
         // --- point_cloud ---
         if (frame.has_point_cloud) {
-            rust::Slice<const uint8_t> pc_slice{
+            // Unlike the two images, a cloud is a flat array of values: its
-                reinterpret_cast<const uint8_t*>(frame.point_cloud.data()),
+            // grid shape is only in the metadata.
-                frame.point_cloud.size() * sizeof(float)};
-
             auto pc_metadata = new_metadata();
-            pc_metadata->set_string("dtype", "float32");
             pc_metadata->set_string("channels", "XYZRGBA");
             pc_metadata->set_string("units", "mm");
             pc_metadata->set_string("layout", "HWC");
@@ -155,26 +158,22 @@ int main() {
             set_intrinsics(*pc_metadata, frame.calib);
             set_frame_stamps(*pc_metadata, frame);
 
-            report("point_cloud", send_output_with_metadata(dora_node.send_output, "point_cloud",
+            send(dora_node, "point_cloud",
-                                                            pc_slice, std::move(pc_metadata)));
+                 arrow_c::float32(frame.point_cloud.data(), frame.point_cloud.size()),
+                 std::move(pc_metadata));
         }
 
         // --- camera_info ---
         if (frame.camera_info_first) {
-            float camera_data[4] = {frame.calib.fx, frame.calib.fy, frame.calib.cx,
+            const float camera_data[4] = {frame.calib.fx, frame.calib.fy, frame.calib.cx,
-                                    frame.calib.cy};
+                                          frame.calib.cy};
-            rust::Slice<const uint8_t> cam_slice{
-                reinterpret_cast<const uint8_t*>(camera_data), sizeof(camera_data)};
 
             auto cam_metadata = new_metadata();
-            cam_metadata->set_string("dtype", "float32");
-            cam_metadata->set_string("layout", "C");
-            cam_metadata->set_list_int("shape", make_vector_int({4}));
             cam_metadata->set_list_float("distortion", make_distortion(frame.calib.distortion));
             set_frame_stamps(*cam_metadata, frame);
 
-            report("camera_info", send_output_with_metadata(dora_node.send_output, "camera_info",
+            send(dora_node, "camera_info", arrow_c::float32(camera_data, 4),
-                                                            cam_slice, std::move(cam_metadata)));
+                 std::move(cam_metadata));
         }
     }
 

tests/arrow_export.cc

@@ -0,0 +1,36 @@
+// C entry points for tests/test_arrow.py.
+//
+// The Python side cannot call the C++ builders, so each one is wrapped here and
+// the payload is passed in from the test rather than fixed in this file.
+
+#include "arrow/array.h"
+
+#include <cstring>
+
+namespace {
+
+void hand_over(void* array, void* schema, const arrow_c::Exported& exported) {
+    // The caller owns the two structs, exactly as dora does: it reads them and
+    // is then responsible for releasing what they point at.
+    std::memcpy(array, &exported.array, sizeof(exported.array));
+    std::memcpy(schema, &exported.schema, sizeof(exported.schema));
+}
+
+} // namespace
+
+extern "C" {
+
+void zed_export_image(void* array, void* schema, uint32_t width, uint32_t height,
+                      const char* encoding, const uint8_t* data, size_t size) {
+    hand_over(array, schema, arrow_c::image(width, height, encoding, data, size));
+}
+
+void zed_export_float32(void* array, void* schema, const float* values, size_t count) {
+    hand_over(array, schema, arrow_c::float32(values, count));
+}
+
+void zed_export_string(void* array, void* schema, const char* text, size_t size) {
+    hand_over(array, schema, arrow_c::string(text, size));
+}
+
+} // extern "C"

tests/arrow_test.cc

@@ -0,0 +1,54 @@
+// Release has to free the whole tree: the array nodes, the schema nodes, and
+// the storage their buffers point into. Nothing here asserts that directly -
+// the sanitizers tests/run.sh builds this with are the assertion, and a node
+// that leaks per frame is a node that dies on a long run.
+
+#include "arrow/array.h"
+
+#include <cassert>
+#include <cstdint>
+#include <string>
+#include <vector>
+
+namespace {
+
+void release(arrow_c::Exported& exported) {
+    exported.array.release(&exported.array);
+    exported.schema.release(&exported.schema);
+
+    // A released struct marks itself, which is how a consumer knows not to
+    // release it twice.
+    assert(exported.array.release == nullptr);
+    assert(exported.schema.release == nullptr);
+}
+
+} // namespace
+
+int main() {
+    const std::vector<uint8_t> pixels(4 * 4 * 3, 7);
+    const std::vector<float> depth(4 * 4, 1500.0F);
+    const std::string state = "streaming";
+
+    for (int i = 0; i < 100; ++i) {
+        arrow_c::Exported image =
+            arrow_c::image(4, 4, "bgr8", pixels.data(), pixels.size());
+        assert(image.array.length == 1);
+        assert(image.array.n_children == 4);
+        release(image);
+
+        arrow_c::Exported depth_image =
+            arrow_c::image(4, 4, "32FC1", reinterpret_cast<const uint8_t*>(depth.data()),
+                           depth.size() * sizeof(float));
+        release(depth_image);
+
+        arrow_c::Exported values = arrow_c::float32(depth.data(), depth.size());
+        assert(values.array.length == static_cast<int64_t>(depth.size()));
+        release(values);
+
+        arrow_c::Exported text = arrow_c::string(state.data(), state.size());
+        assert(text.array.length == 1);
+        release(text);
+    }
+
+    return 0;
+}

tests/run.sh

@@ -0,0 +1,29 @@
+#!/usr/bin/env bash
+# Build and run the payload tests.
+#
+# They cover src/arrow only, which is the part of the node that has no ZED SDK,
+# no CUDA and no dora dependency - so this runs anywhere, and is what CI runs.
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO_DIR="$(dirname "$SCRIPT_DIR")"
+OUT_DIR="${REPO_DIR}/build/tests"
+
+mkdir -p "$OUT_DIR"
+
+# Sanitized: the release callbacks have to free the whole exported tree, and a
+# leak there would be one per frame.
+g++ -std=c++17 -Wall -Wextra -Werror -g -fsanitize=address,undefined \
+    -I "${REPO_DIR}/src" \
+    -o "${OUT_DIR}/arrow_test" \
+    "${REPO_DIR}/src/arrow/array.cc" "${REPO_DIR}/tests/arrow_test.cc"
+"${OUT_DIR}/arrow_test"
+echo "ok  arrow_test"
+
+# Unsanitized: pyarrow is the consumer standing in for dora, and loads the same
+# code as a shared library.
+g++ -std=c++17 -Wall -Wextra -Werror -O2 -shared -fPIC \
+    -I "${REPO_DIR}/src" \
+    -o "${OUT_DIR}/libzed_arrow.so" \
+    "${REPO_DIR}/src/arrow/array.cc" "${REPO_DIR}/tests/arrow_export.cc"
+uv run --no-project --with pyarrow python "${REPO_DIR}/tests/test_arrow.py"

tests/test_arrow.py

@@ -0,0 +1,129 @@
+"""Round-trip every exported payload through a real Arrow consumer.
+
+The node builds its Arrow arrays by hand against the C data interface rather
+than linking Arrow C++, so this is what shows that what dora imports on the
+other side of `send_arrow_output` is what went in. It needs neither a camera nor
+the ZED SDK.
+"""
+
+import ctypes
+import pathlib
+import sys
+
+import pyarrow as pa
+
+LIBRARY = pathlib.Path(__file__).resolve().parent.parent / "build" / "tests" / "libzed_arrow.so"
+
+# ArrowArray is ten machine words and ArrowSchema nine, and both want the
+# 8-byte alignment an array of uint64 gives.
+WORDS = 16
+# Index of the `release` field in ArrowArray: a consumer that took ownership
+# zeroes it, which is how this test sees the move happen.
+RELEASE = 8
+
+IMAGE_TYPE = pa.struct([
+    ("width", pa.uint32()),
+    ("height", pa.uint32()),
+    ("encoding", pa.utf8()),
+    ("data", pa.large_binary()),
+])
+
+
+def load():
+    """Load the test library and declare the argument types of its exports."""
+    library = ctypes.CDLL(str(LIBRARY))
+
+    library.zed_export_image.argtypes = [
+        ctypes.c_void_p, ctypes.c_void_p, ctypes.c_uint32, ctypes.c_uint32,
+        ctypes.c_char_p, ctypes.c_char_p, ctypes.c_size_t,
+    ]
+    library.zed_export_float32.argtypes = [
+        ctypes.c_void_p, ctypes.c_void_p, ctypes.POINTER(ctypes.c_float), ctypes.c_size_t,
+    ]
+    library.zed_export_string.argtypes = [
+        ctypes.c_void_p, ctypes.c_void_p, ctypes.c_char_p, ctypes.c_size_t,
+    ]
+    return library
+
+
+def structs():
+    """Allocate the ArrowArray and ArrowSchema the producer fills in."""
+    return (ctypes.c_uint64 * WORDS)(), (ctypes.c_uint64 * WORDS)()
+
+
+def imported(array, schema):
+    """Import the pair as pyarrow does, taking ownership of both."""
+    result = pa.Array._import_from_c(ctypes.addressof(array), ctypes.addressof(schema))
+    assert array[RELEASE] == 0, "the consumer should have taken ownership of the array"
+    return result
+
+
+def test_image(library):
+    """An image row carries its own dimensions, encoding and bytes."""
+    array, schema = structs()
+    pixels = bytes(range(12))
+
+    library.zed_export_image(
+        ctypes.byref(array), ctypes.byref(schema), 2, 2, b"bgr8", pixels, len(pixels)
+    )
+    result = imported(array, schema)
+
+    assert result.type == IMAGE_TYPE, result.type
+    assert result.to_pylist() == [
+        {"width": 2, "height": 2, "encoding": "bgr8", "data": pixels}
+    ]
+
+
+def test_depth(library):
+    """A depth map is an image whose bytes are float32 millimetres."""
+    array, schema = structs()
+    millimetres = [1500.0, 0.0, 250.5, 4000.0]
+    payload = pa.array(millimetres, type=pa.float32()).buffers()[1].to_pybytes()
+
+    library.zed_export_image(
+        ctypes.byref(array), ctypes.byref(schema), 2, 2, b"32FC1", payload, len(payload)
+    )
+    row = imported(array, schema).to_pylist()[0]
+
+    assert row["encoding"] == "32FC1"
+    assert pa.Array.from_buffers(
+        pa.float32(), 4, [None, pa.py_buffer(row["data"])]
+    ).to_pylist() == millimetres
+
+
+def test_float32(library):
+    """Intrinsics and the point cloud go out as plain float32 values."""
+    array, schema = structs()
+    values = (ctypes.c_float * 4)(700.5, 700.5, 640.0, 360.0)
+
+    library.zed_export_float32(ctypes.byref(array), ctypes.byref(schema), values, 4)
+    result = imported(array, schema)
+
+    assert result.type == pa.float32()
+    assert result.to_pylist() == [700.5, 700.5, 640.0, 360.0]
+
+
+def test_string(library):
+    """The camera state goes out as one UTF-8 string."""
+    array, schema = structs()
+
+    library.zed_export_string(ctypes.byref(array), ctypes.byref(schema), b"streaming", 9)
+    result = imported(array, schema)
+
+    assert result.type == pa.utf8()
+    assert result.to_pylist() == ["streaming"]
+
+
+def main():
+    """Run every test, reporting the first failure."""
+    library = load()
+
+    for test in (test_image, test_depth, test_float32, test_string):
+        test(library)
+        print(f"ok  {test.__name__}")
+
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())