Split the opencv-plot viewer into its own dataflow #19

Merged
cristhian merged 1 commits from split-plot-dataflow into main 2026-09-08 17:23:15 +00:00
5 changed files with 69 additions and 25 deletions
Showing only changes of commit 483da05a98 - Show all commits
+10 -6
View File
@@ -1,5 +1,5 @@
# Checks the dataflow against the typed contract in dora-node.yml. This runs no # Checks both dataflows against the typed contract in dora-node.yml. This runs
# node, so it needs neither a camera nor the ZED SDK. # no node, so it needs neither a camera nor the ZED SDK.
name: validate name: validate
on: [push, pull_request] on: [push, pull_request]
@@ -26,7 +26,11 @@ jobs:
- run: dora --version - run: dora --version
# strict_types: true in dataflow.yml turns a type warning into a non-zero # strict_types: true in each dataflow turns a type warning into a
# exit, so a mismatched annotation fails here rather than at runtime. # non-zero exit, so a mismatched annotation fails here rather than at
- name: Validate the dataflow # runtime. The plot file is checked too: it wires the same outputs, and a
run: dora validate dataflow.yml # rename that breaks only the viewer should still fail the build.
- name: Validate the dataflows
run: |
dora validate dataflow.yml
dora validate dataflow_plot.yml
+9 -5
View File
@@ -28,9 +28,12 @@ For a build without installing the SDK stack on the host, see the `Dockerfile` (
```bash ```bash
dora up dora up
dora start dataflow.yml dora start dataflow.yml # camera alone
dora start dataflow_plot.yml # camera plus the opencv-plot viewer
``` ```
The two dataflows both declare the camera node, so run one at a time.
Camera behavior is configured entirely through environment variables (see README.md for the full list: `ZED_RESOLUTION`, `ZED_FPS`, `ZED_DEPTH_MODE`, etc.). Camera behavior is configured entirely through environment variables (see README.md for the full list: `ZED_RESOLUTION`, `ZED_FPS`, `ZED_DEPTH_MODE`, etc.).
## Architecture ## Architecture
@@ -41,10 +44,11 @@ Camera behavior is configured entirely through environment variables (see README
- **`src/zed/camera.cc`** — Implementation. SDK enum parsers live in an anonymous namespace. `parse_depth_mode` runs in `Config::from_env()`, not in the constructor, so an unrecognised `ZED_DEPTH_MODE` exits before the node joins the dataflow rather than falling back to a default. - **`src/zed/camera.cc`** — Implementation. SDK enum parsers live in an anonymous namespace. `parse_depth_mode` runs in `Config::from_env()`, not in the constructor, so an unrecognised `ZED_DEPTH_MODE` exits before the node joins the dataflow rather than falling back to a default.
- **`src/main.cc`** — Dora event loop and output publishing only. On any input event (intended to be driven by a tick timer; the id is not inspected): calls `camera.grab()`, then sends `image_bgr` and `depth`, plus `point_cloud` when `ZED_POINT_CLOUD` is on and `camera_info` on the first frame of a camera session (`camera_info_pending`, re-armed on reconnection). `pack_depth` copies the depth map into a tightly packed `[H, W]` float buffer (sl::Mat rows can be padded) and clears `inf`/`nan` to `0`. `set_intrinsics` and `set_frame_stamps` stamp every output of a frame with the same intrinsics, `frame_id` and the two timestamps (`device_timestamp_ns`, `host_timestamp_ns`; both Unix-epoch nanoseconds). Point cloud output varies by mode: `FULL` sends the raw `sl::Mat` pointer (no copy); `HALF` downsamples by 2 in each dimension. The loop never touches the SDK: it drains `take_state_change` and `take_frame` from the capture thread and publishes. On exit it calls `capture.stop()` and waits 2 s for the thread; if the thread is still inside a blocking SDK call it gives up and `_Exit(0)`s rather than let dora time out and kill the node. - **`src/main.cc`** — Dora event loop and output publishing only. On any input event (intended to be driven by a tick timer; the id is not inspected): calls `camera.grab()`, then sends `image_bgr` and `depth`, plus `point_cloud` when `ZED_POINT_CLOUD` is on and `camera_info` on the first frame of a camera session (`camera_info_pending`, re-armed on reconnection). `pack_depth` copies the depth map into a tightly packed `[H, W]` float buffer (sl::Mat rows can be padded) and clears `inf`/`nan` to `0`. `set_intrinsics` and `set_frame_stamps` stamp every output of a frame with the same intrinsics, `frame_id` and the two timestamps (`device_timestamp_ns`, `host_timestamp_ns`; both Unix-epoch nanoseconds). Point cloud output varies by mode: `FULL` sends the raw `sl::Mat` pointer (no copy); `HALF` downsamples by 2 in each dimension. The loop never touches the SDK: it drains `take_state_change` and `take_frame` from the capture thread and publishes. On exit it calls `capture.stop()` and waits 2 s for the thread; if the thread is still inside a blocking SDK call it gives up and `_Exit(0)`s rather than let dora time out and kill the node.
- **`DoraTargets.cmake`** — Handles building or fetching dora-rs and generating the CXX bridge files. Two paths: local (`DORA_ROOT_DIR`) or git clone. Produces `dora_node_api_cxx` library and the `node_bridge.cc` glue. - **`DoraTargets.cmake`** — Handles building or fetching dora-rs and generating the CXX bridge files. Two paths: local (`DORA_ROOT_DIR`) or git clone. Produces `dora_node_api_cxx` library and the `node_bridge.cc` glue.
- **`scripts/build.sh`** — Configure/compile/install in one shell-free-callable step. Referenced by `dataflow.yml`, the `Dockerfile` and the README. - **`scripts/build.sh`** — Configure/compile/install in one shell-free-callable step. Referenced by both dataflows, the `Dockerfile` and the README.
- **`dora-node.yml`** — Node manifest (`apiVersion: 1`): typed input/output contract, entrypoint, build command. `dora validate` injects these types into any dataflow that reaches the node by `path:`; a `git:`-sourced node does **not** pick it up, which is why `dataflow.yml` builds from the checkout. Type URNs name the element type (`std/core/v1/UInt8`, `.../Float32`, `.../String`) because every payload is a raw buffer described by its metadata, not an Arrow struct. - **`dora-node.yml`** — Node manifest (`apiVersion: 1`): typed input/output contract, entrypoint, build command. `dora validate` injects these types into any dataflow that reaches the node by `path:`; a `git:`-sourced node does **not** pick it up, which is why both dataflows build from the checkout. Type URNs name the element type (`std/core/v1/UInt8`, `.../Float32`, `.../String`) because every payload is a raw buffer described by its metadata, not an Arrow struct.
- **`dataflow.yml`** — Example dora dataflow, built from this checkout with `path:` + `build:` and `strict_types: true`. A consumer wires the node from a tag instead (see README) — dora 1.0 pins a source with the mutually exclusive `tag:`, `rev:` or `branch:`. - **`dataflow.yml`** — Example dora dataflow: the camera alone, no viewer and no Python dependency. Built from this checkout with `path:` + `build:` and `strict_types: true`. A consumer wires the node from a tag instead (see README) — dora 1.0 pins a source with the mutually exclusive `tag:`, `rev:` or `branch:`.
- **`.gitea/workflows/validate.yml`** — CI gate running `dora validate dataflow.yml` on push and pull request. Needs no camera and no ZED SDK. - **`dataflow_plot.yml`** — The same camera node plus an `opencv-plot` viewer on `image_bgr`. Split out because the viewer pulls dora-hub over `pip` and needs a display. dora 1.0 has no cross-dataflow wiring (an input's `source` must resolve to a node in the same descriptor), so the camera node is duplicated here rather than referenced, and the two files are run one at a time.
- **`.gitea/workflows/validate.yml`** — CI gate running `dora validate` on both dataflows on push and pull request. Needs no camera and no ZED SDK.
- **`Dockerfile`** — Ubuntu 24.04 x86-64 build environment (CUDA 12.8, ZED SDK 5.2, Rust). No ARM64/Jetson image; build natively there. No camera needed; run the compile with `--gpus all`. Sets `NVIDIA_DRIVER_CAPABILITIES=compute,utility,video` — the `video` capability is what makes the container runtime mount NVDEC/NVENC, which `libsl_zed.so` links against. Rust lives at `/usr/local/{rustup,cargo}` rather than `/root` so the build can run as the invoking user. `docker build -t dora-zed-build . && docker run --rm --gpus all --user "$(id -u):$(id -g)" -v "$PWD:/src" -v dora-zed-cargo:/usr/local/cargo/registry dora-zed-build`. - **`Dockerfile`** — Ubuntu 24.04 x86-64 build environment (CUDA 12.8, ZED SDK 5.2, Rust). No ARM64/Jetson image; build natively there. No camera needed; run the compile with `--gpus all`. Sets `NVIDIA_DRIVER_CAPABILITIES=compute,utility,video` — the `video` capability is what makes the container runtime mount NVDEC/NVENC, which `libsl_zed.so` links against. Rust lives at `/usr/local/{rustup,cargo}` rather than `/root` so the build can run as the invoking user. `docker build -t dora-zed-build . && docker run --rm --gpus all --user "$(id -u):$(id -g)" -v "$PWD:/src" -v dora-zed-cargo:/usr/local/cargo/registry dora-zed-build`.
## Key APIs ## Key APIs
+20 -7
View File
@@ -23,9 +23,22 @@ nodes:
- camera_state - camera_state
``` ```
See [`dataflow.yml`](dataflow.yml) for a complete runnable example with a camera display ### Runnable examples
node. That file builds the node from the checkout (`path:` + `build:`) rather than from a
tag, which is what lets `dora validate` pick up the manifest described below. Two dataflows live here, and they are run one at a time — dora has no cross-dataflow
wiring, so each declares the camera node itself:
| File | Contents | Run |
|---|---|---|
| [`dataflow.yml`](dataflow.yml) | The camera alone | `dora run dataflow.yml` |
| [`dataflow_plot.yml`](dataflow_plot.yml) | The camera plus an `opencv-plot` viewer on `image_bgr` | `dora run dataflow_plot.yml` |
The viewer is a separate file because it pulls [dora-hub](https://github.com/dora-rs/dora-hub)
and installs it with `pip`, and it needs a display. Neither is a cost the camera should
carry on a headless robot, so `dataflow.yml` has no Python dependency at all.
Both build the node from the checkout (`path:` + `build:`) rather than from a tag, which
is what lets `dora validate` pick up the manifest described below.
### Node manifest ### Node manifest
@@ -46,10 +59,10 @@ payload is a raw buffer whose shape and dtype are in the metadata, so the URN na
element type rather than a struct. The `tick` input is `std/core/v1/UInt64`, matching how element type rather than a struct. The `tick` input is `std/core/v1/UInt64`, matching how
dora types timer inputs. dora types timer inputs.
`strict_types: true` in `dataflow.yml` makes a type warning a non-zero exit. `strict_types: true` in both dataflows makes a type warning a non-zero exit.
[`.gitea/workflows/validate.yml`](.gitea/workflows/validate.yml) runs [`.gitea/workflows/validate.yml`](.gitea/workflows/validate.yml) validates both on every
`dora validate dataflow.yml` on every push and pull request; it needs no camera and no push and pull request; it needs no camera and no ZED SDK, since validation never runs the
ZED SDK, since validation never runs the node. node.
## Dependencies ## Dependencies
+3 -7
View File
@@ -1,6 +1,9 @@
# The repository's own example: builds and runs the node from this checkout, so # The repository's own example: builds and runs the node from this checkout, so
# `dora validate` picks up the typed contract in dora-node.yml next to it. # `dora validate` picks up the typed contract in dora-node.yml next to it.
# A consumer wires the node from a tag instead — see README.md. # A consumer wires the node from a tag instead — see README.md.
#
# The camera alone, with no viewer and no Python dependency. For a run that
# also displays the stream, use dataflow_plot.yml.
strict_types: true strict_types: true
nodes: nodes:
@@ -15,10 +18,3 @@ nodes:
- point_cloud - point_cloud
- camera_info - camera_info
- camera_state - camera_state
- id: plot
git: https://github.com/dora-rs/dora-hub.git
build: pip install ./node-hub/opencv-plot
path: opencv-plot
inputs:
image: zed-camera/image_bgr
+27
View File
@@ -0,0 +1,27 @@
# The camera plus an opencv-plot viewer on image_bgr.
#
# Separate from dataflow.yml because the viewer pulls dora-hub and installs it
# with pip, and needs a display: the camera on its own should cost neither.
# dora has no cross-dataflow wiring, so the camera node is declared here too
# rather than referenced — run this file or dataflow.yml, not both.
strict_types: true
nodes:
- id: zed-camera
path: bin/dora_zed
build: scripts/build.sh
inputs:
tick: dora/timer/millis/100
outputs:
- image_bgr
- depth
- point_cloud
- camera_info
- camera_state
- id: plot
git: https://github.com/dora-rs/dora-hub.git
build: pip install ./node-hub/opencv-plot
path: opencv-plot
inputs:
image: zed-camera/image_bgr