diff --git a/.gitea/workflows/validate.yml b/.gitea/workflows/validate.yml index 9b99aee..fa68938 100644 --- a/.gitea/workflows/validate.yml +++ b/.gitea/workflows/validate.yml @@ -1,5 +1,6 @@ -# Two gates, neither of which needs a camera or a display: the dataflow still -# matches the typed contract in dora-node.yml, and the node's own tests pass. +# Two gates, neither of which needs a camera or a display: the example dataflow +# still matches the typed contract in dora-node.yml and still runs end to end, +# and the node's own unit tests pass. name: validate # A pull request would otherwise run this twice on the same commit: once for @@ -14,11 +15,22 @@ env: DORA_TARGET: x86_64-unknown-linux-gnu jobs: - validate: + dataflow: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 + # The opencv-python wheel links against libGL at import time, so cv2 + # cannot even be imported on the bare runner image. + - name: Install the OpenCV shared libraries + run: | + sudo apt-get update + sudo apt-get install -y --no-install-recommends libgl1 libglib2.0-0 + + - uses: astral-sh/setup-uv@v5 + with: + enable-cache: true + - name: Install the dora CLI run: | curl -fsSL -o dora-cli.tar.gz \ @@ -36,13 +48,19 @@ jobs: - name: Validate the dataflow run: dora validate dataflow.yml + # CI=true is already set by the runner, which keeps the viewer from trying + # to open a window; SOURCE_FRAMES bounds the source so the run ends on its + # own. Everything the dataflow needs is in the checkout. + - name: Run the dataflow headless + env: + SOURCE_FRAMES: 30 + run: dora run dataflow.yml --uv + test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - # The opencv-python wheel links against libGL at import time, so cv2 - # cannot even be imported on the bare runner image. - name: Install the OpenCV shared libraries run: | sudo apt-get update @@ -59,6 +77,4 @@ jobs: - run: uv run ruff check . - # CI=true is already set by the runner, which is what keeps the node from - # trying to open a window on a headless machine. - run: uv run pytest -q diff --git a/AGENTS.md b/AGENTS.md index 5540eb3..7df0012 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -4,49 +4,37 @@ This file provides guidance to AI coding assistants when working with code in th ## What this is -A dora-rs Python node that displays a video stream with overlays: it keeps a current frame, bbox set and text string, and every `image` event redraws all three into an OpenCV window. It is a display sink — it produces no outputs. +A dora-rs Python node that displays a video stream with overlays: it keeps a current frame, bbox set and text string, and every `image` event redraws all three into an OpenCV window. It is a display sink — it produces no outputs. [README.md](README.md) documents the inputs, the encodings and the environment variables; this file covers only what the code and the README do not say. -Based on [`opencv-plot`](https://github.com/dora-rs/dora-hub/tree/main/node-hub/opencv-plot) from dora-hub, brought up to the dora 1.0 node contract and packaged with `uv`. +Everything the example dataflow needs is in this checkout. Do not reintroduce a dependency on another repository — the example must keep running with no camera, no detector and no network. **It is Python on purpose.** The node was first written in Rust against the `opencv` crate; that crate generates its bindings from the OpenCV headers installed on the build machine, so it needs `libopencv-dev` and `libclang-dev` on every machine and every architecture and does not cross-compile. Every dependency here ships prebuilt wheels for linux-x86_64, linux-aarch64 and macos-arm64, so `uv sync` is the whole install. Do not reintroduce a dependency that has to be compiled from source. -## Build +## Commands ```bash -uv sync # runtime deps -uv sync --group dev # plus pytest and ruff +uv sync --group dev # deps plus pytest and ruff +uv run pytest && uv run ruff check . # no camera/dataflow/display needed +dora run dataflow.yml --uv # the example, in a window +CI=true SOURCE_FRAMES=30 dora run dataflow.yml --uv # the same, headless ``` -`dora build --uv` / `dora run --uv` rewrites the manifest's `build: pip install .` to `uv pip install .` and gives the node its own managed venv under `.dora/python-envs//`, so what it imports at runtime is what was installed at build time. - -**Requirements:** dora 1.0+ (verified with `dora-cli 1.0.1`), Python 3.11+ (`dora-rs` 1.0.1 requires it), `uv`, and a display — or `CI=true`. - -## Run - -```bash -dora run dataflow.yml --uv -``` - -## Test - -```bash -uv run pytest # 14 tests, no camera/dataflow/display needed -uv run ruff check . -``` +`dora build --uv` / `dora run --uv` rewrites each manifest's `build: pip install .` to `uv pip install .` and gives the node its own managed venv under `.dora/python-envs//`, so what it imports at runtime is what was installed at build time. ## Architecture -- **`dora_plot/main.py`** — Argument parsing and the dora event loop, nothing else. `image` decodes and redraws; `bbox` and `text` only store state. The env var wins over the CLI flag for `PLOT_WIDTH`/`PLOT_HEIGHT`, matching the upstream node. `Node(args.name)` rather than `Node()` so the node also runs standalone as a dynamic node. -- **`dora_plot/decode.py`** — `to_bgr()`: every supported encoding to a writable BGR array. `width`/`height` are only consulted for the raw encodings (`bgr8`, `rgb8`, `yuv420`); the rest carry their own dimensions, so a missing dimension is an error only where it actually matters. Two behaviours are deliberate, not bugs: `bgr8` **copies** (the payload may be a read-only view on shared memory, and the frame is annotated in place), and `yuv420` converts with `COLOR_YUV420p2RGB` rather than `...2BGR` — the upstream node converts to RGB and then displays it as BGR, and that channel swap is part of the behaviour callers already see. -- **`dora_plot/plot.py`** — `Plot` (the overlay state and the window) and `Bboxes`. `show()` draws, resizes and displays, and returns whether `q` was pressed. Resizing only happens when *both* `PLOT_WIDTH` and `PLOT_HEIGHT` are set. `headless` (from `CI=true`) skips only the window: every input is still decoded, which is what makes the node safe in a pipeline test. +- **`dora_plot/main.py`** — Argument parsing and the dora event loop, nothing else. The env var wins over the CLI flag for `PLOT_WIDTH`/`PLOT_HEIGHT`, matching the upstream node. `Node(args.name)` rather than `Node()` so the node also runs standalone as a dynamic node. +- **`dora_plot/decode.py`** — `to_bgr()`: every supported encoding to a writable BGR array. Two behaviours are deliberate, not bugs: `bgr8` **copies** (the payload may be a read-only view on shared memory, and the frame is annotated in place), and `yuv420` converts with `COLOR_YUV420p2RGB` rather than `...2BGR` — the upstream node converts to RGB and then displays it as BGR, and that channel swap is part of the behaviour callers already see. +- **`dora_plot/plot.py`** — `Plot` (the overlay state and the window) and `Bboxes`. `headless` (from `CI=true`) skips only the window: every input is still decoded, which is what makes the node safe in a pipeline test. - **`dora_plot/bbox.py`** — Parses the `bbox` struct into xyxy corners. `xywh` is centre-based (x, y are the box centre). `_flat()` unwraps the single-element list that `pa.array([{...}])` produces around each field, and reads a field stored flat as-is. -- **`dora-node.yml`** — Node manifest (`apiVersion: 1`). `dora validate` injects these types into any dataflow that reaches the node by `path:`; a `git:`-sourced node does **not** pick it up, which is why the dataflow builds from the checkout. `image` and `text` carry type URNs naming the element type (`std/core/v1/UInt8`, `.../String`) because each payload is a raw buffer described by its metadata, not an Arrow struct. `bbox` is deliberately **untyped**: senders disagree on whether its numeric fields are 32- or 64-bit, and the node reads either. -- **`dataflow.yml`** — Example dataflow: a webcam and a one-shot caption feeding the viewer, with `strict_types: true`. Note the `output_types:` on the two `git:`-sourced nodes — their own manifests are not injected, so without those annotations strict validation fails on an untyped upstream. `bbox` is left unwired on purpose; every bbox source is a detector, and wiring one would put a model download in front of a webcam preview. -- **`.gitea/workflows/validate.yml`** — CI gate running `dora validate` on the dataflow on push and pull request. Needs no camera and no display. +- **`examples/source.py`** — The synthetic source node: an animated frame, the box around the square in it, and a caption. It exists so the example dataflow exercises all three inputs without a camera or a detector, and so CI can run the dataflow rather than only validate it. `SOURCE_FRAMES` bounds the run, which is what lets CI wait for a clean exit instead of killing it on a timeout. +- **`dora-node.yml`** — Node manifest (`apiVersion: 1`). `dora validate` injects these types into any dataflow that reaches the node by `path:`; a `git:`-sourced node does **not** pick it up, which is why the example builds from the checkout. `image` and `text` carry type URNs naming the element type because each payload is a raw buffer described by its metadata, not an Arrow struct. `bbox` is deliberately **untyped**: senders disagree on whether its numeric fields are 32- or 64-bit, and the node reads either. +- **`dataflow.yml`** — The example and the CI gate, with `strict_types: true`. `source` carries `output_types:` because it has no manifest of its own; `bbox` is annotated on neither side, matching the manifest. +- **`.gitea/workflows/validate.yml`** — CI: validates and runs the dataflow headless, and runs the unit tests. Needs no camera and no display. ## A caveat on typed optional inputs -`dora validate --strict` injects a contract type for *every* typed manifest input and then warns `input_types key "..." not found in inputs list` for any the dataflow does not wire. So a typed optional input forces the example dataflow to wire it. That is why `dataflow.yml` wires `text` from `pyarrow-sender` — dropping it would fail CI. If a typed input is added to the manifest, either wire it in `dataflow.yml` or leave it untyped. +`dora validate --strict` injects a contract type for *every* typed manifest input and then warns `input_types key "..." not found in inputs list` for any the dataflow does not wire. So a typed optional input forces the example dataflow to wire it. If a typed input is added to the manifest, either wire it in `dataflow.yml` or leave it untyped. ## Rules diff --git a/README.md b/README.md index 24017a7..b3d2207 100644 --- a/README.md +++ b/README.md @@ -27,44 +27,35 @@ nodes: Run the dataflow with `--uv` (`dora run dataflow.yml --uv`) so the node gets its own managed environment. +Wire `image` from any node publishing one of the encodings below, and `bbox` from any +detector publishing the struct below. Both inputs are optional: an `image` alone is a +plain viewer. + ### Runnable example -[`dataflow.yml`](dataflow.yml) wires a webcam and a fixed caption from dora-hub into -the viewer: +[`dataflow.yml`](dataflow.yml) is self-contained. [`examples/source.py`](examples/source.py) +is a synthetic source node that emits an animated frame, the box around the square +moving in it, and a caption — the three inputs this node draws — so the example needs +no camera, no detector and no network: ```bash dora run dataflow.yml --uv ``` -It builds the node from this checkout (`path:` + `build:`) rather than from a tag, +A window opens with a square bouncing across a gradient, boxed and labelled. Press `q` +to quit. + +It builds both nodes from this checkout (`path:` + `build:`) rather than from a tag, which is what lets `dora validate` pick up the manifest described below. -`bbox` is left unwired there, because every bbox source is a detector and pulling one -in would put a model download in front of a webcam preview. To see boxes, add -[`dora-yolo`](https://github.com/dora-rs/dora-hub/tree/main/node-hub/dora-yolo) between -the camera and the viewer: +Set `SOURCE_FRAMES` to stop the source after a fixed number of frames instead of running +until you quit; that plus `CI=true` is how the same dataflow runs headless as a test: -```yaml - - id: detector - git: https://github.com/dora-rs/dora-hub.git - build: pip install ./node-hub/dora-yolo - path: dora-yolo - inputs: - image: camera/image - outputs: - - bbox - - - id: plot - path: dora-plot - build: pip install . - inputs: - image: camera/image - bbox: detector/bbox +```bash +CI=true SOURCE_FRAMES=30 dora run dataflow.yml --uv ``` -To display a [dora-zed](https://gitea.liria.cl/liria/dora-zed) camera instead, wire -`image: zed-camera/image_bgr` — that node publishes `bgr8` with the `width`, `height` -and `encoding` metadata this one expects. +`SOURCE_WIDTH` and `SOURCE_HEIGHT` (default 640x480) set the frame size. ### Node manifest @@ -160,5 +151,6 @@ display to show the window on (or `CI=true` to run without one). uv run pytest ``` -The tests cover decoding, the bbox format conversion and the headless draw path, so they -need neither a camera, a dataflow nor a display. +Unit tests over decoding, the bbox format conversion and the headless draw path; they +need neither a camera, a dataflow nor a display. The headless dataflow run above covers +the rest — the manifest, the wiring and the event loop. diff --git a/dataflow.yml b/dataflow.yml index 38bf7f5..0d40ac0 100644 --- a/dataflow.yml +++ b/dataflow.yml @@ -1,41 +1,34 @@ -# The repository's own example: a webcam and a fixed caption feeding the viewer. -# The plot node is built from this checkout with `path:` + `build:`, so -# `dora validate` picks up the typed contract in dora-node.yml next to it. +# The repository's own example, and its end-to-end test: a synthetic source +# feeding the viewer. Everything it needs is in this checkout -- no camera, no +# detector, no network -- so it is also what CI runs. # -# Run it with `--uv` so every Python node gets its own managed environment. -# A consumer wires the node from a tag instead — see README.md. +# Both nodes are built from the checkout with `path:` + `build:`, which is what +# lets `dora validate` pick up the typed contract in dora-node.yml. A consumer +# wires the viewer from a tag instead -- see README.md. # -# `bbox` is left unwired: every bbox source is a detector, and pulling one in -# would put a model download in the way of a webcam preview. README.md shows -# the wiring. +# Run it with `--uv` so every node gets its own managed environment. strict_types: true nodes: - - id: camera - git: https://github.com/dora-rs/dora-hub.git - build: pip install ./node-hub/opencv-video-capture - path: opencv-video-capture + - id: source + path: examples/source.py + # The source needs dora-rs, numpy and pyarrow; installing the project gives + # it exactly the versions pinned in uv.lock. + build: pip install . inputs: - tick: dora/timer/millis/100 + tick: dora/timer/millis/33 outputs: - image + - bbox + - text output_types: image: std/core/v1/UInt8 - - - id: caption - git: https://github.com/dora-rs/dora-hub.git - build: pip install ./node-hub/pyarrow-sender - path: pyarrow-sender - env: - DATA: "'dora-plot'" - outputs: - - data - output_types: - data: std/core/v1/String + text: std/core/v1/String - id: plot path: dora-plot build: pip install . inputs: - image: camera/image - text: caption/data + image: source/image + bbox: source/bbox + text: source/text diff --git a/examples/source.py b/examples/source.py new file mode 100644 index 0000000..711c2a1 --- /dev/null +++ b/examples/source.py @@ -0,0 +1,73 @@ +"""Synthetic source node: this repository's stand-in for a camera and a detector. + +Emits an animated frame, the box around the square moving in it, and a caption -- +the three inputs dora-plot draws -- so the example dataflow needs no camera, no +detector and no network. +""" + +import os + +import numpy as np +import pyarrow as pa +from dora import Node + +WIDTH = int(os.getenv("SOURCE_WIDTH", "640")) +HEIGHT = int(os.getenv("SOURCE_HEIGHT", "480")) +# 0 runs until the viewer quits; a positive count is what lets CI run the +# dataflow to completion instead of killing it on a timeout. +FRAMES = int(os.getenv("SOURCE_FRAMES", "0")) + +BOX = 96 +SPEED = 6 +CAPTION = "dora-plot synthetic source" + + +def main() -> None: + """Send one image, bbox and text per tick.""" + node = Node("source") + sent = 0 + + for event in node: + if event["type"] != "INPUT": + continue + + frame, corners = render(sent) + node.send_output( + "image", + pa.array(frame.ravel()), + {"width": WIDTH, "height": HEIGHT, "encoding": "bgr8"}, + ) + node.send_output( + "bbox", + pa.array([{ + "bbox": corners, + "conf": np.array([0.99]), + "labels": np.array(["square"]), + }]), + {"format": "xyxy"}, + ) + node.send_output("text", pa.array([CAPTION])) + + sent += 1 + if FRAMES and sent >= FRAMES: + break + + +def render(index: int) -> tuple[np.ndarray, np.ndarray]: + """Draw frame `index` and return it with the xyxy corners of the square in it.""" + frame = np.zeros((HEIGHT, WIDTH, 3), dtype=np.uint8) + frame[:, :, 0] = np.linspace(40, 140, HEIGHT, dtype=np.uint8)[:, None] + + travel = WIDTH - BOX + # A triangle wave over `index`: the square starts at the left edge, runs to + # the right edge, and comes back. + left = travel - abs((index * SPEED) % (2 * travel) - travel) + top = (HEIGHT - BOX) // 2 + frame[top : top + BOX, left : left + BOX] = (60, 200, 255) + + corners = np.array([left, top, left + BOX, top + BOX], dtype=np.float64) + return frame, corners + + +if __name__ == "__main__": + main()