liria/dora-image-view
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

make the node standalone and rename it to dora-image-view #3

Merged
cristhian merged 5 commits from standalone-dora-plot into main 2026-09-15 09:33:12 +00:00
Conversation 0 Commits 5 Files Changed 15
+151 -89
5 changed files with 151 additions and 89 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit 22bd1e9e89 - Show all commits
.gitea/workflows/validate.yml
+23 -7
View File
@@ -1,5 +1,6 @@
# Two gates, neither of which needs a camera or a display: the dataflow still
# matches the typed contract in dora-node.yml, and the node's own tests pass.
# Two gates, neither of which needs a camera or a display: the example dataflow
# still matches the typed contract in dora-node.yml and still runs end to end,
# and the node's own unit tests pass.
name: validate
# A pull request would otherwise run this twice on the same commit: once for
@@ -14,11 +15,22 @@ env:
DORA_TARGET: x86_64-unknown-linux-gnu
jobs:
validate:
dataflow:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
# The opencv-python wheel links against libGL at import time, so cv2
# cannot even be imported on the bare runner image.
- name: Install the OpenCV shared libraries
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends libgl1 libglib2.0-0
- uses: astral-sh/setup-uv@v5
with:
enable-cache: true
- name: Install the dora CLI
run: |
curl -fsSL -o dora-cli.tar.gz \
@@ -36,13 +48,19 @@ jobs:
- name: Validate the dataflow
run: dora validate dataflow.yml
# CI=true is already set by the runner, which keeps the viewer from trying
# to open a window; SOURCE_FRAMES bounds the source so the run ends on its
# own. Everything the dataflow needs is in the checkout.
- name: Run the dataflow headless
env:
SOURCE_FRAMES: 30
run: dora run dataflow.yml --uv
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
# The opencv-python wheel links against libGL at import time, so cv2
# cannot even be imported on the bare runner image.
- name: Install the OpenCV shared libraries
run: |
sudo apt-get update
@@ -59,6 +77,4 @@ jobs:
- run: uv run ruff check .
# CI=true is already set by the runner, which is what keeps the node from
# trying to open a window on a headless machine.
- run: uv run pytest -q
AGENTS.md
+16 -28
View File
@@ -4,49 +4,37 @@ This file provides guidance to AI coding assistants when working with code in th
## What this is
A dora-rs Python node that displays a video stream with overlays: it keeps a current frame, bbox set and text string, and every `image` event redraws all three into an OpenCV window. It is a display sink — it produces no outputs.
A dora-rs Python node that displays a video stream with overlays: it keeps a current frame, bbox set and text string, and every `image` event redraws all three into an OpenCV window. It is a display sink — it produces no outputs. [README.md](README.md) documents the inputs, the encodings and the environment variables; this file covers only what the code and the README do not say.
Based on [`opencv-plot`](https://github.com/dora-rs/dora-hub/tree/main/node-hub/opencv-plot) from dora-hub, brought up to the dora 1.0 node contract and packaged with `uv`.
Everything the example dataflow needs is in this checkout. Do not reintroduce a dependency on another repository — the example must keep running with no camera, no detector and no network.
**It is Python on purpose.** The node was first written in Rust against the `opencv` crate; that crate generates its bindings from the OpenCV headers installed on the build machine, so it needs `libopencv-dev` and `libclang-dev` on every machine and every architecture and does not cross-compile. Every dependency here ships prebuilt wheels for linux-x86_64, linux-aarch64 and macos-arm64, so `uv sync` is the whole install. Do not reintroduce a dependency that has to be compiled from source.
## Build
## Commands
```bash
uv sync # runtime deps
uv sync --group dev # plus pytest and ruff
uv sync --group dev # deps plus pytest and ruff
uv run pytest && uv run ruff check . # no camera/dataflow/display needed
dora run dataflow.yml --uv # the example, in a window
CI=true SOURCE_FRAMES=30 dora run dataflow.yml --uv # the same, headless
```
`dora build --uv` / `dora run --uv` rewrites the manifest's `build: pip install .` to `uv pip install .` and gives the node its own managed venv under `.dora/python-envs/<node-id>/`, so what it imports at runtime is what was installed at build time.
**Requirements:** dora 1.0+ (verified with `dora-cli 1.0.1`), Python 3.11+ (`dora-rs` 1.0.1 requires it), `uv`, and a display — or `CI=true`.
## Run
```bash
dora run dataflow.yml --uv
```
## Test
```bash
uv run pytest # 14 tests, no camera/dataflow/display needed
uv run ruff check .
```
`dora build --uv` / `dora run --uv` rewrites each manifest's `build: pip install .` to `uv pip install .` and gives the node its own managed venv under `.dora/python-envs/<node-id>/`, so what it imports at runtime is what was installed at build time.
## Architecture
- **`dora_plot/main.py`** — Argument parsing and the dora event loop, nothing else. `image` decodes and redraws; `bbox` and `text` only store state. The env var wins over the CLI flag for `PLOT_WIDTH`/`PLOT_HEIGHT`, matching the upstream node. `Node(args.name)` rather than `Node()` so the node also runs standalone as a dynamic node.
- **`dora_plot/decode.py`** — `to_bgr()`: every supported encoding to a writable BGR array. `width`/`height` are only consulted for the raw encodings (`bgr8`, `rgb8`, `yuv420`); the rest carry their own dimensions, so a missing dimension is an error only where it actually matters. Two behaviours are deliberate, not bugs: `bgr8` **copies** (the payload may be a read-only view on shared memory, and the frame is annotated in place), and `yuv420` converts with `COLOR_YUV420p2RGB` rather than `...2BGR` — the upstream node converts to RGB and then displays it as BGR, and that channel swap is part of the behaviour callers already see.
- **`dora_plot/plot.py`** — `Plot` (the overlay state and the window) and `Bboxes`. `show()` draws, resizes and displays, and returns whether `q` was pressed. Resizing only happens when *both* `PLOT_WIDTH` and `PLOT_HEIGHT` are set. `headless` (from `CI=true`) skips only the window: every input is still decoded, which is what makes the node safe in a pipeline test.
- **`dora_plot/main.py`** — Argument parsing and the dora event loop, nothing else. The env var wins over the CLI flag for `PLOT_WIDTH`/`PLOT_HEIGHT`, matching the upstream node. `Node(args.name)` rather than `Node()` so the node also runs standalone as a dynamic node.
- **`dora_plot/decode.py`** — `to_bgr()`: every supported encoding to a writable BGR array. Two behaviours are deliberate, not bugs: `bgr8` **copies** (the payload may be a read-only view on shared memory, and the frame is annotated in place), and `yuv420` converts with `COLOR_YUV420p2RGB` rather than `...2BGR` — the upstream node converts to RGB and then displays it as BGR, and that channel swap is part of the behaviour callers already see.
- **`dora_plot/plot.py`** — `Plot` (the overlay state and the window) and `Bboxes`. `headless` (from `CI=true`) skips only the window: every input is still decoded, which is what makes the node safe in a pipeline test.
- **`dora_plot/bbox.py`** — Parses the `bbox` struct into xyxy corners. `xywh` is centre-based (x, y are the box centre). `_flat()` unwraps the single-element list that `pa.array([{...}])` produces around each field, and reads a field stored flat as-is.
- **`dora-node.yml`** — Node manifest (`apiVersion: 1`). `dora validate` injects these types into any dataflow that reaches the node by `path:`; a `git:`-sourced node does **not** pick it up, which is why the dataflow builds from the checkout. `image` and `text` carry type URNs naming the element type (`std/core/v1/UInt8`, `.../String`) because each payload is a raw buffer described by its metadata, not an Arrow struct. `bbox` is deliberately **untyped**: senders disagree on whether its numeric fields are 32- or 64-bit, and the node reads either.
- **`dataflow.yml`** — Example dataflow: a webcam and a one-shot caption feeding the viewer, with `strict_types: true`. Note the `output_types:` on the two `git:`-sourced nodes — their own manifests are not injected, so without those annotations strict validation fails on an untyped upstream. `bbox` is left unwired on purpose; every bbox source is a detector, and wiring one would put a model download in front of a webcam preview.
- **`.gitea/workflows/validate.yml`** — CI gate running `dora validate` on the dataflow on push and pull request. Needs no camera and no display.
- **`examples/source.py`** — The synthetic source node: an animated frame, the box around the square in it, and a caption. It exists so the example dataflow exercises all three inputs without a camera or a detector, and so CI can run the dataflow rather than only validate it. `SOURCE_FRAMES` bounds the run, which is what lets CI wait for a clean exit instead of killing it on a timeout.
- **`dora-node.yml`** — Node manifest (`apiVersion: 1`). `dora validate` injects these types into any dataflow that reaches the node by `path:`; a `git:`-sourced node does **not** pick it up, which is why the example builds from the checkout. `image` and `text` carry type URNs naming the element type because each payload is a raw buffer described by its metadata, not an Arrow struct. `bbox` is deliberately **untyped**: senders disagree on whether its numeric fields are 32- or 64-bit, and the node reads either.
- **`dataflow.yml`** — The example and the CI gate, with `strict_types: true`. `source` carries `output_types:` because it has no manifest of its own; `bbox` is annotated on neither side, matching the manifest.
- **`.gitea/workflows/validate.yml`** — CI: validates and runs the dataflow headless, and runs the unit tests. Needs no camera and no display.
## A caveat on typed optional inputs
`dora validate --strict` injects a contract type for *every* typed manifest input and then warns `input_types key "..." not found in inputs list` for any the dataflow does not wire. So a typed optional input forces the example dataflow to wire it. That is why `dataflow.yml` wires `text` from `pyarrow-sender` — dropping it would fail CI. If a typed input is added to the manifest, either wire it in `dataflow.yml` or leave it untyped.
`dora validate --strict` injects a contract type for *every* typed manifest input and then warns `input_types key "..." not found in inputs list` for any the dataflow does not wire. So a typed optional input forces the example dataflow to wire it. If a typed input is added to the manifest, either wire it in `dataflow.yml` or leave it untyped.
## Rules
README.md
+20 -28
View File
@@ -27,44 +27,35 @@ nodes:
Run the dataflow with `--uv` (`dora run dataflow.yml --uv`) so the node gets its own
managed environment.
Wire `image` from any node publishing one of the encodings below, and `bbox` from any
detector publishing the struct below. Both inputs are optional: an `image` alone is a
plain viewer.
### Runnable example
[`dataflow.yml`](dataflow.yml) wires a webcam and a fixed caption from dora-hub into
the viewer:
[`dataflow.yml`](dataflow.yml) is self-contained. [`examples/source.py`](examples/source.py)
is a synthetic source node that emits an animated frame, the box around the square
moving in it, and a caption — the three inputs this node draws — so the example needs
no camera, no detector and no network:
```bash
dora run dataflow.yml --uv
```
It builds the node from this checkout (`path:` + `build:`) rather than from a tag,
A window opens with a square bouncing across a gradient, boxed and labelled. Press `q`
to quit.
It builds both nodes from this checkout (`path:` + `build:`) rather than from a tag,
which is what lets `dora validate` pick up the manifest described below.
`bbox` is left unwired there, because every bbox source is a detector and pulling one
in would put a model download in front of a webcam preview. To see boxes, add
[`dora-yolo`](https://github.com/dora-rs/dora-hub/tree/main/node-hub/dora-yolo) between
the camera and the viewer:
Set `SOURCE_FRAMES` to stop the source after a fixed number of frames instead of running
until you quit; that plus `CI=true` is how the same dataflow runs headless as a test:
```yaml
- id: detector
git: https://github.com/dora-rs/dora-hub.git
build: pip install ./node-hub/dora-yolo
path: dora-yolo
inputs:
image: camera/image
outputs:
- bbox
- id: plot
path: dora-plot
build: pip install .
inputs:
image: camera/image
bbox: detector/bbox
```bash
CI=true SOURCE_FRAMES=30 dora run dataflow.yml --uv
```
To display a [dora-zed](https://gitea.liria.cl/liria/dora-zed) camera instead, wire
`image: zed-camera/image_bgr` — that node publishes `bgr8` with the `width`, `height`
and `encoding` metadata this one expects.
`SOURCE_WIDTH` and `SOURCE_HEIGHT` (default 640x480) set the frame size.
### Node manifest
@@ -160,5 +151,6 @@ display to show the window on (or `CI=true` to run without one).
uv run pytest
```
The tests cover decoding, the bbox format conversion and the headless draw path, so they
need neither a camera, a dataflow nor a display.
Unit tests over decoding, the bbox format conversion and the headless draw path; they
need neither a camera, a dataflow nor a display. The headless dataflow run above covers
the rest — the manifest, the wiring and the event loop.
dataflow.yml
+19 -26
View File
@@ -1,41 +1,34 @@
# The repository's own example: a webcam and a fixed caption feeding the viewer.
# The plot node is built from this checkout with `path:` + `build:`, so
# `dora validate` picks up the typed contract in dora-node.yml next to it.
# The repository's own example, and its end-to-end test: a synthetic source
# feeding the viewer. Everything it needs is in this checkout -- no camera, no
# detector, no network -- so it is also what CI runs.
#
# Run it with `--uv` so every Python node gets its own managed environment.
# A consumer wires the node from a tag instead — see README.md.
# Both nodes are built from the checkout with `path:` + `build:`, which is what
# lets `dora validate` pick up the typed contract in dora-node.yml. A consumer
# wires the viewer from a tag instead -- see README.md.
#
# `bbox` is left unwired: every bbox source is a detector, and pulling one in
# would put a model download in the way of a webcam preview. README.md shows
# the wiring.
# Run it with `--uv` so every node gets its own managed environment.
strict_types: true
nodes:
- id: camera
git: https://github.com/dora-rs/dora-hub.git
build: pip install ./node-hub/opencv-video-capture
path: opencv-video-capture
- id: source
path: examples/source.py
# The source needs dora-rs, numpy and pyarrow; installing the project gives
# it exactly the versions pinned in uv.lock.
build: pip install .
inputs:
tick: dora/timer/millis/100
tick: dora/timer/millis/33
outputs:
- image
- bbox
- text
output_types:
image: std/core/v1/UInt8
- id: caption
git: https://github.com/dora-rs/dora-hub.git
build: pip install ./node-hub/pyarrow-sender
path: pyarrow-sender
env:
DATA: "'dora-plot'"
outputs:
- data
output_types:
data: std/core/v1/String
text: std/core/v1/String
- id: plot
path: dora-plot
build: pip install .
inputs:
image: camera/image
text: caption/data
image: source/image
bbox: source/bbox
text: source/text
examples/source.py
+73
View File
@@ -0,0 +1,73 @@
"""Synthetic source node: this repository's stand-in for a camera and a detector.
Emits an animated frame, the box around the square moving in it, and a caption --
the three inputs dora-plot draws -- so the example dataflow needs no camera, no
detector and no network.
"""
import os
import numpy as np
import pyarrow as pa
from dora import Node
WIDTH = int(os.getenv("SOURCE_WIDTH", "640"))
HEIGHT = int(os.getenv("SOURCE_HEIGHT", "480"))
# 0 runs until the viewer quits; a positive count is what lets CI run the
# dataflow to completion instead of killing it on a timeout.
FRAMES = int(os.getenv("SOURCE_FRAMES", "0"))
BOX = 96
SPEED = 6
CAPTION = "dora-plot synthetic source"
def main() -> None:
"""Send one image, bbox and text per tick."""
node = Node("source")
sent = 0
for event in node:
if event["type"] != "INPUT":
continue
frame, corners = render(sent)
node.send_output(
"image",
pa.array(frame.ravel()),
{"width": WIDTH, "height": HEIGHT, "encoding": "bgr8"},
)
node.send_output(
"bbox",
pa.array([{
"bbox": corners,
"conf": np.array([0.99]),
"labels": np.array(["square"]),
}]),
{"format": "xyxy"},
)
node.send_output("text", pa.array([CAPTION]))
sent += 1
if FRAMES and sent >= FRAMES:
break
def render(index: int) -> tuple[np.ndarray, np.ndarray]:
"""Draw frame `index` and return it with the xyxy corners of the square in it."""
frame = np.zeros((HEIGHT, WIDTH, 3), dtype=np.uint8)
frame[:, :, 0] = np.linspace(40, 140, HEIGHT, dtype=np.uint8)[:, None]
travel = WIDTH - BOX
# A triangle wave over `index`: the square starts at the left edge, runs to
# the right edge, and comes back.
left = travel - abs((index * SPEED) % (2 * travel) - travel)
top = (HEIGHT - BOX) // 2
frame[top : top + BOX, left : left + BOX] = (60, 200, 255)
corners = np.array([left, top, left + BOX, top + BOX], dtype=np.float64)
return frame, corners
if __name__ == "__main__":
main()