dora-image-view/README.md

# dora-image-view

Dora node that overlays bounding boxes, labels and text on an image input and displays
it in an OpenCV window. A display sink for debugging a vision pipeline: it produces no
outputs.

Based on [`opencv-plot`](https://github.com/dora-rs/dora-hub/tree/main/node-hub/opencv-plot)
from dora-hub, brought up to the dora 1.0 node contract and packaged with `uv`.

## Usage

Add to your dataflow:

```yaml
nodes:
  - id: view
    git: ssh://git@gitea.liria.cl:5001/liria/dora-image-view.git
    tag: v0.1.0
    build: pip install .
    path: dora-image-view
    inputs:
      image: camera/image
      bbox: detector/bbox
      text: detector/text
```

Run the dataflow with `--uv` (`dora run dataflow.yml --uv`) so the node gets its own
managed environment.

Wire `image` from any node publishing one of the encodings below, and `bbox` from any
detector publishing the struct below. Both inputs are optional: an `image` alone is a
plain viewer.

### Runnable example

[`dataflow.yml`](dataflow.yml) is self-contained. [`examples/source.py`](examples/source.py)
is a synthetic source node that emits an animated frame, the box around the square
moving in it, and a caption — the three inputs this node draws — so the example needs
no camera, no detector and no network:

```bash
dora run dataflow.yml --uv
```

A window opens with a square bouncing across a gradient, boxed and labelled. Press `q`
to quit.

It builds both nodes from this checkout (`path:` + `build:`) rather than from a tag,
which is what lets `dora validate` pick up the manifest described below.

Set `SOURCE_FRAMES` to stop the source after a fixed number of frames instead of running
until you quit; that plus `CI=true` is how the same dataflow runs headless as a test:

```bash
CI=true SOURCE_FRAMES=30 dora run dataflow.yml --uv
```

`SOURCE_WIDTH` and `SOURCE_HEIGHT` (default 640x480) set the frame size.

### Node manifest

[`dora-node.yml`](dora-node.yml) declares the node's inputs with type URNs. No source
package is shared between node repositories, so it is the only contract a consumer has,
and it is what makes a wiring mistake fail at build time instead of at runtime. The
example source has a manifest of its own, so `dora validate` checks the two against
each other:

```
$ dora validate dataflow.yml
  node "source": injected 2 contract type(s) from .../examples/dora-node.yml
  node "view": injected 2 contract type(s) from .../dora-node.yml
All type annotations OK.
```

`image` is `std/core/v1/UInt8` and `text` is `std/core/v1/String`; the payloads are raw
buffers described by their metadata, so the URN names the element type rather than a
struct. `bbox` is deliberately left untyped — senders disagree on whether its numeric
fields are 32- or 64-bit, and the node reads either.

## Behavior

The node keeps a current frame, bbox set and text string. Each input updates its slice
of that state:

- An **`image`** event decodes the frame and immediately redraws: it draws every stored
  bbox (green rectangle plus `label, conf` text), draws the stored text at the top left,
  resizes to `VIEW_WIDTH`x`VIEW_HEIGHT` if both are set, and shows it in a window titled
  `Dora Node: dora-image-view`. Press `q` in the window to quit.
- A **`bbox`** or **`text`** event only updates stored state; it is drawn on the next
  `image` redraw.

Setting `CI=true` suppresses the display: the node still decodes every input but opens
no window. That is what makes it safe to run in a pipeline test.

## Inputs

| Input | Payload | Metadata |
|---|---|---|
| `image` | Flat `UInt8` array | `width`, `height`, `encoding` |
| `bbox` | Arrow struct array, element `[0]` with fields `bbox`, `conf`, `labels` | `format` |
| `text` | Arrow array of size 1 | — |

Supported `encoding` values: `bgr8`, `rgb8`, `jpeg`/`jpg`/`jpe`/`bmp`/`webp`/`png`
(encoded byte stream), `yuv420`, `avif`. `width` and `height` are only needed by the raw
encodings — the rest carry their own dimensions. An unsupported encoding raises.

Supported `format` values: `xyxy` and `xywh` (centre-based, converted to `xyxy`).

```python
node.send_output("image", pa.array(frame.ravel()),
                 {"width": 640, "height": 480, "encoding": "bgr8"})

node.send_output("bbox", pa.array([{
    "bbox": np.array([...]),    # flattened, four values per box
    "conf": np.array([...]),
    "labels": np.array([...]),
}]), {"format": "xyxy"})

node.send_output("text", pa.array(["hello"]))
```

## Outputs

None — it is a display sink.

## Environment variables

| Variable | Default | Meaning |
|---|---|---|
| `VIEW_WIDTH` | image width | Resize the displayed frame to this width, in pixels. |
| `VIEW_HEIGHT` | image height | Resize the displayed frame to this height, in pixels. |

Resizing only happens when both are set. Both are also available as `--view-width` and
`--view-height`; the environment variable wins when both are given.

`CI=true` suppresses the display window; it is not a configuration knob.

## Build

```bash
uv sync
```

Every dependency ships prebuilt wheels for linux-x86_64, linux-aarch64 and macos-arm64,
so the node installs with no system packages and no compiler on any of them — including
AVIF support, which comes from `pillow-avif-plugin` rather than OpenCV.

**Requirements:** dora 1.0+ (verified with `dora-cli 1.0.1`), Python 3.11+, `uv`, and a
display to show the window on (or `CI=true` to run without one). On a bare Linux image
`opencv-python` also needs the libGL shared libraries (`libgl1`, `libglib2.0-0`), which
it links at import time.

## Test

```bash
uv run pytest
```

Unit tests over decoding, the bbox format conversion and the headless draw path; they
need neither a camera, a dataflow nor a display. The headless dataflow run above covers
the rest — the manifest, the wiring and the event loop.