dora-image-view/README.md

# dora-image-view

Dora node that displays a colour image and a depth map side by side in one OpenCV
window, with bounding box and text overlays. A display sink for debugging a
vision pipeline: it produces no outputs.

Based on [`opencv-plot`](https://github.com/dora-rs/dora-hub/tree/main/node-hub/opencv-plot)
from dora-hub, brought up to the dora 1.0 node contract and packaged with `uv`.

## Usage

Add to your dataflow:

```yaml
nodes:
  - id: view
    git: ssh://git@gitea.liria.cl:5001/liria/dora-image-view.git
    tag: v0.1.0
    build: pip install .
    path: dora-image-view
    inputs:
      image: camera/image_bgr
      depth: camera/depth
      bbox: detector/bbox
      text: detector/text
```

Run the dataflow with `--uv` (`dora run dataflow.yml --uv`) so the node gets its own
managed environment.

Every input is optional, and each of `image` and `depth` fills one panel of the window,
tiled left to right in that order. Wiring one of them is a plain viewer;
wiring several shows them side by side, scaled to a common height. `bbox` and `text`
are overlays rather than panels — boxes are drawn on the `image` panel, in its own
pixels, and the caption on the composite.

### Runnable example

[`dataflow.yml`](dataflow.yml) is self-contained. [`examples/source.py`](examples/source.py)
is a synthetic source node that emits an animated frame, the box around the square
moving in it, and a caption — the three inputs this node draws — so the example needs
no camera, no detector and no network:

```bash
dora run dataflow.yml --uv
```

A window opens with a square bouncing across a gradient, boxed and labelled. Press `q`
to quit.

It builds both nodes from this checkout (`path:` + `build:`) rather than from a tag,
which is what lets `dora validate` pick up the manifest described below.

Set `SOURCE_FRAMES` to stop the source after a fixed number of frames instead of running
until you quit; that plus `CI=true` is how the same dataflow runs headless as a test:

```bash
CI=true SOURCE_FRAMES=30 dora run dataflow.yml --uv
```

`SOURCE_WIDTH` and `SOURCE_HEIGHT` (default 640x480) set the frame size.

### Node manifest

[`dora-node.yml`](dora-node.yml) declares the node's inputs with type URNs. No source
package is shared between node repositories, so it is the only contract a consumer has,
and it is what makes a wiring mistake fail at build time instead of at runtime. The
example source has a manifest of its own, so `dora validate` checks the two against
each other:

```
$ dora validate dataflow.yml
  node "source": injected 2 contract type(s) from .../examples/dora-node.yml
  node "view": injected 2 contract type(s) from .../dora-node.yml
All type annotations OK.
```

`image` is `std/core/v1/UInt8`, `depth` is `std/core/v1/Float32` and `text` is
`std/core/v1/String`; the payloads are raw buffers described by their
metadata, so the URN names the element type rather than a struct. `bbox` is deliberately
left untyped — senders disagree on whether its numeric fields are 32- or 64-bit, and the
node reads either.

## Behavior

The node keeps a current frame, bbox set and text string. Each input updates its slice
of that state:

- An **`image`** event decodes the frame and immediately redraws: it draws every stored
  bbox (green rectangle plus `label, conf` text), draws the stored text at the top left,
  resizes to `VIEW_WIDTH`x`VIEW_HEIGHT` if both are set, and shows it in a window titled
  `Dora Node: dora-image-view`. Press `q` in the window to quit.
- A **`depth`** event colour-maps the distances onto the depth panel and redraws.
- A **`bbox`** or **`text`** event only updates stored state; it is drawn on the next
  redraw.

Setting `CI=true` suppresses the display: the node still decodes every input but opens
no window. That is what makes it safe to run in a pipeline test.

## Inputs

| Input | Payload | Metadata |
|---|---|---|
| `image` | Flat `UInt8` array | `width`, `height`, `encoding` |
| `depth` | Flat `Float32` array, `[H, W]` | `shape`, `units` |
| `bbox` | Arrow struct array, element `[0]` with fields `bbox`, `conf`, `labels` | `format` |
| `text` | Arrow array of size 1 | — |

Supported `encoding` values: `bgr8`, `rgb8`, `jpeg`/`jpg`/`jpe`/`bmp`/`webp`/`png`
(encoded byte stream), `yuv420`, `avif`. `width` and `height` are only needed by the raw
encodings — the rest carry their own dimensions. An unsupported encoding raises.

Supported `format` values: `xyxy` and `xywh` (centre-based, converted to `xyxy`).

`depth` is in millimetres unless `units` is `m`. In a depth map, `0`
and any non-finite value mean missing depth and are drawn black. A payload sent as a
raw byte slice — which is what a C++ node does — arrives as `UInt8` despite the
`Float32` contract, and is reinterpreted rather than converted.

```python
node.send_output("image", pa.array(frame.ravel()),
                 {"width": 640, "height": 480, "encoding": "bgr8"})

node.send_output("bbox", pa.array([{
    "bbox": np.array([...]),    # flattened, four values per box
    "conf": np.array([...]),
    "labels": np.array([...]),
}]), {"format": "xyxy"})

node.send_output("text", pa.array(["hello"]))
```

## Outputs

None — it is a display sink.

## Environment variables

| Variable | Default | Meaning |
|---|---|---|
| `VIEW_WIDTH` | composite width | Resize the displayed composite to this width, in pixels. |
| `VIEW_HEIGHT` | composite height | Resize the displayed composite to this height, in pixels. |
| `VIEW_DEPTH_MIN_MM` | auto | Pin the near end of the depth colour range, in millimetres. |
| `VIEW_DEPTH_MAX_MM` | auto | Pin the far end of the depth colour range, in millimetres. |

Resizing only happens when both `VIEW_WIDTH` and `VIEW_HEIGHT` are set. Both are also
available as `--view-width` and `--view-height`; the environment variable wins when both
are given.

Each end of the depth range is auto-ranged over the valid pixels of the current frame
unless it is pinned, so pinning one end leaves the other automatic. Auto-ranging makes
an unconfigured sensor legible at the cost of colours that shift from frame to frame;
pin both ends to compare frames against each other. The range covers the full extent of
the valid pixels rather than a percentile of them, so a small near object against a
large wall stays visible.

`CI=true` suppresses the display window; it is not a configuration knob.

## Build

```bash
uv sync
```

Every dependency ships prebuilt wheels for linux-x86_64, linux-aarch64 and macos-arm64,
so the node installs with no system packages and no compiler on any of them — including
AVIF support, which comes from `pillow-avif-plugin` rather than OpenCV.

**Requirements:** dora 1.0+ (verified with `dora-cli 1.0.1`), Python 3.11+, `uv`, and a
display to show the window on (or `CI=true` to run without one). On a bare Linux image
`opencv-python` also needs the libGL shared libraries (`libgl1`, `libglib2.0-0`), which
it links at import time.

## Test

```bash
uv run pytest
```

Unit tests over decoding, the bbox format conversion and the headless draw path; they
need neither a camera, a dataflow nor a display. The headless dataflow run above covers
the rest — the manifest, the wiring and the event loop.