dora-image-view

Dora node that displays a colour image and a depth map side by side in one OpenCV window, with bounding box, text and camera intrinsics overlays. A display sink for debugging a vision pipeline: it consumes frames and produces no outputs.

Based on opencv-plot from dora-hub, brought up to the dora 1.0 node contract, renamed, and changed to take the typed Arrow image struct rather than a flat buffer plus metadata.

Usage

Add to your dataflow:

nodes:
  - id: view
    git: https://gitea.liria.cl/liria/dora-image-view.git
    tag: v0.1.1
    build: uv pip install .
    path: dora-image-view
    inputs:
      image: camera/image
      depth: camera/depth
      camera_info: camera/camera_info
      bbox: detector/bbox
      text: detector/text

Run the dataflow with --uv (dora run dataflow.yml --uv): the build command is uv pip install ., so the node gets its own managed environment and never installs into an ambient one.

Every input is optional. image and depth each fill a panel, tiled left to right and scaled to a common height, so wiring one is a plain viewer and wiring both shows them side by side. bbox, text and camera_info are overlays rather than panels: boxes on the image panel in its own pixels, caption and camera model on the composite.

Runnable dataflows

Run either with dora run <file> --uv:

File	Contents
dataflow.yml	A synthetic source driving every input. What CI runs.
dataflow_webcam.yml	A real camera, via dora-webcam

dataflow.yml needs no camera, no detector and no network: a window opens with a square bouncing across a gradient, boxed and labelled, beside its depth map. Press q to quit. CI runs it headless on every push, which is what keeps it from rotting.

dataflow_webcam.yml is the worked example of consuming a real producer. A git: node needs anonymous read access to fetch — on Gitea that means a public organization, not just a public repository.

Inputs

Input	Payload	Metadata
image	std/media/v1/Image struct	none — the struct describes itself
depth	std/media/v1/Image struct, one channel	units (defaults to mm)
camera_info	[fx, fy, cx, cy] or a 9-element row-major K	distortion
bbox	Arrow struct {bbox, conf, labels}	format: xyxy or xywh
text	Arrow array of size 1	—

image and depth are both std/media/v1/Image: an Arrow struct of width: UInt32, height: UInt32, encoding: Utf8 and data: LargeBinary. The payload describes itself, so nothing about its shape travels as message metadata.

Image encodings: bgr8, rgb8, yuv420, avif, and jpeg/jpg/jpe/bmp/webp/ png as an encoded byte stream. Depth encodings: 32FC1/float32 and 16UC1/mono16/uint16, one channel per pixel — dora's registry has no depth type, and a depth map is an image whose pixels are distances, as ROS also treats it. 0 and non-finite values mean missing depth and draw black.

camera_info is also read from the metadata of any frame carrying fx, fy, cx and cy, so wiring it is optional even for a camera that publishes it — which covers a camera that publishes it once per session and a viewer started afterwards. The node needs none of it to draw; it is held for whatever turns depth into 3D.

Older payloads are still decoded, so a producer can move at its own pace: a flat UInt8 buffer with width/height/encoding metadata for image, and a flat Float32 buffer with shape/units for depth. Note that dora's type compatibility is directional, so a producer still declaring std/core/v1/UInt8 fails dora validate against an std/media/v1/Image input whatever it sends — its manifest has to move too.

Publishing to it:

image = pa.array(
    [{"width": 640, "height": 480, "encoding": "bgr8", "data": frame.tobytes()}],
    type=pa.struct([
        ("width", pa.uint32()), ("height", pa.uint32()),
        ("encoding", pa.utf8()), ("data", pa.large_binary()),
    ]),
)
node.send_output("image", image, {})

node.send_output("bbox", pa.array([{
    "bbox": np.array([...]),    # flattened, four values per box
    "conf": np.array([...]),
    "labels": np.array([...]),
}]), {"format": "xyxy"})

node.send_output("text", pa.array(["hello"]))

Outputs

None — it is a display sink.

Configuration

Settings are environment variables, which a dataflow can carry per node, keeping them with the wiring rather than in whatever shell starts it:

    env:
      VIEW_DEPTH_MIN_MM: 200
      VIEW_DEPTH_MAX_MM: 600

A node's env: wins over the environment the dataflow was started in; variables it does not set still pass through.

Variable	Default	Meaning
VIEW_WIDTH	composite width	Resize the displayed composite to this width, in pixels.
VIEW_HEIGHT	composite height	Resize the displayed composite to this height, in pixels.
VIEW_DEPTH_MIN_MM	auto	Pin the near end of the depth colour range, in millimetres.
VIEW_DEPTH_MAX_MM	auto	Pin the far end of the depth colour range, in millimetres.

Resizing only happens when both VIEW_WIDTH and VIEW_HEIGHT are set; both are also available as --view-width and --view-height, and the environment variable wins.

Each end of the depth range is auto-ranged over the current frame's valid pixels unless pinned, so pinning one end leaves the other automatic. Auto-ranging makes an unconfigured sensor legible at the cost of colours that shift between frames; pin both ends to compare frames against each other.

CI=true suppresses the window while still decoding every input, which is what makes the node safe inside a pipeline test. It is not a configuration knob.

Build

uv sync

Every dependency ships prebuilt wheels for linux-x86_64, linux-aarch64 and macos-arm64, so the node installs with no system packages and no compiler — including AVIF support, which comes from pillow-avif-plugin rather than OpenCV.

Requirements: dora 1.0+ (verified with dora-cli 1.0.1), Python 3.11+, uv, and a display (or CI=true to run without one). On a bare Linux image opencv-python needs the libGL shared libraries (libgl1, libglib2.0-0), which it links at import time.

Test

uv run pytest

Unit tests over decoding, depth, intrinsics, the bbox conversion and the headless draw path; they need neither a camera, a dataflow nor a display. The headless dataflow.yml run covers the rest — the manifest, the wiring and the event loop.