dora-image-view/README.md

dora-image-view

Dora node that displays a colour image and a depth map side by side in one OpenCV window, with bounding box, text and camera intrinsics overlays. A display sink for debugging a vision pipeline: it produces no outputs.

Based on opencv-plot from dora-hub, brought up to the dora 1.0 node contract and packaged with uv.

Usage

Add to your dataflow:

nodes:
  - id: view
    git: ssh://git@gitea.liria.cl:5001/liria/dora-image-view.git
    tag: v0.1.0
    build: pip install .
    path: dora-image-view
    inputs:
      image: camera/image_bgr
      depth: camera/depth
      camera_info: camera/camera_info
      bbox: detector/bbox
      text: detector/text

Run the dataflow with --uv (dora run dataflow.yml --uv) so the node gets its own managed environment.

Every input is optional, and each of image and depth fills one panel of the window, tiled left to right in that order. Wiring one of them is a plain viewer; wiring several shows them side by side, scaled to a common height. bbox, text and camera_info are overlays rather than panels — boxes are drawn on the image panel, in its own pixels, and the caption on the composite.

Runnable example

dataflow.yml is self-contained. examples/source.py is a synthetic source node that emits an animated frame, the box around the square moving in it, and a caption — the three inputs this node draws — so the example needs no camera, no detector and no network:

dora run dataflow.yml --uv

A window opens with a square bouncing across a gradient, boxed and labelled. Press q to quit.

It builds the viewer from this checkout (path: + build:) rather than from a tag, which is what lets dora validate pick up the manifest described below.

Set SOURCE_FRAMES to stop the source after a fixed number of frames instead of running until you quit; that plus CI=true is how the same dataflow runs headless as a test:

CI=true SOURCE_FRAMES=30 dora run dataflow.yml --uv

SOURCE_WIDTH and SOURCE_HEIGHT (default 640x480) set the frame size.

Node manifest

dora-node.yml declares the node's inputs with type URNs. No source package is shared between node repositories, so it is the only contract a consumer has, and it is what makes a wiring mistake fail at build time instead of at runtime. The example source is typed inline in the dataflow with output_types:, so dora validate checks the two against each other:

$ dora validate dataflow.yml
  node "view": injected 4 contract type(s) from .../dora-node.yml
All type annotations OK.

There is one manifest because this repository ships one node. The example source is a test fixture rather than something anyone installs, so it has none, and runs through the interpreter to keep dora validate from matching the viewer's manifest against it.

image is std/core/v1/UInt8, depth and camera_info are std/core/v1/Float32 and text is std/core/v1/String; the payloads are raw buffers described by their metadata, so the URN names the element type rather than a struct. bbox is deliberately left untyped — senders disagree on whether its numeric fields are 32- or 64-bit, and the node reads either.

Behavior

The node keeps a current frame, bbox set and text string. Each input updates its slice of that state:

• An image event decodes the frame and immediately redraws: it draws every stored bbox (green rectangle plus label, conf text), draws the stored text at the top left, resizes to VIEW_WIDTHxVIEW_HEIGHT if both are set, and shows it in a window titled Dora Node: dora-image-view. Press q in the window to quit.
• A depth event colour-maps the distances onto the depth panel and redraws.
• A camera_info event stores the pinhole model and draws it under the caption.
• A bbox or text event only updates stored state; it is drawn on the next redraw.

Setting CI=true suppresses the display: the node still decodes every input but opens no window. That is what makes it safe to run in a pipeline test.

Inputs

Input	Payload	Metadata
image	std/media/v1/Image struct, or a flat UInt8 array	none for the struct; width, height, encoding for the flat form
depth	Flat Float32 array, [H, W]	shape, units
camera_info	Flat Float32 array, [fx, fy, cx, cy] or a 9-element K	distortion, distortion_model
bbox	Arrow struct array, element [0] with fields bbox, conf, labels	format
text	Arrow array of size 1	—

Two image payload shapes are accepted. The typed one is std/media/v1/Image, an Arrow struct of width: UInt32, height: UInt32, encoding: Utf8 and data: LargeBinary, which carries inside the payload what the flat form carries beside it; when it is received the message metadata is not consulted at all. The flat form is a raw UInt8 buffer described by width, height and encoding metadata, which is what a node declaring std/core/v1/UInt8 sends.

The declared contract is still std/core/v1/UInt8, so a producer sending the struct should keep declaring that until every producer in the dataflow has moved: dora's type compatibility is directional, and a std/core/v1/UInt8 output wired into an std/media/v1/Image input fails dora validate.

Supported encoding values: bgr8, rgb8, jpeg/jpg/jpe/bmp/webp/png (encoded byte stream), yuv420, avif. width and height are only needed by the raw encodings — the rest carry their own dimensions. An unsupported encoding raises.

Supported format values: xyxy and xywh (centre-based, converted to xyxy).

depth is in millimetres unless units is m. In a depth map, 0 and any non-finite value mean missing depth and are drawn black. A payload sent as a raw byte slice — which is what a C++ node does — arrives as UInt8 despite the Float32 contract, and is reinterpreted rather than converted.

Camera intrinsics

camera_info carries the pinhole model the depth map is interpreted through, either as the four values [fx, fy, cx, cy] or as a nine-element row-major K, which is how ROS spells the same model in sensor_msgs/CameraInfo. Distortion coefficients are read from the distortion metadata.

The node displays the model and holds it. It needs none of it to draw a panel; it is there for what reads it next — turning a depth map into a point cloud needs fx, fy, cx and cy for every pixel, and nothing else in a dataflow carries them.

Intrinsics are also picked up from the metadata of any image or depth frame that carries fx, fy, cx and cy, so wiring camera_info is optional even for a camera that publishes it. That matters because a camera typically publishes camera_info once per session: a viewer started after the camera would otherwise never see it.

node.send_output("image", pa.array(frame.ravel()),
                 {"width": 640, "height": 480, "encoding": "bgr8"})

node.send_output("bbox", pa.array([{
    "bbox": np.array([...]),    # flattened, four values per box
    "conf": np.array([...]),
    "labels": np.array([...]),
}]), {"format": "xyxy"})

node.send_output("text", pa.array(["hello"]))

Outputs

None — it is a display sink.

Environment variables

Variable	Default	Meaning
VIEW_WIDTH	composite width	Resize the displayed composite to this width, in pixels.
VIEW_HEIGHT	composite height	Resize the displayed composite to this height, in pixels.
VIEW_DEPTH_MIN_MM	auto	Pin the near end of the depth colour range, in millimetres.
VIEW_DEPTH_MAX_MM	auto	Pin the far end of the depth colour range, in millimetres.

Resizing only happens when both VIEW_WIDTH and VIEW_HEIGHT are set. Both are also available as --view-width and --view-height; the environment variable wins when both are given.

Each end of the depth range is auto-ranged over the valid pixels of the current frame unless it is pinned, so pinning one end leaves the other automatic. Auto-ranging makes an unconfigured sensor legible at the cost of colours that shift from frame to frame; pin both ends to compare frames against each other. The range covers the full extent of the valid pixels rather than a percentile of them, so a small near object against a large wall stays visible.

CI=true suppresses the display window; it is not a configuration knob.

Build

uv sync

Every dependency ships prebuilt wheels for linux-x86_64, linux-aarch64 and macos-arm64, so the node installs with no system packages and no compiler on any of them — including AVIF support, which comes from pillow-avif-plugin rather than OpenCV.

Requirements: dora 1.0+ (verified with dora-cli 1.0.1), Python 3.11+, uv, and a display to show the window on (or CI=true to run without one). On a bare Linux image opencv-python also needs the libGL shared libraries (libgl1, libglib2.0-0), which it links at import time.

Test

uv run pytest

Unit tests over decoding, the bbox format conversion and the headless draw path; they need neither a camera, a dataflow nor a display. The headless dataflow run above covers the rest — the manifest, the wiring and the event loop.