liria/dora-image-view
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

add a depth map panel

Browse Source 
The node showed a colour image only, so debugging a depth pipeline meant
either trusting the numbers or writing a throwaway viewer. It now keeps one
panel per frame stream -- image and depth -- and tiles them side by side,
scaled to a common height. Wiring one is a plain viewer; wiring both shows
them together. bbox and text stay overlays rather than panels.

The contract follows what dora-zed already publishes: float32 [H, W] in
millimetres with 0 for missing. It is sent there as a raw byte slice, so the
payload arrives as UInt8 despite the Float32 contract and is reinterpreted
rather than converted.

Auto-ranging the colormap over the valid pixels' full extent, rather than over
a percentile of them, is deliberate and was not the first attempt: trimming
the tails collapsed on the scene the node exists for. A small object in front
of a large wall is a few percent of the pixels, so both tails landed on the
wall and the object clipped away. Rendering a frame and looking at it found
that; every test passed either way.

Verified with dora-cli 1.0.1: validate passes with both contracts injected,
the dataflow runs both nodes to a clean exit, and a composed frame was
rendered and inspected.
This commit is contained in:
cristhian
2026-09-15 05:06:50 -03:00
parent 407395ab88
commit 0efeacd538
10 changed files with 417 additions and 50 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+37 -16
View File
@@ -1,8 +1,8 @@
# dora-image-view
Dora node that overlays bounding boxes, labels and text on an image input and displays
it in an OpenCV window. A display sink for debugging a vision pipeline: it produces no
outputs.
Dora node that displays a colour image and a depth map side by side in one OpenCV
window, with bounding box and text overlays. A display sink for debugging a
vision pipeline: it produces no outputs.
Based on [`opencv-plot`](https://github.com/dora-rs/dora-hub/tree/main/node-hub/opencv-plot)
from dora-hub, brought up to the dora 1.0 node contract and packaged with `uv`.
@@ -19,7 +19,8 @@ nodes:
build: pip install .
path: dora-image-view
inputs:
image: camera/image
image: camera/image_bgr
depth: camera/depth
bbox: detector/bbox
text: detector/text
```
@@ -27,9 +28,11 @@ nodes:
Run the dataflow with `--uv` (`dora run dataflow.yml --uv`) so the node gets its own
managed environment.
Wire `image` from any node publishing one of the encodings below, and `bbox` from any
detector publishing the struct below. Both inputs are optional: an `image` alone is a
plain viewer.
Every input is optional, and each of `image` and `depth` fills one panel of the window,
tiled left to right in that order. Wiring one of them is a plain viewer;
wiring several shows them side by side, scaled to a common height. `bbox` and `text`
are overlays rather than panels — boxes are drawn on the `image` panel, in its own
pixels, and the caption on the composite.
### Runnable example
@@ -72,10 +75,11 @@ $ dora validate dataflow.yml
All type annotations OK.
```
`image` is `std/core/v1/UInt8` and `text` is `std/core/v1/String`; the payloads are raw
buffers described by their metadata, so the URN names the element type rather than a
struct. `bbox` is deliberately left untyped — senders disagree on whether its numeric
fields are 32- or 64-bit, and the node reads either.
`image` is `std/core/v1/UInt8`, `depth` is `std/core/v1/Float32` and `text` is
`std/core/v1/String`; the payloads are raw buffers described by their
metadata, so the URN names the element type rather than a struct. `bbox` is deliberately
left untyped — senders disagree on whether its numeric fields are 32- or 64-bit, and the
node reads either.
## Behavior
@@ -86,8 +90,9 @@ of that state:
bbox (green rectangle plus `label, conf` text), draws the stored text at the top left,
resizes to `VIEW_WIDTH`x`VIEW_HEIGHT` if both are set, and shows it in a window titled
`Dora Node: dora-image-view`. Press `q` in the window to quit.
- A **`depth`** event colour-maps the distances onto the depth panel and redraws.
- A **`bbox`** or **`text`** event only updates stored state; it is drawn on the next
`image` redraw.
redraw.
Setting `CI=true` suppresses the display: the node still decodes every input but opens
no window. That is what makes it safe to run in a pipeline test.
@@ -97,6 +102,7 @@ no window. That is what makes it safe to run in a pipeline test.
| Input | Payload | Metadata |
|---|---|---|
| `image` | Flat `UInt8` array | `width`, `height`, `encoding` |
| `depth` | Flat `Float32` array, `[H, W]` | `shape`, `units` |
| `bbox` | Arrow struct array, element `[0]` with fields `bbox`, `conf`, `labels` | `format` |
| `text` | Arrow array of size 1 | — |
@@ -106,6 +112,11 @@ encodings — the rest carry their own dimensions. An unsupported encoding raise
Supported `format` values: `xyxy` and `xywh` (centre-based, converted to `xyxy`).
`depth` is in millimetres unless `units` is `m`. In a depth map, `0`
and any non-finite value mean missing depth and are drawn black. A payload sent as a
raw byte slice — which is what a C++ node does — arrives as `UInt8` despite the
`Float32` contract, and is reinterpreted rather than converted.
```python
node.send_output("image", pa.array(frame.ravel()),
{"width": 640, "height": 480, "encoding": "bgr8"})
@@ -127,11 +138,21 @@ None — it is a display sink.
| Variable | Default | Meaning |
|---|---|---|
| `VIEW_WIDTH` | image width | Resize the displayed frame to this width, in pixels. |
| `VIEW_HEIGHT` | image height | Resize the displayed frame to this height, in pixels. |
| `VIEW_WIDTH` | composite width | Resize the displayed composite to this width, in pixels. |
| `VIEW_HEIGHT` | composite height | Resize the displayed composite to this height, in pixels. |
| `VIEW_DEPTH_MIN_MM` | auto | Pin the near end of the depth colour range, in millimetres. |
| `VIEW_DEPTH_MAX_MM` | auto | Pin the far end of the depth colour range, in millimetres. |
Resizing only happens when both are set. Both are also available as `--view-width` and
`--view-height`; the environment variable wins when both are given.
Resizing only happens when both `VIEW_WIDTH` and `VIEW_HEIGHT` are set. Both are also
available as `--view-width` and `--view-height`; the environment variable wins when both
are given.
Each end of the depth range is auto-ranged over the valid pixels of the current frame
unless it is pinned, so pinning one end leaves the other automatic. Auto-ranging makes
an unconfigured sensor legible at the cost of colours that shift from frame to frame;
pin both ends to compare frames against each other. The range covers the full extent of
the valid pixels rather than a percentile of them, so a small near object against a
large wall stays visible.
`CI=true` suppresses the display window; it is not a configuration knob.