cristhian
5a0f13d4db
Nothing in a dataflow carries fx, fy, cx and cy except the camera, and every pixel of a depth map needs them to become a 3D point. The node now accepts them so whatever computes that next has them to hand. They are read from two places because dora-zed publishes them twice: the camera_info message, and the metadata of every frame. The per-frame copy is not redundant -- camera_info goes out once per camera session, so a viewer started after the camera would otherwise never see it, and reading frame metadata means wiring the input is optional even for a camera that publishes it. A nine-element row-major K is accepted alongside the compact [fx, fy, cx, cy], since that is how ROS spells the same model in sensor_msgs/CameraInfo. The model is drawn under the caption rather than only stored. A depth map read through the wrong intrinsics looks entirely plausible, and this is where that becomes visible. as_float32 moves to a payload module: camera_info arrives as a raw byte slice for the same reason the depth map does, and the reinterpretation is now shared rather than written twice. Verified with dora-cli 1.0.1: validate passes with four contracts injected on each side, the dataflow runs to a clean exit, and a composed frame was rendered and inspected with the model drawn on it.
94 lines
3.7 KiB
YAML
94 lines
3.7 KiB
YAML
# Node manifest: the typed contract a consumer wires against.
|
|
#
|
|
# No source package is shared between node repositories, so this file and its
|
|
# type URNs are the only contract a consumer has. `dora validate` checks a
|
|
# dataflow against it without running the node, so CI needs neither OpenCV nor
|
|
# a display.
|
|
apiVersion: 1
|
|
name: dora-image-view
|
|
namespace: liria
|
|
description: Displays an image and a depth map side by side in an OpenCV window, with bounding box, text and camera intrinsics overlays
|
|
categories: [visualization, debug]
|
|
keywords: [opencv, view, bbox, bounding-box, image, depth, intrinsics, camera-info, visualization, object-detection]
|
|
|
|
runtime: python
|
|
entrypoint: dora-image-view
|
|
# `dora build --uv` / `dora run --uv` rewrites this to `uv pip install .` and
|
|
# gives the node its own managed venv under `.dora/python-envs/`, so the
|
|
# dependencies it sees at runtime are the ones installed here.
|
|
build: pip install .
|
|
dora: ">=1.0"
|
|
|
|
inputs:
|
|
image:
|
|
type: std/core/v1/UInt8
|
|
required: false
|
|
description: >-
|
|
Base image to draw on: a flat UInt8 buffer described by the metadata
|
|
{width, height, encoding}. Supported encodings: bgr8, rgb8,
|
|
jpeg/jpg/jpe/bmp/webp/png (encoded bytes), yuv420, avif. Receiving this
|
|
input triggers a redraw and display.
|
|
bbox:
|
|
required: false
|
|
description: >-
|
|
Bounding boxes to overlay: an Arrow struct with fields {bbox, conf,
|
|
labels} and metadata {format}. Supported formats: xyxy, xywh. Stored and
|
|
drawn on the next image redraw. Deliberately untyped — senders differ in
|
|
whether the numeric fields are 32- or 64-bit, and the node casts either.
|
|
text:
|
|
type: std/core/v1/String
|
|
required: false
|
|
description: >-
|
|
Text string to overlay at the top-left corner. Stored and drawn on the
|
|
next image redraw.
|
|
camera_info:
|
|
type: std/core/v1/Float32
|
|
required: false
|
|
description: >-
|
|
The pinhole model the depth map is interpreted through: [fx, fy, cx, cy],
|
|
or a nine-element row-major K as ROS spells it in sensor_msgs/CameraInfo.
|
|
Distortion coefficients are read from the `distortion` metadata. Stored
|
|
and displayed, and held for readers that turn depth into 3D. A sender
|
|
that also puts fx/fy/cx/cy in each frame's metadata does not need to wire
|
|
this input -- they are picked up from there, which is what covers a
|
|
camera that published its intrinsics before this node started.
|
|
depth:
|
|
type: std/core/v1/Float32
|
|
required: false
|
|
description: >-
|
|
Depth map as a flat Float32 buffer, described by the metadata
|
|
{shape: [height, width], units}. Millimetres unless `units` says `m`;
|
|
0 and non-finite values are missing depth and draw black. Colour-mapped
|
|
into its own panel, which receiving this input redraws.
|
|
|
|
outputs: {}
|
|
|
|
env:
|
|
VIEW_DEPTH_MIN_MM:
|
|
type: float
|
|
description: >-
|
|
Pin the near end of the depth colour range, in millimetres. Each end is
|
|
auto-ranged over the valid pixels of the current frame when it is unset.
|
|
VIEW_DEPTH_MAX_MM:
|
|
type: float
|
|
description: Pin the far end of the depth colour range, in millimetres.
|
|
VIEW_WIDTH:
|
|
type: int
|
|
description: Resize the displayed frame to this width (pixels). Defaults to the image input width.
|
|
VIEW_HEIGHT:
|
|
type: int
|
|
description: Resize the displayed frame to this height (pixels). Defaults to the image input height.
|
|
|
|
example: |
|
|
- id: view
|
|
git: ssh://git@gitea.liria.cl:5001/liria/dora-image-view.git
|
|
tag: v0.1.0
|
|
build: pip install .
|
|
path: dora-image-view
|
|
inputs:
|
|
image: camera/image_bgr
|
|
depth: camera/depth
|
|
camera_info: camera/camera_info
|
|
bbox: detector/bbox
|
|
text: detector/text
|