add camera_info input for the pinhole model

Nothing in a dataflow carries fx, fy, cx and cy except the camera, and every
pixel of a depth map needs them to become a 3D point. The node now accepts
them so whatever computes that next has them to hand.

They are read from two places because dora-zed publishes them twice: the
camera_info message, and the metadata of every frame. The per-frame copy is
not redundant -- camera_info goes out once per camera session, so a viewer
started after the camera would otherwise never see it, and reading frame
metadata means wiring the input is optional even for a camera that publishes
it.

A nine-element row-major K is accepted alongside the compact [fx, fy, cx, cy],
since that is how ROS spells the same model in sensor_msgs/CameraInfo.

The model is drawn under the caption rather than only stored. A depth map read
through the wrong intrinsics looks entirely plausible, and this is where that
becomes visible.

as_float32 moves to a payload module: camera_info arrives as a raw byte slice
for the same reason the depth map does, and the reinterpretation is now shared
rather than written twice.

Verified with dora-cli 1.0.1: validate passes with four contracts injected on
each side, the dataflow runs to a clean exit, and a composed frame was
rendered and inspected with the model drawn on it.
This commit is contained in:
2026-09-15 05:09:07 -03:00
parent 0efeacd538
commit 5a0f13d4db
12 changed files with 268 additions and 33 deletions
+14 -2
View File
@@ -7,9 +7,9 @@
apiVersion: 1
name: dora-image-view
namespace: liria
description: Displays an image and a depth map side by side in an OpenCV window, with bounding box and text overlays
description: Displays an image and a depth map side by side in an OpenCV window, with bounding box, text and camera intrinsics overlays
categories: [visualization, debug]
keywords: [opencv, view, bbox, bounding-box, image, depth, visualization, object-detection]
keywords: [opencv, view, bbox, bounding-box, image, depth, intrinsics, camera-info, visualization, object-detection]
runtime: python
entrypoint: dora-image-view
@@ -41,6 +41,17 @@ inputs:
description: >-
Text string to overlay at the top-left corner. Stored and drawn on the
next image redraw.
camera_info:
type: std/core/v1/Float32
required: false
description: >-
The pinhole model the depth map is interpreted through: [fx, fy, cx, cy],
or a nine-element row-major K as ROS spells it in sensor_msgs/CameraInfo.
Distortion coefficients are read from the `distortion` metadata. Stored
and displayed, and held for readers that turn depth into 3D. A sender
that also puts fx/fy/cx/cy in each frame's metadata does not need to wire
this input -- they are picked up from there, which is what covers a
camera that published its intrinsics before this node started.
depth:
type: std/core/v1/Float32
required: false
@@ -77,5 +88,6 @@ example: |
inputs:
image: camera/image_bgr
depth: camera/depth
camera_info: camera/camera_info
bbox: detector/bbox
text: detector/text