add camera_info input for the pinhole model

Nothing in a dataflow carries fx, fy, cx and cy except the camera, and every
pixel of a depth map needs them to become a 3D point. The node now accepts
them so whatever computes that next has them to hand.

They are read from two places because dora-zed publishes them twice: the
camera_info message, and the metadata of every frame. The per-frame copy is
not redundant -- camera_info goes out once per camera session, so a viewer
started after the camera would otherwise never see it, and reading frame
metadata means wiring the input is optional even for a camera that publishes
it.

A nine-element row-major K is accepted alongside the compact [fx, fy, cx, cy],
since that is how ROS spells the same model in sensor_msgs/CameraInfo.

The model is drawn under the caption rather than only stored. A depth map read
through the wrong intrinsics looks entirely plausible, and this is where that
becomes visible.

as_float32 moves to a payload module: camera_info arrives as a raw byte slice
for the same reason the depth map does, and the reinterpretation is now shared
rather than written twice.

Verified with dora-cli 1.0.1: validate passes with four contracts injected on
each side, the dataflow runs to a clean exit, and a composed frame was
rendered and inspected with the model drawn on it.
This commit is contained in:
2026-09-15 05:09:07 -03:00
parent 0efeacd538
commit 5a0f13d4db
12 changed files with 268 additions and 33 deletions
+6 -1
View File
@@ -12,7 +12,7 @@
apiVersion: 1
name: source
namespace: liria
description: Synthetic image, depth, bbox and text source for the dora-image-view example
description: Synthetic image, depth, camera_info, bbox and text source for the dora-image-view example
categories: [debug]
keywords: [example, test, synthetic]
@@ -39,6 +39,11 @@ outputs:
text:
type: std/core/v1/String
description: A fixed caption.
camera_info:
type: std/core/v1/Float32
description: >-
[fx, fy, cx, cy] for the frames above. Published once, on the first tick,
the way a real camera publishes it once per session.
depth:
type: std/core/v1/Float32
description: >-
+27 -4
View File
@@ -5,7 +5,9 @@ moving in it, and a caption -- every input dora-image-view draws -- so the examp
dataflow needs no camera, no detector and no network.
The depth map describes the same scene as the image: a flat back wall with the square
standing in front of it.
standing in front of it. `camera_info` is published once, on the first tick, the way a
real camera publishes it once per session -- which is exactly the case the viewer has
to cope with by reading intrinsics from frame metadata as well.
"""
import os
@@ -27,9 +29,17 @@ CAPTION = "dora-image-view synthetic source"
WALL_MM = 2000.0
SQUARE_MM = 800.0
# A plausible pinhole for a frame this size.
FX = FY = float(WIDTH)
CX, CY = WIDTH / 2, HEIGHT / 2
DISTORTION = [0.0, 0.0, 0.0, 0.0, 0.0]
# Attached to every frame as well as published once, which is what dora-zed does
# and what lets a viewer that started late still learn the camera model.
INTRINSICS_METADATA = {"fx": FX, "fy": FY, "cx": CX, "cy": CY, "distortion": DISTORTION}
def main() -> None:
"""Send one image, depth map, bbox and text per tick."""
"""Send one image, depth map, bbox and text per tick, and camera_info once."""
node = Node("source")
sent = 0
@@ -37,11 +47,18 @@ def main() -> None:
if event["type"] != "INPUT":
continue
if sent == 0:
node.send_output(
"camera_info",
pa.array(np.array([FX, FY, CX, CY], dtype=np.float32)),
{"shape": [4], "dtype": "float32", "layout": "C", "distortion": DISTORTION},
)
frame, corners = render(sent)
node.send_output(
"image",
pa.array(frame.ravel()),
{"width": WIDTH, "height": HEIGHT, "encoding": "bgr8"},
{"width": WIDTH, "height": HEIGHT, "encoding": "bgr8", **INTRINSICS_METADATA},
)
node.send_output(
"bbox",
@@ -56,7 +73,13 @@ def main() -> None:
node.send_output(
"depth",
pa.array(depth.ravel()),
{"shape": [HEIGHT, WIDTH], "dtype": "float32", "layout": "HW", "units": "mm"},
{
"shape": [HEIGHT, WIDTH],
"dtype": "float32",
"layout": "HW",
"units": "mm",
**INTRINSICS_METADATA,
},
)
node.send_output("text", pa.array([CAPTION]))