add camera_info input for the pinhole model
Nothing in a dataflow carries fx, fy, cx and cy except the camera, and every pixel of a depth map needs them to become a 3D point. The node now accepts them so whatever computes that next has them to hand. They are read from two places because dora-zed publishes them twice: the camera_info message, and the metadata of every frame. The per-frame copy is not redundant -- camera_info goes out once per camera session, so a viewer started after the camera would otherwise never see it, and reading frame metadata means wiring the input is optional even for a camera that publishes it. A nine-element row-major K is accepted alongside the compact [fx, fy, cx, cy], since that is how ROS spells the same model in sensor_msgs/CameraInfo. The model is drawn under the caption rather than only stored. A depth map read through the wrong intrinsics looks entirely plausible, and this is where that becomes visible. as_float32 moves to a payload module: camera_info arrives as a raw byte slice for the same reason the depth map does, and the reinterpretation is now shared rather than written twice. Verified with dora-cli 1.0.1: validate passes with four contracts injected on each side, the dataflow runs to a clean exit, and a composed frame was rendered and inspected with the model drawn on it.
This commit is contained in:
@@ -12,7 +12,7 @@
|
||||
apiVersion: 1
|
||||
name: source
|
||||
namespace: liria
|
||||
description: Synthetic image, depth, bbox and text source for the dora-image-view example
|
||||
description: Synthetic image, depth, camera_info, bbox and text source for the dora-image-view example
|
||||
categories: [debug]
|
||||
keywords: [example, test, synthetic]
|
||||
|
||||
@@ -39,6 +39,11 @@ outputs:
|
||||
text:
|
||||
type: std/core/v1/String
|
||||
description: A fixed caption.
|
||||
camera_info:
|
||||
type: std/core/v1/Float32
|
||||
description: >-
|
||||
[fx, fy, cx, cy] for the frames above. Published once, on the first tick,
|
||||
the way a real camera publishes it once per session.
|
||||
depth:
|
||||
type: std/core/v1/Float32
|
||||
description: >-
|
||||
|
||||
+27
-4
@@ -5,7 +5,9 @@ moving in it, and a caption -- every input dora-image-view draws -- so the examp
|
||||
dataflow needs no camera, no detector and no network.
|
||||
|
||||
The depth map describes the same scene as the image: a flat back wall with the square
|
||||
standing in front of it.
|
||||
standing in front of it. `camera_info` is published once, on the first tick, the way a
|
||||
real camera publishes it once per session -- which is exactly the case the viewer has
|
||||
to cope with by reading intrinsics from frame metadata as well.
|
||||
"""
|
||||
|
||||
import os
|
||||
@@ -27,9 +29,17 @@ CAPTION = "dora-image-view synthetic source"
|
||||
WALL_MM = 2000.0
|
||||
SQUARE_MM = 800.0
|
||||
|
||||
# A plausible pinhole for a frame this size.
|
||||
FX = FY = float(WIDTH)
|
||||
CX, CY = WIDTH / 2, HEIGHT / 2
|
||||
DISTORTION = [0.0, 0.0, 0.0, 0.0, 0.0]
|
||||
# Attached to every frame as well as published once, which is what dora-zed does
|
||||
# and what lets a viewer that started late still learn the camera model.
|
||||
INTRINSICS_METADATA = {"fx": FX, "fy": FY, "cx": CX, "cy": CY, "distortion": DISTORTION}
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""Send one image, depth map, bbox and text per tick."""
|
||||
"""Send one image, depth map, bbox and text per tick, and camera_info once."""
|
||||
node = Node("source")
|
||||
sent = 0
|
||||
|
||||
@@ -37,11 +47,18 @@ def main() -> None:
|
||||
if event["type"] != "INPUT":
|
||||
continue
|
||||
|
||||
if sent == 0:
|
||||
node.send_output(
|
||||
"camera_info",
|
||||
pa.array(np.array([FX, FY, CX, CY], dtype=np.float32)),
|
||||
{"shape": [4], "dtype": "float32", "layout": "C", "distortion": DISTORTION},
|
||||
)
|
||||
|
||||
frame, corners = render(sent)
|
||||
node.send_output(
|
||||
"image",
|
||||
pa.array(frame.ravel()),
|
||||
{"width": WIDTH, "height": HEIGHT, "encoding": "bgr8"},
|
||||
{"width": WIDTH, "height": HEIGHT, "encoding": "bgr8", **INTRINSICS_METADATA},
|
||||
)
|
||||
node.send_output(
|
||||
"bbox",
|
||||
@@ -56,7 +73,13 @@ def main() -> None:
|
||||
node.send_output(
|
||||
"depth",
|
||||
pa.array(depth.ravel()),
|
||||
{"shape": [HEIGHT, WIDTH], "dtype": "float32", "layout": "HW", "units": "mm"},
|
||||
{
|
||||
"shape": [HEIGHT, WIDTH],
|
||||
"dtype": "float32",
|
||||
"layout": "HW",
|
||||
"units": "mm",
|
||||
**INTRINSICS_METADATA,
|
||||
},
|
||||
)
|
||||
node.send_output("text", pa.array([CAPTION]))
|
||||
|
||||
|
||||
Reference in New Issue
Block a user