add camera_info input for the pinhole model
Nothing in a dataflow carries fx, fy, cx and cy except the camera, and every pixel of a depth map needs them to become a 3D point. The node now accepts them so whatever computes that next has them to hand. They are read from two places because dora-zed publishes them twice: the camera_info message, and the metadata of every frame. The per-frame copy is not redundant -- camera_info goes out once per camera session, so a viewer started after the camera would otherwise never see it, and reading frame metadata means wiring the input is optional even for a camera that publishes it. A nine-element row-major K is accepted alongside the compact [fx, fy, cx, cy], since that is how ROS spells the same model in sensor_msgs/CameraInfo. The model is drawn under the caption rather than only stored. A depth map read through the wrong intrinsics looks entirely plausible, and this is where that becomes visible. as_float32 moves to a payload module: camera_info arrives as a raw byte slice for the same reason the depth map does, and the reinterpretation is now shared rather than written twice. Verified with dora-cli 1.0.1: validate passes with four contracts injected on each side, the dataflow runs to a clean exit, and a composed frame was rendered and inspected with the model drawn on it.
This commit is contained in:
+11
-3
@@ -1,7 +1,8 @@
|
||||
"""A dora display sink: overlays bounding boxes and text on an image input.
|
||||
|
||||
`image` and `depth` each fill one panel of the window and redraw. `bbox` and
|
||||
`text` events only update stored state, which the next redraw draws.
|
||||
`image` and `depth` each fill one panel of the window and redraw. `bbox`,
|
||||
`text` and `camera_info` events only update stored state, which the next redraw
|
||||
draws.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
@@ -10,7 +11,7 @@ import os
|
||||
import numpy as np
|
||||
from dora import Node
|
||||
|
||||
from dora_image_view import bbox, decode, depth
|
||||
from dora_image_view import bbox, decode, depth, intrinsics
|
||||
from dora_image_view.view import View
|
||||
|
||||
|
||||
@@ -37,6 +38,11 @@ def main() -> None:
|
||||
event_id = event["id"]
|
||||
metadata = event["metadata"]
|
||||
|
||||
# A sender that puts intrinsics on every frame keeps them current even
|
||||
# if the camera_info message itself was published before this node
|
||||
# started, which is when dora-zed sends it.
|
||||
view.intrinsics = intrinsics.from_metadata(metadata) or view.intrinsics
|
||||
|
||||
panel = _decode_panel(event_id, event["value"], metadata)
|
||||
|
||||
if panel is not None:
|
||||
@@ -44,6 +50,8 @@ def main() -> None:
|
||||
view.panels[name] = frame
|
||||
if view.show():
|
||||
break
|
||||
elif event_id == "camera_info":
|
||||
view.intrinsics = intrinsics.parse(event["value"], metadata)
|
||||
elif event_id == "bbox":
|
||||
view.bboxes = bbox.parse(event["value"], metadata.get("format", "xyxy").lower())
|
||||
elif event_id == "text":
|
||||
|
||||
Reference in New Issue
Block a user