add camera_info input for the pinhole model
Nothing in a dataflow carries fx, fy, cx and cy except the camera, and every pixel of a depth map needs them to become a 3D point. The node now accepts them so whatever computes that next has them to hand. They are read from two places because dora-zed publishes them twice: the camera_info message, and the metadata of every frame. The per-frame copy is not redundant -- camera_info goes out once per camera session, so a viewer started after the camera would otherwise never see it, and reading frame metadata means wiring the input is optional even for a camera that publishes it. A nine-element row-major K is accepted alongside the compact [fx, fy, cx, cy], since that is how ROS spells the same model in sensor_msgs/CameraInfo. The model is drawn under the caption rather than only stored. A depth map read through the wrong intrinsics looks entirely plausible, and this is where that becomes visible. as_float32 moves to a payload module: camera_info arrives as a raw byte slice for the same reason the depth map does, and the reinterpretation is now shared rather than written twice. Verified with dora-cli 1.0.1: validate passes with four contracts injected on each side, the dataflow runs to a clean exit, and a composed frame was rendered and inspected with the model drawn on it.
This commit is contained in:
@@ -5,6 +5,8 @@ from dataclasses import dataclass, field
|
||||
import cv2
|
||||
import numpy as np
|
||||
|
||||
from dora_image_view.intrinsics import Intrinsics
|
||||
|
||||
WINDOW = "Dora Node: dora-image-view"
|
||||
GREEN = (0, 255, 0)
|
||||
WHITE = (255, 255, 255)
|
||||
@@ -40,6 +42,7 @@ class View:
|
||||
bboxes: Bboxes = field(default_factory=Bboxes)
|
||||
text: str = ""
|
||||
panels: dict[str, np.ndarray] = field(default_factory=dict)
|
||||
intrinsics: Intrinsics | None = None
|
||||
|
||||
def show(self) -> bool:
|
||||
"""Draw the overlays, tile the panels, and display them.
|
||||
@@ -79,6 +82,13 @@ class View:
|
||||
# On the composite, not on a panel: the caption describes the frame as a
|
||||
# whole, and a panel it was drawn on might not be the leftmost one.
|
||||
cv2.putText(frame, self.text, (20, 20), FONT, FONT_SCALE, WHITE, 1)
|
||||
|
||||
# Showing the camera model is the point of receiving it here: a depth
|
||||
# map read through the wrong intrinsics looks entirely plausible, and
|
||||
# this is where that becomes visible.
|
||||
if self.intrinsics is not None:
|
||||
cv2.putText(frame, str(self.intrinsics), (20, 40), FONT, FONT_SCALE, WHITE, 1)
|
||||
|
||||
return frame
|
||||
|
||||
def _label(self, panel: np.ndarray, name: str) -> np.ndarray:
|
||||
|
||||
Reference in New Issue
Block a user