diff --git a/AGENTS.md b/AGENTS.md index 17ac4f7..04319fb 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -4,7 +4,7 @@ This file provides guidance to AI coding assistants when working with code in th ## What this is -A dora-rs Python node that displays a video stream with overlays: it keeps one panel per frame stream (`image`, `depth`) plus a bbox set and text string, and every frame event redraws the lot as one side-by-side composite in an OpenCV window. It is a display sink — it produces no outputs. [README.md](README.md) documents the inputs, the encodings and the environment variables; this file covers only what the code and the README do not say. +A dora-rs Python node that displays a video stream with overlays: it keeps one panel per frame stream (`image`, `depth`) plus a bbox set, a text string and the camera model, and every frame event redraws the lot as one side-by-side composite in an OpenCV window. It is a display sink — it produces no outputs. [README.md](README.md) documents the inputs, the encodings and the environment variables; this file covers only what the code and the README do not say. Everything the example dataflow needs is in this checkout. Do not reintroduce a dependency on another repository — the example must keep running with no camera, no detector and no network. @@ -26,7 +26,9 @@ CI=true SOURCE_FRAMES=30 dora run dataflow.yml --uv # the same, headless - **`dora_image_view/main.py`** — Argument parsing and the dora event loop, nothing else. The env var wins over the CLI flag for `VIEW_WIDTH`/`VIEW_HEIGHT`. The upstream node spells these `PLOT_WIDTH`/`PLOT_HEIGHT`; they were renamed with the node, so a dataflow copied from upstream needs its env keys changed. `Node(args.name)` rather than `Node()` so the node also runs standalone as a dynamic node. - **`dora_image_view/decode.py`** — `to_bgr()`: every supported encoding to a writable BGR array. Two behaviours are deliberate, not bugs: `bgr8` **copies** (the payload may be a read-only view on shared memory, and the frame is annotated in place), and `yuv420` converts with `COLOR_YUV420p2RGB` rather than `...2BGR` — the upstream node converts to RGB and then displays it as BGR, and that channel swap is part of the behaviour callers already see. - **`dora_image_view/view.py`** — `View` (the panels, the overlay state and the window) and `Bboxes`. `headless` (from `CI=true`) skips only the window: every input is still decoded and composed, which is what makes the node safe in a pipeline test. Panels are tiled in the fixed order in `PANELS`, not in arrival order, so a panel does not jump when a stream drops a frame. Boxes are drawn on the `image` panel before scaling, because they are in image pixels; the caption is drawn on the composite, because the panel it would otherwise land on may not be the leftmost one. -- **`dora_image_view/depth.py`** — `to_millimetres()` and `colorize()`. Decoding normalises to float32 millimetres with 0 for missing, so the colouring only ever sees one representation. Two things are deliberate: `as_float32()` **reinterprets** a `uint8` payload rather than converting it, because a C++ node sending a raw byte slice produces a `UInt8` arrow array despite declaring `Float32`; and the auto range is the valid pixels' full min/max, **not** a percentile. Trimming the tails looks safer and breaks the exact scene this node exists for — a small object in front of a large wall is a few percent of the pixels, so both tails land on the wall and the object clips away. A rendered check caught that; do not reintroduce percentile ranging. +- **`dora_image_view/depth.py`** — `to_millimetres()` and `colorize()`. Decoding normalises to float32 millimetres with 0 for missing, so the colouring only ever sees one representation. One thing is deliberate: the auto range is the valid pixels' full min/max, **not** a percentile. Trimming the tails looks safer and breaks the exact scene this node exists for — a small object in front of a large wall is a few percent of the pixels, so both tails land on the wall and the object clips away. A rendered check caught that; do not reintroduce percentile ranging. +- **`dora_image_view/payload.py`** — `as_float32()`: reinterprets a `uint8` payload rather than converting it, because a C++ node sending a raw byte slice produces a `UInt8` arrow array despite declaring `Float32`. Shared by `depth` and `intrinsics`, which both meet it. +- **`dora_image_view/intrinsics.py`** — `Intrinsics`, `parse()` for the `camera_info` payload and `from_metadata()` for the copy senders attach to each frame. Both exist because dora-zed publishes the same numbers twice, and the per-frame copy is the one that survives a viewer starting after the camera did — `camera_info` goes out **once per session**. A nine-element row-major K is accepted alongside the compact four values, since that is how ROS spells the same model. The node needs none of this to draw; it is held for whatever turns depth into 3D, and drawn so a wrong camera model is visible rather than merely plausible. - **`dora_image_view/bbox.py`** — Parses the `bbox` struct into xyxy corners. `xywh` is centre-based (x, y are the box centre). `_flat()` unwraps the single-element list that `pa.array([{...}])` produces around each field, and reads a field stored flat as-is. - **`examples/source.py`** — The synthetic source node: an animated frame, a depth map of the same scene (a flat wall with the square in front of it), the box around the square, and a caption. It exists so the example dataflow exercises all three inputs without a camera or a detector, and so CI can run the dataflow rather than only validate it. `SOURCE_FRAMES` bounds the run, which is what lets CI wait for a clean exit instead of killing it on a timeout. - **`dora-node.yml`** — Node manifest (`apiVersion: 1`). `dora validate` injects these types into any dataflow that reaches the node by `path:`; a `git:`-sourced node does **not** pick it up, which is why the example builds from the checkout. `image` and `text` carry type URNs naming the element type because each payload is a raw buffer described by its metadata, not an Arrow struct. `bbox` is deliberately **untyped**: senders disagree on whether its numeric fields are 32- or 64-bit, and the node reads either. @@ -42,7 +44,7 @@ CI=true SOURCE_FRAMES=30 dora run dataflow.yml --uv # the same, headless ## A caveat on typed optional inputs -`dora validate --strict` injects a contract type for *every* typed manifest input and then warns `input_types key "..." not found in inputs list` for any the dataflow does not wire. So a typed optional input forces the example dataflow to wire it. If a typed input is added to the manifest, either wire it in `dataflow.yml` or leave it untyped. This is why `examples/source.py` publishes a depth map at all: `depth` is typed, so the example has to drive it. +`dora validate --strict` injects a contract type for *every* typed manifest input and then warns `input_types key "..." not found in inputs list` for any the dataflow does not wire. So a typed optional input forces the example dataflow to wire it. If a typed input is added to the manifest, either wire it in `dataflow.yml` or leave it untyped. This is why `examples/source.py` publishes a depth map and a camera_info at all: both are typed, so the example has to drive them. ## Checking what it draws diff --git a/README.md b/README.md index 34f9fee..c54d76a 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ # dora-image-view Dora node that displays a colour image and a depth map side by side in one OpenCV -window, with bounding box and text overlays. A display sink for debugging a +window, with bounding box, text and camera intrinsics overlays. A display sink for debugging a vision pipeline: it produces no outputs. Based on [`opencv-plot`](https://github.com/dora-rs/dora-hub/tree/main/node-hub/opencv-plot) @@ -21,6 +21,7 @@ nodes: inputs: image: camera/image_bgr depth: camera/depth + camera_info: camera/camera_info bbox: detector/bbox text: detector/text ``` @@ -30,7 +31,7 @@ managed environment. Every input is optional, and each of `image` and `depth` fills one panel of the window, tiled left to right in that order. Wiring one of them is a plain viewer; -wiring several shows them side by side, scaled to a common height. `bbox` and `text` +wiring several shows them side by side, scaled to a common height. `bbox`, `text` and `camera_info` are overlays rather than panels — boxes are drawn on the `image` panel, in its own pixels, and the caption on the composite. @@ -75,8 +76,8 @@ $ dora validate dataflow.yml All type annotations OK. ``` -`image` is `std/core/v1/UInt8`, `depth` is `std/core/v1/Float32` and `text` is -`std/core/v1/String`; the payloads are raw buffers described by their +`image` is `std/core/v1/UInt8`, `depth` and `camera_info` are `std/core/v1/Float32` and +`text` is `std/core/v1/String`; the payloads are raw buffers described by their metadata, so the URN names the element type rather than a struct. `bbox` is deliberately left untyped — senders disagree on whether its numeric fields are 32- or 64-bit, and the node reads either. @@ -91,6 +92,7 @@ of that state: resizes to `VIEW_WIDTH`x`VIEW_HEIGHT` if both are set, and shows it in a window titled `Dora Node: dora-image-view`. Press `q` in the window to quit. - A **`depth`** event colour-maps the distances onto the depth panel and redraws. +- A **`camera_info`** event stores the pinhole model and draws it under the caption. - A **`bbox`** or **`text`** event only updates stored state; it is drawn on the next redraw. @@ -103,6 +105,7 @@ no window. That is what makes it safe to run in a pipeline test. |---|---|---| | `image` | Flat `UInt8` array | `width`, `height`, `encoding` | | `depth` | Flat `Float32` array, `[H, W]` | `shape`, `units` | +| `camera_info` | Flat `Float32` array, `[fx, fy, cx, cy]` or a 9-element K | `distortion`, `distortion_model` | | `bbox` | Arrow struct array, element `[0]` with fields `bbox`, `conf`, `labels` | `format` | | `text` | Arrow array of size 1 | — | @@ -117,6 +120,23 @@ and any non-finite value mean missing depth and are drawn black. A payload sent raw byte slice — which is what a C++ node does — arrives as `UInt8` despite the `Float32` contract, and is reinterpreted rather than converted. +### Camera intrinsics + +`camera_info` carries the pinhole model the depth map is interpreted through, either as +the four values `[fx, fy, cx, cy]` or as a nine-element row-major `K`, which is how ROS +spells the same model in `sensor_msgs/CameraInfo`. Distortion coefficients are read from +the `distortion` metadata. + +The node displays the model and holds it. It needs none of it to draw a panel; it is +there for what reads it next — turning a depth map into a point cloud needs `fx`, `fy`, +`cx` and `cy` for every pixel, and nothing else in a dataflow carries them. + +Intrinsics are also picked up from the metadata of any `image` or `depth` frame that +carries `fx`, `fy`, `cx` and `cy`, so **wiring `camera_info` is optional even for a +camera that publishes it**. That matters because a camera typically publishes +`camera_info` once per session: a viewer started after the camera would otherwise never +see it. + ```python node.send_output("image", pa.array(frame.ravel()), {"width": 640, "height": 480, "encoding": "bgr8"}) diff --git a/dataflow.yml b/dataflow.yml index 22731c8..c94f3d4 100644 --- a/dataflow.yml +++ b/dataflow.yml @@ -20,6 +20,7 @@ nodes: outputs: - image - depth + - camera_info - bbox - text @@ -29,5 +30,6 @@ nodes: inputs: image: source/image depth: source/depth + camera_info: source/camera_info bbox: source/bbox text: source/text diff --git a/dora-node.yml b/dora-node.yml index f9dc7bf..0c29cad 100644 --- a/dora-node.yml +++ b/dora-node.yml @@ -7,9 +7,9 @@ apiVersion: 1 name: dora-image-view namespace: liria -description: Displays an image and a depth map side by side in an OpenCV window, with bounding box and text overlays +description: Displays an image and a depth map side by side in an OpenCV window, with bounding box, text and camera intrinsics overlays categories: [visualization, debug] -keywords: [opencv, view, bbox, bounding-box, image, depth, visualization, object-detection] +keywords: [opencv, view, bbox, bounding-box, image, depth, intrinsics, camera-info, visualization, object-detection] runtime: python entrypoint: dora-image-view @@ -41,6 +41,17 @@ inputs: description: >- Text string to overlay at the top-left corner. Stored and drawn on the next image redraw. + camera_info: + type: std/core/v1/Float32 + required: false + description: >- + The pinhole model the depth map is interpreted through: [fx, fy, cx, cy], + or a nine-element row-major K as ROS spells it in sensor_msgs/CameraInfo. + Distortion coefficients are read from the `distortion` metadata. Stored + and displayed, and held for readers that turn depth into 3D. A sender + that also puts fx/fy/cx/cy in each frame's metadata does not need to wire + this input -- they are picked up from there, which is what covers a + camera that published its intrinsics before this node started. depth: type: std/core/v1/Float32 required: false @@ -77,5 +88,6 @@ example: | inputs: image: camera/image_bgr depth: camera/depth + camera_info: camera/camera_info bbox: detector/bbox text: detector/text diff --git a/dora_image_view/depth.py b/dora_image_view/depth.py index 86733ce..a75aeed 100644 --- a/dora_image_view/depth.py +++ b/dora_image_view/depth.py @@ -11,6 +11,8 @@ import os import cv2 import numpy as np +from dora_image_view.payload import as_float32 + # Perceptually uniform and readable in both directions, unlike JET. COLORMAP = cv2.COLORMAP_TURBO @@ -85,21 +87,6 @@ def resolve_range(depth_mm: np.ndarray) -> tuple[float, float]: return float(low), float(high) -def as_float32(storage) -> np.ndarray: - """Read the payload as float32, whether it arrived as floats or as bytes. - - A node that sends the map as a raw byte slice -- dora-zed does -- produces a - UInt8 arrow array even though its manifest declares Float32, so the buffer - has to be reinterpreted rather than converted. - """ - buffer = storage.to_numpy() - - if buffer.dtype == np.uint8: - return buffer.view(np.float32) - - return buffer.astype(np.float32, copy=False) - - def _hw(metadata: dict, size: int) -> tuple[int, int]: shape = metadata.get("shape") diff --git a/dora_image_view/intrinsics.py b/dora_image_view/intrinsics.py new file mode 100644 index 0000000..81868fb --- /dev/null +++ b/dora_image_view/intrinsics.py @@ -0,0 +1,89 @@ +"""The pinhole camera model a depth map is interpreted through. + +The node does not need intrinsics to draw anything -- it displays them, and +holds them for whatever reads them next. Back-projecting a depth map into a +point cloud is the obvious such reader: every pixel needs fx, fy, cx and cy to +become an XYZ point, and nothing else in a dataflow carries them. + +Two senders are handled because dora-zed publishes the same numbers twice: once +per camera session on `camera_info`, and again in the metadata of every frame. +The per-frame copy is what makes a viewer that started after the camera did +still end up with intrinsics. +""" + +from dataclasses import dataclass + +from dora_image_view.payload import as_float32 + +# [fx, fy, cx, cy], the four numbers dora-zed publishes. Also accepted is a +# nine-element row-major K, which is how ROS spells the same model in +# sensor_msgs/CameraInfo. +COMPACT = 4 +K_MATRIX = 9 + +# Indices of fx, fy, cx, cy in a row-major K: +# [fx, 0, cx, +# 0, fy, cy, +# 0, 0, 1] +K_INDICES = (0, 4, 2, 5) + + +@dataclass(frozen=True) +class Intrinsics: + """A pinhole model: focal lengths and principal point, both in pixels.""" + + fx: float + fy: float + cx: float + cy: float + distortion: tuple[float, ...] = () + model: str = "" + + def __str__(self) -> str: + """Render for the overlay, short enough to fit across a panel.""" + return f"fx {self.fx:.1f} fy {self.fy:.1f} cx {self.cx:.1f} cy {self.cy:.1f}" + + +def parse(storage, metadata: dict) -> Intrinsics: + """Read one `camera_info` payload. + + Accepts the four-element [fx, fy, cx, cy] dora-zed sends and the + nine-element row-major K that ROS does. + """ + values = as_float32(storage) + + if values.size >= K_MATRIX: + fx, fy, cx, cy = (float(values[index]) for index in K_INDICES) + elif values.size >= COMPACT: + fx, fy, cx, cy = (float(value) for value in values[:COMPACT]) + else: + raise RuntimeError( + f"camera_info holds {values.size} values, " + f"expected {COMPACT} as [fx, fy, cx, cy] or {K_MATRIX} as K", + ) + + return Intrinsics(fx, fy, cx, cy, _distortion(metadata), _model(metadata)) + + +def from_metadata(metadata: dict) -> Intrinsics | None: + """Read intrinsics a sender attached to a frame, or None if it attached none.""" + try: + fx, fy, cx, cy = ( + float(metadata["fx"]), + float(metadata["fy"]), + float(metadata["cx"]), + float(metadata["cy"]), + ) + except (KeyError, TypeError, ValueError): + return None + + return Intrinsics(fx, fy, cx, cy, _distortion(metadata), _model(metadata)) + + +def _distortion(metadata: dict) -> tuple[float, ...]: + values = metadata.get("distortion") or () + return tuple(float(value) for value in values) + + +def _model(metadata: dict) -> str: + return str(metadata.get("distortion_model", "")) diff --git a/dora_image_view/main.py b/dora_image_view/main.py index fb5f0c9..3b052ed 100644 --- a/dora_image_view/main.py +++ b/dora_image_view/main.py @@ -1,7 +1,8 @@ """A dora display sink: overlays bounding boxes and text on an image input. -`image` and `depth` each fill one panel of the window and redraw. `bbox` and -`text` events only update stored state, which the next redraw draws. +`image` and `depth` each fill one panel of the window and redraw. `bbox`, +`text` and `camera_info` events only update stored state, which the next redraw +draws. """ import argparse @@ -10,7 +11,7 @@ import os import numpy as np from dora import Node -from dora_image_view import bbox, decode, depth +from dora_image_view import bbox, decode, depth, intrinsics from dora_image_view.view import View @@ -37,6 +38,11 @@ def main() -> None: event_id = event["id"] metadata = event["metadata"] + # A sender that puts intrinsics on every frame keeps them current even + # if the camera_info message itself was published before this node + # started, which is when dora-zed sends it. + view.intrinsics = intrinsics.from_metadata(metadata) or view.intrinsics + panel = _decode_panel(event_id, event["value"], metadata) if panel is not None: @@ -44,6 +50,8 @@ def main() -> None: view.panels[name] = frame if view.show(): break + elif event_id == "camera_info": + view.intrinsics = intrinsics.parse(event["value"], metadata) elif event_id == "bbox": view.bboxes = bbox.parse(event["value"], metadata.get("format", "xyxy").lower()) elif event_id == "text": diff --git a/dora_image_view/payload.py b/dora_image_view/payload.py new file mode 100644 index 0000000..2c44b3e --- /dev/null +++ b/dora_image_view/payload.py @@ -0,0 +1,18 @@ +"""Reading raw dora payloads, independently of what they mean.""" + +import numpy as np + + +def as_float32(storage) -> np.ndarray: + """Read a payload as float32, whether it arrived as floats or as bytes. + + A node that sends its payload as a raw byte slice -- which is what a C++ + node does -- produces a UInt8 arrow array even though its manifest declares + Float32, so the buffer has to be reinterpreted rather than converted. + """ + buffer = storage.to_numpy() + + if buffer.dtype == np.uint8: + return buffer.view(np.float32) + + return buffer.astype(np.float32, copy=False) diff --git a/dora_image_view/view.py b/dora_image_view/view.py index bfed187..3b5fb13 100644 --- a/dora_image_view/view.py +++ b/dora_image_view/view.py @@ -5,6 +5,8 @@ from dataclasses import dataclass, field import cv2 import numpy as np +from dora_image_view.intrinsics import Intrinsics + WINDOW = "Dora Node: dora-image-view" GREEN = (0, 255, 0) WHITE = (255, 255, 255) @@ -40,6 +42,7 @@ class View: bboxes: Bboxes = field(default_factory=Bboxes) text: str = "" panels: dict[str, np.ndarray] = field(default_factory=dict) + intrinsics: Intrinsics | None = None def show(self) -> bool: """Draw the overlays, tile the panels, and display them. @@ -79,6 +82,13 @@ class View: # On the composite, not on a panel: the caption describes the frame as a # whole, and a panel it was drawn on might not be the leftmost one. cv2.putText(frame, self.text, (20, 20), FONT, FONT_SCALE, WHITE, 1) + + # Showing the camera model is the point of receiving it here: a depth + # map read through the wrong intrinsics looks entirely plausible, and + # this is where that becomes visible. + if self.intrinsics is not None: + cv2.putText(frame, str(self.intrinsics), (20, 40), FONT, FONT_SCALE, WHITE, 1) + return frame def _label(self, panel: np.ndarray, name: str) -> np.ndarray: diff --git a/examples/dora-node.yml b/examples/dora-node.yml index 93439cf..d1ad921 100644 --- a/examples/dora-node.yml +++ b/examples/dora-node.yml @@ -12,7 +12,7 @@ apiVersion: 1 name: source namespace: liria -description: Synthetic image, depth, bbox and text source for the dora-image-view example +description: Synthetic image, depth, camera_info, bbox and text source for the dora-image-view example categories: [debug] keywords: [example, test, synthetic] @@ -39,6 +39,11 @@ outputs: text: type: std/core/v1/String description: A fixed caption. + camera_info: + type: std/core/v1/Float32 + description: >- + [fx, fy, cx, cy] for the frames above. Published once, on the first tick, + the way a real camera publishes it once per session. depth: type: std/core/v1/Float32 description: >- diff --git a/examples/source.py b/examples/source.py index 704b741..75935f7 100644 --- a/examples/source.py +++ b/examples/source.py @@ -5,7 +5,9 @@ moving in it, and a caption -- every input dora-image-view draws -- so the examp dataflow needs no camera, no detector and no network. The depth map describes the same scene as the image: a flat back wall with the square -standing in front of it. +standing in front of it. `camera_info` is published once, on the first tick, the way a +real camera publishes it once per session -- which is exactly the case the viewer has +to cope with by reading intrinsics from frame metadata as well. """ import os @@ -27,9 +29,17 @@ CAPTION = "dora-image-view synthetic source" WALL_MM = 2000.0 SQUARE_MM = 800.0 +# A plausible pinhole for a frame this size. +FX = FY = float(WIDTH) +CX, CY = WIDTH / 2, HEIGHT / 2 +DISTORTION = [0.0, 0.0, 0.0, 0.0, 0.0] +# Attached to every frame as well as published once, which is what dora-zed does +# and what lets a viewer that started late still learn the camera model. +INTRINSICS_METADATA = {"fx": FX, "fy": FY, "cx": CX, "cy": CY, "distortion": DISTORTION} + def main() -> None: - """Send one image, depth map, bbox and text per tick.""" + """Send one image, depth map, bbox and text per tick, and camera_info once.""" node = Node("source") sent = 0 @@ -37,11 +47,18 @@ def main() -> None: if event["type"] != "INPUT": continue + if sent == 0: + node.send_output( + "camera_info", + pa.array(np.array([FX, FY, CX, CY], dtype=np.float32)), + {"shape": [4], "dtype": "float32", "layout": "C", "distortion": DISTORTION}, + ) + frame, corners = render(sent) node.send_output( "image", pa.array(frame.ravel()), - {"width": WIDTH, "height": HEIGHT, "encoding": "bgr8"}, + {"width": WIDTH, "height": HEIGHT, "encoding": "bgr8", **INTRINSICS_METADATA}, ) node.send_output( "bbox", @@ -56,7 +73,13 @@ def main() -> None: node.send_output( "depth", pa.array(depth.ravel()), - {"shape": [HEIGHT, WIDTH], "dtype": "float32", "layout": "HW", "units": "mm"}, + { + "shape": [HEIGHT, WIDTH], + "dtype": "float32", + "layout": "HW", + "units": "mm", + **INTRINSICS_METADATA, + }, ) node.send_output("text", pa.array([CAPTION])) diff --git a/tests/test_dora_image_view.py b/tests/test_dora_image_view.py index 1fe81be..e12e9c9 100644 --- a/tests/test_dora_image_view.py +++ b/tests/test_dora_image_view.py @@ -7,7 +7,7 @@ import numpy as np import pytest from PIL import Image -from dora_image_view import bbox, decode, depth +from dora_image_view import bbox, decode, depth, intrinsics from dora_image_view.view import Bboxes, View WIDTH, HEIGHT = 12, 8 @@ -224,3 +224,62 @@ def test_depth_range_of_an_empty_frame_does_not_divide_by_zero(): low, high = depth.resolve_range(np.zeros((2, 2), dtype=np.float32)) assert high > low + + +def test_camera_info_reads_the_compact_four_value_form(): + values = np.array([600.0, 601.0, 320.0, 240.0], dtype=np.float32) + + model = intrinsics.parse(FakeStorage(values), {"distortion": [0.1, 0.2]}) + + assert (model.fx, model.fy, model.cx, model.cy) == (600.0, 601.0, 320.0, 240.0) + assert model.distortion == (0.1, 0.2) + + +def test_camera_info_reads_a_ros_k_matrix(): + # Row-major K: [fx, 0, cx, 0, fy, cy, 0, 0, 1]. + k = np.array( + [600.0, 0.0, 320.0, 0.0, 601.0, 240.0, 0.0, 0.0, 1.0], + dtype=np.float32, + ) + + model = intrinsics.parse(FakeStorage(k), {}) + + assert (model.fx, model.fy, model.cx, model.cy) == (600.0, 601.0, 320.0, 240.0) + + +def test_camera_info_reinterprets_a_payload_sent_as_bytes(): + values = np.array([600.0, 601.0, 320.0, 240.0], dtype=np.float32) + + model = intrinsics.parse(FakeStorage(values.view(np.uint8)), {}) + + assert model.fx == 600.0 + assert model.cy == 240.0 + + +def test_camera_info_that_is_too_short_is_an_error(): + with pytest.raises(RuntimeError, match="camera_info"): + intrinsics.parse(FakeStorage(np.zeros(3, dtype=np.float32)), {}) + + +def test_intrinsics_are_read_from_frame_metadata(): + model = intrinsics.from_metadata( + {"fx": 600.0, "fy": 601.0, "cx": 320.0, "cy": 240.0}, + ) + + assert model is not None + assert (model.fx, model.cy) == (600.0, 240.0) + + +def test_frame_metadata_without_intrinsics_yields_none(): + assert intrinsics.from_metadata({"width": 640, "height": 480}) is None + + +def test_the_camera_model_is_drawn_on_the_composite(): + view = View(headless=True) + view.panels["image"] = np.zeros((40, 80, 3), dtype=np.uint8) + + without = view.compose().copy() + view.intrinsics = intrinsics.Intrinsics(600.0, 601.0, 320.0, 240.0) + with_model = view.compose() + + assert not np.array_equal(without, with_model), "the model should be drawn"