liria/dora-image-view
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

use std/media/v1/Image for image and depth

Browse Source 
Both inputs now declare dora's Arrow struct {width, height, encoding, data}
rather than a flat buffer plus loose metadata. The payload describes itself, so
nothing about its shape has to be agreed out of band, and the frame travels as
one Arrow value instead of a buffer whose meaning lives in a sidecar dict.

Depth uses the same type. dora's registry has no depth type, and a depth map is
an image whose pixels are distances -- the reading ROS takes as well -- so it
arrives with a single-channel encoding: 32FC1/float32 or 16UC1/mono16/uint16.
`units` stays in message metadata because the struct has no field for it.

decode.IMAGE_TYPE is the single definition of the struct; the example source
and the tests build payloads from it rather than respelling the schema.

The older shapes are still decoded -- a flat UInt8 buffer with
width/height/encoding for image, a flat Float32 buffer with shape/units for
depth -- so a producer's payload can move before its manifest does. That is
only half a migration path: type compatibility is directional, so a producer
still declaring std/core/v1/UInt8 fails validate against an
std/media/v1/Image input regardless of what it sends, and has to update its
manifest too.

Verified: 41 tests, validate clean with 4 contracts injected, the dataflow runs
to a clean exit, and a composite rendered through the struct path is identical
to the one rendered before the migration.
This commit is contained in:
cristhian
2026-09-15 05:27:40 -03:00
parent a6fa3ce32b
commit 8a7bda5867
8 changed files with 153 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+19 -13
View File
@@ -106,25 +106,31 @@ no window. That is what makes it safe to run in a pipeline test.
| Input | Payload | Metadata |
|---|---|---|
| `image` | `std/media/v1/Image` struct, **or** a flat `UInt8` array | none for the struct; `width`, `height`, `encoding` for the flat form |
| `depth` | Flat `Float32` array, `[H, W]` | `shape`, `units` |
| `image` | `std/media/v1/Image` struct | none — the struct describes itself |
| `depth` | `std/media/v1/Image` struct, one channel | `units` (defaults to `mm`) |
| `camera_info` | Flat `Float32` array, `[fx, fy, cx, cy]` or a 9-element K | `distortion`, `distortion_model` |
| `bbox` | Arrow struct array, element `[0]` with fields `bbox`, `conf`, `labels` | `format` |
| `text` | Arrow array of size 1 | — |
Two image payload shapes are accepted. The typed one is `std/media/v1/Image`, an Arrow
struct of `width: UInt32`, `height: UInt32`, `encoding: Utf8` and `data: LargeBinary`,
which carries inside the payload what the flat form carries beside it; when it is
received the message metadata is not consulted at all. The flat form is a raw `UInt8`
buffer described by `width`, `height` and `encoding` metadata, which is what a node
declaring `std/core/v1/UInt8` sends.
`image` and `depth` are both `std/media/v1/Image`: an Arrow struct of `width: UInt32`,
`height: UInt32`, `encoding: Utf8` and `data: LargeBinary`. The payload describes
itself, so nothing about its shape has to travel as message metadata.
The declared contract is still `std/core/v1/UInt8`, so a producer sending the struct
should keep declaring that until every producer in the dataflow has moved: dora's type
compatibility is directional, and a `std/core/v1/UInt8` output wired into an
`std/media/v1/Image` input fails `dora validate`.
Depth is an image whose pixels are distances — dora's type registry has no depth type,
and this is how ROS spells it too. Its encodings are `32FC1`/`float32` and
`16UC1`/`mono16`/`uint16`, one channel per pixel, and `units` metadata still switches
millimetres to metres.
Supported `encoding` values: `bgr8`, `rgb8`, `jpeg`/`jpg`/`jpe`/`bmp`/`webp`/`png`
Older payloads are still decoded, so a producer can move at its own pace: a flat `UInt8`
buffer with `width`/`height`/`encoding` metadata for `image`, and a flat `Float32`
buffer with `shape`/`units` metadata for `depth`.
**Migrating a producer.** dora's type compatibility is directional — `Image` satisfies a
`Bytes` input, but a `std/core/v1/UInt8` output wired into an `std/media/v1/Image` input
is a `dora validate` error. A producer still declaring `std/core/v1/UInt8` or
`std/core/v1/Float32` therefore has to change its manifest as well as its payload.
Supported `image` `encoding` values: `bgr8`, `rgb8`, `jpeg`/`jpg`/`jpe`/`bmp`/`webp`/`png`
(encoded byte stream), `yuv420`, `avif`. `width` and `height` are only needed by the raw
encodings — the rest carry their own dimensions. An unsupported encoding raises.