liria/dora-image-view
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

accept std/media/v1/Image payloads

Browse Source 
dora's typed image is an Arrow struct of width, height, encoding and data --
it carries inside the payload what this node has been reading out of message
metadata. The decoder now takes either shape: a struct describes itself and
its metadata is ignored, a flat UInt8 buffer is described by width/height/
encoding beside it, and everything after unpacking is common.

The declared contract stays std/core/v1/UInt8, deliberately. Type
compatibility is directional -- Image satisfies a Bytes input, but a
std/core/v1/UInt8 output wired into an std/media/v1/Image input is a validate
error -- and dora-zed declares image_bgr as std/core/v1/UInt8. Flipping this
manifest now would break the camera. Accepting both shapes first is what lets
the producers move before the contract does.

output_framing is not part of this. It selects the wire format, raw or
arrow-ipc, and is not required for images: a struct payload was verified to
arrive identically under both framings, and validate does not ask for
arrow-ipc when an output is annotated std/media/v1/Image. It earns its keep
when the wire must be self-describing, such as crossing languages or
recording.
This commit is contained in:
cristhian
2026-09-15 05:25:18 -03:00
parent 92abe8a446
commit a6fa3ce32b
5 changed files with 133 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+13 -1
View File
@@ -106,12 +106,24 @@ no window. That is what makes it safe to run in a pipeline test.
| Input | Payload | Metadata |
|---|---|---|
| `image` | Flat `UInt8` array | `width`, `height`, `encoding` |
| `image` | `std/media/v1/Image` struct, **or** a flat `UInt8` array | none for the struct; `width`, `height`, `encoding` for the flat form |
| `depth` | Flat `Float32` array, `[H, W]` | `shape`, `units` |
| `camera_info` | Flat `Float32` array, `[fx, fy, cx, cy]` or a 9-element K | `distortion`, `distortion_model` |
| `bbox` | Arrow struct array, element `[0]` with fields `bbox`, `conf`, `labels` | `format` |
| `text` | Arrow array of size 1 | — |
Two image payload shapes are accepted. The typed one is `std/media/v1/Image`, an Arrow
struct of `width: UInt32`, `height: UInt32`, `encoding: Utf8` and `data: LargeBinary`,
which carries inside the payload what the flat form carries beside it; when it is
received the message metadata is not consulted at all. The flat form is a raw `UInt8`
buffer described by `width`, `height` and `encoding` metadata, which is what a node
declaring `std/core/v1/UInt8` sends.
The declared contract is still `std/core/v1/UInt8`, so a producer sending the struct
should keep declaring that until every producer in the dataflow has moved: dora's type
compatibility is directional, and a `std/core/v1/UInt8` output wired into an
`std/media/v1/Image` input fails `dora validate`.
Supported `encoding` values: `bgr8`, `rgb8`, `jpeg`/`jpg`/`jpe`/`bmp`/`webp`/`png`
(encoded byte stream), `yuv420`, `avif`. `width` and `height` are only needed by the raw
encodings — the rest carry their own dimensions. An unsupported encoding raises.