Publish a depth output #11

New Issue

2026-09-04T20:35:17Z

cristhian commented

2026-09-04 20:35:17 +00:00

Feature: Publish a depth output

User Story

As a perception node,
I want a per-pixel depth map in millimetres,
so that I can fit a table plane and locate objects without receiving a point cloud.

Requirements

Retrieve sl::MEASURE::DEPTH on each successful grab.
New output depth: float32, shape [H, W], layout HW, units millimetres.
Replace inf and nan with 0 before publishing.
H and W match image_bgr for the same frame.
Metadata: dtype, layout, shape, units, fx, fy, cx, cy, distortion, frame_id, and the timestamps defined in the timestamp issue.
Document the output and its metadata in README.md.

Acceptance Criteria

depth is published on every frame on which image_bgr is published.
shape metadata matches the payload length: H * W * 4 bytes.
The payload contains no inf and no nan.
Depth dimensions equal image dimensions for the same frame_id.
README.md documents the output.

Notes

The existing point_cloud output is not a substitute. Consumers back-project depth to 3D
themselves using the intrinsics; reference implementation
littlehand/perception/table.py::back_project(depth_mm, camera_matrix), called from
littlehand/perception/stages/table.py.

Zero is used for missing depth rather than nan to match the reference driver
(littlehand/cameras/zed.py), which clears inf and nan to 0 after retrieval. Holes
are meaningful and must not be interpolated unless ZED_DEPTH_FILL is set.

# Feature: Publish a depth output ## User Story As a perception node, I want a per-pixel depth map in millimetres, so that I can fit a table plane and locate objects without receiving a point cloud. ## Requirements - Retrieve `sl::MEASURE::DEPTH` on each successful grab. - New output `depth`: `float32`, shape `[H, W]`, layout `HW`, units millimetres. - Replace `inf` and `nan` with `0` before publishing. - `H` and `W` match `image_bgr` for the same frame. - Metadata: `dtype`, `layout`, `shape`, `units`, `fx`, `fy`, `cx`, `cy`, `distortion`, `frame_id`, and the timestamps defined in the timestamp issue. - Document the output and its metadata in `README.md`. ## Acceptance Criteria - [ ] `depth` is published on every frame on which `image_bgr` is published. - [ ] `shape` metadata matches the payload length: `H * W * 4` bytes. - [ ] The payload contains no `inf` and no `nan`. - [ ] Depth dimensions equal image dimensions for the same `frame_id`. - [ ] `README.md` documents the output. ## Notes The existing `point_cloud` output is not a substitute. Consumers back-project depth to 3D themselves using the intrinsics; reference implementation `littlehand/perception/table.py::back_project(depth_mm, camera_matrix)`, called from `littlehand/perception/stages/table.py`. Zero is used for missing depth rather than `nan` to match the reference driver (`littlehand/cameras/zed.py`), which clears `inf` and `nan` to `0` after retrieval. Holes are meaningful and must not be interpolated unless `ZED_DEPTH_FILL` is set.

cristhian added this to the v0.2.0 Pure camera source on dora 1.0 milestone 2026-09-04 20:35:17 +00:00

cristhian added the feature priority:high labels 2026-09-04 20:35:17 +00:00

cristhian referenced a pull request that will close this issue

2026-09-08 12:39:27 +00:00

v0.2.0: pure camera source on dora 1.0 #18

cristhian closed this issue

2026-09-08 15:06:39 +00:00

Sign in to join this conversation.