liria/dora-zed
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

add depth configuration parity with the reference driver

Browse Source 
parse_depth_mode returned NEURAL for any unrecognised string, so a typo
silently changed depth quality, and NEURAL_PLUS and NEURAL_LIGHT could not be
selected at all. All six modes are now reachable and an unknown name exits
non-zero listing the accepted values.

Parsing moves into Config::from_env() and main reads the configuration before
init_dora_node(), so a bad value fails with its own message instead of after
the node has registered with the dataflow.

Add ZED_CONFIDENCE_THRESHOLD (60) and ZED_TEXTURE_CONFIDENCE_THRESHOLD (100),
which were never set, and change the defaults to 1080 / 200 mm / 600 mm to
match the reference driver perception was tuned against. The resolved settings
are logged at startup.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7g2cBJ5e3r15ZbJ9WZn8c
This commit is contained in:
cesar
2026-09-08 09:20:01 -03:00
co-authored by Claude Opus 5
parent 15134dc12e
commit 617b8ceb50
5 changed files with 61 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+1 -1
View File
@@ -37,7 +37,7 @@ Camera behavior is configured entirely through environment variables (see README
- **`src/util/env.h`** — Shared inline env-variable parsing helpers (`get_env_int`, `get_env_str`, `get_env_bool`, `get_env_float`).
- **`src/zed/camera.h`** — `zed::Config`, `zed::Calibration`, and `zed::Camera` class. Camera wraps the ZED SDK lifecycle: open (with retry loop), warmup, grab (image + depth + point cloud), calibration, device timestamp, and close. Exposes `handle()` for engine access.
- **`src/zed/camera.cc`** — Implementation. SDK enum parsers live in an anonymous namespace.
- **`src/zed/camera.cc`** — Implementation. SDK enum parsers live in an anonymous namespace. `parse_depth_mode` runs in `Config::from_env()`, not in the constructor, so an unrecognised `ZED_DEPTH_MODE` exits before the node joins the dataflow rather than falling back to a default.
- **`src/main.cc`** — Dora event loop. Parses `ZED_PC_MODE` (`FULL`/`HALF`) once before the loop. On any input event (intended to be driven by a tick timer; the id is not inspected): calls `camera.grab()`, then sends `image_bgr`, `depth`, `point_cloud` and `camera_info`. `pack_depth` copies the depth map into a tightly packed `[H, W]` float buffer (sl::Mat rows can be padded) and clears `inf`/`nan` to `0`. `set_intrinsics` and `set_frame_stamps` stamp every output of a frame with the same intrinsics, `frame_id` and the two timestamps (`device_timestamp_ns`, `host_timestamp_ns`; both Unix-epoch nanoseconds). Point cloud output varies by mode: `FULL` sends the raw `sl::Mat` pointer (no copy); `HALF` downsamples by 2 in each dimension. Handles reconnection on grab failure.
- **`DoraTargets.cmake`** — Handles building or fetching dora-rs and generating the CXX bridge files. Two paths: local (`DORA_ROOT_DIR`) or git clone. Produces `dora_node_api_cxx` library and the `node_bridge.cc` glue.
- **`scripts/build.sh`** — Configure/compile/install in one shell-free-callable step. Referenced by `dataflow.yml`, the `Dockerfile` and the README.
README.md
+11 -5
View File
@@ -93,16 +93,22 @@ All behavior is controlled via environment variables.
| Variable | Default | Description |
|---|---|---|
| `ZED_RESOLUTION` | `720` | `720`, `1080`, or `2000` |
| `ZED_RESOLUTION` | `1080` | `720`, `1080`, or `2000` |
| `ZED_FPS` | `15` | Frame rate |
| `ZED_DEPTH_MODE` | `NEURAL` | `PERFORMANCE`, `QUALITY`, `NEURAL`, or `NONE` |
| `ZED_DEPTH_MIN_MM` | `10` | Minimum depth (mm) |
| `ZED_DEPTH_MAX_MM` | `500` | Maximum depth (mm) |
| `ZED_DEPTH_FILL` | `false` | Enable depth fill mode |
| `ZED_DEPTH_MODE` | `NEURAL` | `NEURAL`, `NEURAL_PLUS`, `NEURAL_LIGHT`, `PERFORMANCE`, `QUALITY`, or `NONE`. An unrecognised value is a startup error, not a fallback. |
| `ZED_DEPTH_MIN_MM` | `200` | Minimum depth (mm) |
| `ZED_DEPTH_MAX_MM` | `600` | Maximum depth (mm) |
| `ZED_CONFIDENCE_THRESHOLD` | `60` | Reject depth less certain than this (1–100, lower is stricter) |
| `ZED_TEXTURE_CONFIDENCE_THRESHOLD` | `100` | Reject depth on surfaces less textured than this (1–100) |
| `ZED_DEPTH_FILL` | `false` | Interpolate depth over holes. Overrides both confidence thresholds. |
| `ZED_FLIP` | `ON` | `ON` or `OFF` |
| `ZED_WARMUP_FRAMES` | `30` | Frames to discard on startup |
| `ZED_RETRY_INTERVAL_SECONDS` | `5` | Reconnection interval (s) |
The resolved depth mode, both confidence thresholds and the depth range are logged at
startup. The defaults match the reference camera driver the perception stack was tuned
against, so depth quality on the robot matches what it expects.
### Point cloud
| `ZED_PC_MODE` | Shape | Description |
src/main.cc
+4 -2
View File
@@ -91,9 +91,11 @@ int64_t host_timestamp_ns() {
} // namespace
int main() {
auto dora_node = init_dora_node();
// Read the environment before joining the dataflow, so a bad configuration
// fails with its own message rather than after the node has registered.
auto config = zed::Config::from_env();
auto dora_node = init_dora_node();
zed::Camera camera(config);
camera.open();
camera.warmup();
src/zed/camera.cc
+38 -9
View File
@@ -2,10 +2,12 @@
#include "util/env.h"
#include <chrono>
#include <cstdlib>
#include <iostream>
#include <iterator>
#include <string>
#include <thread>
#include <utility>
namespace {
@@ -21,17 +23,30 @@ sl::RESOLUTION parse_resolution(int resolution) {
}
}
const std::pair<const char*, sl::DEPTH_MODE> DEPTH_MODES[] = {
{"NEURAL", sl::DEPTH_MODE::NEURAL},
{"NEURAL_PLUS", sl::DEPTH_MODE::NEURAL_PLUS},
{"NEURAL_LIGHT", sl::DEPTH_MODE::NEURAL_LIGHT},
{"PERFORMANCE", sl::DEPTH_MODE::PERFORMANCE},
{"QUALITY", sl::DEPTH_MODE::QUALITY},
{"NONE", sl::DEPTH_MODE::NONE},
};
// A typo here silently changes depth quality if it falls back to a default, and
// the resulting depth is wrong in a way that only shows up downstream, so an
// unrecognised name stops the node instead.
sl::DEPTH_MODE parse_depth_mode(const std::string& mode) {
if (mode == "PERFORMANCE") {
return sl::DEPTH_MODE::PERFORMANCE;
for (const auto& entry : DEPTH_MODES) {
if (mode == entry.first) {
return entry.second;
}
if (mode == "QUALITY") {
return sl::DEPTH_MODE::QUALITY;
}
if (mode == "NONE") {
return sl::DEPTH_MODE::NONE;
std::cerr << "Unknown ZED_DEPTH_MODE '" << mode << "'. Accepted values:";
for (const auto& entry : DEPTH_MODES) {
std::cerr << " " << entry.first;
}
return sl::DEPTH_MODE::NEURAL;
std::cerr << std::endl;
std::exit(1);
}
sl::FLIP_MODE parse_flip(const std::string& mode) {
@@ -59,10 +74,13 @@ Config Config::from_env() {
Config c;
c.resolution = util::get_env_int("ZED_RESOLUTION", c.resolution);
c.fps = util::get_env_int("ZED_FPS", c.fps);
c.depth_mode = util::get_env_str("ZED_DEPTH_MODE", c.depth_mode);
c.depth_mode = parse_depth_mode(util::get_env_str("ZED_DEPTH_MODE", "NEURAL"));
c.depth_min_mm = util::get_env_int("ZED_DEPTH_MIN_MM", c.depth_min_mm);
c.depth_max_mm = util::get_env_int("ZED_DEPTH_MAX_MM", c.depth_max_mm);
c.depth_fill = util::get_env_bool("ZED_DEPTH_FILL", c.depth_fill);
c.confidence_threshold = util::get_env_int("ZED_CONFIDENCE_THRESHOLD", c.confidence_threshold);
c.texture_confidence_threshold =
util::get_env_int("ZED_TEXTURE_CONFIDENCE_THRESHOLD", c.texture_confidence_threshold);
c.flip = util::get_env_str("ZED_FLIP", c.flip);
c.warmup_frames = util::get_env_int("ZED_WARMUP_FRAMES", c.warmup_frames);
int retry_seconds = util::get_env_int("ZED_RETRY_INTERVAL_SECONDS", 5);
@@ -78,7 +96,7 @@ Camera::Camera(const Config& config) : config_(config) {
init_params_.camera_resolution = parse_resolution(config_.resolution);
init_params_.camera_fps = config_.fps;
init_params_.depth_mode = parse_depth_mode(config_.depth_mode);
init_params_.depth_mode = config_.depth_mode;
init_params_.coordinate_units = sl::UNIT::MILLIMETER;
init_params_.camera_image_flip = parse_flip(config_.flip);
init_params_.depth_minimum_distance = config_.depth_min_mm;
@@ -86,6 +104,17 @@ Camera::Camera(const Config& config) : config_(config) {
runtime_params_.enable_depth = true;
runtime_params_.enable_fill_mode = config_.depth_fill;
// The two thresholds discard the depth the SDK is least certain of, which is
// the depth at object edges: those pixels straddle the object and whatever is
// behind it, and belong to neither.
runtime_params_.confidence_threshold = config_.confidence_threshold;
runtime_params_.texture_confidence_threshold = config_.texture_confidence_threshold;
std::cout << "ZED depth: mode=" << sl::toString(init_params_.depth_mode)
<< ", range=" << config_.depth_min_mm << "-" << config_.depth_max_mm << " mm"
<< ", confidence=" << config_.confidence_threshold
<< ", texture_confidence=" << config_.texture_confidence_threshold
<< ", fill=" << (config_.depth_fill ? "on" : "off") << std::endl;
}
Camera::~Camera() {
src/zed/camera.h
+6 -4
View File
@@ -8,12 +8,14 @@
namespace zed {
struct Config {
int resolution = 720;
int resolution = 1080;
int fps = 15;
std::string depth_mode = "NEURAL";
int depth_min_mm = 10;
int depth_max_mm = 500;
sl::DEPTH_MODE depth_mode = sl::DEPTH_MODE::NEURAL;
int depth_min_mm = 200;
int depth_max_mm = 600;
bool depth_fill = false;
int confidence_threshold = 60;
int texture_confidence_threshold = 100;
std::string flip = "ON";
int warmup_frames = 30;
int retry_interval_ms = 5000;