Hardware testing showed the tick-driven recovery state machine did not work.
async_grab_camera_recovery does not make grab() prompt on a disconnected ZED-M
under SDK 5.3: grabs took ~3 s each while the link was down and then blocked
indefinitely. The event loop stopped turning, so Stop went unanswered, the
30 s patience timer and the backoff were never reached, and replugging the
camera did nothing. Measured: 122 s frozen, then dora stop needed 10 s and a
forced kill.
Move everything that can block onto a capture thread. zed::Capture owns the
camera and runs open, warmup, grab, blank and stall detection, backoff and
escalation; it packs each frame into owned buffers and hands the newest one to
the reader by swapping under a mutex. The dora event loop now only drains that
buffer and publishes, so it never calls into the SDK and stays responsive
however long the SDK blocks.
The newest frame wins and uncollected frames are dropped, which is what a
consumer wants from a camera source. Swapping rather than assigning returns the
previous frame's buffers to the capture thread, so steady state does not
reallocate. The point cloud is now copied out of the SDK buffer rather than
published in place, since the reader is another thread.
On exit the loop waits 2 s for the thread and then _Exit()s if it is still
inside the SDK, so Stop is always answered promptly.
This also removes the blocking warmup from the event loop: it stays a blocking
burst, which is harmless on the capture thread.
Verified on a ZED-M: unplug leaves the node responsive and logs the backoff
1, 2, 5 then 10 s; replug gives "camera recovered after 13 attempt(s)" and
publishing resumes without restarting the dataflow; dora stop with the camera
unplugged returns in 0.70 s with the thread joining gracefully.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7g2cBJ5e3r15ZbJ9WZn8c
The README and AGENTS intros still described the v0.1.0 output set. Also
removes a copy of the point-cloud rationale that ended up orphaned after the
recovery section landed above it, and narrows the timestamp section to the
frame outputs, since camera_state carries only the host clock.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7g2cBJ5e3r15ZbJ9WZn8c
No source package is shared between node repositories, so the manifest and its
type URNs are the only contract a consumer has. Declaring them makes a wiring
mistake fail at validate time with the producer and consumer URNs, instead of
at runtime.
Every payload this node sends is a raw buffer whose shape and dtype live in the
metadata rather than an Arrow struct, so the URNs name the element type:
UInt8 for image_bgr, Float32 for depth, camera_info and point_cloud, String for
camera_state. tick is UInt64, matching how dora types timer inputs.
dataflow.yml now builds the node from the checkout with path: + build:. A
git:-sourced node does not pick up an adjacent manifest, so validating the
previous form checked nothing; with path: the CLI reports "injected 6 contract
type(s)" and a deliberately mismatched annotation exits non-zero under
strict_types. The README keeps the tag: form for consumers.
The workflow needs no camera and no ZED SDK, since validation never runs the
node. Also ignores the git/ and out/ working directories dora build creates,
and drops the ignore rules for the deleted test assets.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7g2cBJ5e3r15ZbJ9WZn8c
Camera::open() retried forever inside a fixed sleep, so a camera that did not
come back left the node stuck there: it stopped reading dora events and no
longer honoured Stop or AllInputsClosed.
Recovery is now a state machine over ticks. open() makes one attempt and
returns; the loop decides when to try again from a 1/2/5/10 s backoff deadline,
and nothing sleeps, so events keep being read while the camera is down. The
first attempt closes the handle; past that the device is wedged and the node
escalates to sl::Camera::reboot(), guarded to the ZED 2 and newer since it does
nothing on the ZED and ZED Mini. Reopening pins the serial so recovery gets the
same physical camera back.
Set async_grab_camera_recovery so a link hiccup returns CAMERA_REBOOTING
immediately instead of blocking the grab for ~17 s, and enable_image_validity_
check so the SDK flags frames a stalled camera hands back looking valid. Those
transient errors no longer drop the handle, since the SDK is already
reconnecting it.
An all-black frame now counts as a failed grab rather than being published, and
a camera delivering nothing usable for 5 s (30 s while the SDK reports it is
reconnecting) triggers recovery. A new camera_state output publishes
opening/streaming/recovering so a downstream node can tell "no frame yet" from
"camera lost".
ZED_RETRY_INTERVAL_SECONDS is replaced by the backoff.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7g2cBJ5e3r15ZbJ9WZn8c
ZED_POINT_CLOUD, off by default, gates both retrieveMeasure(XYZRGBA) and the
point_cloud output; when off the measure is not retrieved at all. At 1080p a
full cloud is ~14.7 MB per frame and costs capture time whether or not anything
consumes it, and consumers that need 3D can back-project depth instead.
camera_info now goes out once per camera session rather than on every frame:
fx, fy, cx, cy and the distortion coefficients are fixed for a given camera and
resolution, so repeating them per frame adds no information. It rides the first
frame after the camera opens, and again after a reconnection, so it still
carries that frame's timestamps.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7g2cBJ5e3r15ZbJ9WZn8c
parse_depth_mode returned NEURAL for any unrecognised string, so a typo
silently changed depth quality, and NEURAL_PLUS and NEURAL_LIGHT could not be
selected at all. All six modes are now reachable and an unknown name exits
non-zero listing the accepted values.
Parsing moves into Config::from_env() and main reads the configuration before
init_dora_node(), so a bad value fails with its own message instead of after
the node has registered with the dataflow.
Add ZED_CONFIDENCE_THRESHOLD (60) and ZED_TEXTURE_CONFIDENCE_THRESHOLD (100),
which were never set, and change the defaults to 1080 / 200 mm / 600 mm to
match the reference driver perception was tuned against. The resolved settings
are logged at startup.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7g2cBJ5e3r15ZbJ9WZn8c
Publish sl::MEASURE::DEPTH as float32 [H, W] millimetres on every frame that
carries image_bgr, with the intrinsics, frame_id and timestamps the other
outputs carry.
The map is copied out rather than published in place: sl::Mat rows can be
padded, so the buffer is not necessarily H * W floats. inf and nan are cleared
to 0 in the same pass, matching the reference driver — a hole is meaningful and
a consumer cannot back-project a non-finite value.
Consumers that need 3D can now back-project depth with the published
intrinsics instead of receiving a point cloud.
Also drops the stale objects entry from the dataflow outputs, missed when
detection was removed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7g2cBJ5e3r15ZbJ9WZn8c
steady_clock's epoch is unspecified and typically counts from process start or
boot, so the published timestamp_ns could not be compared against another
node's timestamps, wall clock or a recording — and it recorded when the node
processed the frame rather than when the camera captured it.
Publish two keys on every output instead: device_timestamp_ns from
sl::Camera::getTimestamp(TIME_REFERENCE::IMAGE), and host_timestamp_ns from
system_clock. Both are Unix-epoch nanoseconds, set as dora timestamp
parameters. The SDK clock source is pinned to SYSTEM_CLOCK at construction,
since it is process-wide and decides the epoch of the device value.
Both are kept because the wall clock can step when NTP synchronises after boot
on a Jetson, while the device timestamp is what two nodes reading the same
frame can agree on.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7g2cBJ5e3r15ZbJ9WZn8c
dora-zed publishes camera data only. The two engines under src/engines/, the
objects output, the ZED_ENGINE/ZED_DETECTION_*/ZED_ONNX_* variables and the
ZED_PC_MODE=ROI crop are gone, along with the standalone engine test and the
model export script.
This drops the unconditional ONNX Runtime requirement from CMakeLists, the
Dockerfile and the install scripts, so the node now configures on a machine
with no ONNX Runtime — which is what blocked the Jetson build, since there is
no official ARM64 GPU binary.
zed_native is dropped rather than kept: it is bound to the live camera handle
and cannot run in a node that receives images over a dataflow. The removed
code remains reachable at tag v0.1.0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7g2cBJ5e3r15ZbJ9WZn8c
dora 1.0 runs a node's build command directly rather than through a shell, so
the mkdir/cd/&& sequence in dataflow.yml could not work. Move it to
scripts/build.sh and invoke that instead.
Default CMAKE_BUILD_TYPE to Release so DoraTargets.cmake picks the release
cargo profile and the binary does not link debug dora bindings.
Pin the dataflow source with tag: — 1.0 replaced git_checkout with the
mutually exclusive tag/rev/branch.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7g2cBJ5e3r15ZbJ9WZn8c
