Depth Anything Jetson

DA3 Model and Outputs

Properties1
tagsdepth-anything-v3, model-selection, inference, coordinates

3 min read

DA3 Model and Outputs

Model selection

The official Depth Anything 3 repository provides any-view, monocular, metric, and nested series. Its model table lists:

ModelParametersRelevant capabilityProject role
DA3 Small0.08BRelative depth, pose estimation, pose conditioningRequired model
DA3 Base0.12BSame any-view task familyOutside required scope
DA3 Large 1.10.35BHigher-quality any-view geometryToo risky for 8 GB
DA3 Metric Large0.35BMetric monocular depth onlyStretch metric branch
DA3 Nested Giant-Large 1.11.40BAny-view geometry plus metric depthInappropriate for the primary device

DA3 Small uses the Apache 2.0 license and is the only required checkpoint. Gaussian output requires larger variants and is outside the primary pipeline.

Locked inference configuration

  • Checkpoint: depth-anything/DA3-SMALL
  • Model series: any-view
  • Precision: PyTorch FP16
  • Primary resolution: process_res=336
  • Resize mode: upper_bound_resize
  • Primary window: 4 frames with 2-frame overlap
  • Fallback: 3 frames with 2-frame overlap at process_res=280
  • Reference view: middle
  • Pose implementation: camera decoder with use_ray_pose=False
  • Confidence filter: retain values at or above each frame’s 40th percentile

The official Python API defaults to 504 px and recommends the middle reference strategy for video. The reduced resolutions are explicit Jetson memory tradeoffs.

Output contract

For N input views, require:

OutputShapeMeaning
processed_images[N, H, W, 3]RGB aligned to the model outputs
depth[N, H, W]Relative depth
conf[N, H, W]Prediction confidence
extrinsics[N, 3, 4]OpenCV/Colmap-style world-to-camera transforms
intrinsics[N, 3, 3]Per-view camera matrices
flowchart LR
    Window["N-frame overlapping<br/>RGB window"] --> Model["DA3 Small<br/>any-view inference"]
    Model --> RGB["processed_images<br/>N x H x W x 3"]
    Model --> Depth["relative depth<br/>N x H x W"]
    Model --> Confidence["confidence<br/>N x H x W"]
    Model --> Intrinsics["intrinsics<br/>N x 3 x 3"]
    Model --> Extrinsics["world-to-camera poses<br/>N x 3 x 4"]
    RGB --> Validate["Validate shapes,<br/>grids, and finite values"]
    Depth --> Validate
    Confidence --> Validate
    Intrinsics --> Validate
    Extrinsics --> Validate
    Validate --> Filter["Per-frame 40th-percentile<br/>confidence filter"]
    Filter --> Backproject["Back-project retained pixels<br/>and invert camera poses"]
    Backproject --> Cloud["Colored local point cloud<br/>in the window frame"]

For visual context, the official Depth Anything 3 project page demonstrates video reconstruction, recovered cameras, and large-scale SLAM. The official interactive demo exposes point-cloud, camera, metric-depth, and novel-view outputs. These demonstrate the broader DA3 family, not the locked DA3 Small configuration or its Orin Nano update rate.

Reject a prediction when:

  • Any required output is missing or has the wrong shape
  • Depth is entirely non-finite or non-positive
  • Confidence, intrinsics, or extrinsics contain non-finite values
  • Processed RGB dimensions do not match depth and confidence

Invalid individual depth pixels are removed rather than rejecting an otherwise valid window.

Confidence filtering

For each frame:

  1. Select finite, positive-depth pixels with finite confidence.
  2. Compute the 40th percentile of those confidence values.
  3. Retain pixels at or above the threshold.
  4. Preserve the threshold and retained-point count in the window log.

Confidence is a relative filtering signal, not a calibrated probability.

Coordinates and back-projection

Use DA3-returned intrinsics and processed RGB coordinates. Independent OpenCV calibration is diagnostic and does not replace predicted intrinsics in the unconditioned pipeline.

For retained pixel (u, v):

X_camera = (u - cx) × Z / fx
Y_camera = (v - cy) × Z / fy
Z_camera = relative depth

Convert each [3, 4] extrinsic into a homogeneous OpenCV world-to-camera matrix. Invert it before transforming camera points:

point_window = inverse(T_camera_window) × point_camera

Attach color from the matching location in processed_images.

Single-window normalization

Normalize the first accepted window so the median retained depth is 1.0 internal unit. Apply the same factor to its points and camera translations. This is a numerical convention only; it does not make the output metric.

Pose modes

The camera decoder (use_ray_pose=False) is the baseline because the official documentation describes it as faster. Ray pose is an optional evaluation after the baseline succeeds and must still satisfy memory and update-rate requirements.