Depth Anything Jetson

Live 3D Reconstruction on a Jetson Orin Nano Using Depth Anything V3

Properties3
descriptionA capstone project investigating bounded, incremental monocular 3D reconstruction on an 8 GB edge device.
tagscapstone, jetson, depth-anything-v3, 3d-reconstruction
aliasesDepth Anything Jetson

4 min read

This capstone investigates whether Depth Anything V3 Small can turn a live monocular camera stream into a continually updated, room-scale 3D reconstruction on an 8 GB Jetson Orin Nano Super.

PlatformModelTarget update rate
Jetson Orin Nano Super, 8 GBDepth Anything V3 SmallOne accepted 3D map update every 2–5 seconds

Intended result

The visual target is Richard Palethorpe’s depth-anything.cpp voxel reconstruction demo: a source camera view beside a persistent colored voxel world that can be replayed and explored from an independent 3D viewpoint.

The depth-anything.cpp DA3 Studio displaying a colored voxel reconstruction of a building beside its source camera frame

Voxel reconstruction reference by Richard Palethorpe. Select the image to view the original demo.

Visual reference—not a hardware performance baseline

The reference uses phone time-lapse footage and a desktop-oriented depth-anything.cpp and SLAM pipeline. This project aims for a slower, bounded Jetson adaptation: deliberate movement through one small room, a responsive RGB preview, and a persistent point or voxel map updated after each accepted inference window.

A successful final demonstration will make recognizable structures—such as a doorway, desk, and wall corner—appear incrementally while preserving previously accepted geometry. Moderate noise, seams, and blocky voxels are acceptable when the room remains spatially coherent.

See Final Demo Target and Reference Systems for a detailed comparison with this demo and related systems.

Research question

How effectively can Depth Anything V3 Small be adapted for incremental, spatially coherent point-cloud reconstruction on an 8 GB edge device?

The challenge is not simply whether the model can run. The project must determine whether small overlapping predictions can be aligned reliably, accumulated without exhausting memory, and updated frequently enough to support a useful live demonstration.

Core approach

RGB camera
    ↓
Small overlapping image windows
    ↓
Depth Anything V3 Small
depth + confidence + camera geometry
    ↓
Confidence-filtered colored point cloud
    ↓
Sim(3) alignment through shared frames
    ↓
Bounded persistent point or voxel map

The primary configuration uses four frames with two retained overlap frames at 336 px. If that does not fit the memory budget, the documented fallback uses three frames with two retained overlap frames at 280 px.

Capstone contribution

The contribution is a Jetson-specific sliding-window reconstruction pipeline and evaluation, not merely a model deployment. The work measures:

  • multi-frame inference within 8 GB of shared memory;
  • consistency and alignment across independently predicted windows;
  • update latency and bounded point-cloud growth;
  • reconstruction quality at reduced input resolution; and
  • sustained memory, power, and thermal behavior.

Full SLAM, loop closure, Gaussian splatting, survey-grade metric accuracy, and desktop-class real-time performance are outside the required scope.

Outcome tiers

Minimum viable

  • Run DA3 Small on recorded frames using the primary or fallback configuration.
  • Produce a confidence-filtered colored point cloud from one window.
  • Align at least two overlapping windows and accumulate a recorded room sweep.
  • Export the final cloud, trajectory, configuration, and logs.

Target

  • Use live camera input with a responsive RGB preview.
  • Produce at least one accepted map update every five seconds.
  • Maintain a persistent point or voxel view bounded to 30 accepted windows and 1,000,000 points.
  • Complete a recognizable one-room reconstruction and a 15-minute stability run.

Stretch

  • Increase the accepted update rate beyond the required target.
  • Reduce visible seams with optional ICP refinement.
  • Add loop closure, pose-graph optimization, or a remote WebGL viewer.
  • Investigate TensorRT, C++ deployment, or Gaussian splatting after the core system succeeds.

Current status

Planning and feasibility validation

Next technical milestones:

  1. Run DA3 Small FP16 on the Jetson and measure 2-, 3-, and 4-frame memory use.
  2. Validate depth, confidence, intrinsics, pose outputs, and coordinate conventions.
  3. Export one confidence-filtered colored point cloud.
  4. Test shared-frame consistency and align two overlapping windows with Sim(3).
  5. Accumulate a bounded recorded room sweep before integrating live capture.

Project notes