New pre-print: SEAR — Simple and Efficient Adaptation of Visual Geometric Transformers for RGB+Thermal 3D Reconstruction.
Foundational models like VGGT excel with RGB inputs but struggle to align RGB and thermal images when processed jointly, often producing disjoint reconstructions. SEAR bridges this modality gap with minimal parameter updates (LoRA adapters, thermal camera tokens, and a batching strategy), enabling reliable multimodal pose estimation and reconstruction even under challenging conditions such as low lighting and dense smoke.
- 30%+ improvement in AUC@30 for camera pose estimation over state-of-the-art baselines
- ~10 FPS — 200× faster than the closest competitor, using <5% of the original model’s parameters
- A new dataset of 9 scenes (~2,000 images) with distinct RGB/thermal trajectories, released on Zenodo
Pre-print | Code | Weights | Dataset | cite
This work is part of the INSULATED project on thermal digital twins for building retrofit.