Spline-Based Boundary Representations for Sparse View Reconstruction and Simulation Using Isogeometric Analysis
INFORMATION
Authors: Davor Dobrota, Vsevolod Skorokhodov, Chenghao Xu, Olga Fink, Malcolm Mielle
Status: arXiv pre-print
In this work, Davor Dobrota, Vsevolod Skorokhodov, Chenghao Xu, Olga Fink and I developed FORGE-SIM, a method that directly reconstructs a multi-patch B-spline boundary representation from a sparse set of posed RGB images — models that are natively compatible with both CAD and simulation workflows, without manual intervention.
Image-based reconstruction has become remarkably good at recovering visually detailed 3D models. But those models are not what simulation needs: numerical solvers require explicit, watertight, and smooth geometries, and surfaces extracted from reconstructions typically lack all three. Converting a reconstruction into a simulation-ready model today means hours of manual CAD re-modeling and mesh repair — a long-standing barrier between computer vision and numerical analysis.
FORGE-SIM removes that barrier by optimizing the spline representation itself: the output is a compact, smooth, watertight B-spline boundary representation that the solver consumes directly, with no intermediate mesh export or format conversion.
Method
FORGE-SIM starts from a cubed-sphere initialization (six B-spline patches with shared edge control points, giving an approximately uniform and isotropic distribution of degrees of freedom), which is affine-aligned to the reference masks before optimization. Shape and appearance are then recovered by alternating three optimization passes:
-
Shape pass: the B-spline control points are optimized with L-BFGS against a differentiable rendering loss, with normal-continuity (G¹) and curvature regularization across patch boundaries — no graph-Laplacian preconditioning needed, because a point on a cubic patch depends on up to sixteen control points, spreading gradients over a much larger neighborhood than mesh vertices.
-
Texture pass: with the shape frozen, per-patch textures are optimized with Adam (sigmoid-projected to stay in the spline’s parameter domain).
-
Physical-field inpainting: observation-derived fields — an initial thermal state captured by infrared thermography, or semantic information — are projected onto the reconstructed model in the same spline basis, so the IGA solver consumes them unmodified.
The result is a single representation that serves simultaneously as reconstruction target and computational domain: geometry, appearance, and physical fields all live in one B-spline basis, and the same basis-function evaluations, control points, and tensor primitives used during optimization are reused directly by the isogeometric analysis (IGA) solver.
Results
On four textureless meshes (Suzanne, Bunny, Spot, Armadillo) with known ground truth, IGA-based heat-flow and modal simulations run directly on the FORGE-SIM reconstruction agree with FEM-based simulations on the ground-truth mesh to within:
| Mesh | Init. energy % | Gen. rate % | FL² t₀ % | FL² t_N % | Relaxation λ₁ % | Median freq. % | MAC ↑ |
|---|---|---|---|---|---|---|---|
| Suzanne | 0.07 | 2.06 | 1.97 | 1.41 | 0.72 | 2.47 | 0.53 |
| Bunny | 1.35 | 1.21 | 6.95 | 1.07 | 0.77 | 5.35 | 0.43 |
| Spot | 1.99 | 1.78 | 7.02 | 3.35 | 5.93 | 7.94 | 0.34 |
| Armadillo | 5.26 | 3.93 | 14.36 | 6.89 | 19.85 | 8.72 | 0.32 |
Simulation error tracks reconstruction error rather than diverging independently: for the Armadillo — the hardest shape, with thin limbs — the errors grow smoothly and remain stable over the simulated horizon, and the field and relaxation-rate errors stay under 7% and 20%. The geometric recovery step, not the simulation formulation, is the binding constraint on end-to-end fidelity.
Reconstruction quality
Forcing simulation-grade constraints does not trade away visual fidelity. FORGE-SIM achieves a lower Chamfer distance than Large Steps (a leading mesh-based shape-optimization method) on 7 of 8 objects with known ground-truth topology — by close to an order of magnitude on the Armadillo (0.035 vs 0.337) — and matches or achieves the best LPIPS among all tested baselines on 7 of 8 test scenes:
| Method | BuildNet3D 1 | 2 | 3 | 4 | Suzanne | Bunny | Spot | Armadillo |
|---|---|---|---|---|---|---|---|---|
| FORGE-SIM | 0.167 | 0.322 | 0.094 | 0.096 | 0.001 | 0.012 | 0.019 | 0.035 |
| Large Steps | 0.434 | 0.460 | 0.115 | 0.140 | 0.016 | 0.088 | 0.0065 | 0.337 |
(Chamfer distance ↓, lower is better; bold = better.)
It also remains competitive on novel-view synthesis (PSNR/SSIM/LPIPS) against Nerfacto, NeuS, and Gaussian splatting — a classic vision task the spline representation was not designed for.
On real-world scenes, thermal and modal simulations run on FORGE-SIM reconstructions of buildings captured with infrared thermography, demonstrating the complete image-to-simulation pipeline on realistic data.
What’s next
- Beyond genus-0: the six-patch cubed-sphere parametrization restricts reconstructions to genus-0 solids — through-holes, handles, and multi-body assemblies cannot currently be represented. Adaptive representations such as T-splines or hierarchical B-splines are the natural extension.
- Fine geometric detail: B-splines can over-smooth features critical for infrastructure analysis or industrial quality control.
- Pose and scale robustness: on real scenes, poses come from VGGT and metric scale from a single calibration factor per scene — fusing RGB, thermal, and inertial measurements (as in SEAR) would push toward a fully autonomous digitize-and-simulate workflow.
- Speed: 4–6 hours of per-scene optimization on a single workstation GPU — fine for offline digital twins, a constraint for time-sensitive inspection.
- If any of these directions interest you, see Collaborate.