# 3D Gaussian Splatting and Simulation Reconstruction

> Photogrammetry, 3D Gaussian Splatting and Unreal Engine real-time rendering combined to turn real sites into interactive scenes that enter the simulation data loop, quantified across every registered view of a public benchmark.

- 来源：神太再现 · 渲元数科（Imitatio Artis）· 项目成果
- 页面：https://k2sim.cn/projects/gaussian-splatting-reconstruction
- 日期：2026-07-31
- 标签：3DGS、NeRF、Unreal Engine、3D reconstruction、Real-time rendering、Digital twin

---
This project extends our hybrid **photogrammetry + 3D Gaussian Splatting** reconstruction pipeline into a product that runs inside Unreal Engine. A reconstructed site is not just something to look at: it becomes a level that dynamic objects, sensors and data capture tasks run inside.

**IAGaussianRenderer** targets Unreal Engine 5.8 and loads standard 3DGS `.ply` files and ordinary coloured point clouds directly. It connects reconstruction output to UE scenes, Blueprints and our data pipeline, so a real site can be used for digital twins, failure reproduction, algorithm training and closed-loop evaluation.

## What it looks like

Real-time walkthrough of a reconstructed Gaussian scene:

![Real-time walkthrough of a 3D Gaussian reconstruction](/static/videos/cesium-google-3d.mp4)

The comparisons below are from a public benchmark. Left is the real photograph, right is the IAGaussianRenderer output in UE 5.8. Top row is the best view, bottom row the median view.

Truck scene: best view **24.77 dB**, median view **22.44 dB**.

![Truck scene, best and median views compared against the source photographs](/static/image/projects/ia-gaussian-renderer/truck_best_median_comparison.webp)

Train scene: best view **27.32 dB**, median view **21.47 dB**.

![Train scene, best and median views compared against the source photographs](/static/image/projects/ia-gaussian-renderer/train_best_median_comparison.webp)

## Measured quality

Validated on the **Truck** and **Train** scenes of the Tanks and Temples dataset, using the pretrained models released by INRIA. Every registered COLMAP camera pose is rendered at the source photograph's resolution and compared against its paired image. All figures are means over **all** views, with no cherry-picking.

| Metric | Truck (mean of 251 views) | Train (mean of 301 views) | Better |
| --- | --- | --- | --- |
| PSNR | **22.4547 dB** | **21.4530 dB** | Higher |
| SSIM | **0.7982** | **0.7946** | Higher |
| LPIPS (AlexNet) | **0.1228** | **0.1770** | Lower |
| LPIPS (VGG) | **0.2066** | **0.2582** | Lower |

These are per-view paired metrics computed the same way as the figures reported in the 3DGS literature, so they can be compared directly.

## Where the quality falls down

A single best view says very little. We publish the low scorers too. Truck ranges from 24.77 dB at its best view to **13.14 dB** at its worst; Train from 27.32 dB down to **13.03 dB**.

![Train scene worst view: photograph, reference render and engine output](/static/image/projects/ia-gaussian-renderer/train_00264_gt_ref_engine.webp)

The worst views share three causes: insufficient training-view coverage of that region, strong exposure differences between the source photographs, and locally unresolved geometry. All three are addressable at capture time, which is why we surface them rather than hide them.

## Where it fits

Teams that need a specific real location inside their simulator: a customer site where the robot keeps failing, an intersection the fleet handles badly, a facility that cannot be visited repeatedly. Reconstruction turns one capture session into an environment the team can re-enter indefinitely, with sensors and dynamic objects added on top.

## Deliverables

- Capture planning for the site, sized to the quality target
- The reconstructed scene as a 3DGS asset plus the UE 5.8 plugin to load it
- A per-view quality report covering all registered views, not a selected subset
- Integration into the simulation level and data capture pipeline

## Talk to us

Tell us the site you need reproduced and the fidelity your task requires, and we will scope the capture and reconstruction work.