Data flywheel Simulation data Hardware co-development SDG Data governance

Fused Data Production and the Data Flywheel

Lets simulation data start training and evaluation before the hardware exists, then reuses the same scenes, governance and validation pipeline once real-world data flows back in, forming a data flywheel that keeps compounding.

2026-01-30 5 min read
Article contents12

This project serves intelligent hardware, robotics and industrial UAV teams with a fused data platform that runs end to end acrosssimulation data production, real-world data intake, failure reproduction and closed-loop evaluation.

The problem it solves is not "where do the files live". It is a sharper engineering question: how does an algorithm team start training and validating before the machine has been prototyped and large-scale field collection is done; and once the product is deployed, how does first-hand real-world data enter the existing pipeline instead of requiring a second data system to be built alongside it.

Core value: simulation first, hardware in step

Simulation data and real-world data do not substitute for each other. They play different roles and share one pipeline for scenes, data, evaluation and tuning:

  • Simulation first: generate nominal, boundary and long-tail scene data ahead of time around the machine, its sensors and the business goal, so training and validation move from "waiting for hardware" to running in parallel.
  • Hardware calibration: data from real deployments calibrates sensor characteristics, environmental distributions and model error, which makes the next round of scene and data generation more targeted.
  • Failure feedback: failure cases from customer sites are rebuilt as reproducible experiments, turned into targeted datasets, and the fix is verified against the same evaluation baseline.

Platform capabilities

1. SimReady assets and a digital twin of the machine

The flywheel starts from a reusable digital foundation. The platform manages device models, sensor configurations, environment assets and business scene definitions in one place, so the same machine and the same scenes can be called by data collection, algorithm training, evaluation and customer demos alike.

The existing 3D asset platform continues to serve as that foundation:

  • Ingests OBJ, FBX, GLTF/GLB, STL, PLY, STEP, IFC, OSGB, LAS, 3D Tiles, PCD and other source formats.
  • Handles mesh decimation, level-of-detail generation, texture processing and streaming tiling automatically.
  • Provides online preview, 3D annotation, measurement, sectioning, version comparison and collaborative review.

2. Integrated simulation data production

The platform covers the full path from scene definition to simulation run, data capture, automatic annotation, cleaning and quality inspection, version management and distribution. Algorithm engineers and training agents can request nominal, boundary and failure-condition data against a task goal rather than depending on a one-off manual collection campaign.

Every production task keeps its scene parameters, sensor configuration, engine version, random seed, data labels and quality results, so the provenance of the data is traceable and the experimental conditions are reproducible.

3. Real-world data intake and distribution calibration

Once real-world data enters the platform it uses the same task taxonomy, scene tags and versioning rules as simulation data. Real samples are used to:

  • Calibrate the noise and latency models of cameras, LiDAR, IMU and GPS.
  • Compare simulated against real distributions and identify the environments and operating conditions still to be covered.
  • Link field logs, anomalous frames and failure cases to the corresponding model, software and hardware versions.
  • Turn a field problem into a repeatable simulation experiment through 3D reconstruction or scene parameterisation.

4. Failure-driven targeted generation

The system does more than archive failure cases: it turns them into the next round of data production tasks. An algorithm defect, an evaluation regression or a customer-site problem can trigger scene variant generation, sensor perturbation, fault injection and data capture, producing a targeted training set and regression set.

The fixed model returns to the same evaluation baseline for comparison, which avoids the trap of "the training data was updated, and the validation standard drifted with it".

5. Data governance and engineering collaboration

  • One catalogue and one version scheme: links assets, scenes, datasets, models, experiments and evaluation results.
  • Quality gates: record completeness, annotation quality, distribution coverage and task status.
  • Permissions and audit: access controlled by team, project and data type, with an operation log retained.
  • Agent interface: data production, retrieval, quality inspection, distribution and result analysis are all callable through a structured API.
  • Visual review: inspect 3D assets, samples, annotations and version differences in the browser, which lowers the cost of cross-team communication.

Typical development workflows

A new product programme

Before field collection is complete, build the digital twin and the simulation scenes from the machine design and the target application, and start the first round of training and closed-loop evaluation. Real-world data then calibrates and supplements that work rather than making the algorithm team start from nothing.

A product OTA update

Perception, planning and control can iterate in parallel against one evaluation baseline. Each model or software update is bound to its data version and regression results, which reduces the risk of discovering that modules interfere with each other only at integration time.

A long-tail problem on a customer site

Customer feedback, logs and failure cases are quickly restored into reproducible scenes; targeted fine-tuning data is generated and regression-verified. Firefighting on site becomes an online engineering capability that keeps accumulating.

Deployment and security

The platform can be deployed on a corporate intranet, a local GPU cluster, a private cloud or a hybrid environment. Core data, models and business processes can stay inside an environment the customer controls, and fine-grained permissions, log auditing and version tracking meet long-term engineering and security requirements.

What is ultimately delivered is not a data management product but a development data production line: started by simulation, calibrated by real hardware, and driven onward by its own failure cases.