Data Engine
SUMMARY
The Data Engine is the data infrastructure of the Telekinesis Agentic OS. It standardizes how data is represented, generated, collected, and transformed into training-ready datasets, creating the continuous learning loop behind every Skill and Agent.
Data Types
Strongly typed contracts for point clouds, bounding boxes, frames, meshes and more; shared across all Skills and Agents.
Explore →
Synthetic Data
Photorealistic training datasets generated with Extreme Domain Randomization (EDR) for robust sim-to-real transfer. Free on Kaggle.
Explore →What is the Data Engine?
- Synthetic data seeds the first Skills.
- Skills are trained on the data.
- Trained Skills deploy and execute on real robots.
- Every run is captured as new data — which feeds back into training.
The Data Engine is the data infrastructure layer of the Telekinesis Agentic OS. It provides the end-to-end infrastructure to generate Synthetic Data for training and deploying Skills, and continuously collecting data to improve those Skills.
This forms a closed loop. Synthetic data trains the first Skills, those Skills deploy to real robots, and every deployment is captured as new data that retrains them. The loop compounds: every deployment generates new data, every new dataset improves Skills, and better Skills unlock more applications — which generate even more data.
Underpinning the loop are standardized data types: fixed contracts that let Skills, Agents, and models exchange data without translation.
1. Standardized Data Types as Contracts
Every Skill is a strongly typed function — outputs = skill(inputs) — where each input and output is a Datatype: point clouds, bounding boxes, frames, meshes, transforms, trajectories. The Data Engine defines these Datatypes, and they are the contracts the rest of the OS is built on. One Skill's output is another Skill's input; an Agent composes Skills into code without writing conversion glue; and models train on the same schema they run on.
Because every signal is typed, the Data Engine can also fix its shape. A robot produces asynchronous, event-driven data — sensors, cameras, and control loops firing at different rates. Learning systems need the opposite: synchronous, aligned, fixed-size batches. The Data Engine fuses those raw streams into batched, tabular datasets that are ready to train on, with no per-project logging or alignment code.
2. Synthetic Data Generation
Physical AI has a cold-start problem: a new Skill needs training data, but with no deployment yet, there is nothing to produce it. The Data Engine breaks the deadlock by generating the data itself — it bootstraps new Skills before any real-world deployment exists.
It produces photorealistic, perfectly labeled synthetic datasets at scale, with no manual collection or annotation. Extreme Domain Randomization (EDR) aggressively varies lighting, texture, geometry, material, and pose, forcing models to learn what is invariant about a task so they transfer cleanly from simulation to real hardware.
The result: a Skill arrives at its first deployment already capable, not blank. Synthetic data seeds the loop on day zero and covers the long tail of rare or unsafe cases that are impractical to capture on physical robots.
All datasets are free on Kaggle. Browse Synthetic Datasets →
3. Skill Training & Deployment
Because the data is already contract-shaped and batch-ready, it flows straight into training. Iris, the Telekinesis model-training module, trains and fine-tunes models on synthetic and collected data and packages them as Skills. From there they behave like any other Skill: called directly, or composed by a Physical AI Agent under the Code-as-Policy paradigm.
The same code runs in simulation and on real robots, so trained Skills deploy to production — manufacturing, logistics, aerospace — as Industrial Applications. Deployment is where the loop starts producing both work and data.
4. Continuous Data Collection
Every run in production is captured. As Skills execute on real hardware, the Data Engine records each one in the same Datatypes the models were trained on — typed inputs and outputs, sensor streams, transforms, trajectories, and outcomes — with no separate collection stack. Replay failures against successful runs to debug, then feed the captured data back into training.
That closes the loop: synthetic data → training → deployment → collection → better Skills.
Why It Matters
The Data Engine is the data infrastructure layer of the Telekinesis Agentic OS — the memory shared by every Skill and Agent, the backbone for composing them, and the training-ready pipeline behind the models they run.
Because it closes the loop, the system improves with use: every deployment produces contract-shaped data, that data trains better Skills, and better Skills earn more deployments. The robots you ship today generate the data that improves the robots you ship next.