Validate in Isaac Sim

PREVIEW

Simulation validation is a preview feature. It is off by default and is enabled per user in Tzara Settings.

With simulation enabled, Tzara can go beyond generating an application: it inspects your Isaac Sim scene, runs the application against the simulated cell, checks whether the task actually succeeded, and uses the failures to repair the code and run it again — all in the same conversation, before anything touches real hardware.

How It Works

1. Inspect. Tzara connects to the Telekinesis Isaac Sim bridge and reads the active scene: robots, grippers, cameras, object poses, and bounding boxes.
2. Generate. It writes the application and a validation script in your workspace, using the scene information and your request.
3. Define checks. Before the first run, Tzara decides which measurable checks prove the task succeeded — for example, "part within tolerance of the place pose". These checks stay fixed across repairs.
4. Run. After your approval, the validation script runs the application in simulation and writes a report with the result of each check.
5. Repair. If a check fails, Tzara reads the report and logs, fixes the code, and runs again — up to the run limit (3 runs per message by default).

A run passes only when every required check passes. A script that merely exits without errors is not a pass.

Requirements

• NVIDIA Isaac Sim with the Telekinesis Isaac Sim bridge running and a stage open. See Install Isaac Sim Support.
• A Python environment with the Telekinesis application and device packages installed. Tzara runs the validation script with this interpreter.
• A trusted workspace folder where Tzara saves the application and validation files. It can be a new, empty folder.

Set Up Simulation in Tzara Settings

Open Tzara Settings (gear icon in the Session Manager, or Tzara: Settings).

1. Choose the output folder. Under Output workspace, click Select output folder…, or use the folder open in VS Code. If the window is in restricted mode, click Manage workspace trust and trust the folder.
2. Enable simulation. Turn on Enable scene inspection and simulation.
3. Connect to the bridge. Set Isaac Sim bridge URL (default http://127.0.0.1:8766) and Simulation Python executable. Leave the executable empty to use the interpreter selected in VS Code.
4. Click Save simulation settings.
5. Pick a scene (optional). Under Isaac Sim scene (USD), click Browse… or paste the scene's path on the Isaac Sim machine. Use open scene clears it so the currently open scene is used. Selecting a scene doesn't load it in Isaac Sim; the generated application uses the path.
6. Send a new message. Settings apply to your next request. If a request is waiting, stop it first.

INFO

Saving the settings doesn't test the connection. Tzara checks the bridge when it inspects the scene at the start of a simulation request.

Example Prompt

Create a pick and place application for the robot in the open Isaac Sim scene.
Pick the part from the table and place it in the bin. Validate it in simulation,
check that the part ends up inside the bin, and repair any failures.

Ask explicitly for simulation validation and say what success means. The clearer the expected outcome, the better the checks Tzara writes.

Approving Runs

Before each simulation run, Tzara shows an approval card in the chat (Allow once / Reject), unless you have enabled Automatically approve actions or allowed it with a tool policy. Connecting to the bridge to inspect the scene can also ask for approval; inspection only reads the scene.

For unattended iteration in a simulation-only setup, you can allow the simulation tools in tzara.tools.policy (user settings):

"tzara.tools.policy": [
  { "tool": "inspect_simulation", "action": "allow" },
  { "tool": "run_simulation", "action": "allow" }
]

Runs are not sandboxed

The validation script runs on your machine with the selected interpreter and your user account's permissions. Make sure the application is configured for simulation — not a real robot — before approving a run.

Results and Logs

Each run keeps its results in your workspace under .tzara/simulation/<run-id>/: the run result, the validation report, and the stdout and stderr logs. Tzara uses these to diagnose failures, and you can open them to see exactly what happened.

When the run limit is reached without a pass, Tzara stops and reports the unresolved findings. Send another message to continue — the run limit resets with each new message.

Reset the scene between runs

Tzara doesn't reset Isaac Sim between runs. If a run times out or leaves objects in the wrong place, restore the scene before running again.

VS Code Settings

Setting	Default	Description
tzara.simulation.enabled	false	Enable Isaac Sim inspection and validation tools in trusted workspaces.
tzara.simulation.baseUrl	http://127.0.0.1:8766	Telekinesis Isaac Sim bridge endpoint.
tzara.simulation.pythonPath	""	Python executable used to run validation scripts. Empty uses the interpreter selected in VS Code.
tzara.simulation.scenePath	""	USD scene path on the Isaac Sim machine. Empty uses the currently open scene.
tzara.simulation.maxRuns	3	Maximum simulation runs per message, including repair attempts (1–10).
tzara.sourceRoots	{}	Optional. Map of names to extra local source folders (for example, your own checkout of an application package) that Tzara can read as reference. Read-only; it doesn't install packages.

Limitations

• Checks are written by the agent. A pass means the checks in the report passed. Review the checks to confirm they measure what you care about; simulation is not proof that the real cell will behave the same way.
• One scene at a time. Runs act on the live Isaac Sim scene.
• Real hardware is your decision. Moving from simulation to a physical robot is a separate step that you review and run yourself.