Over the past year, letting Claude Code or Codex write game code has gone from a novelty to a habit. Refactors, whole systems, shader and gameplay tweaks — an agent turns out diffs at a speed no human matches. The question is no longer whether the diff works. It’s whether it’s still fast.

The bottleneck when AI writes the code

When writing code gets cheap, the bottleneck moves from writing to verifying. Whether a feature works is something tests answer on their own. Whether that code just pushed real-device frame time up by a few milliseconds is not something a green test suite will tell you.

Once an agent is landing dozens of diffs a day, hand-profiling each one stops being realistic. An unnoticed allocation creeps onto the render thread; an update loop quietly gets more expensive; the GC starts running more often. Each of these passes its unit tests, slips past a visual code review, and only shows up as felt degradation once it’s on a real device.

The speed you gained on writing with an agent turns verifying into the new bottleneck.

So you can’t leave performance checking as a step that depends on a reviewer’s alertness or a veteran’s gut. It has to become a machine-side verdict that runs at the same speed as the agent.

Put the regression gate on the machine side

The idea is plain. Collect performance telemetry for each build, compare it against the last known-good build, and fail CI when it has regressed past a threshold. Instead of a human staring at numbers and deciding, a machine judges whether a regression exists and returns it as a pass/fail on the pipeline.

Framedash’s framedash perf-diff reports both P50 and P95 for two builds’ frame time, memory, and GPU time. Add --fail-on-regression and it exits 1 when the candidate build’s P50 regresses past the threshold. The gate decides on the P50 comparison; P95 is reported alongside as context, not gated on.

framedash perf-diff --baseline "$BASE_SHA" --candidate "$GITHUB_SHA" \
  --threshold 5 --fail-on-regression

Beyond frame time, memory, and GPU time, it also compares map load time (load_time_ms) and disk I/O (io.*), all treated as lower-is-better metrics. You can narrow the gate to a single metric with --metric, or limit the comparison scope with --map and --platform.

The pivot here is the build_id that serves as the unit of comparison. Unless it’s unambiguous which build a number belongs to, comparing builds against each other doesn’t hold together in the first place.

The workflow end to end

Here’s how the code an agent wrote makes its way through the regression gate.

Agent edits code Profile test CI regression gate

When the gate fails, the agent itself reads the telemetry to narrow down the cause.

The command that runs this whole flow in CI is framedash run-profile-test. It writes out the FRAMEDASH_* automated-session variables, launches the profiling build you point it at, waits for that telemetry to ingest, then runs the perf-diff gate against a baseline.

framedash run-profile-test \
  --command "./Build/Game.exe -nullrhi -ExecCmds='Automation RunTest Perf'" \
  --scenario nightly --api-key-file ci-read.key \
  --baseline "$BASE_SHA" --threshold 5 --fail-on-regression

On the game side, you call the automated-session API once in your automated-test entry point. When the SDK calls BeginAutomatedSessionFromEnvironment(), it reads the FRAMEDASH_BUILD_ID / FRAMEDASH_GIT_BRANCH / FRAMEDASH_GIT_COMMIT / FRAMEDASH_TEST_SCENARIO variables that run-profile-test wrote out, and from then on every event carries the CI build along with its branch, commit, and scenario. No per-event tagging code.

TelemetrySDK.Instance.BeginAutomatedSessionFromEnvironment();
// ... run the profiling scenarios ...
TelemetrySDK.Instance.EndAutomatedSession();

One operational detail: a short or headless profiling run can exit before the SDK’s periodic flush. Let the run stay alive until its telemetry is actually sent (each SDK’s CI and headless guide covers how); run-profile-test then waits for that telemetry to ingest before it gates.

The build_id is recorded as a top-level field; branch, commit, and scenario ride the ci.branch / ci.commit / ci.scenario attributes. That makes the build_id from the agent’s commit directly the candidate for a regression comparison.

One thing to keep straight is the keys. The gate reads telemetry with an analytics:read key, while the launched game sends telemetry with a separate events:write ingest key. To avoid a name clash, pass the gate key with --api-key-file rather than the FRAMEDASH_API_KEY environment variable, so FRAMEDASH_API_KEY stays free as the game’s ingest key.

When the gate fails, where to look

When perf-diff fails, what you learn is which metric regressed and by how much. Where it regressed isn’t in that number.

That’s what the performance heatmap is for. It overlays FPS, frame time, GPU time, and memory usage cell by cell on your game map. You can filter by device or build profile, so you can pin down which area of which map got heavier in the regressed build, as a location on the map.

The spatial view is drawn from position-qualified events. When your profiling scenarios report player position with a registered map ID, the SDK attaches camera orientation (yaw and pitch) to those events automatically. With that in place, you can chase something as specific as “P95 spikes in one corner of desert_ruins on build 1042, looking north.” The regression becomes a reproducible investigation target.

Let the agent read the telemetry itself

There’s still one human left in the flow above: opening the heatmap, narrowing the cause, and writing the fix.

You can hand that step to the agent too. The Framedash MCP server exposes read-only tools and resources over your telemetry to an LLM. Heatmap grid data, dashboard KPIs, even raw SQL — the agent can pull them directly from a natural-language instruction.

claude mcp add framedash \
  -e FRAMEDASH_API_KEY=fd_xxx \
  -e FRAMEDASH_PROJECT_ID=your-project-uuid \
  -- npx -y @framedash/mcp-server

Now you can close the loop by letting the agent that caused the regression read the very spot it broke and fix it. It reads the spatial overview with get_heatmap, isolates the candidate build’s hotspots with a query filtered by build_id, forms a hypothesis, and edits the code. The tools are read-only, so the agent can’t alter or delete your telemetry. Whether the fix actually cleared the regression is judged again, objectively, by the same CI gate on the next build.

CI regression gatePASS / FAILA machine judges whether it regressed build over build

Heatmap and MCPCAUSEThe agent reads where it regressed

The verdict goes to the machine, reading the cause goes to the agent. What’s left for a human is where to draw the threshold, and a final look at whether the fix is sound.

Getting started

All you need is to add the SDK to your game and pass a build_id from CI, then add one gate to your pipeline. SDK integration is a few lines, and the supported engines are Unity, UE5, and Godot (C#).

The concrete SDK and CI setup is laid out in CI-Integrated Profiling.
The full command list is in the CLI Reference.
To put it in an agent’s hands, the Claude Code plugin installs the MCP server and skills in one go.

claude plugin marketplace add crane-valley/framedash-claude-plugin
claude plugin install framedash@framedash

Match your performance checks to your agent's speed Supported engines are Unity, UE5, and Godot (C#). You can start without a credit card.

Start for free UnityUE5Godot (C#)