Build & Evals

Playground

The Playground is one place to test and certify an agent or skill — edit its definition, run it before you commit the change, grade it against its evals, and check the gates that decide whether it can ship.

Before the Playground, testing was scattered: a prompt bench on the agent page, another in the designer, eval authoring on its own route, a sandbox bench on the skill page — and none of them could answer the question a builder actually has: is this change safe to ship? The Playground consolidates all of it and adds the missing piece — running an edit before it is published.

The Playground: edit the definition, then Test, Evals and Certify.

Drafts: test an edit before you commit it

Editing a definition in the Playground saves a draft. A draft is a real, unpublished version of the agent — which is what lets you actually run it. Runs are pinned to a specific version, so an "in-memory-only" edit could never be executed; making the draft a version is what keeps testing on the one governed execution path rather than a second one.

A draft is invisible to everything but you It is never the agent's current version, so nothing routes to it. It does not appear in version history, diffs, or rollback. Each builder gets their own draft of a given agent, so two people editing the same agent never overwrite each other's work.

The four panels

Panel	What it does
Definition	Edit the Agent.md in a full editor. Save draft validates and stores the change; Commit version publishes it.
Test	Run a prompt against the draft (or the published version) and watch the answer, tool calls, and cost stream in — the same runtime chat uses.
Evals	Grade the agent's eval suite against the draft or the published version. The same suite the promotion gate runs against a candidate.
Certify	A read-out of every gate that will decide the promotion: definition validates, security scan, eval coverage floors, the eval run for this version, and ring.

A Draft / Published toggle chooses which definition the Test, Evals, and Certify panels act on. The default is your draft the moment one exists — so "test my edit" always means the edit, never the version everyone else is using.

Certification reads the gates — it never invents a score

Every row in the Certify panel is the same check the promotion pipeline enforces: the security scan verdict, the eval coverage floors (a write-capable agent needs refusal and red-team cases), and the eval run for this exact version. Because it reads those gates rather than computing its own, a green Playground and a rejected promotion cannot disagree.

Honest "pending", not a false green An unscanned, ungraded draft reports pending — not certified. A gate you have not run is never quietly counted as passed.

Committing a draft

Commit version publishes the draft through the normal version path — the security scan, skill and MCP resolution, and the eval-regression check all run, exactly as they would for any new version. The draft then steps aside, but is retired, not deleted: the test runs and eval results that justified the commit still point at it, and that evidence must remain readable after you act on it.

Skills in the Playground

A skill can be selected too. Its entrypoints run in the sandbox from the Test panel, and its definition is shown for reference. Editing a skill's bundle in place — the draft story above, applied to skills — is a planned follow-up.