CANONIC

RL environments for creative agents

What a lab would receive

A bounded, versioned creative-production workflow an agent can be trained and evaluated in, with the reward and the constraint costs reported separately and never blended. One phase of that is built and tested. The rest is a plan, and this page marks which is which on every claim.

Read the tags first

Nothing below is described without one of these four words. They are the same four the internal blueprint uses, so the page and the plan cannot drift.

Implemented v0
Code exists in this repository and has automated tests.
Planned
A required capability, not yet implemented.
Spike-gated
A time-boxed feasibility test decides whether it is adopted at all.
Deferred
Explicitly out of the first release.

What is not true yet

This block exists because every other sentence on this page would be easier to read generously without it. These are the five things a reader would reasonably assume from a page titled “RL environments”, and none of them are true today.

  1. No agent has ever run in it.

    Not one. There are no recorded episode traces anywhere in this project. The runs/ records are generation and judgement calls from the benchmark sweep — a different object entirely. Every number on the rest of this site comes from that sweep, not from this environment.

  2. It does not grade creative quality.

    The entire required-beats reward is beat.casefold() in text.casefold() over the concatenated text of every action. An agent that pastes the beat strings verbatim scores 1.0. There is no judge, no rubric, and no quality signal of any kind in this package.

  3. Five of the seven tool verbs are inert.

    Only update_story_state and submit have handlers. The other five are a validated, closed vocabulary — the payload rules are enforced and the calls are recorded — but nothing happens when they are called. No Show Bible is wired in, so query_bible has nothing to query.

  4. The separate verifiers export scores nothing.

    sitcom_v1 packages the eight sitcom rubric dimensions at their published weights and its entry point resolves from a clean install. But all eight reward methods return 0.0 by design — a consuming harness is expected to supply the judge — and load() currently returns zero tasks, because the corpus export it reads has not been written.

  5. The stored seed does nothing.

    It is recorded in the trace and honoured on replay, but nothing in the core consumes it. Determinism here is trivially true: there is no stochasticity to control. That will stop being trivial the moment a judge or a simulator enters, which is exactly when the claim would start to matter — so it is worth marking as unearned now.

The deterministic core Implemented v0

canonic-sitcom-room is a standalone Python package whose only runtime dependency is pydantic. It builds as a wheel independently of this repository, and its source is free of network and model-provider dependencies — which is what makes reset, state transitions, and replay deterministic rather than merely usually reproducible.

The lifecycle

frozen Scenario
      │
      ▼
room.reset(seed) ─────────────► SanitizedState
      │
      ▼
CreativeAction ─► room.step() ─► StepResult
      │                             │
      │              reward and cost, never merged
      ▼
submit ─► EpisodeOutcome ─► EpisodeTrace + EnvironmentManifest
                                  │
                                  ▼
                  replay only against the same scenario hash

An episode must be reset before it accepts actions. A successful submit terminates it; a later action raises. Replay refuses a trace whose scenario version, id, or content hash differs from the scenario it is handed — it does not warn and continue.

The tool vocabulary, and what each verb actually does

Seven verbs in a closed enum. The Behaviour today column is the honest one: five of them are vocabulary with no handler behind it.
Verb Handler Behaviour today
update_story_state yes Mutates a declared character's field. An unknown character is a cost, not a silent write.
submit yes Terminates the episode and produces the outcome. A later action raises.
inspect_brief none Accepted, validated, recorded in the trace. No handler.
query_bible none Requires a query string. No Show Bible is wired in; nothing is returned.
plan_episode none Requires text. The text is recorded and counts toward the beat scan.
draft_scene none Requires text. Recorded; no draft state is kept.
revise_scene none Requires text. No revision lineage is tracked.

Reward and cost are separate vectors

A capability reward and a constraint cost are reported as two vectors and are never combined into one number. That is not a stylistic preference: a blended figure lets a high quality score hide a gamed or unsafe path, and lets a constraint violation be laundered into a slightly weaker creative score. Both directions destroy the signal you were training on.

RewardVector
required_beats — fraction of beat strings found, by case-insensitive substring match. submitted — 1.0 on submission.
CostVector
invalid_state_mutation — an update naming a character the scenario never declared. action_budget_overrun — one per action past the budget. The budget overrun does not terminate the episode, so the failure stays visible in the outcome instead of disappearing into a truncated run.

What the tests prove Implemented v0

  • Seeded reset and replay produce identical step results.
  • An update naming an undeclared character increments a cost and changes no state.
  • A step after submission raises rather than silently extending the episode.
  • An exported trace replays against its manifest and reproduces the same outcome.

Four contract tests. That is the whole of what is verified today.

One episode, end to end

This is not a demo. It is the fixture from the contract test tests/environment/test_writers_room.py, reproduced exactly, and it is worth reading closely for a reason given after it.

office-fire-drill-v1 · sitcom-room-v1 · action budget 5

A fire drill becomes a referendum on who is in charge.

Required beats
  • Michael asserts authority
  • Dwight escalates the drill
Cast
Michael (manager) · Dwight (salesman)
  1. 1 plan_episode Michael asserts authority.
  2. 2 draft_scene Dwight escalates the drill.
  3. 3 submit Final scene.

outcome — required_beats 1.0 · submitted 1.0 · cost 0.0

Read the plan step again. It says “Michael asserts authority.”, which is the first required beat, spelled the same way. The draft step is the second beat. That is why this episode scores a perfect 1.0 — not because the writing is good, but because the beat strings appear in the text. Nothing in this environment can currently tell that apart from good writing. Publishing the walkthrough without that sentence would be the overclaim.

Distribution

Private wheel Implemented v0
Builds and runs locally with no network access. Pinned version, pydantic ≥ 2.9 the only runtime requirement.
verifiers export Implemented v0
A packaging surface, not a second implementation. Verified on 2026-08-07: the package builds to a wheel, installs into a clean virtual environment, and its entry point resolves. Rubric weights survive the export rather than being flattened. It loads zero tasks today — see the fourth denial above.
OpenEnv adapter Spike-gated
A Gym-style, container-first interoperability target whose own documentation marks it experimental. Adopted only if one representative trajectory keeps reset/replay determinism, state isolation, manifest identity, and full reward and cost visibility without duplicating core logic. Otherwise dropped.
Remote server, containers, concurrency Planned
After the local core and the adapter spike earn it. The first environment has no arbitrary-code action space, so sandboxing infrastructure is not on the critical path.
Hosted inference, rollout fleet, multi-agent writer/critic Deferred
Out of the first release by decision. The moat is environment quality and trustworthy measurement, not commodity orchestration.

The road to a pilot

Each phase names the gate that has to pass before the next one starts. Seven of the eight are unstarted.

Delivery phases with their status and exit gate. Only phase 0 has shipped.
Phase Status Deliverable Exit gate
0. Deterministic core Implemented v0 Package, strict contracts, reset/step, reward and cost vectors, trace and replay. Tests pass; standalone wheel builds.
1. Compatibility spikes Planned A representative multi-step verifiers adapter; optional OpenEnv wrapper. Written keep/fallback decision; an adapter may not omit cost or provenance.
2. Scenario simulator Planned 40–60 frozen interactive scenarios, compiler, backed state, revision lineage, scenario manifest. Same seed and trace yields equivalent state and outcomes twice.
3. Graders and probes Planned Outcome graders, pinned quality-judge integration, hidden interactive probes, evaluation card. Valid traces pass; each adversarial fixture trips a low score or a visible cost.
4. Lab execution kit Planned Private package, rollout client, trajectory export, replay CLI, batch evaluation and resume. A lab can install, run five seeded scenarios, export, and replay without repository access.
5. Frozen evaluation and baselines Planned Held-out suite, three baselines, intervals, failure report, expert review sample. Reproducible report with no train/eval overlap and no cross-domain creativity claim.
6. Reward-robustness proof Planned A bounded 1–3B open-model run before any 7–8B run, on a pre-registered budget. Held-out improvement while costs and probes stay acceptable.
7. Lab pilot Planned Release package, reward contract, validation report, commercial and legal terms. A customer completes a one-to-two-week evaluation or training cycle.

What ships with a pilot Planned

A pilot delivers six artifacts. None of them exist as a customer-ready package today; they are listed so the shape of the deal is legible rather than to suggest it is on a shelf.

  1. Pinned environment package — a version, a wheel, a release manifest.
  2. Scenario manifest — every scenario id, version, and content hash in the release.
  3. Reward contract — what each reward and cost channel measures, and what it does not.
  4. Evaluation card — protocol, exclusions, uncertainty, and known failure modes.
  5. Baseline report — reference results with intervals, run on the exact release.
  6. Usage guide — install, roll out, export traces, replay.

One deliberate absence: there is no cross-domain creativity score in that package and there will not be one. A number that averages a sitcom scene against a book opening reads as authoritative and means nothing — the same reason this benchmark publishes six boards and refuses to add them up.

If you are building here

The useful conversation right now is about which creative workflow is worth instrumenting first, and what your post-training loop would need to consume. If that is a conversation you want, the benchmark on the rest of this site is the evidence that we can build the measurement half of it.

See the sweep · See what it gets wrong