Checking your agent¶
Preview
Framework v2 is not yet available to all accounts.
Five checks tell you whether an agent will behave as its Brief says, before a customer finds out. They run in this order, from cheapest to most thorough, and all of them are visible in the console.
| Check | Question it answers | When it runs |
|---|---|---|
| Readiness | Is everything set up? | Continuously; shown on the Capabilities page and the readiness card. |
| Lint | Is the Brief consistent with itself and with the capabilities? | On every save. |
| Preview | What will the agent read in a given Playbook on a given channel? | On demand. |
| Test conversations | Does it behave the way I expect when I talk to it? | On demand, in the emulator or the tester app. |
| Evaluations | Does it still pass the conversations I care about, after this change? | On demand and before going live. |
Readiness¶
Readiness answers "is it set up?" for each capability, from one check
that the Capabilities page, the readiness card, and the runtime all
share. A capability is Ready, Needs attention (it works but
something is missing, such as no representatives) or Off. A Playbook
is ready when every capability in its uses is ready.
For Mia, the readiness card is empty except when the Glow front desk's Slack connection expires; then Notifications shows Needs attention with a Reconnect action, and Talk to a person is listed as not enterable until it is fixed.
Lint¶
Lint reads the Brief and reports anything inconsistent, every time you save. Each finding has a severity, a plain message, the place in the Brief it points at, and where possible a one-click fix. Errors block the save; warnings and notes are shown beside it.
| Finding | Severity | Example |
|---|---|---|
| A Playbook uses a capability that does not exist | Error | uses: [calender] |
| A message template is referenced but is not in Content | Error | The booking Flow sends Booking confirmed and Content has Booking confirmation |
| No default Playbook, or two | Error | |
| A Playbook talks about a capability it does not use | Warning | "This Playbook talks about booking but does not use Scheduling" |
| A Playbook declares no capabilities at all | Note | "Book a consultation call draws on Scheduling and People but declares none", with a one-click Declare fix that saves the suggested list |
| A Playbook uses a capability its body never draws on | Note | "knowledge is declared but nothing here refers to it" |
| A template placeholder does not match a People field | Warning | {person.treatment} when the field is treatmentInterest |
| A Flow's path no longer matches its Playbook body | Warning | "Body changed since the path was generated. Regenerate?" |
The consistency findings are the ones that catch real mistakes. When
Mia's Qualify body was edited to say "offer to book a call" without
adding Scheduling to uses, lint flagged it on save; without that,
Mia would have talked about booking in a Playbook where she could not
book. The Copilot can run a closer reading of the same rules on request
and returns findings in the same list.
Lint runs on the Definition page, on import, when a template is applied,
and through the API (GET …/lint, see the
Agent API) and the Anychat MCP
server, so a partner keeping Briefs in version control can run it in
their own checks. The same rules run on the Generic runtime, where the
findings about uses and Flows are informational.
Preview what the agent sees¶
Preview what the agent sees, on the Definition page, shows the agent's complete reading for a Playbook and channel you choose: the Brief with that Playbook's body in full and the others by name and when-to-use, each used capability's "what this adds" text, and the channel line. It is the same assembly a real turn does, minus the conversation. Use it to confirm the one promise for yourself: what is on the page is what the agent reads.
Pick Answer treatment questions on SMS and you see Knowledge's text, no Scheduling text (that Playbook does not use it), and the SMS line that says plain text and a short length.
Test conversations¶
Talk to the agent yourself. The console's emulator shows the conversation the way RCS, SMS, or another channel would render it, and the tester app lets you invite colleagues to try the agent on their own phones before launch. Both are marked as test traffic and stay out of your reporting. Every test turn keeps a full turn record, so when the agent does something odd you open the record instead of guessing.
Evaluations¶
An evaluation is an example conversation with checks: what the person says, turn by turn, and what must be true afterwards. "After the person picks a time and confirms, a call is booked and the confirmation template was sent." "When the person asks for medical advice, the reply contains no diagnosis and offers a consultation."
Evaluations attach to the thing they test: a Playbook, a Flow, or an Objective. Templates ship with theirs, so Mia arrived with booking and opt-in evaluations already attached, and you add your own as you learn what matters for Glow. Run them from the console before going live and after any change to the Brief; readiness for launch is configuration ready and evaluations passing. Because they run against the turn record, an evaluation can check things a transcript cannot show: that a Policy fired, that a template went out verbatim, that a Playbook was chosen for the right reason.
Evaluations also let you compare settings you would otherwise guess at. Run the same set against a faster, cheaper model and a stronger one and you can see whether the difference is worth paying for.
What to check when¶
Before launch: readiness clean, lint clean, evaluations passing, and one person on your side has had a real test conversation on the channel you are launching on. After a Brief edit: lint (automatic) and the evaluations for the Playbook you touched. After a capability change: readiness, then the evaluations for every Playbook that uses it.