Scenario
A scenario is one made-up caller:
An agent can have at most 10 scenarios. On an empty tab, Suggest 3
scenarios writes three from the agent’s instructions: a typical caller, a
hurried caller who gives details out of order, and a caller with an unusual
request. It takes the fields to collect from the ones the agent already
collects. You can edit or delete the scenarios afterwards.
Running and the result
Run all plays every scenario; the ▶ icon next to a scenario plays only that one. A run usually takes from a few seconds to a minute. The caller has at most 12 turns. A scenario passed when the goal was reached and the agent collected every field on the list. Expand it to see the verdict, each field with its value or “not collected”, and the whole transcript. A result is about the draft that was saved when the run started. After any change in the editor, an old result is marked Result from an earlier version: run the tests again to check the current draft. Without a draft, the published agent is tested. Could not be played means an error, not a verdict, e.g. no API key for the model provider the agent uses.Publishing
When a scenario on the current draft failed or could not be played, Publish asks first whether to publish anyway. Tests never block publishing.What tests do not check
- The voice. The conversation is written: no pronunciation, latency or interruptions.
- Tools. The agent does not really book, send an SMS or transfer the call. In the transcript it writes in square brackets what it would do, e.g. “[booking a visit: tomorrow 10:00, Anna Smith]”, and assumes it worked.
- Transitions between flow stages. The agent gets the global prompt, the greeting and every conversation stage in the order a call walks them.