New provider: \<CLI name\>
Public docs: \<url\>
Public docs: <url>
Someone evaluating a CLI fills this in. Status is yes, no, or partly. Write the flag or event that proves each item, and a link to that CLI's public docs. Do not write account names, keys, or model ids that look real.
Copy the table. Leave status blank until you check the CLI.
| # | Must or nice | Status | Flag or event | Public docs |
|---|---|---|---|---|
| P1 Non-interactive mode that runs one task and exits | must | |||
| P2 Prompt from stdin, or from a file, not only argv | must | |||
| P3 Line-delimited JSON events while it works, with a final result event | must | |||
| P4 Honest exit codes, and a final result that says success or error | must | |||
| P5 Tool permission control matching read_only, edit, and interview, without skipping checks | must | |||
| P6 Model selection by flag | must | |||
| P7 Auth from its own login or an environment variable, with a clear "not signed in" error | must | |||
| P8 Usage in the final result (input, output, cache tokens); per-call usage for context size | nice | |||
| P9 A session id, and resume by id | nice | |||
| P10 Control of which MCP servers and skills load, and a way to list them | nice | |||
| P11 A way to turn off its own sandbox when the factory sandbox wraps it, and documented state folders | nice | |||
| P12 --help and --version that Probe can read without a model call | must | |||
| P13 Ends cleanly on SIGTERM (Unix) or Job Object close (Windows) | must | |||
| P14 Doesn't send the prompt to a cloud agent or remote worker by default | must |
Verdict
- Every must item is
yes→ can be a built-in provider. Add a runner package, a fake, a flags file, and pass the conformance suite. - Any must item is
no(orpartly) → use a custom command step. See Using a CLI that isn't a built-in provider.
Filled examples for Claude Code and Cursor are in the checklist appendix.
Adding a provider
Use this checklist to judge whether a third agent CLI can become a built-in provider. Fill in the [template](new-provider-template.md). Public material only: do not write account names or keys.
Provenance
Every step attempt stores what it ran on: the provider, CLI version, model, step and protocol versions, and hashes of the prompt, request, instructions, skills, MCP servers, config, and env names. Only hashes and names are stored. Env values, secret values, prompt text, and file contents are not.