Skip to Content
kusaw docsv1.0.0
CLI reference

kusaw bench

Capture a run and replay it as a benchmark

kusaw bench

Capture a run and replay it as a benchmark

Usage

Text
kusaw bench [flags]

Flags

NameShortTypeDefaultDescription
--configstringPath to kusaw.yaml (overrides KUSAW_CONFIG and parent search)
--factorystringfactory name from kusaw list
--help-hboolfalsehelp for bench

Subcommands

kusaw bench add

Capture a finished run as a benchmark task

Text
kusaw bench add <run> [flags]
NameShortTypeDefaultDescription
--checkstringArrayHeld-out check as = (repeatable)
--configstringPath to kusaw.yaml (overrides KUSAW_CONFIG and parent search)
--factorystringfactory name from kusaw list
--help-hboolfalsehelp for add
--namestringBenchmark name (default: the run id)
--originstringunknownTask origin: fresh, private, public, or unknown

Example:

Text
kusaw bench add demo --name demo-bench --origin fresh

kusaw bench add-review

Store a planted-bug review benchmark

Text
kusaw bench add-review <name> [flags]
NameShortTypeDefaultDescription
--basestringBase commit or ref the patch applies to
--bugsstringbugs.yaml with planted bugs
--configstringPath to kusaw.yaml (overrides KUSAW_CONFIG and parent search)
--factorystringfactory name from kusaw list
--help-hboolfalsehelp for add-review
--originstringfreshTask origin: fresh, private, public, or unknown
--patchstringUnified diff of the change under review
--repostringRepo label from kusaw.yaml

Example:

Text
kusaw bench add-review demo --repo demo --base HEAD --patch change.patch --bugs bugs.yaml --origin fresh

kusaw bench cancel

Stop a benchmark batch

Text
kusaw bench cancel <batch> [flags]
NameShortTypeDefaultDescription
--configstringPath to kusaw.yaml (overrides KUSAW_CONFIG and parent search)
--factorystringfactory name from kusaw list
--help-hboolfalsehelp for cancel

Example:

Text
kusaw bench cancel demo-batch

kusaw bench init-recipes

Copy the routing and reviewer-model overlays into the variants directory

Text
kusaw bench init-recipes [flags]
NameShortTypeDefaultDescription
--configstringPath to kusaw.yaml (overrides KUSAW_CONFIG and parent search)
--factorystringfactory name from kusaw list
--help-hboolfalsehelp for init-recipes

Example:

Text
kusaw bench init-recipes

kusaw bench label-finding

Mark a review finding as real or noise

Text
kusaw bench label-finding <batch> <try> <finding-no> real|noise [flags]
NameShortTypeDefaultDescription
--configstringPath to kusaw.yaml (overrides KUSAW_CONFIG and parent search)
--factorystringfactory name from kusaw list
--help-hboolfalsehelp for label-finding
--notestringOptional note stored with the label

Example:

Text
kusaw bench label-finding demo-batch 1 1 real

kusaw bench list

List benchmark tasks

Text
kusaw bench list [flags]
NameShortTypeDefaultDescription
--configstringPath to kusaw.yaml (overrides KUSAW_CONFIG and parent search)
--factorystringfactory name from kusaw list
--help-hboolfalsehelp for list

Example:

Text
kusaw bench list

kusaw bench report

Print a benchmark comparison table

Text
kusaw bench report [batch] [flags]
NameShortTypeDefaultDescription
--configstringPath to kusaw.yaml (overrides KUSAW_CONFIG and parent search)
--factorystringfactory name from kusaw list
--help-hboolfalsehelp for report
--jsonboolfalsePrint JSON
--kint3k for pass@k and pass^k
--markdownboolfalseAlso write docs/benchmarks/-.md
--tasksstringComma-separated benchmark task names
--variantsstringComma-separated variant names

Example:

Text
kusaw bench report demo-batch

kusaw bench run

Queue baseline and variant replays

Text
kusaw bench run <name> [flags]
NameShortTypeDefaultDescription
--budgetfloat6410Batch budget in USD
--configstringPath to kusaw.yaml (overrides KUSAW_CONFIG and parent search)
--factorystringfactory name from kusaw list
--help-hboolfalsehelp for run
--repeatint3Replays of each variant, from 1 to 20
--variantsstringComma-separated variant names, after baseline

Example:

Text
kusaw bench run demo-bench

kusaw bench run-review

Replay a review step against planted bugs

Text
kusaw bench run-review <name> [flags]
NameShortTypeDefaultDescription
--budgetfloat645Batch budget in USD
--configstringPath to kusaw.yaml (overrides KUSAW_CONFIG and parent search)
--count-infoboolfalseCount severity info findings as claimed bugs
--factorystringfactory name from kusaw list
--help-hboolfalsehelp for run-review
--matchstringCustom matcher command (replaces the location matcher)
--repeatint3Replays of each variant, from 1 to 20
--stepstringReview step id to replay
--variantsstringComma-separated variant names, after baseline

Example:

Text
kusaw bench run-review demo --step review-standards --repeat 3 --budget 5.00

kusaw bench set-origin

Set a benchmark task's origin

Text
kusaw bench set-origin <name> <origin> [flags]
NameShortTypeDefaultDescription
--configstringPath to kusaw.yaml (overrides KUSAW_CONFIG and parent search)
--factorystringfactory name from kusaw list
--help-hboolfalsehelp for set-origin

Example:

Text
kusaw bench set-origin demo-bench fresh

kusaw bench show

Show one benchmark task and its batches

Text
kusaw bench show <name> [flags]
NameShortTypeDefaultDescription
--configstringPath to kusaw.yaml (overrides KUSAW_CONFIG and parent search)
--factorystringfactory name from kusaw list
--help-hboolfalsehelp for show

Example:

Text
kusaw bench show demo-bench

Exit codes

CodeNameDescription
0okSuccess.
1errorRuntime error.
2usageUsage or validation error.
3rejectedControl request rejected; the error string is printed verbatim.
4not picked upControl request not picked up within the wait time while serve runs.

Example

Text
kusaw bench list

kusaw atlas

Build and inspect the local code index

kusaw claude-args

Print or run the Claude Code argv the built-ins use

On this page

kusaw benchUsageFlagsSubcommandskusaw bench addkusaw bench add-reviewkusaw bench cancelkusaw bench init-recipeskusaw bench label-findingkusaw bench listkusaw bench reportkusaw bench runkusaw bench run-reviewkusaw bench set-originkusaw bench showExit codesExample