JSON output
Use marginci test ... --json to consume a comparison in scripts or CI. It writes one JSON document to stdout followed by a newline, including on errors. Application output is saved in attempt logs; --verbose progress uses stderr. The human verdict, JSON verdict and process exit code agree.
Reports use schema version 2, scope declared_workload. Consumers should accept additional fields and use reason codes rather than parsing human-readable messages.
Completed reports
| Field | Meaning |
|---|---|
schema_version, scope |
2, declared_workload |
run_id, operation |
Run identifier and configured operation name |
verdict, exit_code |
Lowercase pass/fail/inconclusive/error; exit codes 0/1/2/3 |
baseline_revision |
Immutable baseline commit |
candidate_revision |
HEAD plus working-content hash; use comparison_identity.candidate_commit for the commit and candidate_snapshot_sha256 for the snapshot digest |
comparison_identity |
Effective config SHA-256, commits, snapshot digest/dirty flag, resolved commands, comparison-file hashes, success protocol and environment policy |
baseline, candidate |
Costs, attempts and success counts for each revision, described below |
measured_regression |
Fraction computed from unrounded costs; 0.20 means a 20% increase. null if undefined or incomplete |
max_regression |
Configured threshold as a fraction |
providers |
Provider cost breakdown |
coverage |
complete, declared_cases, completed_pairs and selected case_ids |
comparison |
Comparison details; method is observed_cost_per_success |
reasons |
Objects with stable code and human-readable detail |
artifacts |
Local run-directory path |
pairs |
Completed test case pairs and their attempts |
Each side reports operations, attempts, successful_operations, known_economic_value_usd, known_economic_value_usd_decimal, total_economic_value_usd, cost_per_success, unit_cost_status and provider/provenance totals. The economic-value fields contain calculated cost from configured rates or supported provider estimates; they are not invoice charges. known_economic_value_usd is a subtotal when measurement is incomplete.
An operation with complete measurements can fail its success check and still contribute cost. Complete coverage alone does not establish a defined comparison: both sides also need successes and the baseline cost must be positive. Unknown or nonfinite results use null, never NaN, Infinity or an invented zero. Partial or unpaired attempts remain in run files and side subtotals.
Handle reasons and errors
Common reason codes include OPERATION_TIMEOUT, OPERATION_CANCELLED, INCONCLUSIVE_BUDGET_LIMIT, INCOMPLETE_MEASUREMENT, NO_SUCCESSFUL_OPERATIONS, ZERO_BASELINE_VALUE, UNREPRESENTABLE_RATIO and INCONCLUSIVE_PROVIDER_USAGE. Use Cost calculation and results to decide what to fix before rerunning.
ERROR uses a smaller report: schema_version, scope, verdict, exit_code and reasons. Revisions or costs are not invented when setup prevents measurement.
init, doctor, help and version with --json also emit one document, but use their own response structures. They are not economic reports. A successful doctor does not install dependencies, make paid calls or produce PASS.
Saved files
Stable file names are report.json, config.json, revisions.json, comparison-identity.json and source-files.json. The attempt and request-record formats may gain fields. Keep run files private and review them before sharing.