Budgets and Replay
Budget
mace.spec.Budget holds three independent limits:
Field |
Default |
Limits |
|---|---|---|
|
|
Iterations per run |
|
|
Accumulated LLM cost in US dollars |
|
|
Wall-clock seconds since the checksum check |
Each limit must be positive, and max_iterations and max_wall_s must be integers.
Before each iteration, run_mace_loop() compares elapsed time and accumulated cost with their limits and ends the run with budget_exceeded when either is exceeded.
An iteration already in flight runs to completion.
A run whose max_iterations iterations all end without a pass also ends with budget_exceeded.
The cost total sums every planner, task, and triage call the run has made, which mace.usage records one by one (see Results and Metrics).
Each call’s cost comes from the usage field of its reply: OpenCode, Antigravity, and Vertex replies carry it, while Claude replies do not and count as 0.0.
The post-mortem runs after the last budget check, so its cost counts toward compute_usd but never ends a run.
max_wall_s also bounds every build and simulation.
Each gets the smaller of its default timeout, 7200 s for a build and 3600 s for a run, and the time left in the budget.
A build or simulation that would start after the budget runs out does not start, and its task fails with failure reason budget_exhausted.
Gate-Workload Checksums
mace/workloads/CHECKSUMS holds the SHA-256 digest of each C program in mace/workloads/, in sha256sum format.
run_mace_loop() calls verify_checksums() right after it records the run, before any LLM call or checkout access.
A changed file, a file missing from CHECKSUMS, or a listed file that no longer exists raises ValueError, and the run ends with checksum_mismatch and no iterations.
Run Status
start_run() records each run in the runs table with status running, and finish_run() sets the final value:
Status |
Meaning |
|---|---|
|
Every task in an iteration passed, or one task’s design passed every gate workload |
|
An iteration returned no task results |
|
|
|
A limit ran out before any iteration passed |
|
A C program in |
|
The core descriptor’s core fails the Generic Tile support check |
|
An exception escaped the loop; it is recorded, then re-raised |
Deterministic Replay
integrate_parallel() tags each remote call of a config, workload, or rtl task with mace.replay.tag_for():
f"{run_id}/iter{iteration}/{task_id}/{phase}" # phase: "prompt", "build", "run", or "run:<workload>"
The first gate workload’s run takes the phase run, and each later one run:<workload>.
A prompt phase appears only when LoopOptions.task_prompts is on, which the fast path turns off by default.
run_mace_loop() passes its run ID and iteration number, so the remote calls of every run are tagged.
Tags take effect once a caller enables CHIA’s cache and bypass.
A later call with the same run ID, iteration, task id, and phase then resolves to the entry the original call wrote.
run_mace_loop() draws a new run ID for each run, so running the loop again writes new entries.
enable_caching(cache_dir_path, *, size=8, units="GB", yaml_path=None) starts CHIA’s cache actor, which stores the return value of every tagged call to a function the YAML file marks cache: true.
enable_replay(yaml_path, func_names) registers cache_provider for the named functions; their calls that the YAML file marks bypass: true are then served from the cache without running.
A bypassed call with no cache entry fails with a KeyError naming the missing tag.
The tier-1 test mace/test/cluster/replay_e2e_test.py checks this round trip on a toy CHIA function.
The drivers in the repository call neither function.
Replay reproduces decisions: the LLM’s QueryResult and the build and run artifacts that decide whether each task passed.
CHIA’s cache holds return values only, so replay does not re-apply edits an agent made through its tools.
Planner, triage, and post-mortem calls, an rtl task’s agent call, and unit_test tasks run locally and carry no tag.