MACE Evaluation

Contents:

• Methodology
  • Approaches
  • Workloads
  • Time basis
  • Environment
• Results
  • One-shot baseline
  • MACE loop
  • What made multi-tile meshes pass
  • Coverage
  • Recovery
• Cost
  • Total
  • Per run
  • Pricing assumptions
  • Measuring tokens
• Evaluation Harness
  • Task suite
  • Methods
  • Batch runs
  • Seeded faults
  • Plan-check replay
  • Reverted fixes
  • Source faults
  • Co-design searches
  • Reports
• Directed-Test Evaluation
  • The mutation check
  • Methods
  • The held-out bugs
  • Running it
• RTL-Parameter Searches
  • The program
  • The finish time
  • The parameters
  • How an edit works
  • The storage budget
  • Methods
  • Trades
  • The verdict
  • Counting simulations
  • Running a batch
  • What a run records