AI Agents
Building a Coding Agent from Scratch, Part 6: How to Build the Evaluation. Have It Build Something That Runs, Auto-Grade It, and Doubt the Grader Itself
The grader's 10 points for tests were 0/0 for three whole days, invisible in the score table. How we built a rig that has the agent build a web system and grades it by running it, the five-kinds-times-three pass/fail criteria, running the control first, and contracts decided only by the tests.