Claim Safety

This is part 5 of the Agent production system series. Previous: From Xbox SLAs to agent fleets. Words I use below Agent — a program that writes words or code for you, in a chat Claim — a sentence that says a number, a job title, or who made a thing (true or false) Evidence — the proof you can point at (a file, a date, a source) Metric — a number inside a claim Allowlist — the short list of numbers I may print in public; if it is not on the list, it does not ship Authorship — who made a thing (running someone else’s tool is not the same as making it) Gate — a check that can say no (a wish in the instructions is not a gate) Those seven words are the whole toolkit. Now the point. ...

This is part 1 of the Agent production system series. Start with the system map if you haven’t read it yet. On the interactive diagram: Eval gates. Demo agents look smart until they touch a real repo. In a demo, the agent writes the code, the code runs once, everyone claps. Nobody checks what happens the second time, or the tenth, or the time the agent decides the fastest way past a failing test is to delete the test. Then you learn the hard lesson: intelligence without a gate is just a faster way to ship a bad change. ...