A team called me in because their agent kept "hallucinating CRM data". That was the description. The reality was more boring.
The agent had six tools. One of them, get_contact, hit an internal API. The API returned 200 OK with a body of {} whenever the contact ID didn't exist. No error, no null, just an empty object. The agent read that as "this contact exists and has no fields", filled in the blanks from the conversation context, and wrote the result to the CRM.
The model was doing exactly what it should. It was handed a tool that lied, and it believed it.
Make silence impossible
When I audit a tool layer, the first thing I look for is what happens on the boring failure paths: not found, unauthorised, rate limited, timed out, returned the wrong shape. If any of those come back as a successful empty result, the agent has no way to distinguish "nothing there" from "I got nothing useful".
The fix in that codebase was about twenty lines. Every tool response got an explicit envelope:
{
"status": "not_found",
"message": "No contact with id 4821",
"retryable": false
}
status is required. The wrapper throws if a tool returns anything else, which means a bad tool fails at the boundary rather than three steps later inside the model's reasoning. Suddenly the agent stops inventing contacts and starts telling the user the record doesn't exist. Which is the correct behaviour, and it took less time to fix than writing the ticket.
Errors are instructions
There's a second benefit that people miss: a well-written error is the cheapest prompt you will ever write.
"Rate limited, retry after 30 seconds" is a better instruction than anything you can put in a system prompt. "The user is not authorised to view billing: ask them to confirm the account owner" tells the agent exactly what to do next. Compare that to a generic 500 and you can see why agents thrash.
I write tool errors for the model as the primary reader, not the developer. Developers get the stack trace in the logs. The model gets a short, actionable sentence.
Assertions before the call, not after
The other half is input validation. If update_record requires an account_id, reject the call when it's missing instead of letting the database insert a row with a null and discovering it in a reconciliation report a week later.
This is unglamorous schema work. It is also the single highest-leverage change I make in most codebases. Typed tool inputs and explicit status codes turn a class of flaky, hard-to-reproduce agent bugs into loud, immediate, obvious ones.
What to do on Monday
- Pick your five most-used tools.
- For each one, write down every way it can fail. All of them, including the ones that "can't happen".
- Make each failure return a distinct, human-readable status.
- Throw on anything unrecognised rather than passing it through.
- Add a test that calls the tool with a bad ID and asserts the agent reports a problem instead of inventing an answer.
That last test is the one that catches regressions, because someone will "helpfully" make the API return a default again in six months.
The prompt matters. But in my experience, a mediocre prompt on top of honest tools beats a beautiful prompt on top of tools that lie.
