Agent tool misuse

What it is

The tool is legitimate and the agent is authorised to call it. The fault is in how it is used: the wrong operation, the wrong target, the wrong order, or a chain of individually harmless calls that together do harm. No attacker is required; ambiguity and model error are enough. Where the agent calls a tool that does not exist or invents the result, see function-calling hallucination.

How it happens

• An ambiguous instruction ("clean up the test data") is carried out against production.
• The agent passes an unvalidated parameter, such as a wildcard, to a delete or update operation.
• Tool output is treated as an instruction for the next call.
• The agent combines a read tool and a send tool to move data out of the organisation.

Why it matters

Tools are where an agent's output becomes a change in the world. Unlike a wrong answer, a wrong tool call may have no undo.

Warning signs

• Tool calls carry parameters broader than the request implied.
• Destructive operations run without a prior read or a dry run.
• The same tool is called repeatedly with small variations.
• Error rates on tool calls rise after a model or prompt change.

Who owns it

Engineering owns the design of each tool's interface and its validation. The business owner of the process the tool touches decides which operations need approval.

In practice

An operations agent asked to archive closed accounts older than seven years builds a query that omits the date condition and archives every closed account. The same pattern generalises to any tool whose scope is set by a parameter the agent composes.