Prompting AI Agents: Task Decomposition and Tool Use
Agents fail at planning, not at prose. How to write instructions that survive tool errors, ambiguous requests, and the urge to do five things at once.
InnovProduct Team
Library Team
A single-turn prompt is judged on one output. An agent is judged on a trajectory: a sequence of decisions, tool calls, and recoveries that may run for a dozen steps. That changes what good instructions look like. The hardest part of an agent is not producing good prose on the first try, it is staying oriented after the fourth unexpected result, and the prompt has to carry that load.
Decompose the task, not the sentence
Agents fail when the instruction mixes a goal with an implicit plan and the model improvises the plan. Make the steps explicit, and make each one checkable. "Research the topic and write a summary" is one sentence and an unbounded plan. "List the sources, extract the three main claims, note where they disagree, then write 200 words" is four steps the model can verify as it goes.
- Name the steps in order, and say what counts as done for each.
- State what to do when a step produces nothing usable, so the model does not stall.
- Cap the number of steps or tool calls, so a loop cannot run indefinitely.
- Define the stopping condition explicitly: what output means the task is complete.
Design tools that cannot be misused
The tool schema is half the prompt. A tool with a vague description, ambiguous parameter names, or an unconstrained string will be called wrong, and no amount of instruction tuning fixes a bad interface. Write descriptions that say when to use the tool and when not to, keep parameter names unambiguous, and make the allowed values enumerable wherever possible.
{
"name": "search_orders",
"description": "Look up orders by customer email and date range. Use ONLY when the user asks about a specific existing order. Do NOT use for questions about policies, pricing, or general how-to. Returns at most 50 results; if the user needs more, narrow the date range first.",
"parameters": {
"type": "object",
"properties": {
"email": {
"type": "string",
"description": "Customer email address exactly as provided. If absent, ask for it rather than guessing."
},
"start_date": {
"type": "string",
"description": "Inclusive start date in YYYY-MM-DD. Required."
},
"end_date": {
"type": "string",
"description": "Inclusive end date in YYYY-MM-DD. Required."
}
},
"required": ["email", "start_date", "end_date"]
}
}The description earns its length. "Returns at most 50 results" prevents the agent from reporting a truncated list as complete. "Ask for it rather than guessing" prevents a confident hallucinated email address. Every clause here is a failure mode you closed at the interface rather than at the prompt.
Every constraint you can express in the tool schema is one the model cannot forget halfway through a long trajectory. Put rules there first, and prose second.
Plan for failure explicitly
The defining difference between a demo agent and a working one is what happens when a tool errors. An agent that retries the identical call on the identical arguments has learned nothing. Instruct the recovery: retry once with corrected arguments if the error names the problem, switch to a fallback source if the tool is unavailable, and surface the problem to the user rather than continuing on incomplete data. Silently proceeding after a failed lookup is the most damaging behavior an agent can have, because the user cannot tell the difference between "no such order" and "the lookup failed".
- Distinguish "the call failed" from "the answer is empty" in the instructions, and never treat them as the same.
- Forbid repeating a call with identical arguments, so a retry loop cannot burn a budget.
- Require the agent to say what it could not do, rather than omitting the gap.
- Cap retries per tool, and cap total tool calls for the task.
A worked agent prompt
A useful agent prompt is long, and that is not a problem as long as the structure is stable. Put the policy at the top, the available state in the middle, and the output contract at the bottom. The model reads the same policy on every turn, which means a defect in that policy is a defect in every trajectory the agent ever runs, and you will see it in the trace rather than the final message.
ROLE
You triage inbound support requests. You classify, you do not answer.
OBJECTIVE
Produce one decision per request that a human reviewer can audit in under 30 seconds.
HARD RULES
1. Choose exactly one category from the allowed list. Never invent a category.
2. If the request matches no category, use "unclassified" and set needs_human = true.
3. Never include customer contact details in the summary.
4. Never call the refund tool. Draft only; a human approves.
ALLOWED CATEGORIES
billing | account_access | bug_report | feature_request | spam
OUTPUT
JSON matching the response schema. No prose before or after the object.Notice what the prompt does not contain. It does not describe what support is, it does not restate the company mission, and it does not explain why refunds are sensitive. Those belong in documents the agent can consult when a case is unusual, not in a rule block it re-reads on every turn. The distinction between policy that is always true and knowledge that is sometimes needed is the difference between a prompt you can cache and a prompt you cannot.
Where these prompts fail in the wild
| Symptom | Root cause | Fix in the prompt |
|---|---|---|
| Wrong tool called | Descriptions overlap | Rewrite so each names a distinct outcome and trigger |
| Loop repeats the same call | No stopping condition | State the stop condition and the max iteration count |
| Fabricated tool result | Results not marked untrusted | Label tool output as data, never as instructions |
| Correct on turn one, drifts later | State lives outside the context | Restate the current objective and open items each turn |
| Passes review, fails in production | Tested on happy paths only | Evaluate the trajectory, not the final message |
If your agent works in a demo and breaks in production, the transcript is the evidence. Read the three turns before the bad decision, not the final message, because the defect is almost always upstream of the answer you are looking at.
Keep state in the context, not in your head
Long trajectories lose coherence because the model is re-reading a growing transcript. Restate the goal periodically, and record decisions and open questions in a short structured scratchpad the agent updates as it goes. The scratchpad is what lets the agent recover after an error three steps back, rather than restarting from a partially understood context.
SCRATCHPAD (maintain this; update it after every tool call)
GOAL: what the user actually wants, in one sentence
DONE WHEN: the condition that means the task is finished
DONE SO FAR: numbered list of completed steps with their results
OPEN QUESTIONS: anything unresolved; ask the user before continuing
FAILED ATTEMPTS: tool, arguments, error. Do not repeat these.
NEXT: the single next action, with the reason
Rules:
- Keep it under 150 words. Prune completed detail, keep decisions.
- Update it before and after any tool call that changes state.
- If an entry in FAILED ATTEMPTS is relevant to NEXT, choose a different approach.An agent is not a prompt with retries attached. It is a loop with state, and the instructions have to survive the fifth iteration.
Test the trajectory, not the last message
Evaluating only the final output hides most agent failures, because a lucky final answer can follow a trajectory full of hallucinated parameters and ignored errors. Evaluate the trace: how many tool calls it took, whether any argument was invented, whether a failure was reported, and whether the stated goal is still the user goal at the end. This is the same discipline as a frozen evaluation set applied to a moving target, and it is where most agent regressions hide.
How many tools should an AI agent have?
Fewer than feels comfortable. Every additional tool is another way to route wrongly, and the model degrades as the toolset grows. Group related actions into one tool with a parameter instead of adding near-duplicate tools, and keep the descriptions specific about when not to use them.
Should an agent ask clarifying questions or guess?
Ask, for anything that changes the outcome, and require the instruction to say so explicitly. Guessing an email address, an account identifier, or a date range produces confident work that is wrong in a way the user often cannot detect.
How do I stop an agent from looping?
Cap tool calls per task in the instruction, forbid repeating an identical call, and require the agent to switch approach after a failure rather than retry. Record failed attempts in the scratchpad so the loop is visible to the model as well as to you.
Do I still need few-shot examples for agents?
Far fewer than for classification. A trajectory example is long and expensive, so use one or two for a genuinely tricky interaction pattern and spend the rest of the budget on explicit rules, a good scratchpad format, and tight tool descriptions.
Related prompt engineering guides
Agent prompting is where the other practices stop being optional.
- Best practices for training AI models with promptsThe pillar overview, including the frozen eval set that makes agent regressions detectable.
- How to evaluate LLM prompts with frozen test setsHow to score a trajectory instead of only the final message, where most agent bugs hide.
- Few-shot prompting: how many examples you needWhy agents usually need one or two trajectory examples rather than a full example set.
Agent skills that arrive with their contracts
Agent skills in the library declare their tools, their stopping conditions, and their failure behavior up front, so you can see the whole loop before running it. Browse agent skills alongside workflow templates.
Browse agent skills