AI Agent Development: Architectures, Tools, Protocols, and Guardrails for Production

AI agent development is the engineering of systems in which a large language model plans and carries out multi-step tasks by calling tools such as APIs, databases, search, or code execution, observing the results, and deciding what to do next until the goal is met. Agents can handle tasks such as researching a supplier, reconciling records, or resolving a support case end to end, but they need careful design to be reliable and safe.

Agents vs. workflows

Anthropic's engineering guide "Building effective agents" draws a useful distinction. Workflows orchestrate LLMs and tools through predefined code paths. Agents let the LLM direct its own process and tool use. Its advice is to start with the simplest solution and add complexity only when it improves results. Many "agent" projects are better built as workflows: prompt chaining, routing, or parallel steps with fixed logic.

Core agent architecture

1. Model: an LLM capable of reliable tool use and reasoning.
2. Instructions: the goal, constraints, available tools, and when to stop or ask for help.
3. Tools: well-documented functions with typed inputs and clear error messages.
4. Loop: reason, call a tool, observe the result, and repeat, in the spirit of the ReAct pattern.
5. Memory and state: conversation history, task state, and, where needed, retrieval over past interactions. See RAG explained.
6. Guardrails and human checkpoints.
7. Observability: traces of every step, tool call, and cost.

Tools and the Model Context Protocol

The Model Context Protocol (MCP), an open standard introduced by Anthropic in 2024 and since adopted widely across the industry, defines a common way to connect AI applications to tools and data sources. Building tools as MCP servers lets several agents and AI clients reuse them. Whatever the protocol, design each tool to do one thing well, validate inputs, and return concise, informative results.

Frameworks and platforms

Options include open-source frameworks such as LangGraph (from the LangChain team), cloud agent services such as Google's Agent Builder and Agent Development Kit on Vertex AI, Amazon Bedrock Agents, and Microsoft's Azure AI agent service, as well as direct use of model providers' tool-calling APIs. Choose based on control, observability, hosting, and how well the framework fits your stack. A thin custom orchestration layer is often enough.

Guardrails: what can go wrong

The OWASP Top 10 for LLM Applications highlights risks that are especially serious for agents:

• Prompt injection: malicious instructions in emails, web pages, or documents the agent reads.
• Excessive agency: tools with more permissions than needed, or actions taken without approval.
• Sensitive information disclosure through tool outputs or logs.
• Unbounded consumption: runaway loops and costs.

Mitigations:

• Least-privilege credentials for every tool, scoped to the current user.
• Human approval for irreversible or high-value actions such as payments, deletions, and external emails.
• Limits on steps, time, and spend per task.
• Treat all tool outputs and retrieved content as untrusted input.
• Sandbox code execution.

Our application security guide covers the wider secure development process.

Evaluating agents

• Build task suites with clear success criteria, such as "invoice matched to the correct purchase order."
• Measure task success rate, steps taken, cost, latency, and how often humans intervene.
• Review traces of failures to improve tools and instructions.
• Re-run the suite on every model, prompt, or tool change.

Common agent design patterns

Most production agents use one of a small number of patterns, and choosing the simplest pattern that solves the problem makes the system easier to test and trust.

• Router: a model classifies the request and sends it to the right specialized workflow, tool, or prompt. Simple, predictable, and often enough.
• Tool-using loop (ReAct style): the model reasons about the next step, calls a tool, observes the result, and repeats until it can answer. Flexible, but needs step limits and careful tool design.
• Plan and execute: the model writes a plan first, then executes each step, possibly revising the plan. Useful for longer tasks such as research or report generation.
• Multi-agent: several specialized agents, such as a researcher, a writer, and a reviewer, collaborate under an orchestrator. Powerful for complex workflows, but harder to debug and more expensive.
• Human-in-the-loop: the agent prepares work, such as a drafted email or a proposed refund, and a person approves it before anything happens.

Start with a router or a deterministic workflow with a few model-powered steps. Move to open-ended loops only when the task genuinely requires flexible decision-making.

Designing good tools

An agent is only as reliable as its tools. Each tool should do one clear thing, have a descriptive name and description written for the model, and accept parameters defined by a strict schema. Return concise, structured results, including clear error messages the model can act on, such as "Order not found: check the order number format". Make read-only tools the default, and separate tools that change data so they can carry stricter permissions and confirmation steps. Enforce authorization inside each tool using the end user's identity, never relying on the model to decide who may see what. Log every tool call with its inputs, outputs, and duration.

Memory and state

Agents need different kinds of memory. Short-term memory is the current conversation and the results of recent tool calls, held in the model's context. Working state, such as a task's progress, should be stored in your application, not only in the prompt, so tasks can resume after failures. Long-term memory, such as user preferences or past interactions, should be stored deliberately with clear rules about what is kept, for how long, and who can see it. Be cautious about storing personal data in memory, and give users a way to see or clear it where appropriate. Summarize long conversations to keep context focused and costs under control.

Cost and latency control

Agents call models repeatedly, so costs and response times can grow quickly. Set a maximum number of steps and tool calls per task, and a time limit. Use smaller, faster models for routine steps such as routing and extraction, and larger models only for difficult reasoning. Cache results of expensive tool calls when data does not change often. Stream progress updates to users for longer tasks, so they know the agent is working. Track cost per completed task, not just per model call, and alert when tasks exceed expected limits, which often indicates a loop or a broken tool.

Security threats specific to agents

Agents introduce risks beyond those of chatbots, because they can take actions. The most important threat is indirect prompt injection: instructions hidden in emails, web pages, documents, or tool results that try to make the agent do something the user did not intend, such as sending data to an attacker. Defenses include treating all external content as untrusted, keeping a strict separation between instructions and data, limiting which tools are available for each task, requiring confirmation for sensitive actions, restricting outbound communications, and monitoring for unusual tool usage. The OWASP Top 10 for LLM Applications describes these risks, including excessive agency, and is a useful checklist. See also our application security guide.

Example: an accounts payable agent

A finance team receives supplier invoices by email. An agent reads each new invoice, extracts the details with structured output, looks up the supplier and purchase order in the ERP system through read-only tools, and checks that quantities and amounts match. If everything matches within tolerance, it prepares the invoice for posting and adds it to an approval queue. If something does not match, it drafts a query email to the supplier for a finance team member to review and send. The agent cannot pay invoices or change supplier bank details; those actions remain with people. Every step is logged, and a weekly sample is audited. The team spends its time on exceptions instead of data entry.

Rolling out agents safely

1. Shadow mode: the agent processes real tasks but its outputs are only compared with what people did, without taking effect.
2. Assist mode: the agent prepares actions, and a person approves each one.
3. Limited autonomy: low-risk actions within defined limits are automated, with sampling and monitoring.
4. Expanded autonomy: more actions are automated as evidence of reliability accumulates.

Moving through these stages builds confidence with data rather than assumptions, and gives you a clear point to stop if quality falls short.

When not to use an agent

If a task follows the same steps every time, a conventional workflow or script is cheaper, faster, and more reliable than an agent. If a task needs only a single model call, such as summarizing a document, a simple prompt is enough. Use agents when the path to a result genuinely varies with the input and requires choosing between tools. For knowledge-based question answering, a RAG system or chatbot is often the better fit, and our enterprise AI guide covers the governance agents need at scale.

Observability for agents

Debugging an agent requires seeing every step it took. Record a trace for each task that includes the user request, each model call with its prompt version and output, each tool call with inputs, outputs, errors, and timing, any human approvals, and the final result. Tracing tools built on standards such as OpenTelemetry, or LLM-specific observability platforms, make these traces searchable. Build dashboards for task success rate, average steps per task, tool error rates, escalations to humans, latency, and cost per task. Review failed and unusually long traces regularly; they reveal missing tools, confusing tool descriptions, and gaps in instructions.

Evaluating agents in more depth

Agent evaluation goes beyond checking final answers. For a representative set of tasks, measure whether the task was completed correctly, whether the agent chose appropriate tools in a sensible order, whether it stayed within permissions and limits, how many steps and how much cost it needed, and whether it asked for help when it should have. Include tasks designed to tempt the agent into unsafe behavior, such as documents containing hidden instructions. Run the evaluation suite automatically whenever you change prompts, tools, models, or orchestration logic, and compare results against the previous version before deploying.

Governance and accountability

Every agent should have a named business owner responsible for its behavior, a documented list of the tools and data it can access, and clear rules for which actions require human approval. Keep an inventory of deployed agents, review their permissions periodically, and remove access that is no longer needed. When an agent makes a mistake, treat it like any other operational incident: understand the cause, fix the tool, prompt, or permission involved, and add a test so it does not happen again.

Where to start

Pick a process with a clear definition of done, frequent repetition, existing APIs, and low risk if a step goes wrong, such as triaging support tickets, preparing data for reports, or drafting responses for review. Build it first as a workflow with a few model-powered steps, measure it in shadow mode, and add autonomy gradually as the evidence supports it.

Build AI agents with Vertex Cyber Tech

We design agentic systems that combine our AI/ML expertise with security engineering, so autonomy comes with controls. Start with our AI partner checklist or book a consultation.

Related guides: for practical next steps, read our AI and business process automation guide and our AI voice agent development guide.

Sources and further reading

• Anthropic: Building effective agents
• Yao et al., "ReAct: Synergizing Reasoning and Acting in Language Models"
• Model Context Protocol
• OWASP Top 10 for LLM Applications