Designing Enterprise Conversational Agents
Paper written by Perplexity to understand the considerations of enterprise conversational agentic architecture
Domain capabilities, agent-facing tools, specialist reasoning, and the boundaries that hold them together
The central conclusion of our discussion is that enterprise agent architecture is not a choice between domain-level MCP servers and agent-level tools. It is a set of related but independent decisions about business ownership, interface design, reasoning, authority, execution, and operations.
Those decisions should be made together, but they should not be collapsed into one another.
A domain can own a capability without owning an agent. An agent can consume a task-specific interface without owning the underlying business logic. A specialist can investigate a problem without being allowed to change an account. A shared contract can run in isolated deployments. And a conversational agent can own the customer experience without personally performing every investigation.
The organizing principle is separation: separate responsibilities where they have different purposes, risks, lifecycles, or performance characteristics, while preserving a coherent end-to-end experience.
This paper develops that principle through the full architecture we discussed. It is written for listening, so the argument proceeds in connected sections rather than relying on tables or diagrams.
1. The starting point: MCP as a backend for agents
Your framing of MCP as a backend for frontend, but for agents, changes the architecture discussion in an important way.
Strictly speaking, MCP is the protocol. An MCP server can implement an agent-facing backend. The protocol specifies how capabilities are exposed and invoked; it does not dictate whether those capabilities should mirror domain APIs, compose several services, or represent a task-oriented operation.[1]
The architectural analogy is the traditional backend-for-frontend pattern. A mobile application and a desktop application may use the same underlying business systems but require different interfaces. A client-specific backend adapts shared capabilities to each consumer’s needs. Microsoft’s description of the BFF pattern explicitly emphasizes tailoring the backend interface to its frontend consumer.[2]
Agents deserve the same consideration.
An agent is not necessarily best served by the API surface that was designed for a web application, an integration platform, or another domain service. Its interface should reflect the work it is being asked to perform.
Suppose an agent must explain why a customer’s bill increased. A domain-oriented interface might expose invoice retrieval, account details, adjustments, product prices, and recent orders.
Those are useful primitives. But the agent must now decide which systems to query, correlate the records, interpret billing periods, identify partial-period charges, and determine whether the available evidence is sufficient.
An agent-oriented interface might instead expose an operation called “investigate bill change.” Behind that operation, ordinary code retrieves the relevant records, correlates them, calculates differences, and returns structured evidence.
The difference is not simply that one interface has fewer tools. The difference is where integration responsibility resides.
A good agent-facing backend removes predictable integration work from model reasoning. It does not necessarily remove judgment. It creates a clearer space in which judgment can operate.
2. The decisions that must remain independent
There are at least six decisions to make.
First: who owns the business capability?
Billing should ordinarily own the meaning of a valid billing adjustment. Orders should own order state and fulfillment rules. Network Operations should own authoritative network diagnostics and relevant operational constraints.
Second: what interface should the consuming agent see?
The answer may be a domain primitive, a task-oriented composition, or a combination of both.
Third: where should reasoning happen?
It may belong in the conversational agent, a specialist investigator, or nowhere at all because the work is deterministic.
Fourth: who has authority?
The ability to reason about a credit does not imply permission to apply one.
Fifth: where does the component run?
A shared interface can be deployed regionally, isolated by workload, or divided into separate read and write execution pools.
Sixth: who operates the complete experience?
A task that crosses three services still needs an accountable owner for its end-to-end outcome.
If these decisions are bundled together, the architecture becomes unnecessarily rigid. “Billing owns the capability” can quietly turn into “all billing traffic must pass through one Billing Agent, one MCP server, one release pipeline, and one shared capacity pool.”
None of those conclusions follows automatically.
3. Domain-oriented tools versus agent-oriented tools
The strongest argument for domain-oriented tools is reuse.
A coherent domain operation can serve many consumers. It can provide stable semantics, consistent validation, and a natural ownership boundary.
The strongest argument for agent-oriented tools is task fit.
An agent should not repeatedly reconstruct the same integration recipe simply because the enterprise has exposed only its internal service boundaries.
These positions are compatible.
The domain should own business correctness. The agent-facing layer should own the shape of the task contract.
For example, a billing domain may provide authoritative operations for invoice retrieval, credit eligibility, and credit execution. A customer-service backend may compose them into investigation and preview operations tailored to the conversational experience.
The facade should adapt and compose. It should not become a second, unofficial implementation of billing policy.
That distinction is critical. If the agent-facing layer independently reimplements eligibility rules, the enterprise now has two competing definitions of business truth.
There is also a danger at the opposite extreme: an overly broad task tool called “resolve customer issue.”
Such an interface may sound convenient, but it obscures scope, side effects, failure modes, and authority. It can become an unbounded application hidden behind a friendly tool name.
A stronger design uses bounded operations: investigate the issue, preview eligible remedies, and execute a particular approved remedy.
The abstraction should simplify the task without concealing its risks.
4. Domain agents are a separate question
Once domain capabilities exist, the next temptation is to create a domain agent for each domain.
Sometimes that is useful. Sometimes it merely adds a model call between the user and a service.
The distinction between a service, a workflow, and an agent matters here.
A service performs a defined operation.
A workflow follows controlled execution paths, possibly with branches, approvals, and model calls.
An agent dynamically decides what to do next based on the objective and what it discovers.
Anthropic’s architectural guidance makes this same distinction between predefined workflows and agents that direct their own process. It also recommends starting with simpler patterns and adding agentic complexity when needed.[3]
A Billing Agent is justified when it performs meaningful billing investigation: choosing evidence, testing explanations, identifying contradictions, and deciding whether more inquiry is necessary.
A component that simply selects among three known APIs may be better implemented as a router or workflow.
The key question is not whether billing is important enough to deserve an agent. The question is whether the work requires a distinct, reusable reasoning capability.
5. Domain boundaries and reasoning boundaries do not always align
Business domains organize data, rules, and accountability.
Reasoning tasks organize problem solving.
Those boundaries may overlap, but they are not identical.
Consider a customer who upgraded a plan, lost a discount, and received an unexpected charge. The explanation may depend on Orders, Product, Billing, and Customer data.
A Billing Agent may be too narrow. A general Customer Agent may be too broad.
The useful specialist may be a billing-dispute investigator that consumes capabilities across domains.
This leads to a practical principle: use domain boundaries to organize business authority, and use task boundaries to justify reasoning agents.
Do not reproduce the organizational chart as a hierarchy of language models.
A conversational agent that calls a Customer Agent, which calls a Billing Agent, which calls an Orders Agent, may look modular. But each handoff can introduce latency, context loss, duplicated reasoning, and ambiguous responsibility.
Introduce another agent because its reasoning adds value, not because another organizational box exists.
6. The conversational–investigation boundary
The conversational layer and the investigation layer have different jobs.
The conversational layer owns understanding and serving the user’s intent.
The investigation layer owns establishing what the evidence supports.
Neither automatically owns permission to change business state.
A user says, “My bill went up. Can you fix it?”
That statement contains an observation, a desired outcome, and perhaps an assumption.
The observation is that the bill increased.
The desired outcome may be an explanation, correction, reduction, or some combination.
The assumption may be that an error occurred.
A poorly framed delegation says, “Find the billing error and fix it.” That instruction assumes the conclusion and mixes investigation with execution.
A better delegation says, “Determine the causes of the increase. Distinguish expected charges, possible errors, and unresolved items. Return evidence and permitted next steps. Do not modify the account.”
The conversational agent preserves the user’s objective. The investigator tests what is true.
That boundary protects both accuracy and authority.
7. Who owns clarification?
Investigators often need information from the user.
By default, the investigator should identify the missing information and explain why it matters. The conversational layer should decide how and when to ask.
For example, the investigator may report that two invoices fit the description and the disputed billing period is unclear.
The conversational agent then asks a natural question: “Are you referring to the September bill or the October bill?”
This avoids multiple agents competing for the user’s attention.
There are exceptions. A network diagnostic specialist may need a sustained technical interview. In that case, interaction can be delegated explicitly.
But the transfer should specify what task the specialist owns, which questions it can ask, when control returns, and how the user can interrupt.
A background worker should not silently become a second owner of the conversation.
8. Evidence must survive the handoff
One of the most dangerous failures is confidence inflation across boundaries.
An investigator reports, “The discount probably expired, but the contract record was unavailable.”
The conversational agent says, “Your discount expired.”
The qualification disappeared.
To prevent that, an investigation should return more than polished prose. It should return findings, evidence references, relevant dates, missing evidence, unresolved contradictions, and the status of each conclusion.
A user statement, an observed record, an inference, a recommendation, and an executed action must remain distinct.
“The customer says a discount was promised” is not the same as “the discount was contractually promised.”
“A credit is eligible” is not the same as “a credit was applied.”
Condensed handoffs are valuable, but summaries are lossy. Anthropic’s context-engineering guidance discusses the risk that aggressive compaction can discard important details, and describes structured notes and external references as ways to preserve continuity.[4]
The right design is concise findings backed by retrievable evidence.
9. Authority should not travel casually with reasoning
Security is not determined by where an MCP server runs.
It is determined by which actor can perform which operation, on which resource, on whose behalf, under what conditions.
The architecture must distinguish the end user, the calling workload, the acting agent, and any delegated authority.
Tool visibility is not authorization.
A model-selected tool call is not approval.
A specialist recommendation is not a policy decision.
The MCP security guidance warns about broad scopes, token passthrough, confused-deputy risks, and session-related security failures. These are particularly relevant when a server acts as an intermediary to downstream systems.[5]
A good enterprise design therefore separates investigation, recommendation, approval, execution, and outcome recording.
Sensitive approval should bind to the intended action and its material arguments. If the recipient, amount, or target changes, the prior approval should not silently authorize the new action.
Execution should also revalidate relevant state. An eligible remedy may become invalid between preview and execution.
Business authority belongs at trusted execution boundaries, not merely in instructions telling the model to behave responsibly.
10. Latency is a dependency-path problem
Latency is not simply the sum of network hops.
It includes queueing, input processing, reasoning, output generation, tool execution, and coordination.
The most expensive architectural mistake may be an unnecessary sequential reasoning stage, not an extra service call.
A fine-grained tool design can force the model to retrieve invoices, think, retrieve orders, think, retrieve eligibility, think, and finally answer.
A task-oriented backend may perform the known retrieval and correlation sequence in code, returning one structured evidence package.
However, a well-designed generalist can also request independent tools in parallel. Parallel tool use is supported as a way to reduce latency when operations do not depend on one another.[6]
Therefore, a facade does not automatically outperform direct tools. Its advantage may be making the efficient execution plan consistent and testable.
A domain specialist introduces delegation and synthesis stages. It earns those stages if it avoids substantially more wasted investigation, improves task quality, or enables useful parallel work.
The comparison must be made at the complete-task level.
11. Parallelism trades elapsed time against total work
Parallel investigation can shorten the wait.
If three independent investigations take four, six, and eight seconds, the parallel stage finishes roughly when the eight-second branch finishes, rather than after eighteen seconds of sequential work.
But the enterprise still pays for all three investigations.
Latency follows the longest dependency path. Cost follows aggregate work.
Parallel agents can also duplicate retrieval, pursue overlapping hypotheses, or continue after enough evidence has been found.
Useful controls include fan-out limits, non-overlapping assignments, cancellation, stop conditions, and explicit treatment of missing or slow branches.
Parallelism is not free acceleration. It is a resource-allocation decision.
12. Token and thinking budgets change the economic unit
An agent call is not a meaningful unit of cost by itself.
The relevant work includes input processing, reasoning generation, visible output, tool execution, and repeated context transfer.
For documented reasoning models from OpenAI and Claude, reasoning contributes to billed output usage, even when the user sees only a short final answer. Exact controls and accounting differ by provider and model.[7][8]
This means that a concise answer can still represent substantial compute.
It also means that a single high-thinking generalist may cost more than several tightly scoped calls—or much less than specialists that duplicate the same reasoning.
There are four separate budgets to manage: input context, reasoning, visible output, and total task execution.
A context window is not a task-spend limit. A per-request thinking allowance is not a workflow-wide allowance.
If a task makes six model requests, a locally configured budget can be consumed repeatedly. Adding specialists multiplies the opportunities.
The orchestrator must therefore govern aggregate spending across the user’s task.
13. Spend thinking where it adds value
A useful allocation strategy is to keep routine coordination lightweight and concentrate reasoning where uncertainty is highest.
Simple routing should not automatically receive deep thinking.
Known retrieval should usually be a direct tool operation.
Predictable integration should be code.
Ambiguous diagnosis may deserve substantial reasoning.
Presenting a validated result may need much less.
Cross-domain contradictions may justify an explicit synthesis stage.
The principle is to place deep reasoning where the evidence is richest and the decision is hardest, rather than at every handoff.
A task-oriented backend can improve economics by removing reasoning work entirely. That is different from giving the model a smaller allowance and hoping it performs the same task.
Programmatic tool-calling guidance describes a related benefit: execute tool sequences and filter intermediate data outside the model to avoid repeated inference and unnecessary context.[9]
But lower thinking is not always cheaper overall. If it creates mistakes, retries, escalations, or repeat contacts, total outcome cost can rise.
The target is the lowest compute allocation that meets the required quality and risk standard.
14. Budgets must follow the task across agents
A practical enterprise budget envelope should include spending, elapsed time, maximum model calls, tool-operation limits, concurrency limits, and an escalation reserve.
Child agents should receive allocations from that envelope rather than fresh, independent allowances.
For concurrent work, reserve capacity before dispatch. Otherwise, several specialists can each spend the same supposedly remaining budget.
Dollar or normalized-compute budgets are more meaningful than one universal token count when models have different prices.
Reconcile reservations against actual usage. Preserve enough capacity to return a useful partial result or escalation status.
Also recognize enforcement limits. If usage is reported only after a call, the system needs headroom. It should not claim a hard global cap that in-flight requests can exceed.
15. Throttling must reflect downstream work
One agent-facing tool invocation may trigger many backend operations.
A specialist may generate several model requests and dozens of tool calls.
Therefore, a request-per-second limit on the MCP endpoint is insufficient.
Budgets should exist at the user-task level, agent-profile level, tenant level, domain-operation level, and downstream-service level.
Retries need a designated owner. If the agent, facade, and backend all retry independently, a slowdown can become a retry storm.
Microsoft’s throttling guidance emphasizes controlling load at defined boundaries and accounting for shared downstream dependencies and traffic amplification.[10][11]
Separate interactive and batch traffic where necessary. Protect shared systems without forcing every workload into the same capacity pool.
Shared contracts and isolated execution capacity are compatible.
16. State, interruption, and recovery
Conversation state, investigation state, business state, and UI state have different lifecycles.
The conversation records what the user asked and clarified.
The investigation records evidence, hypotheses, and outstanding checks.
Business systems record authoritative transactions.
The UI records what is displayed.
Closing a panel should not accidentally cancel a durable task. Browser navigation should not replay a credit. Reconnecting should recover task status rather than restart the investigation.
User interruption also needs explicit semantics.
“Stop talking,” “stop investigating,” “cancel the pending action,” and “abandon the task” are different commands.
If the user changes their objective, an old investigation result should not automatically become the answer to the new request. Task and intent versions can make that relationship explicit.
For cross-service writes, partial completion requires deliberate recovery. Saga-style approaches provide a framework for local transactions, compensating actions, and reconciliation, but compensation is not necessarily equivalent to erasing an action.[12]
17. Observability and evaluation must cross boundaries
A trace should connect the user request to delegation, tool invocation, domain operation, approval, transaction, and final outcome.
It should preserve who acted, under what authority, using which contract version, with what evidence, at what cost, and with what result.
Operational telemetry, security audit, and conversational analytics should not be treated as one indiscriminate log. They have different access and retention needs.
Evaluation should compare complete tasks at the same quality bar.
Test direct domain tools, deterministic task backends, specialist agents, and parallel specialists against representative workloads.
Measure correct completion, latency distributions, aggregate model usage, backend fan-out, retries, unauthorized-action rejection, human escalation, and cost per correct outcome.
Run realistic concurrency tests. An architecture that looks excellent with one user and warm caches may fail under shared capacity constraints.
Change management must include prompts, models, tool descriptions, schemas, policies, and workflow definitions. A technically compatible tool change can still alter model behavior.
18. Ownership and the enterprise operating model
Domain teams should own business semantics, invariants, authoritative data, and transaction correctness.
Agent or product teams should own the task contract, conversational behavior, user outcomes, and task-level evaluation.
Platform teams should provide common identity, runtime, telemetry, registry, deployment, and budget mechanisms.
Risk and security teams should establish policy and approval requirements.
But the composite experience still needs one accountable owner.
“The APIs worked” is not an adequate response when the customer’s task failed.
Shared governance does not require one central gateway in every execution path. Federated ownership can coexist with common standards and a shared control plane.
Likewise, an agent-specific interface does not require a separate deployment for every prompt revision. Stable role or capability profiles are usually a more useful unit.
19. The resulting architectural posture
The default architecture that emerges is a conversational experience supported by durable orchestration.
Simple bounded operations go directly to tools.
Known business processes go to workflows.
Adaptive investigations go to specialists.
Specialists and conversational agents consume task-appropriate interfaces, which in turn reuse domain-owned capabilities.
Authorization, business invariants, and transaction controls remain at trusted execution boundaries.
Deployment topology follows latency, trust, scale, regional constraints, and failure isolation—not organizational labels.
Domain agents are optional reasoning capabilities, not mandatory gateways.
Agent-facing backends simplify the agent’s task without becoming shadow domain systems.
Thinking budgets follow the user’s objective across the system rather than resetting at every agent boundary.
And the conversation remains responsible for communicating what has been established, what remains uncertain, and what can happen next.
Closing perspective
The deepest architectural question is not, “Where should we put MCP?” It is, “What responsibility should the model carry, and what responsibility should the enterprise system carry?”
Models are valuable where intent is ambiguous, evidence is incomplete, and the next investigative step cannot be fully predetermined.
Code and workflows are valuable where the sequence is known, correctness is testable, and execution must be controlled.
Domains provide business authority.
Task-oriented interfaces provide usability for agents.
Specialists provide bounded reasoning.
The conversational layer provides continuity with the human.
The architecture succeeds when those parts cooperate without impersonating one another: conversation does not become evidence, evidence does not become authority, recommendation does not become execution, and modularity does not become an excuse for uncontrolled cost and latency.
Sources
[1] Tools https://modelcontextprotocol.io/specification/2026-07-28/server/tools
[2] Backends for Frontends Pattern - Azure Architecture Center https://learn.microsoft.com/en-us/azure/architecture/patterns/backends-for-frontends
[3] Building Effective AI Agents - Anthropic https://www.anthropic.com/engineering/building-effective-agents
[4] Effective context engineering for AI agents - Anthropic https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
[5] Security Best Practices - Model Context Protocol https://modelcontextprotocol.io/docs/2025-11-25/tutorials/security/security_best_practices
[6] Parallel tool use - Claude Platform Docs https://platform.claude.com/docs/en/agents-and-tools/tool-use/parallel-tool-use
[7] Reasoning models | OpenAI API https://developers.openai.com/api/docs/guides/reasoning?api-mode=responses
[8] platform.claude.com https://platform.claude.com/docs/en/build-with-claude/extended-thinking.md
[9] Programmatic tool calling - Claude Platform Docs https://platform.claude.com/docs/en/agents-and-tools/tool-use/programmatic-tool-calling
[10] Throttling Pattern - Azure Architecture Center | Microsoft Learn https://learn.microsoft.com/en-us/azure/architecture/patterns/throttling
[11] Design throttling to improve resilience - Microsoft Learn https://learn.microsoft.com/en-us/azure/well-architected/design-guides/throttling
[12] Saga Design Pattern - Azure Architecture Center | Microsoft Learn https://learn.microsoft.com/en-us/azure/architecture/patterns/saga