What Is an Agent Mesh?

Large language models (LLMs) are incredibly powerful pieces of technology, leveraging mathematical reasoning models to create agentic powerhouses capable of everything from writing articles to developing complex codebases. As LLMs become more powerful, many are starting to look at what a future might look like where services are much more agentic and how they might connect with one another.

One solution — the agent mesh — has started surfacing as an answer. In essence, the concept seems simple — a mesh of agents that can work together and discover one another. But, as with most things in the AI space, it’s still largely experimental.

With that in mind, let’s dive into the world of the agent mesh, and figure out what it is, how it works, and whether or not you need one.

What is an Agent Mesh?

The concept of an agent mesh is simple, but its execution is anything but. Conceptually, all an agent mesh is a runtime or infrastructural layer that allows agents to discover each other, communicate with one another, and delegate tasks to each other in a coordinated way according to a specific policy or specification. If that sounds like a service mesh for AI agents, that’s not entirely accidental — in many ways, as much as agentic implementations are looking like API systems, so too are their interconnections.

The basic idea stems from the fact that LLM agents are increasingly being used to run multistep and complex workflows, coordinating data transfer and cooperative implementations, which might even invoke additional LLM agents. This orchestration layer, then, is an agreed-upon system that allows these agents to work with one another without human intervention.

Building systems with multiple AI agents introduces coordination and security challenges, and the concept of the agent mesh aims to solve some critical key issues, including:

• Trust and authorization between agents: Not all agents should be able to call or influence all others.
• Service discovery: Agents need to find the right tool or agent for a job.
• State management: Shared memory is needed so that agents don’t duplicate work or make inconsistent decisions.
• Scalability: Developers need a way to scale multi-agent interactions across workflows without building brittle orchestration logic.

On the practical side of things, this is where the concept gets murkier. To date, a handful of providers claim to provide an agent mesh, but their implementations are different structurally and practically. Let’s look at a few examples.

Also read: How to Turn Any API Into an MCP Server Lyzr’s AgentMesh Architecture

Lyzr’s solution to the problem is to create an ecosystem dependent on a marketplace solution. In this approach, the marketplace represents a single store of all available agents and, thereby, their workflows, processes, and focuses. In essence, any agent within this architecture can browse and choose an agent from this marketplace, offloading additional functionality, requests, and so forth to appropriate flows and internal systems.

The issue with this approach, of course, is that this is a Lyzr-specific implementation — it represents a decided vendor lock-in guarantee, restricting agentic interactions to only those within the marketplace. Even if you can go outside of the marketplace, you’re much better off sticking to internal resources, which can mean increased long-term complexity and difficulty using open-source or public systems without a marketplace equivalent or front-end.

Solo.io

Solo.io, a provider known for its work in the service mesh space, has taken on this issue with its proposed agent mesh pattern. In essence, this approach uses control planes and MCP servers to combine LLMs, API gateways, and policy engines through an agent gateway. If this sounds like how Solo.io handles service meshes, you’re exactly right — for better or for worse, this is an agentic take on that exact service mesh paradigm.

The main drawbacks of this approach are not within the practical implementation — the idea of an agent gateway is solid, and makes sense given the different circumstances, environments, and flows inherent in multi-agent workflows. Where the approach is somewhat problematic is when one considers the high complexity of the system.

All of this abstraction is very complex, and for users who are connecting one or two agents together, the idea of building out a huge MCP server network and agent gateway solution seems a bit heavy. Even when providers do have the current need and agent count to use something like this, the multi-layered wrapping of heavy control layers could inhibit autonomy and flexibility while improving connective tissue, a balancing act that nobody has really figured out perfectly quite yet.

Solace

Solace, a provider known for its PubSub and event mesh solutions, has suggested a solution that leverages event-driven architecture approaches for agentic mesh connections. The idea is that every agentic request is fundamentally an event, and that these events can be used to route inputs and requests through an orchestration layer using both prescriptive and dynamic workflows.

It makes perfect sense that the event-driven solution would come from Solace, but the reality is that event-driven isn’t always agent-friendly. Solace is essentially retrofitting its event services to agent-to-agent communication, but LLM agents don’t always operate in an event-delineated format. When agents are synchronous and request/response-oriented, this kind of framework might not play well with the overall system.

Even if you’re not using synchronous solutions, the lack of stateful semantic memory states in Solace’s approach means that multi-agent workflows must be from point A to point B and point C without episodic, shared memory states or narrative context.

The Current State of Agentic Meshes

The criticisms above are not given to say that these solutions are bad. Quite the contrary, in fact, they are foundational mesh concepts that will drive the industry forward with creative and useful structures that allow for multi-agent connections. The core problem, however, is that all of these offerings, like LLMs themselves, are in the early conceptual stages, and as such, they suffer from early-stage development issues.

The reality is that so few people are using multi-agent systems at the scale that would require such solutions, and when they are, they are likely using bespoke routing solutions. Unlike the service mesh, which is in a moderately mature stage of use due to the complexity brought on by microservices, we are still very much in the monolithic stage of agentic development, with most users adopting one or two core agents in their pipeline.

That’s going to change relatively quickly, however, especially if the evolution of microservices is anything to reference. Organizations building enterprise copilots that span departments are starting to coordinate multiple fine-tuned LLM agents for tasks like sales enablement, HR automation, and IT support — and these use cases will directly benefit from and require structured agent-to-agent coordination and visibility. While the need is nascent, it is there and evolving — and will likely become a strong value proposition in the next few years.

While there’s no consensus on a single definition or implementation yet, the trend is clear — as the complexity of agent-based systems grows, so does the need for orchestration layers beyond basic prompt chaining or function calling.

Future Outlooks For Agent Mesh

While the “agent mesh” may come across as marketing jargon right now, the same could be said for the service mesh paradigm of earlier years. The rise of this concept is pointing to a real architectural need — a runtime fabric for coordinating autonomous agents at scale. Like the service mesh before it, the agent mesh will abstract away many of the low-level coordination problems of the agentic workflow so that developers can focus on building intelligent behavior.

One area where these agent meshes will see significant attention is in the evolution of agentic security. Agents and LLMs come with significant concerns — ephemerality, state management, workload attestation, identity management, context injection, and policy enforcement, just to name a few. The security issues around LLM agents are growing ever more complex, and right now, there’s no one golden solution. As the agentic mesh approach is figured out, security must be a part of this evolution, setting the stage for both interconnected systems and trust around their outputs.

Whether the term sticks or evolves, the core challenges it describes — agent discovery, communication, and governance — are here to stay. As enterprises move beyond single-agent use cases, expect agent meshes to become a foundational part of the AI infrastructure stack.