Systems | Development | Analytics | API | Testing

Most teams building multi-agent systems hit the same wall at roughly the same point. The prototype works. Agents chain together, tasks complete, the demo impresses the room. Then someone asks: "What happens when this runs a thousand times a day? What happens when an agent calls an external API that's down? How do we know what the agents actually did?" That's when the architecture conversation starts. Here's the framing that clarifies most of these questions.

Every team running production LLMs has had the same idea: not every request needs the frontier model. Intelligent model routing (or LLM routing)— choosing a model per request on criteria such as task complexity, cost, latency, or quality — enables more efficient model usage.

You've probably seen the demos. An AI model kicks off a task, hands pieces of it to other AI models, and somehow the whole thing gets done. Emails drafted, code reviewed, reports summarized — all without a human in the loop. While a single agent doing one thing is impressive, the true paradigm shift occurs when transitioning from single-agent to multi-agent AI systems. It looks like magic. It isn't.

You Can't Govern What You Can't See A decade ago, cloud and API sprawl got ahead of governance, and enterprises spent years trying to account for costs they'd never tracked. Today, we're seeing the same pattern around AI, with hundreds of customers proxying traffic via Kong AI Gateway, which includes LLM, MCP, and agent connectivity. *AI spending will reach $2.59 trillion in 2026.* I regularly like to share what we're seeing in production at Kong.

You've diagnosed the problem. Your agentic AI initiatives are stalling — not because the models are wrong, but because the integration layer underneath them wasn't built for this. Batch data, rigid schemas, fragmented governance, no real-time event delivery. Now the question is: what do you actually build, and how do you build it without tearing down the infrastructure you already have?

If you've been around long enough to remember when iPaaS was the answer to everything, you know the pattern. New paradigms arrive. Someone realizes that wiring it into existing infrastructure is harder than the demos suggested. An integration layer gets built. That layer slowly becomes load-bearing. Eventually, the integration layer becomes the bottleneck. We're at that moment again — except this time, the new paradigm is agentic AI, and the bottleneck is forming faster than usual.

You swapped in a better model. You fine-tuned it. You threw more tokens at the problem. And still — your agents hallucinate, break under load, and deliver answers that were accurate about three hours ago. The model isn't the problem. Gartner recently flagged that up to 40% of enterprise agentic AI initiatives are at risk of failure. Executives see that number and immediately audit their LLM provider. Their infrastructure team. Their prompts.

The new Konnect Dev Portal Toolkit extension for Visual Studio Code gives you a live, portal-accurate preview of your content right beside the file you are editing. You watch a page take shape exactly as it will appear to your developers, as you type, without ever leaving your local editor.

*You wouldn't expose a REST API without a gateway. What about Kafka?* You would never expose a REST API without an API gateway in front of it. Authentication, rate limiting, observability, access control — these aren't optional extras. They're mandatory, and we've spent the last decade building the API gateway pattern to solve exactly this problem. So here's the question worth sitting with: would you expose Kafka or any event stream without a gateway?

If you've been following the agentic AI wave, you've probably noticed that the conversation tends to center on the agents themselves: which LLM to use, which orchestration framework to pick, which use cases to tackle first. But a growing body of analyst research is pointing to a different bottleneck, one that's hiding in plain sight: integration. Forrester's David Mooter argues that integration must sit at the center of your AI strategy — not as plumbing, but as a strategic capability.