Systems | Development | Analytics | API | Testing

Engineering is in the middle of an almighty shift. Thanks to AI code-generation solutions, Engineers are being asked to take on a different and wider set of responsibilities in order to be more productive. It’s what’s increasingly being coined as Agentic Engineering - using AI agents to accelerate engineering & operations work while maintaining human oversight, quality and rigour.

You can now drive Lenses from Cursor, VS Code, IBM Bob or Claude Code without running any extra piece of infrastructure locally. Lenses MCP offers secure tools across topics, schemas, Kafka Connect, SQL processors, consumer groups, datasets and pod logs: everything an engineer would normally click through in the Lenses UI, now reachable from any MCP-compatible client over HTTP.

If you've written a line of code in the last 18 months, you already know this. Tools like Claude, Codex, Bob, Kiro and Cursor have made agentic software engineering the default. Most developers today are writing prompts as much as they are writing code. That shift changes what ‘developer experience’ means. Clean UIs, useful tools and good docs are still the foundation but the focus has shifted to ensuring a coding agent actually knows what it is doing, in the hands of a developer.

Adding large language models (LLMs) and artificial intelligence (AI) to real-time event streams comes down to one thing: picking the right boundary between data transport and model compute. Where you run inference determines your system's resilience, latency, and cost. This article is for data engineers, streaming architects, and developers who want to add AI capabilities to their Apache Kafka event backbone without destabilizing production consumer groups or blowing through API rate limits.

Most engineering teams adopt Apache Kafka for one simple reason: it works. It scales effortlessly, it is incredibly reliable, and it powers real-time systems across almost every industry. But as your Kafka usage expands across different teams, regions, and external consumers, success creates a brand new problem. Kafka is a massive data firehose, and without the right nozzle, it quickly becomes unmanageable.