Systems | Development | Analytics | API | Testing

We’re introducing enhanced visibility for streaming workload performance on Confluent Cloud, making it easier for developers and operators to understand, troubleshoot, and optimize real-time applications. As Apache Kafka has become the backbone of data streaming, many teams rely on Confluent Cloud for its scale, elasticity, and reduced operational burden.

Last week, a fleet of autonomous robotaxis in China suddenly stopped working—at scale. Over a hundred vehicles stalled across a city, stranding passengers in traffic and raising immediate concerns about safety, reliability, and trust in autonomous systems. This wasn’t just a bad day for self-driving cars. It was a distributed systems failure, one that happened in the physical world, not just in dashboards.

Performance testing without observability is like diagnosing a sick patient using only a thermometer. You get one number. You miss everything that matters. Observability-driven performance testing combines load testing with metrics, logs and distributed tracing to identify not just when performance degrades, but exactly why.

Observability is the key to understanding and improving MCP servers. These servers connect AI agents to tools, but without visibility, issues like slow responses, errors, or security risks can go undetected. Observability helps track how agents interact with tools, pinpoint failures, and optimize performance.

Most teams already know a lot about production. They have dashboards. They have traces. They have alerts. They have enough telemetry to explain what happened after an incident and enough graphs to argue about it for the rest of the week. Then they go to test a change and start from scratch. The integration tests hit a hand-written mock that returns {"status": "ok"}. The load tests replay a CSV somebody exported months ago. Staging is close enough to production right up until it matters.

In the current enterprise technology landscape, we’re witnessing an industry-wide scramble. As organizations shift from monolithic architectures to complex environments leveraging heterogeneous infrastructures, cloud-based data platforms are hitting a visibility—i.e., observability—wall. Their response has been a wave of reactive, multi-billion-dollar acquisitions designed to "bolt-on" the observability that they lack natively.

There's a common belief in the observability space: if you just collect more data, you'll have what you need to debug any issue. The reality is more frustrating: even with 100% unsampled observability, you're still missing critical debugging data. There's a common belief in the observability space: if you just collect more data, you'll have what you need to debug any issue. The reality is more frustrating: even with 100% unsampled observability, you're still missing critical debugging data.

For years, the Kubernetes sidecar pattern has been a practical way to capture observability data. Running a collector alongside each application pod gave us deep visibility into traffic, including full request and response payloads across supported protocols. However, as cloud-native environments have grown more complex, the limitations of sidecars—such as resource overhead, operational complexity, and scaling challenges—have become more apparent.