Systems | Development | Analytics | API | Testing

dbt is the most commonly used tool by data engineers to define SQL transformations (as models), write tests, generate documentation, and deploy through CI/CD and now it’s available with Confluent Cloud too! The magic of dbt is that it brings the engineering rigor to modern data work and data engineering, regardless of the underlying compute source - Snowflake, BigQuery, Databricks, Redshift or Confluent. You can find out more about the launch in our Q2 Confluent Cloud Launch post and the keynote.

Moving a retrieval-augmented generation (RAG) prototype from a Python notebook into production isn't an API orchestration challenge. It's a distributed systems problem. For engineering managers and data platform leads, the build-versus-buy decision on streaming infrastructure will dictate your artificial intelligence (AI) feature velocity for the next three to five years. This guide assumes you've already prototyped a RAG pipeline.

The EU AI Act's general provisions are already in force, and high-risk AI system obligations apply from August 2026. The National Institute of Standards and Technology (NIST) AI Risk Management Framework and its Generative AI Profile set the baseline for what auditors expect, framing governance around four functions: identify, measure, manage, and monitor. Deploying artificial intelligence (AI) agents in regulated environments isn't a sandbox experiment anymore. It's a strict governance challenge.

Your team just shipped a microservices refactor. Services are smaller, deployments are faster, and boundaries are clearer. Then, during a design review, someone inevitably suggests: “We should use Kafka.”That suggestion might be the exact architectural breakthrough you need—or it could quietly introduce months of unnecessary operational complexity.This article serves as a practical decision framework.

We are proud to announce the release of Apache Kafka 4.3. This release contains many new features and improvements. This blog post will highlight some of the more prominent ones. For a full list of changes, be sure to check the release notes. With 25 KIPs and over 600 commits since 4.2.0, this release introduces many new features, improvements and bug fixes to all the components. See the Upgrading to 4.3 section in the documentation for the list of notable changes and detailed upgrade steps.

For most go-to-market (GTM) teams, understanding what’s really happening with a customer right now is harder than it should be. Usage data lives in one system, renewals in another, support escalations somewhere else—and field notes are scattered across tools and docs. By the time someone pieces together a full picture, it’s already out of date. As we began using our own data platform internally, this fragmentation became impossible to ignore.

People often imagine that to provide a cloud service for a piece of open source software is a simple matter of packaging up the open source and putting it in Kubernetes. We knew when we set out to build Confluent Cloud that a true cloud-native offering of Apache Kafka as a service would be much, much more than that.