Apache Kafka 3.8 is here! On behalf of the Kafka community, Danica Fine highlights key release updates, with KIPs from Kafka Core, Kafka Streams, and Kafka Connect. Kafka Core: Kafka Streams: Kafka Connect.

In this video, we'll look at some techniques for evolving events by analyzing a specific use case in a banking fraud detection system.

When developing or debugging a stream processing pipeline with Flink SQL, it’s common to inspect each processing step's output to ensure data is being transformed properly. However, comprehending the resulting data stream's structure, distribution, and characteristics entails executing multiple ad-hoc SQL queries, which can be time-consuming and tedious. Additionally, isolating specific subsets of the stream for analysis or debugging often involves even more queries, adding to the complexity and time required.

Learn how to leverage the native monitoring capabilities of the Python Kafka producer along with Confluent Cloud’s Metrics API while exploring how linger.ms affects latency and batch sizes.