Systems | Development | Analytics | API | Testing

Scaling Kafka with WebSockets

Kafka is a highly popular realtime data streaming platform, renowned for handling massive volumes of data with minimal latency. Typical use cases include handling user activity tracking, log aggregation and IoT telemetry. Kafka’s architecture, based on distributed partitions, allows it to scale horizontally across multiple brokers. But although Kafka excels at high data throughput, scaling it to manage thousands of client connections can be costly and complex.

How to Visualize Real-Time Data from Apache Kafka using Apache Flink SQL and Streamlit

Data visualization is cool, but have you tried setting up a chart of real-time data? In this video, Lucia Cerchie shows you how to create a live visualization of market data. She starts by producing data to a topic in Confluent Cloud from an Alpaca API websocket, then processes that data with Flink SQL, and finally uses a Streamlit component for a real-time visualization.

What Made Current 2024 Unforgettable? Hear From Our Attendees | Current 2024

In this recap video from Current 2024, attendees share their favorite moments from the event. From insightful talks on data streaming innovation to hands-on workshops and networking opportunities, hear what participants found most valuable.

Windowing with Table-Valued Functions | Apache Flink SQL

Apache Flink SQL makes it easy to implement analytics that summarize important attributes of real-time data streams. There are four different types of time-based windows in Flink SQL: tumbling, hopping, cumulating, and session. Learn how these various window types behave, and how to work with the table-valued functions that are at the heart of Flink SQL’s support for windowing.

How Thrivent Uses Real-Time Data for AI-Driven Fraud Detection

In today’s fast-paced financial services landscape, customers have a shorter attention span than ever. To meet clients’ growing demands for real-time access to information and keep innovating in areas like fraud detection and personalized financial advice, Thrivent needed to overhaul its data infrastructure. With data scattered across siloed legacy systems, diverse tech stacks, and multiple cloud environments, the challenge was a bit daunting. But by adopting Confluent Cloud, Thrivent was able to unify its disparate data systems into a single source of truth.

SQL for data exploration in a multi-Kafka world

Every enterprise is modernizing their business systems and applications to respond to real-time data. Within the next few years, we predict that most of an enterprise's data products will be built using a streaming fabric – a rich tapestry of real-time data, abstracted from the infrastructure it runs on. This streaming fabric spans not just one Apache Kafka cluster, but dozens, hundreds, maybe even thousands of them.