Systems | Development | Analytics | API | Testing

Kafka Live Stream #26 | Rapid development for real-time analytics with @Implydata

How can engineers enable real-time insights when working with high-throughput, data-intensive streams? In this 30-minute session, Imply and Lenses.io show you how to Enable self-service access for developers working with critical, high-velocity data flows in #apache #kafka Ingest and normalize complex data structures, enabling real-time analytics at scale via modern databases like #druid.

Running Apache Kafka at the Edge Requires Confluent's Enterprise-Grade Data Streaming Platform

Modern edge computing is transforming industries including manufacturing, healthcare, transportation, defense, retail, energy, and much more—pushing data management to far-reaching data sources to enable connected, low latency operations and enhanced decision making. These new use cases shift workloads to the left—requiring real-time data streaming and processing at the edge, right where the data is generated.

Microservice Pitfalls: Solving the Dual-Write Problem | Designing Event-Driven Microservices

When building a distributed system, developers are often faced with something known as the dual-write problem. It occurs whenever the system needs to perform individual writes to separate systems that can't be transactionally linked. This situation creates the potential for data loss if the developer isn't careful. However, techniques such as the Transactional Outbox Pattern and Event Sourcing can be used to guard against the potential for data loss while also providing added resilience to the system.

Tabs or spaces? Merge vs. rebase? Let's settle it with confluent-kafka-javascript

Tabs or spaces? Merge vs. rebase? Flink SQL vs. KStreams? Let’s Settle This is powered by a new Kafka JavaScript client from Confluent: confluent-kafka-javascript (early access). Find out how Lucia used it to make the website in the video above.

What is a Headless Data Architecture?

The headless data architecture. Is it a fad? Some marketecture? Or something real? In this video, Adam Bellemare takes you through the basics of the headless data architecture and why it’s beginning to emerge as its own respective pattern. Driven by the decoupling of data computation from storage, the headless data architecture provides the basis for a modular data ecosystem. Stream your data for near real-time low latency use cases, or convert it to an Iceberg table for analytical use cases.

How to Turn a REST API Into a Data Stream with Kafka and Flink

In the space of APIs for consuming up-to-date data (say, events or state available within an hour of occurring) many API paradigms exist. There are file- or object-based paradigms, e.g., S3 access. There’s database access, e.g., direct Snowflake access. Last, we have decoupled client-server APIs, e.g., REST APIs, gRPC, webhooks, and streaming APIs.

Ensuring the performance of your Kafka-dependent applications

In today’s data-driven world, Apache Kafka has emerged as an essential component in building real-time data pipelines and streaming applications. Its fault tolerance, scalability, and ability to handle high throughput makes it a great choice for businesses handling high volumes of data.