Apache Kafka® 3.7 is here! On behalf of the Kafka community, Danica Fine highlights key release updates, with KIPs from Kafka Core, Kafka Streams, and Kafka Connect. Kafka Core: Kafka Streams: Kafka Connect: Many more KIPs are a part of this release. See the blog post for more details.

#kafka #apachekafka #secops #securityoperations

Unlock the secrets to complex schema evolution with Gilles Philippart! Discover how migration rules can transform what is usually a daunting task into a painless process with no breaking changes.

Event Sourcing is a pattern of storing an object's state as a series of events. Each time the object is updated a new event is written to an append-only log. When the object is loaded from the database, the events are replayed in order, reapplying the necessary changes. The benefit of this approach is that it stores a full history of the object. This can be valuable for debugging, auditing, building new models, and a variety of other situations. It is also a technique that can be used to solve the dual-write problem when working with event-driven architectures.

See how to build a streaming data pipeline to a cloud data warehouse for real-time analytics. Leverage Confluent Data Streaming Platform to.

How to manage consumer offsets in #apachekafka.

The transactional outbox pattern leverages database transactions to update a microservice's state and an outbox table. Events in the outbox will be sent to an external messaging platform such as Apache Kafka. This technique is used to overcome the dual-write problem which occurs when you have to write data to two separate systems such as a database and Apache Kafka. The database transactions can be used to ensure atomic writes between the two tables. From there, a separate process can consume the outbox and update the external system as required.