Systems | Development | Analytics | API | Testing

August 2020

Archive data from to S3 with the new Kafka Connect connector

The new open-source #ApacheKafka Connect sink connector for #S3 gives you full control on how to sink data to S3 and save money on long term storage costs in #Kafka. The connector has the ability to flush data out in a number of different formats including #AVRO, #JSON, #Parquet and #Binary as well as ability to create S3 buckets based on partitions, metadata fields and value fields.

Introducing the Apache Kafka App Catalog

Working with Apache Kafka and real-time applications comes with challenges. Visibility into the deployed applications and their dependency on what we call the “data fabric” is one of them (For the sake of this blog, it means Kafka and all its state and configuration). If you’ve built a multi-tenant real-time data platform with Kafka, where teams are deploying applications outside your jurisdiction, this is where the pain is particularly acute. It goes something like this.

The best of Kafka Summit 2020

After a self-isolated and event-free spring, some of us around the world welcomed a more promising summer. You might be taking some time away on a socially distanced holiday. You might be taking some time away from the day-to-day at home. But if a cold beer in the sun isn't enough to make up for these difficult months, the premier event for the Streaming Data Community is back! Kafka Summit has gone virtual this year and that means you can attend the event from anywhere.