Systems | Development | Analytics | API | Testing

Kafka

Stream Processing Simplified: An Inside Look at Flink for Kafka Users

There was a huge amount of buzz about Apache Flink® at this year’s Kafka Summit London. From an action-packed keynote to standing-room only breakout sessions, it's clear that the Apache Kafka® community is hungry to learn more about Flink and how the stream processing framework fits into the modern data streaming stack.

How Let's Encrypt Powers Confluent Cloud to Automate Its Certificate Operations

Since the inception of our cloud journey, we have extensively utilized Let's Encrypt because it has been very reliable, fully automated, open, and free. Today, we’re proud to become an official sponsor of Let’s Encrypt. In this blog post, we’re celebrating this event by explaining our journey with Let’s Encrypt, how we integrate with their service, and why we chose them.

Ably Kafka Connector 3.0: Increased throughput, improved error handling, Confluent Cloud accreditation

We are excited to announce the release of the Ably Kafka Connector 3.0. Version 3 brings a host of improvements, including: Overall, the Ably Kafka connector v3.0 makes the management of Kafka pipelines extension to millions of web and mobile users simpler and more reliable.

Lenses 5.2: for a healthier Kafka with less manual effort

Kafka adoption is growing fast. Very fast. At Lenses, we’re pushing out new features to increase developer productivity, reduce manual effort & improve the cost & hygiene of operating your Kafka platform. Only a few weeks since Lenses 5.1, yet here we are again with more goodies in our release 5.2.

Lenses 5.1 - A 1st class ticket to be event-driven in AWS

Hello again. We strive to improve the productivity of developers building event-driven applications on the technology choices that best fit your organization. AWS continues to be a real powerhorse for our customers. Not just for running the workloads, but in supporting them with their native services: MSK Kafka, MSK Connect and now increasingly Glue Schema Registry. This is bringing a strong alternative to Confluent and their Kafka infrastructure offerings.

Secret rotation for Kafka Connect connectors with AWS Secret Manager

With version 5.1, Lenses is now offering enterprise support for our popular open-source Secret Provider to customers. In this blog, we’ll explain how secrets for Kafka Connect connectors can be safely protected using Secret Managers and walk you through configuring the Lenses S3 Sink Connector with the Lenses Secret Provider plugin and AWS Secret Manager.

The Glue Schema that binds Apache Kafka

With increased applications developed by different engineering teams on Kafka comes increased need for data governance. JSON is often used when streaming projects bootstrap but this quickly becomes a problem as your applications iterate, changing the data structures with add new fields, removing old and even changing data formats. It makes your applications brittle, chaos ensues as downstream consumers fall over due to miss data and SREs curse you.

From Kinesis to Kafka

At the beginning of 2021, a brand new data team was assembled to build a real-time data platform for Kong’s SaaS platform, Konnect. Our mission is to provide top-notch real-time API analytics features for Konnect customers. V1 platform architecture The initial backend consisted of three main components: an ingestion service to handle incoming telemetry requests, an intermediate message queue, and a real-time OLAP data storage.