Systems | Development | Analytics | API | Testing

September 2024

Expanding Confluent's Integration with Microsoft Azure: Create and Manage Confluent Resources Directly from the Azure Portal with Confluent's Fully Managed Connectors (Preview)

We are thrilled to announce yet another milestone in our integration capabilities with Microsoft Azure. Now, you can manage Confluent resources (Preview) directly from the Azure portal. This new capability not only simplifies the setup and management process but also empowers you to leverage the full potential of Confluent's data streaming platform on Azure.

Atomic Tessellator: Revolutionizing Computational Chemistry with Data Streaming

Computational chemistry relies on large volumes of complex data in order to provide insights into new applications, whether it’s for electric vehicles or new battery development. With the emergence of generative AI (GenAI), the rapid, scalable processing of this data has become possible and critical to investigate previously unexplored areas in catalysis and materials science.

How to source data from AWS DynamoDB to Confluent using the Open-Source Connector

This is a one-minute video showing an animated architectural diagram of an integration between Amazon DynamoDB and Confluent Cloud using an open-source Kafka connector. The integration allows you to avoid maintaining custom code, and gives you the ability to automatically discover and adapt to changes in DynamoDB tables. All details are provided.

Handling the Producer Request: Kafka Producer and Consumer Internals, Part 2

Welcome to the second installment of our blog series to understand the inner workings of the beautiful black box that is Apache Kafka. We’re diving headfirst into Kafka to see how we actually interact with the cluster through producers and consumers. Along the way, we explore the configurations that affect each step of this epic journey and the metrics that we can use to more effectively monitor the process.

How to source data from AWS DynamoDB to Confluent using Kinesis Data Streams and Connect

This is a one-minute video showing an animated architectural diagram of an integration between Amazon DynamoDB and Confluent Cloud using Kinesis Data Streams and the Kinesis Data Streams connector. It’s a fully managed and serverless solution that reduces operational complexity and leverages scalability and cost-effectiveness.

Current 2024 Keynote Day 1 - Data Streaming in the Age of AI

Get ready to dive into the future of data! Jay Kreps and Confluent's top minds are set to reveal the next game-changing evolution of data infrastructure that effectively leverages AI and ubiquitous automation. Discover how leaders in professional services, media, and automotive industries are harnessing the power of Confluent's Data Streaming Platform and Data Products to revolutionize their operations. This is your chance to see how real-time data is driving innovation, transforming decisions, and propelling businesses into the future. The future is here—let's ignite it together!

Current 2024 Day 2 Keynote - The Rise of Data Streaming Engineers

What does the rise of data streaming mean for software and data professionals? A once-in-a-generation paradigm shift is giving rise to a brand new role: the Data Streaming Engineer. Come hear from your fellow technologists how this new role is bringing together the work of application developers, data engineers, data scientists, and architects, and how it might describe the work you're already doing. Along the way, we'll get a survey of the current state of the technologies underlying the emerging Data Streaming Platform, and how you can prepare to use them in the coming year.

Unlocking Real-Time Insights and Making Migrations a Breeze: 13 New Partner Solutions and Offerings Launched

Whether you’re looking to jump-start a new use case, or accelerate your migration journey, collaborating closely with Confluent and our partners enables you to fulfill the increasing need for real-time customer experiences and applications, opening doors to new growth and success opportunities.

Confluent Supercharges Apache Flink Offering with New Developer Tools and Enterprise-Ready Security

Confluent adds Table API support for Apache Flink® making it even easier for developers to use Java or Python to build streaming applications. New private networking and encryption features safeguard data streams for enterprises in highly regulated industries.

Introducing Confluent's OEM Program: Deliver Data Streaming Faster and Unlock Revenue Growth

As real-time experiences become integral to everyday life and data-driven technologies like GenAI drive the next wave of innovation, one thing is clear: real-time data is the key to enterprise growth. The Confluent OEM Program empowers MSPs, CSPs, and ISVs to quickly unlock new revenue streams by integrating the industry-leading data streaming platform into their customer offerings.

New in Confluent Cloud: Making Serverless Flink a Developer's Best Friend, Protecting Sensitive Data, and More

The Q3 Cloud Bundle Launch comes to you from Current 2024, where data streaming industry experts have come together to show you why data streaming is critical today, especially in the age of AI, and how it will become even more important in shaping tomorrow’s businesses. This year’s Current event attracted over 2,500 attendees, both in-person and virtual, and featured 140+ learning sessions by industry experts.

Build, Manage, and Monitor Data Streaming Applications, All Within Your Favorite IDE

Today, we’re excited to announce the Early Access Program for Confluent for VS Code! Confluent has integrated support for Visual Studio with Confluent for VS Code that streamlines workflows, accelerates development cycles, and enhances real-time data processing, all within a unified environment.

The Confluent Q3 '24 Launch

The Confluent Q3 ’24 Launch is packed with the new features and tools needed to make stream processing a developer’s best friend. We’re also introducing Client-Side Field Level Encryption, WarpStream, and more! Our quarterly launches provide a single resource to learn about the accelerating number of new features we’re bringing to Confluent Cloud, our cloud-native data streaming platform.

Real-Time Insurance Claims Processing With Confluent

The insurance industry has undergone a massive transformation over the last 20-30 years. Customer service and business processes that were once done on paper or over the phone are now completed via web and mobile experiences. As a result, manual paperwork and tasks have gradually become automated via software solutions. For the insurance customer, this transition has led to faster and more predictable experiences.

New research from Confluent sees IT leaders share their biggest AI implementation challenges

12th September 2024 - Skills shortages are the #1 challenge facing IT leaders looking to implement artificial intelligence in 2024. That's according to research from Confluent, released ahead of this year's BigData LDN event. The research, which surveyed over 500 UK IT leaders, explores the top challenges facing IT departments when it comes to adopting and implementing AI.

How Developers Can Use Generative AI to Improve Data Quality

It sounds counterintuitive—using a technology that has trust issues to create more trustworthy data. But smart engineers can put generative AI to work to improve the quality of their data, allowing them to build more accurate and trustworthy AI-powered applications.

Shift left to write data once, read as tables or streams

Shift Left is a rethink of how we circulate, share and manage data in our organizations using DataStreams, Change Data Capture, FlinkSQL and Tableflow. It addresses the challenges with multi-hop and medallion architectures using batch pipelines by shifting the data preparation, cleaning and schemas to the point where data is created and as a result, you can build fresh trustworthy datasets as streams for operational use cases or Apache Iceberg tables for analytical use cases.

How to source data from AWS DynamoDB to Confluent using DynamoDB Streams and AWS Lambda

This is a one-minute video showing an animated architectural diagram of the integration between Amazon DynamoDB and Confluent Cloud using DynamoDB Streams and AWS Lambda. Details of the integration are provided via narration.

Confluent + WarpStream = Large-Scale Streaming in your Cloud

I’m excited to announce that Confluent has acquired WarpStream, an innovative Kafka-compatible streaming solution with a unique architecture. We’re excited to be adding their product to our portfolio alongside Confluent Platform and Confluent Cloud to serve customers who want a cloud-native streaming offering in their own cloud account.

How Producers Work: Kafka Producer and Consumer Internals, Part 1

I shouldn’t have to convince anyone that Apache Kafka is an incredibly useful and powerful technology. As a distributed event streaming platform, it’s adept at storing your event data and serving it up for downstream consuming applications to make sense of that information––in real time or as close to real time as your use case permits. The real beauty of Kafka as a technology is that it can do it with very little effort on your part. In effect, it’s a black box.

Connect with Confluent: Celebrating One Year and 50+ Integrations

In just 12 short months, the Connect with Confluent (CwC) technology partner program has transformed from a new, ambitious initiative to expand the data streaming ecosystem into a thriving portfolio that’s rapidly increasing the breadth and value of real-time data. The program now provides a portfolio of 50+ integrations, each one amplifying the capabilities of Confluent's unified data streaming platform for Apache Kafka and Apache Flink.

Let Flink Cook: Mastering Real-Time Retrieval-Augmented Generation (RAG) with Flink

Commercial and open source large language models (LLMs) are evolving rapidly, enabling developers to create innovative generative AI-powered business applications. However, transitioning from prototype to production requires integrating accurate, real-time, domain-specific data tailored to your business needs and deploying at scale with robust security measures.