How to Analyze Data from a REST API with Flink SQL

How to Analyze Data from a REST API with Flink SQL

Join Lucia Cerchie in a coding walkthrough, bridging the gap between REST APIs and data streaming. Together we’ll transform the OpenSky Network's live API into a data stream using Kafka and Flink SQL.

To see more data streaming in action, check out the demos on Confluent Developer: https://cnfl.io/3X9niaV

Not only do we change the REST API into a data stream in this walkthrough, but we clean up the data on the way! We use Flink SQL to make it more readable and clean, and in that way we keep more of the business logic away from the client code.

RELATED RESOURCES
► Flink 101 course: https://cnfl.io/3Kt3zLO
► Building Flink Apps in Java course: https://cnfl.io/4bOdN5E
► Give Confluent Cloud a try: https://cnfl.io/3VtRrAH
► Flink SQL tutorials: https://cnfl.io/3VsQRTH
► Flink SQL reference guide from Confluent: https://cnfl.io/45i4J6r
► Confluent CLI: https://cnfl.io/3wUqWee
► confluent-flink-quickstart: https://cnfl.io/4e6uwCC
► OpenSky Network terms of use: https://openskynetwork.github.io/opensky-api/
OpenSky Network citation: Matthias Schäfer, Martin Strohmeier, Vincent Lenders, Ivan Martinovic and Matthias Wilhelm."Bringing Up OpenSky: A Large-scale ADS-B Sensor Network for Research". In Proceedings of the 13th IEEE/ACM International Symposium on Information Processing in Sensor Networks (IPSN), pages 83-94, April 2014.

CHAPTERS

00:00 - Intro

00:41 - Step 1: Pre-requisites

00:57 - Step 2: Setup and source connector launch

01:54 - Step 3: Connector validation

02:46 - Step 4: Cleaning up the tables

03:37 - Step 5: Teardown

04:20 - Outro

ABOUT CONFLUENT
Confluent is pioneering a fundamentally new category of data infrastructure focused on data in motion. Confluent’s cloud-native offering is the foundational platform for data in motion – designed to be the intelligent connective tissue enabling real-time data, from multiple sources, to constantly stream across the organization. With Confluent, organizations can meet the new business imperative of delivering rich, digital front-end customer experiences and transitioning to sophisticated, real-time, software-driven backend operations. To learn more, please visit www.confluent.io.

#flinksql #apacheflink #apachekafka #flink #kafka #confluent #apis