How to Analyze Data from a REST API with Flink SQL
Join Lucia Cerchie in a coding walkthrough, bridging the gap between REST APIs and data streaming. Together we’ll transform the OpenSky Network's live API into a data stream using Kafka and Flink SQL.
To see more data streaming in action, check out the demos on Confluent Developer: https://cnfl.io/3X9niaV
Not only do we change the REST API into a data stream in this walkthrough, but we clean up the data on the way! We use Flink SQL to make it more readable and clean, and in that way we keep more of the business logic away from the client code.
RELATED RESOURCES
► Flink 101 course: https://cnfl.io/3Kt3zLO
► Building Flink Apps in Java course: https://cnfl.io/4bOdN5E
► Give Confluent Cloud a try: https://cnfl.io/3VtRrAH
► Flink SQL tutorials: https://cnfl.io/3VsQRTH
► Flink SQL reference guide from Confluent: https://cnfl.io/45i4J6r
► Confluent CLI: https://cnfl.io/3wUqWee
► confluent-flink-quickstart: https://cnfl.io/4e6uwCC
► OpenSky Network terms of use: https://openskynetwork.github.io/opensky-api/
OpenSky Network citation: Matthias Schäfer, Martin Strohmeier, Vincent Lenders, Ivan Martinovic and Matthias Wilhelm."Bringing Up OpenSky: A Large-scale ADS-B Sensor Network for Research". In Proceedings of the 13th IEEE/ACM International Symposium on Information Processing in Sensor Networks (IPSN), pages 83-94, April 2014.
CHAPTERS
00:00 - Intro
00:41 - Step 1: Pre-requisites
00:57 - Step 2: Setup and source connector launch
01:54 - Step 3: Connector validation
02:46 - Step 4: Cleaning up the tables
03:37 - Step 5: Teardown
04:20 - Outro
ABOUT CONFLUENT
Confluent is pioneering a fundamentally new category of data infrastructure focused on data in motion. Confluent’s cloud-native offering is the foundational platform for data in motion – designed to be the intelligent connective tissue enabling real-time data, from multiple sources, to constantly stream across the organization. With Confluent, organizations can meet the new business imperative of delivering rich, digital front-end customer experiences and transitioning to sophisticated, real-time, software-driven backend operations. To learn more, please visit www.confluent.io.
#flinksql #apacheflink #apachekafka #flink #kafka #confluent #apis