This hands-on self-paced training course targets Data Engineers who want to process big data using Apache Spark™ Structured Streaming. The course ends with a capstone project building a complete data streaming pipeline using structured streaming.
3-6 hours, 75% hands-on
The course is a series of five self-paced lessons plus a final capstone project building a complete data pipeline using Structured Streaming. Each lesson includes hands-on exercises.
Supported platforms include Azure Databricks, Databricks Community Edition, and non-Azure Databricks.
- If you're planning to use the course on Azure Databricks, select the "Azure Databricks" Platform option.
- If you're planning to use the course on Databricks Community Edition or on a non-Azure version of Databricks, select the "Other Databricks" Platform option.
During this course you:
- Use the interactive Databricks notebook environment
- Ingest streaming log file data
- Aggregate small batches of data with time windows
- Stream data from a Kafka connection
- Use Structured Streaming in conjunction with Databricks Delta
- Visualize streaming live data
- Use Structured Streaming to analyze streaming Twitter data
- Getting Started with Apache Spark™ DataFrames self-paced course (optional, but strongly encouraged)
- Structured Streaming Concepts
- Time Windows
- Using Kafka
- Capstone Project
- Please be sure to use a supported browser.
This self-paced training course may be used by 1 user for 12 months from the date of purchase. It may not be transferred or shared with any other user.