Easy ETL with Auto Loader

Summary

Learn how to use Auto Loader to configure simple and powerful data ingestion for easy ETL in the Databricks Lakehouse.

Description

Databricks Auto Loader is the preferred method for ingesting incremental data landing in cloud object storage into your Lakehouse. This course introduces Auto Loader and demonstrates some of the newer features added to this product. Included are recommended patterns for data ingestion with Auto Loader.

Learning objectives

  • Describe the basic functionality and features of Auto Loader.

  • Use Auto Loader to ingest data to Delta Lake without losing data.

  • Configure automatic schema detection and evolution.

  • Rescue unexpected data arriving in well-structured datasets.

Prerequisites

  • Basic experience with Spark APIs

  • Basic knowledge of Delta Lake

  • Basic experience with Structured Streaming

Learning path

  • This course is part of the Data Engineering learning path.

Proof of completion

  • Upon 80% completion of this course, you will receive a proof of completion. 

 

Part of Learning Pathway(s)