Applied Data Engineering Solutions with Databricks

Applied Data Engineering Solutions with Databricks

Summary

This 2-day course is focused on helping users apply best practices when building a Lakehouse architecture on Databricks. Special emphasis is placed on Delta Lake, Structured Streaming, and data management.

Description

In this course, formerly known as Advanced Data Engineering with Databricks, participants will learn about advanced topics in building and maintaining data engineering workloads on Databricks. This course highlights those features of the Databricks Lakehouse platform that make it well-suited for production data engineering, with an emphasis on Spark 3, Delta Lake, Structured Streaming, and proprietary platform features.

Duration

2 Days

Objectives

Upon completion of this course, students should be able to:

  • Design and implement multi-pipeline multi-hop architecture to enable the Lakehouse paradigm.
  • Deploy Structured Streaming operations that take advantage of Databricks Job scheduling capabilities and avoid workspace limitations.
  • Implement Databricks-native code that leverages platform-specific Delta Lake features to simplify production workloads.
  • Master design patterns that enable common use cases, including change data capture (CDC), slowly changing dimensions (SCD), and managing personally identifiable information (PII).

Audience

Data Engineers

Prerequisites

  • Advanced experience using Apache Spark
  • Advanced experience coding with Python
  • Intermediate experience writing SQL queries
  • Intermediate experience using Databricks platform
  • Intermediate experience using Delta Lake
  • Intermediate experience using Structured Streaming
  • Intermediate knowledge of data engineering concepts

Additional Notes

All ​participants ​will ​need ​:

  • an ​internet ​connection

  • a ​device ​that is compliant with the following supported internet browsers

  • to ​confirm ​​​your ​​​device ​​​can ​​​run ​​​GoToTraining : ​ Validate

  • NOTE: GoToTraining ​is ​our chosen online ​platform ​through which the ​class ​will ​be ​delivered and ​prior ​to ​attendance, ​each ​registrant ​will ​receive ​GoToTraining ​log-in ​instructions.

  • Upcoming Classes

    No classes have been scheduled, but you can always Request a Class.