Course

Airflow: Level 01

Apache Airflow is an open-source platform to programmatically author, schedule and monitor workflows. It was created at Airbnb and currently is a part of Apache Software Foundation. Airflow helps you to create workflows using Python programming language and these workflows can be scheduled and monitored easily with it. If you have many ETL(s) to manage, Airflow is a must-have. In this course you are going to learn everything you need to start using Apache Airflow. Starting from very basic notions such as, what is Airflow and how it works, to advanced concepts such as, how to create plugins and make real dynamic pipelines. This is an introductory course where you’ll learn everything you need to get started with Airflow. The course is designed for: - People who are curious about data engineering. - People who want to learn basic and advanced concepts about Apache Airflow. - People who like hands-on approach.

6 Lessons

 

Outcomes

By the end of the course, learners will be able to:

  • Install and configure Apache Airflow
  • Master core functionalities such as DAGs, Operators, Tasks, Workflows etc
  • Apply advanced concepts of Apache Airflow such as XCOMs, Branching and SubDAGs
  • Use Apache Airflow in a Big Data ecosystem with Hive, Spark, AWS S3 etc
  • Run a script, test, secure and manage backfills
  • Differentiate between different operators, models, Executors of Airflow
  • Perform production deployment of DAGs in Apache Airflow
  • Think, answer and implement solutions using Airflow to real data processing problems

Author: Rakesh MV

Co-Author: Shivanand Ukkali

 

Level: 01
Duration: 18 Hours
Pre-requisites: Prior programming or scripting experience

Python Programming: Level 0

 

What’s next: TBD

Click for information on Earning Credits for this course

Kindly click here to view the course videos (Microsoft Teams folder).
Note: The textual content in the lessons are not exclusive of video contents.