Data Engineer

Course length

12 Weeks | 120 Hours

Data Engineer

Mentor

Jonatas Piscirilo

As The AI Academy co-founder & CTO, Jonatas has been driving technological innovation and working on several initiatives and projects related to data and artificial intelligence in large enterprises. Over 18 years of experience building teams, changes, and large scale solutions.

About this path

The Data Engineer ensures quality data are available in an efficient and repeatable way

 

Syllabus

Week 1
Live session with Mentor
Introduction to Data Engineering

Week 2
Live session with Mentor
Data Ingestion

Week 3
Live session with Mentor
Writing Efficient Code
Writing Functions
Object Oriented Programming

Week 4
Live session with Mentor
Intro to Shell
Data Processing in Shell

Week 5
Live session with Mentor
Intro to Bash Scripting
Unit Testing for Data Science

Week 6
Live session with Mentor
Building Data Pipelines
Intro to Airflow

Week 7
Live session with Mentor
Intro to PySpark
Cleaning Data with PySpark
Big Data Fundamentals with PySpark

Week 8
Live session with Mentor
Intro to Relational DB with SQL
Introduction to MongoDB
Database Design

Week 9
Live session with Mentor
Introduction to Spark SQL
Transactions/Error Handling in SQL

Week 10
Live session with Mentor
Building Triggers in SQL Server
Improving Query performance in SQL S

Week 11
Live session with Mentor
Intro to Scala
Intro to AWS Boto

Week 12
Project