AI & Data
Data Engineering roadmap
Pipelines, batch and streaming, storage architectures, Kafka, data quality and orchestration.
- Intermediate
- about 7 hours
- 7 modules
- 9 steps
What you will learn
1. Data Pipelines
ETL and ELT, batch versus streaming, and orchestrating pipelines.
- ETL, ELT & Pipelines — Data Engineering · ETL, ELT & Pipelines
- Batch & Streaming — Data Engineering · Batch & Streaming
- Orchestration — Data Engineering · Orchestration
2. Data Storage & Formats
Warehouses, lakes and lakehouses, and storing data efficiently with Parquet and partitioning.
- Warehouse, Lake & Lakehouse — Data Engineering · Warehouse, Lake & Lakehouse
- Parquet & Partitioning — Data Engineering · Parquet & Partitioning
3. SQL for Analytics
Summarize data with GROUP BY and HAVING, and answer harder questions with subqueries, CTEs and window functions.
- GROUP BY & HAVING — Databases · GROUP BY & HAVING
- Subqueries, CTEs & Window Functions — Databases · Subqueries, CTEs & Window Functions
4. Data Analysis with NumPy & pandasOptional
Load, clean, join and aggregate tabular data with NumPy and pandas.
- NumPy — Data Analysis with Python · NumPy
- pandas Fundamentals — Data Analysis with Python · pandas Fundamentals
- Data Cleaning — Data Analysis with Python · Data Cleaning
- Merge & Join — Data Analysis with Python · Merge & Join
- GroupBy & Aggregation — Data Analysis with Python · GroupBy & Aggregation
5. BI Modeling & ToolsOptional
Model data for BI and learn the core concepts shared by Power BI, Tableau and Looker.
- Data Modeling for BI — Data Visualization & BI · Data Modeling for BI
- Power BI, Tableau & Looker Concepts — Data Visualization & BI · Power BI, Tableau & Looker Concepts
6. AI-Assisted DevelopmentOptional
Work effectively with coding agents: give them context and instruction files, prompt for code, connect tools with MCP and review what they produce.
- Coding Agents — AI-Assisted Development · Coding Agents
- Context & Instruction Files — AI-Assisted Development · Context & Instruction Files
- Prompting for Code — AI-Assisted Development · Prompting for Code
- Tools & MCP — AI-Assisted Development · Tools & MCP
- Reviewing & Testing AI-Generated Code — AI-Assisted Development · Reviewing & Testing AI-Generated Code
7. Kafka & Data Quality
Stream events with Kafka and keep data trustworthy with schema evolution and quality checks.
- Kafka — Data Engineering · Kafka
- Schema Evolution & Data Quality — Data Engineering · Schema Evolution & Data Quality
Each step unlocks when you answer its questions well enough, and a module you finish counts in every roadmap that shares it.
Start this roadmap for free