Category: Apache Airflow

Production Data Pipelines with Apache Airflow: From DAG Design to Dynamic Task Generation

Posted on 1 min read

After 20 years in this industry, I’ve seen Production Data Pipelines with Apache Airflow evolve from [past state] to [current state]. The fundamentals haven’t changed, but the implementation details have. Let me share what I’ve learned. The Fundamentals Understanding the fundamentals is crucial. Many people skip this and jump to implementation, which leads to problems… Continue reading