Dagster: Open Source Cloud-Native Data Pipeline Orchestrator with Asset Lineage and Quality Checks
Dagster is an open source orchestrator for data pipelines that manages execution order, dependency tracking, and data lineage visualization. It enables defining data as Python assets with dependencies mapped in a lineage graph visible in a web UI. The platform supports automated data quality checks that can block downstream asset materialization if failed, helping prevent broken data propagation. Dagster supports Python 3.9 to 3.14 and offers integrations with popular data tools such as dbt. It provides flexible backfill capabilities for partitioned assets, optimizing runs by allowing either individual or grouped executions. The tool is freely available on GitHub for deployment and integration into data workflows.
Linha do tempo
-
Dagster: Open Source Cloud-Native Data Pipeline Orchestrator with Asset Lineage and Quality Checks
ler a análise desta fonte ocultar
Dagster is an open source orchestrator for data pipelines that manages execution order, dependency tracking, and data lineage visualization. It enables defining data as Python assets with dependencies mapped in a lineage graph visible in a web UI. The platform supports automated data quality checks that can block downstream asset materialization if failed, helping prevent broken data propagation. Dagster supports Python 3.9 to 3.14 and offers integrations with popular data tools such as dbt. It provides flexible backfill capabilities for partitioned assets, optimizing runs by allowing either individual or grouped executions. The tool is freely available on GitHub for deployment and integration into data workflows.