Apache Airflow (ML Edition)
by Apache Software Foundation · open-source · Last verified 2026-03-17
Battle-tested workflow scheduler for authoring, scheduling, and monitoring data and ML pipelines as directed acyclic graphs. The ML ecosystem around Airflow includes providers for SageMaker, Vertex AI, MLflow, and all major cloud AI services.
https://airflow.apache.org ↗B+
B+—Good
Adoption: A+Quality: AFreshness: B+Citations: AEngagement: F
Specifications
- License
- Apache 2.0
- Pricing
- open-source
- Capabilities
- dag-authoring, task-scheduling, dynamic-dags, xcom-communication, provider-ecosystem
- Integrations
- sagemaker, vertex-ai, databricks, snowflake, spark, kubernetes, dbt
- Use Cases
- ml-pipeline-scheduling, etl-orchestration, model-training-jobs, data-quality-checks, feature-engineering
- API Available
- Yes
- SDK Languages
- python
- Deployment
- self-hosted, kubernetes, cloud, docker, managed-astronomer, managed-cloud-composer
- Rate Limits
- N/A (open-source)
- Data Privacy
- Self-hosted; cloud providers vary
- Tags
- workflow, dag, scheduling, mlops, pipeline, apache
- Added
- 2026-03-17
- Completeness
- 100%
Index Score
74.8Adoption
92
Quality
80
Freshness
75
Citations
88
Engagement
0