dagster: Dagster is a data orchestrator purpose-built for data platforms in the MLOps era, helping users define, develop, and operate data assets. It offers a powerful programming model, local development experience, and a robust UI for observing and debugging pipelines in production.; doit: doit is an automation tool designed to bring the power of build-tools to execute any kind of task, serving as both a simple Task Runner and an efficient build-tool. It creates a DAG to ensure incremental builds and supports dynamic task dependencies for complex workflows and pipelines.
Building reliable data platforms
Organizing and running project-related ad hoc tasks.