dagster

Dagster is a data pipeline orchestrator for the whole development lifecycle. Instead of wiring up tasks, you declare the data assets you want (tables, datasets, ML models, reports) as Python functions with the `@dg.asset` decorator, and Dagster works out dependencies from function arguments and runs them at the right time to keep the assets up to date. The README's example builds three assets, a DataFrame, a scikit-learn model and a stats table, that load as a graph in the web UI.

It is meant to be used from local development and unit tests through staging to production. The README describes three roles: a productivity platform (reusable components, early data-quality checks), an orchestration engine (multi-tenant, multi-tool) and a control plane (metadata, cataloging, lineage and diagnostics in one place), plus a library of integrations with common data tools. Python 3.9 to 3.14 is supported, and it installs from PyPI (for example `uv add dagster dagster-webserver dagster-dg-cli`).

It is a data-engineering tool rather than an AI-agent tool, but it is widely used to orchestrate ML and data workflows. This review is based on the README and repository data; we have not run it ourselves.