dagster: Dagster is a data orchestrator purpose-built for data platforms in the MLOps era, helping users define, develop, and operate data assets. It offers a powerful programming model, local development experience, and a robust UI for observing and debugging pipelines in production.; FedML: FedML is a unified and scalable open-source machine learning library powered by TensorOpera AI, enabling training and deployment of AI jobs anywhere at any scale. It offers holistic support for MLOps, scheduling, and high-performance ML libraries, including federated learning, distributed training, and generative AI functionalities.
Building reliable data platforms
Distributed training and fine-tuning of large models (including LLMs)