Luigi 

Luigi is a Python (3.6, 3.7, 3.8, 3.9 tested) package that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization, handling failures, command line integration, and much more. Features Luigi has features such as visualiser page, dependency graph, task, scheduler, explicit dependencies between workflows, easier to write, vastly more […]

Metaflow

[wtm_mlop_cats] Metaflow is a human-friendly Python library that helps scientists and engineers build and manage real-life data science projects. Metaflow was originally developed at Netflix to boost the productivity of data scientists who work on a wide variety of projects from classical statistics to state-of-the-art deep learning.Metaflow provides a unified API to the infrastructure stack […]

Kedro

[wtm_mlop_cats] Kedro is an open-source Python framework for creating reproducible, maintainable and modular data science code. It borrows concepts from software engineering best-practice and applies them to machine-learning code; applied concepts include modularity, separation of concerns and versioning. Features Project TemplateData CatalogPipeline AbstractionCoding StandardsFlexible Deployment Official website Link Tutorial and documentation Click here to view […]

Flyte 

[wtm_mlop_cats] Flyte’s main purpose is to increase the development velocity for data processing and machine learning, enabling large-scale compute execution without the operational overhead. Teams can therefore focus on the business goals and not the infrastructure. Features Container Native Extensible Backend & SDK’s Ergonomic SDK’s in Python, Java & Scala Versioned & Auditable – all […]

MLRun

[wtm_mlop_cats] MLRun is an end-to-end open-source MLOps solution to manage and automate your entire analytics and machine learning lifecycle, from data ingestion, through model development to full pipeline deployment. MLRun is running as a built-in service in Iguazio and is integrated well with other services in the platform. Its primary goal is to ease the […]

Couler 

[wtm_mlop_cats] Couler aims to provide a unified interface for constructing and managing workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow. Couler is included in CNCF Cloud Native Landscape and LF AI Landscape. Features Many workflow engines exist nowadays, e.g. Argo Workflows, Tekton Pipelines, and Apache Airflow. However, their programming […]

Kale

[wtm_mlop_cats] KALE (Kubeflow Automated pipeLines Engine) is a project that aims at simplifying the Data Science experience of deploying Kubeflow Pipelines workflows. Kubeflow is a great platform for orchestrating complex workflows on top Kubernetes and Kubeflow Pipeline provides the mean to create reusable components that can be executed as part of workflows. The self-service nature […]

Prefect

[wtm_mlop_cats] Prefect is a new workflow management system, designed for modern infrastructure and powered by the open-source Prefect Core workflow engine. Users organize Tasks into Flows, and Prefect takes care of the rest. Features Automate all the thingsTest local, deploy globalSimple but powerful Official website Link Tutorial and documentation Click here to view See more […]

Automate Studio

[wtm_mlop_cats] Organizations embarking on intelligent process automation initiatives can rapidly build and deploy AI-powered workflows and integrate resulting insights into business applications and processes. Transform audio, video, text and data content into actionable intelligence, at scale, with no AI expertise. Features Advertising.Data Management.Energy Management.Face & Voice Recognition.Media Management & Monetization.MLOps & ModelOps.Recruitment.Redaction.Advertising.Data Management.Energy Management.Face & […]

ZenML

[wtm_mlop_cats] ZenML is an extensible, open-source MLOps framework to create production-ready machine learning pipelines. It has a simple, flexible syntax, is cloud and tooling agnostic, and has interfaces/abstractions that are catered towards ML workflows. At its core, ZenML pipelines execute ML-specific workflows from sourcing data to splitting, preprocessing, training, all the way to the evaluation […]