Luigi 

Luigi is a Python (3.6, 3.7, 3.8, 3.9 tested) package that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization, handling failures, command line integration, and much more.

Features

Luigi has features such as visualiser page, dependency graph, task, scheduler, explicit dependencies between workflows, easier to write, vastly more extensible, code reuse and can easily re-run individual steps.

Official website

Tutorial and documentation

Enter your contact information to continue reading