Dagster
GitHub Support CommunityData Processing
Dagster is a data orchestrator. It lets you define pipelines (DAGs) in terms of the data flow between logical components called solids. These pipelines can be developed locally and run anywhere.
Features
Dagster models data dependencies between steps in your orchestration graph and handles passing data between them. Optional typing on inputs and outputs helps catch bugs early.
Track what’s produced by your pipelines with Dagster’s Asset Manager, so you can understand how your data was generated and trace issues when it doesn’t look how you expect.