BOOK A MEETING FR Extract, transform, load using visual programming that can run Spark jobs on any environment Create and debug from your browser and export into pure python code! Features Up and running as simple as docker pull Create complex pipelines and flows using drag and drop Debug and preview step by step Integrated …
MLOP Category Archives:
Azkaban
BOOK A MEETING FR Azkaban is a batch workflow job scheduler created at LinkedIn to run Hadoop jobs. Azkaban resolves the ordering through job dependencies and provides an easy to use web user interface to maintain and track your workflows. Features Compatible with any version of Hadoop Easy to use web UI Simple web and …
Argo Workflows
BOOK A MEETING FR Argo Workflows is an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes. Argo Workflows is implemented as a Kubernetes CRD (Custom Resource Definition). Features Workflow: a Kubernetes resource defining the execution of one or more template. Workflows are named. Template: a step, steps or dag. Step: a single …
Apache Nifi
BOOK A MEETING FR Put simply, NiFi was built to automate the flow of data between systems. While the term ‘dataflow’ is used in a variety of contexts, we use it here to mean the automated and managed flow of information between systems. This problem space has been around ever since enterprises had more than …
Airflow
BOOK A MEETING FR Airflow is a platform that lets you build and run workflows. A workflow is represented as a DAG (a Directed Acyclic Graph), and contains individual pieces of work called Tasks, arranged with dependencies and data flows taken into account. Features 1. Pure Python 2. Useful UI 3. Robust Integrations 4. Easy …