Dask is a flexible library for parallel computing in Python.
Dask is composed of two parts:
Dynamic task scheduling optimized for computation. This is similar to Airflow, Luigi, Celery, or Make, but optimized for interactive computational workloads.
“Big Data” collections like parallel arrays, dataframes, and lists that extend common interfaces like NumPy, Pandas, or Python iterators to larger-than-memory or distributed environments. These parallel collections run on top of dynamic task schedulers.
Familiar: Provides parallelized NumPy array and Pandas DataFrame objects
Flexible: Provides a task scheduling interface for more custom workloads and integration with other projects.
Native: Enables distributed computing in pure Python with access to the PyData stack.
Fast: Operates with low overhead, low latency, and minimal serialization necessary for fast numerical algorithms
Scales up: Runs resiliently on clusters with 1000s of cores
Scales down: Trivial to set up and run on a laptop in a single process
Responsive: Designed with interactive computing in mind, it provides rapid feedback and diagnostics to aid humans
1275 Av. des Canadiens-de-Montréal,
Montréal, QC H3B 0G4
Canada
312 Arizona Ave,
Santa Monica, CA 90401,
USA
Gate Avenue Zone D at DIFC – Sheikh Zayed Road
Dubai, United Arab Emirates
1 Al Corniche St, Burj Doha, level 21,
Doha, Qatar
© Copyright 2022. All rights reserved by Rocket Science Development
