API

delayed

Wraps a function or object to produce a Delayed.

dask.delayed.delayed()

Wraps a function or object to produce a Delayed.

Delayed objects act as proxies for the object they wrap, but all operations on them are done lazily by building up a dask graph internally.

Parameters:

obj : object

The function or object to wrap

name : string or hashable, optional

The key to use in the underlying graph. Defaults to hashing content.

pure : bool, optional

Indicates whether calling the resulting Delayed object is a pure operation. If True, arguments to the call are hashed to produce deterministic keys. Default is False.

Examples

Apply to functions to delay execution:

>>> def inc(x):
...     return x + 1

>>> inc(10)
11

>>> x = delayed(inc, pure=True)(10)
>>> type(x) == Delayed
True
>>> x.compute()
11

Can be used as a decorator:

>>> @delayed(pure=True)
... def add(a, b):
...     return a + b
>>> add(1, 2).compute()
3

delayed also accepts an optional keyword pure. If False (default), then subsequent calls will always produce a different Delayed. This is useful for non-pure functions (such as time or random).

>>> from random import random
>>> out1 = delayed(random, pure=False)()
>>> out2 = delayed(random, pure=False)()
>>> out1.key == out2.key
False

If you know a function is pure (output only depends on the input, with no global state), then you can set pure=True. This will attempt to apply a consistent name to the output, but will fallback on the same behavior of pure=False if this fails.

>>> @delayed(pure=True)
... def add(a, b):
...     return a + b
>>> out1 = add(1, 2)
>>> out2 = add(1, 2)
>>> out1.key == out2.key
True

delayed can also be applied to objects to make operations on them lazy:

>>> a = delayed([1, 2, 3])
>>> type(a) == Delayed
True
>>> a.compute()
[1, 2, 3]

Delayed results act as a proxy to the underlying object. Many operators are supported:

>>> (a + [1, 2]).compute()
[1, 2, 3, 1, 2]
>>> a[1].compute()
2

Method and attribute access also works:

>>> a.count(2).compute()
1

Note that if a method doesn’t exist, no error will be thrown until runtime:

>>> res = a.not_a_real_method()
>>> res.compute()  
AttributeError("'list' object has no attribute 'not_a_real_method'")

Methods are assumed to be impure by default, meaning that subsequent calls may return different results. To assume purity, set pure=True. This allows sharing of any intermediate values.

>>> a.count(2, pure=True).key == a.count(2, pure=True).key
True