Skip to content

Lazy and eager

zeki works like Polars' lazy API. A zk.Collection is lazy: arithmetic, filters, group-bys and copulas on it build up a record of how its values are computed, and compute nothing. Calling collect() evaluates it, giving a zk.Collected, which holds the values as data.

zk.Collection zk.Collected
Like Polars' LazyFrame DataFrame
Holds How each value is computed The values
Arithmetic, filters, group-bys, copulas Yes No
Read values out No to_numpy, to_polars, to_pandas, to_list
Structure: dims, rows, column types, lengths Yes Yes
net = book.get_column("gross") * 0.8  # nothing is computed yet
net.dims, net.column_names, net.value_lengths  # but the structure is always known

collected = net.collect()  # one engine run: a zk.Collected holding the values

A collection's structure is always known without evaluating anything: its dimensions, its rows, its column types and their lengths. Only the values wait for collect().

Evaluate together

Each collect() is one run of the engine. To read several collections, collect them together with zk.collect_all, so any work they share is done once:

gross_values, net_values = zk.collect_all(book.get_column("gross"), net)  # shared work done once

Why lazy

Because a model is described before it's evaluated, zeki can:

  • skip what isn't needed. Only the work the collected values depend on is done.
  • avoid holding intermediate values. The engine runs the whole model over a chunk of simulations at a time, in small reused buffers, rather than computing each step over every simulation and keeping the result.
  • apply copulas in the right place. A copula registered after a value was built still applies to it, because nothing has been computed yet. See Dependency and copulas.

A Collected is a snapshot

A Collected holds the values as they were when it was collected. It has no arithmetic: compute with collections, then collect the results. Reading values from a Collection itself is an error that points you to collect():

with pytest.raises(AttributeError, match="collect"):
    net.to_list()  # a Collection holds no values to read

If you register a copula that reorders inputs a Collected already depends on, zeki warns with zk.StaleReadWarning: the Collected is out of date, and collecting again gives the reordered values.

gross = zk.Collection(Business, Gross)
gross.add("motor", 2020, np.array([1.0, 2.0]))
gross.add("property", 2020, np.array([3.0, 4.0]))

early = gross.collect()
with pytest.warns(zk.StaleReadWarning):
    zk.correlate(gross, zk.Copula.gaussian(lambda row_a, row_b: 1.0 if row_a == row_b else 0.6))
# early holds values from before the copula; collecting again gives reordered ones

Register copulas before you collect, and the warning never arises.