Lazy and eager¶
zeki works like Polars' lazy API. A zk.Collection is lazy: arithmetic, filters, group-bys and
copulas on it build up a record of how its values are computed, and compute nothing. Calling
collect() evaluates it, giving a zk.Collected, which holds the values as data.
zk.Collection |
zk.Collected |
|
|---|---|---|
| Like Polars' | LazyFrame |
DataFrame |
| Holds | How each value is computed | The values |
| Arithmetic, filters, group-bys, copulas | Yes | No |
| Read values out | No | to_numpy, to_polars, to_pandas, to_list |
| Structure: dims, rows, column types, lengths | Yes | Yes |
net = book.get_column("gross") * 0.8 # nothing is computed yet
net.dims, net.column_names, net.value_lengths # but the structure is always known
collected = net.collect() # one engine run: a zk.Collected holding the values
A collection's structure is always known without evaluating anything: its dimensions, its rows,
its column types and their lengths. Only the values wait for collect().
Evaluate together¶
Each collect() is one run of the engine. To read several collections, collect them together with
zk.collect_all, so any work they share is done once:
Why lazy¶
Because a model is described before it's evaluated, zeki can:
- skip what isn't needed. Only the work the collected values depend on is done.
- avoid holding intermediate values. The engine runs the whole model over a chunk of simulations at a time, in small reused buffers, rather than computing each step over every simulation and keeping the result.
- apply copulas in the right place. A copula registered after a value was built still applies to it, because nothing has been computed yet. See Dependency and copulas.
A Collected is a snapshot¶
A Collected holds the values as they were when it was collected. It has no arithmetic: compute
with collections, then collect the results. Reading values from a Collection itself is an error
that points you to collect():
with pytest.raises(AttributeError, match="collect"):
net.to_list() # a Collection holds no values to read
If you register a copula that reorders inputs a Collected already depends on, zeki warns with
zk.StaleReadWarning: the Collected is out of date, and collecting again gives the reordered
values.
gross = zk.Collection(Business, Gross)
gross.add("motor", 2020, np.array([1.0, 2.0]))
gross.add("property", 2020, np.array([3.0, 4.0]))
early = gross.collect()
with pytest.warns(zk.StaleReadWarning):
zk.correlate(gross, zk.Copula.gaussian(lambda row_a, row_b: 1.0 if row_a == row_b else 0.6))
# early holds values from before the copula; collecting again gives reordered ones
Register copulas before you collect, and the warning never arises.