model
The life of a model. Every operation builds new lazy values without computing anything; reading evaluates them all in one pass of the engine, with every copula applied so far.
Every part of zeki's Python API as a railroad diagram. Follow a track from left to right: any path through it is valid code. Click a square box to jump to its own diagram.
| Kind | Constant | Array | Jagged |
|---|---|---|---|
| float | FloatC | FloatA | FloatJ |
| int | IntC | IntA | IntJ |
| bool | BoolC | BoolA | BoolJ |
| datetime.date | DateC | DateA | DateJ |
| str | StringC | StringA | StringJ |
A model builds collections of simulated values, combines them lazily, correlates them with copulas, and reads the results. Nothing is computed until a value is read.
The life of a model. Every operation builds new lazy values without computing anything; reading evaluates them all in one pass of the engine, with every copula applied so far.
A collection's type lists its dims and then its value columns, so an annotation says what a variable holds: Collection[ReserveDims, FloatA[Literal["gross_ibnr"]], FloatC[Literal["payment_pattern"]]].
Every value column has a kind and a shape. Nothing converts between kinds or shapes implicitly (an array of length 1 is not a constant), but a constant broadcasts over the arrays and jagged values it meets. The name is a Literal so type checkers see it; at runtime zk.FloatA["gross"] gives the same class.
A collection's dims: a Dims class, NoDims for a collection reduced to a single row, or a list of Dims.
Dims shared by many collections are declared once, as a class; each Dim takes its attribute's name. A Dims class extends its bases' dims. Iterating over the class gives its Dims in order.
One named axis, with str, int or float dim cells. Two dims are equal when their names and kinds are. A float dim holds the parameters a row's values are sampled from, never NaN.
Several value columns declared once, wherever a column type is accepted at runtime; each ValueColumn takes its attribute's name. In annotations, a tuple of column types does the same job: Collection[ReserveDims, *Reserves].
A collection starts from its dim rows with Collection.of and gains columns with with_columns, is grown a row at a time, or is derived from other collections. Its rows are sorted by dim row, and each array or jagged value cell is a node in the lineage.
Anything that gives a collection. The last alternatives make one from others, so a collection is followed by any number of operations.
One dim row per row: a tuple of its dim cells in dim order, or a bare dim cell when there is one dim - or a polars DataFrame with a column per dim.
Growing a collection in place. add takes one row: its dim cells in dim order, then a value per column in column order, or each by name. A dim row already present, or a value that doesn't fit its column, raises ValueError and leaves the collection as it was.
What one value cell holds. A Value read from a collection keeps its lineage, so a copula applied to what it was built from reorders it too.
One value cell's Value, read without computing it: an instance of the column's type, such as FloatC["premium"], holding the cell's node. Raises ValueError if the dim row or column is missing, or the column has another shape.
Values combine as columns do, with Values of their kind and with scalars, giving another Value. Shapes broadcast as for columns, and the result is named after the left operand unless it's the scalar.
Binary operations align their operands by dim (name and kind), not position, and each operand has a single column. The result takes the left operand's column name. Every operation is strict: a dim row of the shared dims on one side only raises, and extend_default adds missing ones. A NoDims collection broadcasts to every row.
What can follow a collection to give a new one. Arithmetic works on floats and ints (ints divide to floats, and // rounds down), < <= > >= on floats, ints and dates, and == and != on every kind. A comparison gives a bool collection, which has no truth value: if a < b: raises. ** is for floats. -collection negates every value. Reductions keep every column; jagged columns reduce their entries before they reduce across rows.
The other side of an operation. A Python int counts as a float alongside a float column; otherwise operands are of one kind.
Functions of several collections. zk.min and zk.max shadow the builtins deliberately. concat puts value columns side by side over the same dim rows (horizontal, the default) or stacks rows over the same dims and columns (vertical).
Each expression grouped by becomes a dim of the result, named after the dim it refers to, its alias, or its keyword; every other dim is reduced away. Grouped by nothing, the result has no dims: collection.sum() is group_by().sum().
An expression over a collection's dim cells, resolved only when it's used: a condition filters a collection, and one giving str or int dim cells groups it. Python's and, or, not and x if c else y raise TypeError; use &, |, ~ and zk.when.
The other side of a dim expression.
An if/else over values: value by value, the first branch whose conditions (comparisons, all of which must hold) hold, from values of one kind.
The other side of a value expression.
A column computed by calling a function once per row, in with_columns: it returns a scalar, a 1-D array or a (counts, values) pair, the same shape every row. Each argument is resolved for the row: a dim expression gives the row's dim cell, a value expression of a constant column the row's value, and anything else is passed as it is.
An expression over value columns, for with_columns and GroupBy.agg. Every column of an agg must reduce across the group's rows; reduce both sides of an operation, or neither.
A collection is lazy: describing it computes nothing. collect() - or zk.collect_all for several at once, in one pass of the engine - evaluates it, with every copula registered so far applied, into a zk.Collected holding the values as data.
A Collected has the collection's dims, rows and column types, and no arithmetic. It is a snapshot: a copula registered afterwards doesn't change it.
to_polars gives a column per dim and then per value column: the kind's dtype for a constant, pl.Array for an array and pl.Array of pl.List for a jagged column. A dim and value column sharing a name raise ValueError. to_pandas takes a single constant or array column. A jagged column reads as a (counts, values) pair.
A collection's structure is always known without computing anything. explain() shows a value's lineage, including the copulas it goes through and whether they have been applied yet.
A collection's dim rows without its values: a snapshot that rows added later don't change. Its zeros build new collections over the same rows.
A copula reorders a float array collection's cells against each other. Every input tied to a cell is reordered by its permutation, so values derived from them, before or after, stay consistent.
Correlates the cells of a FloatA column, lazily: the copula is applied when something depending on it is collected. Without a seed, one is derived from the context's seed and the cells' lineage, so results don't depend on statement order. Cells that already share an input can't be correlated. Warns with zk.StaleReadWarning if values already collected depend on the inputs it will reorder.
Elliptical copulas take a function giving the correlation between two cells, from their rows' dim rows as tuples: called once per pair, giving a positive-definite matrix with values in [-1, 1]. Archimedean copulas take one theta for every pair and give positive dependence only.
The ambient settings a model is built under; only the root seed copula seeds derive from. Contexts nest, a setting left out being inherited, and are local to each thread and asyncio task. The default seed is 0.
Process-wide settings and counters.
timings splits zeki's time into evaluate, copula, export and import, each exclusive of the others; building lazy operations and the caller's own code aren't included.
The DAG underneath the collections, built directly for float arithmetic in zeki.engine (from zeki import engine): float operations over arrays of one length, compiled once and evaluated in one parallel pass. It skips the collections' lineage, dims and kinds, so a large program of plain float arithmetic builds and evaluates with the least overhead. Most models want the collections.
A Graph holds arrays of one length. Inputs are copied in once, so the compiled program owns its data. Compiling freezes the outputs into one program; the graph stays usable, so it can be compiled again for other outputs.
A node of a Graph. Reusing one shares its computation, so a value computed once can feed many others. Operands come from the same graph, or are floats on either side.
The other side of an Expr operation.
Evaluates every output in one pass. Compiled from one Expr, eval gives a 1-D array; from a list, one row per output.