Collection¶
Collection ¶
Bases: Generic[D, *Columns]
A dimensioned collection of constant, array and jagged values, in one or more uniquely named value columns.
Collection[SomeDims, FloatA[Literal["gross"]], FloatC[Literal["rate"]]]
annotates a collection with its Dims class and its columns' types, in
order; a set of columns named once as a tuple can be unpacked:
Collection[SomeDims, *SomeColumns]. Operations keep what they don't
change: arithmetic with a scalar or a collection of the same dims keeps the
dims, the left operand's column name and the larger shape; a reduction keeps
the columns. Where the result's dims or columns can't be known statically
the type says so, and validate checks a collection against dims and
column types and returns it typed by them.
Every column has a type: a kind - Float, Int, Bool, Date
or String, stored at its native width - and a shape - Constant (one
value per row), Array (an array of length values per row) or
Jagged (per row, length entries each holding its own number of
values). Columns of one collection can differ in type and length.
Binary operations align operands by dimension (name and kind), not position.
Each operand must have a single value column - get_column isolates one,
and nothing broadcasts across value columns; the result takes the left operand's
column name (a float operand takes the collection's name). Every
operation is strict: a dim row of the shared dims on one side only raises
ValueError, never dropped or filled. extend_default adds missing dim
rows, so collections covering different rows line up.
Operands must be of the same kind, as nothing is converted implicitly (a
Python int literal does count as a float alongside a float column).
+ - * // apply to floats and ints, / to floats and ints (ints
dividing to floats, as in Python), min/max to
floats, ints and dates, < <= > >= to floats, ints and dates, and
== != to every kind, the comparisons giving bool collections, as
Polars' expressions do; zk.when picks between branches of any one kind.
A collection has no truth value, so if a < b: raises, and since
== compares values, a collection isn't hashable.
Shapes aren't converted either, but a constant (including a Python scalar) broadcasts over an array or jagged value. Arrays combine with arrays of the same length - an array of length 1 is not a constant - and an array applies to a jagged value of its length entry by entry. Jagged values combine value by value, so their counts must agree row by row.
A collection holds lineage, not values: each value cell is a node owned
by the values that refer to it, and operations build nodes. An array or
jagged cell is lazy - its values are computed only when it's collected:
collect(), or zk.collect_all for several collections at once,
evaluates into a zk.Collected, whose to_numpy, to_list and
to_polars read them. A constant cell keeps its lineage too - a
constant column summed across rows (premium.sum()) remembers the
cells it came from, and explain shows them - but, being one number,
its value is computed as soon as it's built. Lineage is freed with the
values, so there is no context to reset. See correlate and
explain.
A collection starts from its dim rows, Collection.of(dims, dim_rows),
with no value columns, and with_columns adds them - computed from
the dims row by row with zk.float or zk.int, or from other
columns; data already in arrays is read by a function the same way. A
collection is also grown in place a row at a time with
Collection(dims, *columns) and add, for columns of any kind. zk.concat puts another
collection's columns alongside. Values already derived from a
collection keep the rows they were built from.
An empty collection over dims with the columns columns
declare - column types such as FloatA[Literal["gross"]], in order -
to fill a row at a time with add. An array or jagged column's
length is set by the first row added.
dim_rows
property
¶
dim_rows: DimRows[D]
The collection's dim rows - its dims and each row's dim cells -
without its values, to build new collections over the same rows:
collection.dim_rows.zeros(FloatA[Literal["name"]], length=...).
A snapshot of the rows when it is read: rows added to the collection
afterwards aren't in it.
value_lengths
property
¶
value_lengths: list[int | None]
Each column's value_length(name), in order, with None for a
constant column.
is_constant
property
¶
Whether the single value column is constant (one value per
row). Raises ValueError if there are several columns.
kinds
property
¶
The value columns' kinds, in order: float, int, bool,
datetime.date or str.
types
property
¶
types: list[type[Value]]
The value columns' types, in order: zk.FloatA, zk.StringC
and so on.
dims
property
¶
dims: list[Dim]
The dims, in row order: rows sort by the first, then the second, and so on.
add ¶
Add one row, in place: its dim row in dim order, then one value per
column in column order - positionally, or by dim and column name.
A constant column takes a scalar, an array column a 1-D array, and a
jagged column a (counts, values) pair of 1-D arrays; every array or
jagged value of a column has the same length. Any column also takes a
Value read from a collection with get_constant, get_array or
get_jagged, of its kind and shape: the value cell then holds that
Value's node, so it keeps its lineage - a copula applied to what it was
built from reorders it too. Raises ValueError at once for a dim row
already in the collection or a value that doesn't fit its column,
leaving the collection as it was.
extend_default ¶
extend_default(
dim_rows: Collection | DimRows | Sequence[object],
*,
float_default: float = 0.0,
int_default: int = 0,
bool_default: bool = False,
date_default: date = min,
string_default: str = "",
) -> Collection[D, *Columns]
This collection with a row of defaults for each dim row of
dim_rows it doesn't already have; its own rows are kept as they
are. dim_rows is another collection, or its dim_rows, over the
same dims (in any order), or a sequence of dim rows in this
collection's dim order (a bare dim cell for a collection with one
dim). Each new cell is its column kind's default, as a constant, an
array of it, or a jagged value of empty entries. Operations between
collections are strict, so this is how to line up collections that
cover different rows - e.g. a.extend_default(b) +
b.extend_default(a).
of
staticmethod
¶
of(
dims: type[Dims] | Sequence[Dim], dim_rows: _DimRows
) -> Collection[Any]
A collection over dims (a Dims class or a sequence of
Dims) and dim_rows, with no value columns yet: with_columns
adds them. dim_rows is a sequence of dim rows in dim order (a bare
dim cell for one dim), or a polars DataFrame with a column per dim,
whose other columns are ignored. Float dims can hold the parameters
each row is sampled from, which zk.float passes a function::
class Parameters(zk.Dims):
line = zk.Dim(str)
mean = zk.Dim(float)
sd = zk.Dim(float)
rng = np.random.default_rng(seed)
gross = zk.Collection.of(Parameters, parameters).with_columns(
gross_ibnr=zk.float(rng.lognormal, zk.dim("mean"), zk.dim("sd"), nsims)
)
drop_dims("mean", "sd") then leaves the values by line alone.
Data already in arrays is read by a function too, keyed by the dims:
zk.float(lambda line: gross[line], zk.dim("line")).
validate ¶
Check the collection has exactly the dims of dims and the value
columns columns declare - column types such as
FloatA[Literal["gross"]], in any order - and return it typed by
them, with its dims in dims's order (the same object if they
already are). Raises ValueError on a mismatch.
get_column ¶
get_column(
name: str | type[ValueColumns],
) -> Collection[D, Value]
A single value column (never a dim column), as in Polars'
get_column. It returns a collection holding just the value column
the name class declares, which must have that type, alongside all
of this collection's dim columns, sharing its dims and dim rows.
This is how to isolate one value column for arithmetic, which needs a
single value column in each operand: gross.get_column(GrossIbnr) * rate.
select_values ¶
select_values(
*columns: str | type[ValueColumns],
) -> Collection[D, *tuple[Any, ...]]
The collection with just the value columns columns, in that
order - each a name, or a column type such as FloatA[Literal["gross"]],
whose type the column must have. The dims and dim rows are shared.
Typed by the column types given; with names, the columns are typed as
any.
drop_values ¶
drop_values(
*columns: str | type[ValueColumns],
) -> Collection[D, *tuple[Any, ...]]
The collection without the value columns columns - each a name,
or a column type whose type the column must have - the rest in order.
The dims and dim rows are shared, so dropping every value column
leaves the dim rows alone. What's left can't be typed statically, so
its columns are typed as any - validate restates them.
drop_dims ¶
drop_dims(*dims: str | Dim) -> Collection[Any, *Columns]
The collection without the dims dims - each a name or a Dim
- over the same values, its other dims in order. The dims dropped
must be redundant, every row keeping a dim row of its own without
them, as parameter dims are once the values sampled from them are
built: sampled.drop_dims("mean", "sd"). Raises ValueError for
a dim the collection doesn't have, or rows that only the dropped dims
told apart - group_by combines those. The dims left aren't known
statically - validate restates them.
with_columns ¶
with_columns(
*expressions: ValueExpr, **named: ValueExpr
) -> Collection[D, *tuple[Any, ...]]
This collection with columns computed from its own, as in Polars: a
column per value expression - in expressions, named by its alias or
the first column it refers to, or by keyword in named - computed row
by row from this collection as it is before any is added:
with_columns(net=zk.value("gross") * zk.value("rate")). A column named
like an existing one replaces it in place, and the rest follow in order;
no name may repeat. Another collection's columns are put alongside with
zk.concat.
A value expression's type isn't known statically, so the columns are
typed as any - validate restates them.
rename ¶
rename(
mapping: Mapping[str, str | type[ValueColumns]],
) -> Collection[D, *tuple[Any, ...]]
The collection with its value columns renamed by mapping, as in
Polars: rename({"gross": "net"}). Each old name maps to its new
one, a str or a column type such as FloatA[Literal["net"]],
whose type the column must have. All are renamed at once, so names can
swap; the columns keep their order. Operations keep the left
operand's column name, so a derived collection is renamed before it's
validated against its own column types - a single column renamed to a
column type keeps its type statically.
value_length ¶
value_length(name: str) -> int | None
The length of the value column name's Values - len() of each
value cell: an array's number of values, or a jagged value's number of
entries (its outer length) - or None if the column is constant.
Raises ValueError if there's no such column. Typed by the column:
an int for an array or jagged column, None for a constant one.
jagged_shape ¶
jagged_shape(name: str) -> JaggedShape
The jagged column name's shape, row by row: its entries hold as
many values as this column's do, however copulas move them - the
counts themselves aren't known until the column is collected. Pass it
to dim_rows.zeros(FloatJ[...], jagged_shape=...) for zeros that
combine with this column value by value. A snapshot, like
dim_rows. Raises ValueError if there's no such column, or it
isn't jagged.
abs ¶
abs() -> Collection[D, *Columns]
Every value's absolute value, in every column. Floats and ints only.
sqrt ¶
sqrt() -> Collection[D, *Columns]
Every value's square root, in every column. Floats only: raises
ValueError for any other kind.
exp ¶
exp() -> Collection[D, *Columns]
e to the power of every value, in every column. Floats only:
raises ValueError for any other kind.
log ¶
log() -> Collection[D, *Columns]
Every value's natural log, in every column: -inf at 0 and NaN
below. Floats only: raises ValueError for any other kind.
sum ¶
sum() -> Collection[NoDims, *Columns]
Every column summed across all the rows, value by value: a
collection with no dims and a single row, its columns keeping their
names, kinds and shapes. Reduce to some of the dims with
group_by(...).sum(). Jagged columns don't reduce across rows;
reduce their entries first, e.g. with jagged_sum. An empty
collection raises a ValueError.
min ¶
min() -> Collection[NoDims, *Columns]
Every column's minimum across all the rows; as for sum.
For the elementwise minimum of two collections, use zk.min.
max ¶
max() -> Collection[NoDims, *Columns]
Every column's maximum across all the rows; as for sum.
For the elementwise maximum of two collections, use zk.max.
jagged_sum ¶
jagged_sum(name: str) -> Collection[D, Value]
The sum of each entry's values in the jagged column name: a
collection of that column alone, as arrays of the same length. An
empty entry sums to 0.
jagged_min ¶
jagged_min(name: str) -> Collection[D, Value]
The least of each entry's values in the jagged column name; as
for jagged_sum. An empty entry gives 0.
jagged_max ¶
jagged_max(name: str) -> Collection[D, Value]
The greatest of each entry's values in the jagged column name;
as for jagged_sum. An empty entry gives 0.
jagged_count ¶
jagged_count(name: str) -> Collection[D, Value]
How many values each entry of the jagged column name holds: a
collection of that column alone, as int arrays of the same length.
clamp ¶
clamp(
lower: Collection[Any, Any] | Scalar,
upper: Collection[Any, Any] | Scalar,
) -> Collection[Any, Any]
The values held between lower and upper, row by row and
value by value; see zk.clamp.
filter ¶
filter(*predicates: DimExpr) -> Collection[D, *Columns]
The rows where every one of predicates holds - conditions on
their dim rows, such as zk.dim("class_of_business").is_in(["motor",
"home"]) and zk.dim("underwriting_year") == 2026, implicitly
and-ed together as in Polars - with every dim and column kept. Values
are shared, not copied.
group_by ¶
The collection grouped by by, for GroupBy.agg to reduce.
Each of by - a dim expression such as zk.dim("class_of_business")
or zk.when(zk.dim("year") < 2025).then(zk.dim("year")).otherwise(2025),
a Dim, or a dim's name - becomes a dim of the result, named after
the first dim it refers to (or its alias), or by keyword in
named_by. Every other dim is reduced away::
collection.group_by(zk.dim("class_of_business")).agg(
total_claims=zk.value("gross_reserves").sum(),
)
Grouped by nothing, every row is one group, and the result has no
dims (NoDims): collection.sum() is collection.group_by().sum().
collect ¶
collect() -> Collected[D, *Columns]
Evaluate the collection, with every copula registered so far
applied, into a zk.Collected: its values as data, to read
(to_numpy, get_array, ...) or convert (to_list,
to_polars, to_pandas). A Collection is lazy: nothing else
evaluates it. zk.collect_all evaluates several collections in one
run.
get_constant ¶
get_constant(
dim_row: tuple[str | int, ...], name: str
) -> (
FloatC[Any]
| IntC[Any]
| BoolC[Any]
| DateC[Any]
| StringC[Any]
)
The Value of the constant column name at dim_row - a tuple of
the row's dim cells, () for a collection with no dims - as an
instance of the column's type, such as FloatC["premium"], holding
the value cell's node in the lineage. Raises ValueError if the
collection has no such dim row or column, or if the column isn't
constant.
Typed from the column's Literal name for the first 8
columns; a column further along gives any constant value.
get_array ¶
get_array(
dim_row: tuple[str | int, ...], name: str
) -> (
FloatA[Any]
| IntA[Any]
| BoolA[Any]
| DateA[Any]
| StringA[Any]
)
The Value of the array column name at dim_row, as an instance
of the column's type, such as FloatA["gross"], holding the value
cell's node in the lineage. Raises ValueError as get_constant
does.
get_jagged ¶
get_jagged(
dim_row: tuple[str | int, ...], name: str
) -> (
FloatJ[Any]
| IntJ[Any]
| BoolJ[Any]
| DateJ[Any]
| StringJ[Any]
)
The Value of the jagged column name at dim_row, as an instance
of the column's type, such as FloatJ["large"], holding the value
cell's node in the lineage. Raises ValueError as get_constant
does.
explain ¶
A readable account of how the values came to be, without computing
them: the inputs and operations they depend on, the tie groups those
inputs form, the copulas that reorder them (applied or not yet), and
their formulas. Cells built by the same operations - a column computed
the same way in every row - share one formula, written with input
for the inputs (and c for constants that differ between cells),
followed by one example cell in full::
formula, every cell: ((input + input) * 0.5)
e.g. ('home',) = ((input 1 + input 2) * 0.5)
A collection of several columns explains each in turn.