zarr_metadata.v3.chunk_grid

zarr_metadata.v3.chunk_grid

Zarr v3 chunk grid metadata types.

Each chunk grid lives in its own submodule:

• regular -- core v3 spec
• rectilinear -- zarr-extensions

The <X>ChunkGridMetadata aliases re-exported here are the canonical type for each grid's permitted JSON shapes. For the underlying <X>ChunkGridObject, <X>ChunkGridConfiguration, etc., import directly from the leaf submodule.

See https://zarr-specs.readthedocs.io/en/latest/v3/core/index.html#chunk-grids

zarr_metadata.v3.chunk_grid.regular

Regular chunk grid (Zarr v3 core spec).

See https://zarr-specs.readthedocs.io/en/latest/v3/core/index.html#regular-grids

REGULAR_CHUNK_GRID_NAME module-attribute

REGULAR_CHUNK_GRID_NAME: Final = 'regular'

The name field value of the regular chunk grid.

RegularChunkGridMetadata module-attribute

RegularChunkGridMetadata = RegularChunkGridObject

Permitted JSON shape for regular chunk grid metadata.

chunk_shape is required and has no default, so only the object form is valid; the short-hand-name form is not permitted by the spec for this grid. https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L528-L537 ("must be an object with the names name and configuration") https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L1562-L1564

RegularChunkGridName module-attribute

RegularChunkGridName = Literal['regular']

Literal type of the name field of the regular chunk grid.

__all__ module-attribute

__all__ = [
    "REGULAR_CHUNK_GRID_NAME",
    "RegularChunkGrid",
    "RegularChunkGridConfiguration",
    "RegularChunkGridMetadata",
    "RegularChunkGridName",
    "RegularChunkGridObject",
    "RegularChunkGridOptions",
]

RegularChunkGrid dataclass

Bases: ChunkGridEntity

The regular chunk grid, coerced from its metadata.

Source code in src/zarr_metadata/v3/chunk_grid/regular.py

@dataclass(frozen=True)
class RegularChunkGrid(ChunkGridEntity):
    """The `regular` chunk grid, coerced from its metadata."""

    configuration: RegularChunkGridOptions

    identifier: ClassVar[str] = REGULAR_CHUNK_GRID_NAME

    def shape_problems(self, array_shape: object) -> tuple[ValidationProblem, ...]:
        """A regular grid must chunk every array dimension."""
        if not isinstance(array_shape, (list, tuple)):
            return ()
        extents = tuple(cast("Sequence[object]", array_shape))
        if len(self.configuration.chunk_shape) == len(extents):
            return ()
        return problem(
            ("chunk_shape",),
            f"chunk_shape has {len(self.configuration.chunk_shape)} entries but shape has "
            f"{len(extents)} dimensions",
            "invalid_value",
        )

    def grid(self, array_shape: object) -> ChunkGrid:
        """One extent per axis, the same for every chunk on that axis."""
        return ChunkGrid.regular(self.configuration.chunk_shape)

configuration instance-attribute

configuration: RegularChunkGridOptions

The record of this entity's members.

An entity with members narrows it to its own record, configuration: GzipOptions, its one positional argument. An entity of a bare name defaults it to the empty record -- configuration: Configuration = field(default_factory=Configuration) -- so that Crc32cCodec() builds; coerce passes the record either way.

identifier class-attribute

identifier: str = REGULAR_CHUNK_GRID_NAME

The name this entity is registered under.

Usually the name the metadata carries. The raw-bytes data types are the exception: every r<N> spelling is one family, so the family gets an invented identifier that no real name can collide with.

__init__

__init__(configuration: RegularChunkGridOptions) -> None

__post_init__

__post_init__() -> None

Refuse a record that is not this entity's own, then the first problem the rules find.

The runtime half of the entity's type, as the record's constructor is of the record's: GzipCodec(BloscOptions(...)) and a family member carrying a name that is not a string are refused before any rule reads them. Then BloscCodec(BloscOptions(clevel=99)) raises on the first problem the rules yield.

Source code in src/zarr_metadata/v3/_entity.py

def __post_init__(self) -> None:
    """Refuse a record that is not this entity's own, then the first problem the rules find.

    The runtime half of the entity's type, as the record's constructor
    is of the record's: `GzipCodec(BloscOptions(...))` and a family
    member carrying a name that is not a string are refused before
    any rule reads them. Then `BloscCodec(BloscOptions(clevel=99))`
    raises on the first problem the rules yield.
    """
    plan = _plan(type(self))
    if not isinstance(self.configuration, plan.record):
        raise MetadataValidationError(
            problem(
                (),
                f"expected a {plan.record.__name__} configuration, got "
                f"{type(self.configuration).__name__}",
            )
        )
    name: object = self.identifier if plan.from_name is None else getattr(self, plan.from_name)
    if not isinstance(name, str):
        raise MetadataValidationError(problem((), f"expected a string name, got {name!r}"))
    first = next(type(self).name_problems(name), None)
    if first is None:
        first = next(self.configuration.problems(), None)
    if first is not None:
        raise MetadataValidationError((first,))

accepts classmethod

accepts(name: str) -> bool

Whether name denotes this entity.

Constant for all but the raw-bytes family, where one class covers every r<N>.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def accepts(cls, name: str) -> bool:
    """Whether `name` denotes this entity.

    Constant for all but the raw-bytes family, where one class covers
    every `r<N>`.
    """
    return name == cls.identifier

canonical

canonical() -> Self

This entity in the simplest form that means the same thing.

A transformation, asked for by canonicalize_array_metadata_v3 and by nothing else. to_json does not apply it, because writing a document back is not the same as asking for it to be rewritten: a reader that reads and writes should not change bytes it was not asked to change.

The default is the entity itself. Override it where two spellings of the entity's members mean the same -- a rectilinear dimension's run-length encoding, a typesize that noshuffle ignores -- and, in an entity that contains entities, to put those in canonical form: self.with_configuration(inner=self.inner.canonical()).

Source code in src/zarr_metadata/v3/_entity.py

def canonical(self) -> Self:
    """This entity in the simplest form that means the same thing.

    A *transformation*, asked for by `canonicalize_array_metadata_v3`
    and by nothing else. `to_json` does not apply it, because writing
    a document back is not the same as asking for it to be rewritten:
    a reader that reads and writes should not change bytes it was not
    asked to change.

    The default is the entity itself. Override it where two spellings
    of the entity's members mean the same -- a rectilinear
    dimension's run-length encoding, a `typesize` that `noshuffle`
    ignores -- and, in an entity that contains entities, to put those
    in canonical form: `self.with_configuration(inner=self.inner.canonical())`.
    """
    return self

coerce classmethod

coerce(value: JSONValue, context: Context) -> Coerced[Self]

value as this entity, or the reasons it is not one: the class's validation routine.

resolve relates a field's name to this class and hands it the field, refined JSON with arrays as tuples, which is what value is; this is what the class does with it. The configuration is parsed against the record the configuration field names, member by member; a member holding another entity is read in context, the scope this reading is happening in. An optional member the document left out is UNSET in the record, so no field's default decides what a document said. The entity is built only when every member of its own read -- its rules are written over a whole configuration -- and handed back only when everything inside it read too.

The envelope is the field's, not the class's, and resolve judges it: a stray member or a must_understand of false is not reported here. Called on a class no scope has registered, this runs with none of registration's refusals having happened.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def coerce(cls, value: JSONValue, context: Context) -> Coerced[Self]:
    """`value` as this entity, or the reasons it is not one: the class's validation routine.

    `resolve` relates a field's name to this class and hands it the
    field, refined JSON with arrays as tuples, which is what `value`
    is; this is what the class does with it. The configuration is
    parsed against the record the `configuration` field names,
    member by member; a member holding another entity is read in
    `context`, the scope this reading is happening in. An optional
    member the document left out is `UNSET` in the record, so no
    field's default decides what a document said. The entity is
    built only when every member of its own read -- its rules are
    written over a whole configuration -- and handed back only when
    everything inside it read too.

    The envelope is the field's, not the class's, and `resolve`
    judges it: a stray member or a `must_understand` of `false` is
    not reported here. Called on a class no scope has registered,
    this runs with none of registration's refusals having happened.
    """
    name, given, envelope = named_configuration(value)
    if name is None or not cls.accepts(name):
        return None, problem((), f"expected the {cls.identifier!r} entity")
    if len(envelope) != 0:
        return None, envelope
    plan = _plan(cls)
    carried: dict[str, str] = {} if plan.from_name is None else {plan.from_name: name}
    if given is None and plan.requires_configuration:
        return None, problem(
            ("configuration",),
            f"{cls.identifier!r} requires a configuration",
            "missing_key",
        )
    reading = _Reading(context, [])
    # A bare name's record has no members, so any key is an unknown one.
    record, own = plan.read({} if given is None else given, ("configuration",), reading)
    found = (*own, *reading.nested)
    if record is None:
        # An unknown key is survivable; a member that could not be
        # read is a hole, and judging around it would be guessing.
        return None, found
    # The rules, asked of the name and of the record before anything
    # is built: a name problem lands on the entity, a configuration
    # problem under the configuration.
    refused = (*cls.name_problems(name), *within((), tuple(record.problems())))
    if len(refused) != 0:
        # Values the spec disallows: reported rather than raised,
        # every one.
        return None, (*found, *refused)
    if any(entry.kind != "unknown_key" for entry in reading.nested):
        # A contained entity could not be read. This entity's own
        # rules ran -- an invalid inner is an `Opaque`, as an
        # out-of-scope one is -- but what is handed back is not an
        # entity that would be asked composition questions it cannot
        # answer.
        return None, found
    return cls.create_unchecked(configuration=record, **carried), found

create_unchecked classmethod

create_unchecked(**fields: object) -> Self

This entity with these fields, built without the constructor's checks.

The one way around them, for a caller that has just made them: coerce, which type-checked the record and ran the rules before building. Every field is given -- the record, and the carried name for a family -- since nothing here applies a default. Anything that has not checked goes through the constructor.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def create_unchecked(cls, **fields: object) -> Self:
    """This entity with these fields, built without the constructor's checks.

    The one way around them, for a caller that has just made them:
    `coerce`, which type-checked the record and ran the rules before
    building. Every field is given -- the record, and the carried
    name for a family -- since nothing here applies a default.
    Anything that has not checked goes through the constructor.
    """
    entity = object.__new__(cls)
    for name, value in fields.items():
        object.__setattr__(entity, name, value)
    return entity

grid

grid(array_shape: object) -> ChunkGrid

One extent per axis, the same for every chunk on that axis.

Source code in src/zarr_metadata/v3/chunk_grid/regular.py

def grid(self, array_shape: object) -> ChunkGrid:
    """One extent per axis, the same for every chunk on that axis."""
    return ChunkGrid.regular(self.configuration.chunk_shape)

name_problems classmethod

name_problems(name: str) -> Iterator[ValidationProblem]

Why name, which accepts claimed, is not a well-formed name of this family.

For a family, whose names carry data -- r<N> -- and which claims a malformed member so that it is reported rather than waved through as an unknown extension. Locations are relative to the entity: (). Default: none, for an entity of one name.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def name_problems(cls, name: str) -> Iterator[ValidationProblem]:
    """Why `name`, which `accepts` claimed, is not a well-formed name of this family.

    For a family, whose names carry data -- `r<N>` -- and which claims
    a malformed member so that it is reported rather than waved
    through as an unknown extension. Locations are relative to the
    entity: `()`. Default: none, for an entity of one name.
    """
    yield from ()

shape_problems

shape_problems(
    array_shape: object,
) -> tuple[ValidationProblem, ...]

A regular grid must chunk every array dimension.

Source code in src/zarr_metadata/v3/chunk_grid/regular.py

def shape_problems(self, array_shape: object) -> tuple[ValidationProblem, ...]:
    """A regular grid must chunk every array dimension."""
    if not isinstance(array_shape, (list, tuple)):
        return ()
    extents = tuple(cast("Sequence[object]", array_shape))
    if len(self.configuration.chunk_shape) == len(extents):
        return ()
    return problem(
        ("chunk_shape",),
        f"chunk_shape has {len(self.configuration.chunk_shape)} entries but shape has "
        f"{len(extents)} dimensions",
        "invalid_value",
    )

to_json

to_json() -> ZarrV3MetadataFieldJSON

This entity as a document would write it.

Written from the configuration record by the same declaration coerce reads it by, each member by the writer its annotation implies: the bare name when every member it holds is absent, the object otherwise, a contained entity through its own to_json, a JSON-valued member copied so the document is not a handle on the entity. Faithful to every member: read a document, write it back, and those come out as they went in. The envelope is written the entity's way -- the bare name when nothing is configured, the object otherwise, no must_understand, which means what absence means -- because an entity alone has no document to be faithful to; ArrayDocumentV3.to_json puts back the spelling the document used. Ask canonical first if you want the simplest equivalent spelling.

An entity whose JSON is not its fields overrides this; none in the package does.

Source code in src/zarr_metadata/v3/_entity.py

def to_json(self) -> ZarrV3MetadataFieldJSON:
    """This entity as a document would write it.

    Written from the configuration record by the same declaration
    `coerce` reads it by, each member by the writer its annotation
    implies: the bare name when every member it holds is absent, the
    object otherwise, a contained entity through its own `to_json`,
    a JSON-valued member copied so the document is not a handle on
    the entity. Faithful to every member: read a document, write it
    back, and those come out as they went in. The envelope is
    written the entity's way -- the bare name when nothing is
    configured, the object otherwise, no `must_understand`, which
    means what absence means -- because an entity alone has no
    document to be faithful to; `ArrayDocumentV3.to_json` puts back
    the spelling the document used. Ask `canonical` first if you
    want the simplest equivalent spelling.

    An entity whose JSON is not its fields overrides this; none in
    the package does.
    """
    plan = _plan(type(self))
    name = self.identifier
    if plan.from_name is not None:
        carried = getattr(self, plan.from_name)
        name = carried if isinstance(carried, str) else name
    configuration = plan.write(self)
    if len(configuration) == 0:
        return name
    return {"name": name, "configuration": configuration}

with_configuration

with_configuration(**changes: object) -> Self

This entity with these configuration members changed.

codec.with_configuration(typesize=UNSET) is the record rebuilt through its constructor, which refuses a member of the wrong type, and the entity rebuilt through its own, which refuses a value the rules disallow -- the same checks as any construction, since pyright cannot see the members through **changes. A name that is not a member is refused the way replace refuses it.

Source code in src/zarr_metadata/v3/_entity.py

def with_configuration(self, **changes: object) -> Self:
    """This entity with these configuration members changed.

    `codec.with_configuration(typesize=UNSET)` is the record rebuilt
    through its constructor, which refuses a member of the wrong
    type, and the entity rebuilt through its own, which refuses a
    value the rules disallow -- the same checks as any construction,
    since pyright cannot see the members through `**changes`. A
    name that is not a member is refused the way `replace` refuses
    it.
    """
    return replace(self, configuration=replace(self.configuration, **changes))

RegularChunkGridConfiguration

Bases: TypedDict

Configuration for the regular chunk grid.

Source code in src/zarr_metadata/v3/chunk_grid/regular.py

class RegularChunkGridConfiguration(TypedDict, closed=True):
    """Configuration for the regular chunk grid."""

    chunk_shape: tuple[int, ...]

chunk_shape instance-attribute

chunk_shape: tuple[int, ...]

RegularChunkGridObject

Bases: TypedDict

Regular chunk grid metadata in object form.

Source code in src/zarr_metadata/v3/chunk_grid/regular.py

class RegularChunkGridObject(TypedDict, closed=True):
    """Regular chunk grid metadata in object form."""

    name: RegularChunkGridName
    configuration: RegularChunkGridConfiguration
    must_understand: NotRequired[bool]

configuration instance-attribute

configuration: RegularChunkGridConfiguration

must_understand instance-attribute

must_understand: NotRequired[bool]

name instance-attribute

name: RegularChunkGridName

RegularChunkGridOptions dataclass

Bases: Configuration

What a regular grid is configured with.

Source code in src/zarr_metadata/v3/chunk_grid/regular.py

@dataclass(frozen=True)
class RegularChunkGridOptions(Configuration):
    """What a `regular` grid is configured with."""

    chunk_shape: tuple[int, ...]

    def problems(self) -> "Iterator[ValidationProblem]":
        for index, extent in enumerate(self.chunk_shape):
            if extent < 1:
                yield ValidationProblem(
                    ("chunk_shape", index),
                    f"expected an integer >= 1, got {extent}",
                    "invalid_value",
                )

chunk_shape instance-attribute

chunk_shape: tuple[int, ...]

__init__

__init__(chunk_shape: tuple[int, ...]) -> None

__post_init__

__post_init__() -> None

Refuse every member of the wrong type, so GzipOptions(level="high") raises.

Source code in src/zarr_metadata/v3/_entity.py

def __post_init__(self) -> None:
    """Refuse every member of the wrong type, so `GzipOptions(level="high")` raises."""
    found = mistyped(self)
    if len(found) != 0:
        raise MetadataValidationError(found)

create_unchecked classmethod

create_unchecked(**members: object) -> Self

This record with these members, built without the constructor's check.

The one way around the check, for a caller that has just made it: the parser, which type-checked every member against the same annotations before building the record. Every field is given -- the parser gives an absent optional member as UNSET -- since nothing here applies a default. Anything that has not checked the members goes through the constructor.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def create_unchecked(cls, **members: object) -> Self:
    """This record with these members, built without the constructor's check.

    The one way around the check, for a caller that has just made
    it: the parser, which type-checked every member against the same
    annotations before building the record. Every field is given --
    the parser gives an absent optional member as `UNSET` -- since
    nothing here applies a default. Anything that has not checked
    the members goes through the constructor.
    """
    record = object.__new__(cls)
    for name, value in members.items():
        object.__setattr__(record, name, value)
    return record

problems

problems() -> Iterator[ValidationProblem]

Every reason these values are not allowed, yielded as found. Default: none.

Source code in src/zarr_metadata/v3/chunk_grid/regular.py

def problems(self) -> "Iterator[ValidationProblem]":
    for index, extent in enumerate(self.chunk_shape):
        if extent < 1:
            yield ValidationProblem(
                ("chunk_shape", index),
                f"expected an integer >= 1, got {extent}",
                "invalid_value",
            )

zarr_metadata.v3.chunk_grid.rectilinear

Rectilinear chunk grid (zarr-extensions).

See https://github.com/zarr-developers/zarr-extensions/blob/4da7b37a84f76e660902f6d3de3eaef0e0febae6/chunk-grids/rectilinear/README.md

RECTILINEAR_CHUNK_GRID_KIND module-attribute

RECTILINEAR_CHUNK_GRID_KIND: Final = ('inline',)

The kind values the rectilinear grid defines.

Only inline so far: the extents are written into the metadata. The member exists so a later kind can put them somewhere else.

RECTILINEAR_CHUNK_GRID_NAME module-attribute

RECTILINEAR_CHUNK_GRID_NAME: Final = 'rectilinear'

The name field value of the rectilinear chunk grid.

RectilinearChunkGridMetadata module-attribute

RectilinearChunkGridMetadata = RectilinearChunkGridObject

Permitted JSON shape for rectilinear chunk grid metadata.

kind and chunk_shapes are required, so only the object form is valid; the short-hand-name form is not permitted by the spec for this grid. https://github.com/zarr-developers/zarr-extensions/blob/4da7b37a84f76e660902f6d3de3eaef0e0febae6/chunk-grids/rectilinear/README.md#L59-L62 https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L1562-L1564

RectilinearChunkGridName module-attribute

RectilinearChunkGridName = Literal['rectilinear']

Literal type of the name field of the rectilinear chunk grid.

RectilinearDimSpec module-attribute

RectilinearDimSpec = int | tuple[int | tuple[int, int], ...]

JSON shape for one dimension's rectilinear spec.

Either a bare integer (uniform shorthand for a regular dimension within a rectilinear grid), or a tuple of integers and/or [value, count] RLE pairs.

__all__ module-attribute

__all__ = [
    "RECTILINEAR_CHUNK_GRID_KIND",
    "RECTILINEAR_CHUNK_GRID_NAME",
    "RectilinearChunkGrid",
    "RectilinearChunkGridConfiguration",
    "RectilinearChunkGridMetadata",
    "RectilinearChunkGridName",
    "RectilinearChunkGridObject",
    "RectilinearChunkGridOptions",
    "RectilinearDimSpec",
    "canonical_chunk_shapes",
    "canonical_dim_spec",
]

RectilinearChunkGrid dataclass

Bases: ChunkGridEntity

The rectilinear chunk grid, coerced from its metadata.

Source code in src/zarr_metadata/v3/chunk_grid/rectilinear.py

@dataclass(frozen=True)
class RectilinearChunkGrid(ChunkGridEntity):
    """The `rectilinear` chunk grid, coerced from its metadata."""

    configuration: RectilinearChunkGridOptions

    identifier: ClassVar[str] = RECTILINEAR_CHUNK_GRID_NAME

    def shape_problems(self, array_shape: object) -> tuple[ValidationProblem, ...]:
        """One spec per dimension, and explicit specs must cover it.

        A bare integer is uniform shorthand, so it covers whatever the
        dimension turns out to be and imposes no sum; an explicit list
        names every chunk, so the names have to add up.
        """
        if not isinstance(array_shape, (list, tuple)):
            return ()
        extents = tuple(cast("Sequence[object]", array_shape))
        if len(self.configuration.chunk_shapes) != len(extents):
            return problem(
                ("chunk_shapes",),
                f"chunk_shapes has {len(self.configuration.chunk_shapes)} entries but shape has "
                f"{len(extents)} dimensions",
                "invalid_value",
            )
        found: list[ValidationProblem] = []
        for dim, (spec, extent) in enumerate(
            zip(self.configuration.chunk_shapes, extents, strict=True)
        ):
            if isinstance(spec, int) or not is_integer(extent):
                continue
            total = _covered_extent(spec)
            if total is not None and total < extent:
                found.extend(
                    problem(
                        ("chunk_shapes", dim),
                        f"chunk sizes sum to {total} but must cover shape[{dim}] extent {extent}",
                        "invalid_value",
                    )
                )
        return tuple(found)

    def grid(self, array_shape: object) -> ChunkGrid:
        """The distinct lengths each axis's chunks take.

        Plural per axis, which is the point of a rectilinear grid: an
        axis of `[30, 34]` gives `{30, 34}`, and anything asking about
        divisibility has to hold for both.
        """
        return ChunkGrid.derived(
            tuple(_axis_lengths(spec) for spec in self.configuration.chunk_shapes)
        )

    def canonical(self) -> Self:
        """Run-length encoded, which is the spelling that does not grow.

        Two dimension specs listing the same extents describe the same
        grid, and the encoded one stays the same size as the array grows.
        """
        return self.with_configuration(
            chunk_shapes=canonical_chunk_shapes(self.configuration.chunk_shapes)
        )

configuration instance-attribute

configuration: RectilinearChunkGridOptions

The record of this entity's members.

An entity with members narrows it to its own record, configuration: GzipOptions, its one positional argument. An entity of a bare name defaults it to the empty record -- configuration: Configuration = field(default_factory=Configuration) -- so that Crc32cCodec() builds; coerce passes the record either way.

identifier class-attribute

identifier: str = RECTILINEAR_CHUNK_GRID_NAME

The name this entity is registered under.

Usually the name the metadata carries. The raw-bytes data types are the exception: every r<N> spelling is one family, so the family gets an invented identifier that no real name can collide with.

__init__

__init__(
    configuration: RectilinearChunkGridOptions,
) -> None

__post_init__

__post_init__() -> None

Refuse a record that is not this entity's own, then the first problem the rules find.

The runtime half of the entity's type, as the record's constructor is of the record's: GzipCodec(BloscOptions(...)) and a family member carrying a name that is not a string are refused before any rule reads them. Then BloscCodec(BloscOptions(clevel=99)) raises on the first problem the rules yield.

Source code in src/zarr_metadata/v3/_entity.py

def __post_init__(self) -> None:
    """Refuse a record that is not this entity's own, then the first problem the rules find.

    The runtime half of the entity's type, as the record's constructor
    is of the record's: `GzipCodec(BloscOptions(...))` and a family
    member carrying a name that is not a string are refused before
    any rule reads them. Then `BloscCodec(BloscOptions(clevel=99))`
    raises on the first problem the rules yield.
    """
    plan = _plan(type(self))
    if not isinstance(self.configuration, plan.record):
        raise MetadataValidationError(
            problem(
                (),
                f"expected a {plan.record.__name__} configuration, got "
                f"{type(self.configuration).__name__}",
            )
        )
    name: object = self.identifier if plan.from_name is None else getattr(self, plan.from_name)
    if not isinstance(name, str):
        raise MetadataValidationError(problem((), f"expected a string name, got {name!r}"))
    first = next(type(self).name_problems(name), None)
    if first is None:
        first = next(self.configuration.problems(), None)
    if first is not None:
        raise MetadataValidationError((first,))

accepts classmethod

accepts(name: str) -> bool

Whether name denotes this entity.

Constant for all but the raw-bytes family, where one class covers every r<N>.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def accepts(cls, name: str) -> bool:
    """Whether `name` denotes this entity.

    Constant for all but the raw-bytes family, where one class covers
    every `r<N>`.
    """
    return name == cls.identifier

canonical

canonical() -> Self

Run-length encoded, which is the spelling that does not grow.

Two dimension specs listing the same extents describe the same grid, and the encoded one stays the same size as the array grows.

Source code in src/zarr_metadata/v3/chunk_grid/rectilinear.py

def canonical(self) -> Self:
    """Run-length encoded, which is the spelling that does not grow.

    Two dimension specs listing the same extents describe the same
    grid, and the encoded one stays the same size as the array grows.
    """
    return self.with_configuration(
        chunk_shapes=canonical_chunk_shapes(self.configuration.chunk_shapes)
    )

coerce classmethod

coerce(value: JSONValue, context: Context) -> Coerced[Self]

value as this entity, or the reasons it is not one: the class's validation routine.

resolve relates a field's name to this class and hands it the field, refined JSON with arrays as tuples, which is what value is; this is what the class does with it. The configuration is parsed against the record the configuration field names, member by member; a member holding another entity is read in context, the scope this reading is happening in. An optional member the document left out is UNSET in the record, so no field's default decides what a document said. The entity is built only when every member of its own read -- its rules are written over a whole configuration -- and handed back only when everything inside it read too.

The envelope is the field's, not the class's, and resolve judges it: a stray member or a must_understand of false is not reported here. Called on a class no scope has registered, this runs with none of registration's refusals having happened.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def coerce(cls, value: JSONValue, context: Context) -> Coerced[Self]:
    """`value` as this entity, or the reasons it is not one: the class's validation routine.

    `resolve` relates a field's name to this class and hands it the
    field, refined JSON with arrays as tuples, which is what `value`
    is; this is what the class does with it. The configuration is
    parsed against the record the `configuration` field names,
    member by member; a member holding another entity is read in
    `context`, the scope this reading is happening in. An optional
    member the document left out is `UNSET` in the record, so no
    field's default decides what a document said. The entity is
    built only when every member of its own read -- its rules are
    written over a whole configuration -- and handed back only when
    everything inside it read too.

    The envelope is the field's, not the class's, and `resolve`
    judges it: a stray member or a `must_understand` of `false` is
    not reported here. Called on a class no scope has registered,
    this runs with none of registration's refusals having happened.
    """
    name, given, envelope = named_configuration(value)
    if name is None or not cls.accepts(name):
        return None, problem((), f"expected the {cls.identifier!r} entity")
    if len(envelope) != 0:
        return None, envelope
    plan = _plan(cls)
    carried: dict[str, str] = {} if plan.from_name is None else {plan.from_name: name}
    if given is None and plan.requires_configuration:
        return None, problem(
            ("configuration",),
            f"{cls.identifier!r} requires a configuration",
            "missing_key",
        )
    reading = _Reading(context, [])
    # A bare name's record has no members, so any key is an unknown one.
    record, own = plan.read({} if given is None else given, ("configuration",), reading)
    found = (*own, *reading.nested)
    if record is None:
        # An unknown key is survivable; a member that could not be
        # read is a hole, and judging around it would be guessing.
        return None, found
    # The rules, asked of the name and of the record before anything
    # is built: a name problem lands on the entity, a configuration
    # problem under the configuration.
    refused = (*cls.name_problems(name), *within((), tuple(record.problems())))
    if len(refused) != 0:
        # Values the spec disallows: reported rather than raised,
        # every one.
        return None, (*found, *refused)
    if any(entry.kind != "unknown_key" for entry in reading.nested):
        # A contained entity could not be read. This entity's own
        # rules ran -- an invalid inner is an `Opaque`, as an
        # out-of-scope one is -- but what is handed back is not an
        # entity that would be asked composition questions it cannot
        # answer.
        return None, found
    return cls.create_unchecked(configuration=record, **carried), found

create_unchecked classmethod

create_unchecked(**fields: object) -> Self

This entity with these fields, built without the constructor's checks.

The one way around them, for a caller that has just made them: coerce, which type-checked the record and ran the rules before building. Every field is given -- the record, and the carried name for a family -- since nothing here applies a default. Anything that has not checked goes through the constructor.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def create_unchecked(cls, **fields: object) -> Self:
    """This entity with these fields, built without the constructor's checks.

    The one way around them, for a caller that has just made them:
    `coerce`, which type-checked the record and ran the rules before
    building. Every field is given -- the record, and the carried
    name for a family -- since nothing here applies a default.
    Anything that has not checked goes through the constructor.
    """
    entity = object.__new__(cls)
    for name, value in fields.items():
        object.__setattr__(entity, name, value)
    return entity

grid

grid(array_shape: object) -> ChunkGrid

The distinct lengths each axis's chunks take.

Plural per axis, which is the point of a rectilinear grid: an axis of [30, 34] gives {30, 34}, and anything asking about divisibility has to hold for both.

Source code in src/zarr_metadata/v3/chunk_grid/rectilinear.py

def grid(self, array_shape: object) -> ChunkGrid:
    """The distinct lengths each axis's chunks take.

    Plural per axis, which is the point of a rectilinear grid: an
    axis of `[30, 34]` gives `{30, 34}`, and anything asking about
    divisibility has to hold for both.
    """
    return ChunkGrid.derived(
        tuple(_axis_lengths(spec) for spec in self.configuration.chunk_shapes)
    )

name_problems classmethod

name_problems(name: str) -> Iterator[ValidationProblem]

Why name, which accepts claimed, is not a well-formed name of this family.

For a family, whose names carry data -- r<N> -- and which claims a malformed member so that it is reported rather than waved through as an unknown extension. Locations are relative to the entity: (). Default: none, for an entity of one name.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def name_problems(cls, name: str) -> Iterator[ValidationProblem]:
    """Why `name`, which `accepts` claimed, is not a well-formed name of this family.

    For a family, whose names carry data -- `r<N>` -- and which claims
    a malformed member so that it is reported rather than waved
    through as an unknown extension. Locations are relative to the
    entity: `()`. Default: none, for an entity of one name.
    """
    yield from ()

shape_problems

shape_problems(
    array_shape: object,
) -> tuple[ValidationProblem, ...]

One spec per dimension, and explicit specs must cover it.

A bare integer is uniform shorthand, so it covers whatever the dimension turns out to be and imposes no sum; an explicit list names every chunk, so the names have to add up.

Source code in src/zarr_metadata/v3/chunk_grid/rectilinear.py

def shape_problems(self, array_shape: object) -> tuple[ValidationProblem, ...]:
    """One spec per dimension, and explicit specs must cover it.

    A bare integer is uniform shorthand, so it covers whatever the
    dimension turns out to be and imposes no sum; an explicit list
    names every chunk, so the names have to add up.
    """
    if not isinstance(array_shape, (list, tuple)):
        return ()
    extents = tuple(cast("Sequence[object]", array_shape))
    if len(self.configuration.chunk_shapes) != len(extents):
        return problem(
            ("chunk_shapes",),
            f"chunk_shapes has {len(self.configuration.chunk_shapes)} entries but shape has "
            f"{len(extents)} dimensions",
            "invalid_value",
        )
    found: list[ValidationProblem] = []
    for dim, (spec, extent) in enumerate(
        zip(self.configuration.chunk_shapes, extents, strict=True)
    ):
        if isinstance(spec, int) or not is_integer(extent):
            continue
        total = _covered_extent(spec)
        if total is not None and total < extent:
            found.extend(
                problem(
                    ("chunk_shapes", dim),
                    f"chunk sizes sum to {total} but must cover shape[{dim}] extent {extent}",
                    "invalid_value",
                )
            )
    return tuple(found)

to_json

to_json() -> ZarrV3MetadataFieldJSON

This entity as a document would write it.

Written from the configuration record by the same declaration coerce reads it by, each member by the writer its annotation implies: the bare name when every member it holds is absent, the object otherwise, a contained entity through its own to_json, a JSON-valued member copied so the document is not a handle on the entity. Faithful to every member: read a document, write it back, and those come out as they went in. The envelope is written the entity's way -- the bare name when nothing is configured, the object otherwise, no must_understand, which means what absence means -- because an entity alone has no document to be faithful to; ArrayDocumentV3.to_json puts back the spelling the document used. Ask canonical first if you want the simplest equivalent spelling.

An entity whose JSON is not its fields overrides this; none in the package does.

Source code in src/zarr_metadata/v3/_entity.py

def to_json(self) -> ZarrV3MetadataFieldJSON:
    """This entity as a document would write it.

    Written from the configuration record by the same declaration
    `coerce` reads it by, each member by the writer its annotation
    implies: the bare name when every member it holds is absent, the
    object otherwise, a contained entity through its own `to_json`,
    a JSON-valued member copied so the document is not a handle on
    the entity. Faithful to every member: read a document, write it
    back, and those come out as they went in. The envelope is
    written the entity's way -- the bare name when nothing is
    configured, the object otherwise, no `must_understand`, which
    means what absence means -- because an entity alone has no
    document to be faithful to; `ArrayDocumentV3.to_json` puts back
    the spelling the document used. Ask `canonical` first if you
    want the simplest equivalent spelling.

    An entity whose JSON is not its fields overrides this; none in
    the package does.
    """
    plan = _plan(type(self))
    name = self.identifier
    if plan.from_name is not None:
        carried = getattr(self, plan.from_name)
        name = carried if isinstance(carried, str) else name
    configuration = plan.write(self)
    if len(configuration) == 0:
        return name
    return {"name": name, "configuration": configuration}

with_configuration

with_configuration(**changes: object) -> Self

This entity with these configuration members changed.

codec.with_configuration(typesize=UNSET) is the record rebuilt through its constructor, which refuses a member of the wrong type, and the entity rebuilt through its own, which refuses a value the rules disallow -- the same checks as any construction, since pyright cannot see the members through **changes. A name that is not a member is refused the way replace refuses it.

Source code in src/zarr_metadata/v3/_entity.py

def with_configuration(self, **changes: object) -> Self:
    """This entity with these configuration members changed.

    `codec.with_configuration(typesize=UNSET)` is the record rebuilt
    through its constructor, which refuses a member of the wrong
    type, and the entity rebuilt through its own, which refuses a
    value the rules disallow -- the same checks as any construction,
    since pyright cannot see the members through `**changes`. A
    name that is not a member is refused the way `replace` refuses
    it.
    """
    return replace(self, configuration=replace(self.configuration, **changes))

RectilinearChunkGridConfiguration

Bases: TypedDict

Configuration for the rectilinear chunk grid.

Source code in src/zarr_metadata/v3/chunk_grid/rectilinear.py

class RectilinearChunkGridConfiguration(TypedDict, closed=True):
    """Configuration for the rectilinear chunk grid."""

    kind: Literal["inline"]
    chunk_shapes: tuple[RectilinearDimSpec, ...]

chunk_shapes instance-attribute

chunk_shapes: tuple[RectilinearDimSpec, ...]

kind instance-attribute

kind: Literal['inline']

RectilinearChunkGridObject

Bases: TypedDict

Rectilinear chunk grid metadata in object form.

Source code in src/zarr_metadata/v3/chunk_grid/rectilinear.py

class RectilinearChunkGridObject(TypedDict, closed=True):
    """Rectilinear chunk grid metadata in object form."""

    name: RectilinearChunkGridName
    configuration: RectilinearChunkGridConfiguration
    must_understand: NotRequired[bool]

configuration instance-attribute

configuration: RectilinearChunkGridConfiguration

must_understand instance-attribute

must_understand: NotRequired[bool]

name instance-attribute

name: RectilinearChunkGridName

RectilinearChunkGridOptions dataclass

Bases: Configuration

What a rectilinear grid is configured with.

Source code in src/zarr_metadata/v3/chunk_grid/rectilinear.py

@dataclass(frozen=True)
class RectilinearChunkGridOptions(Configuration):
    """What a `rectilinear` grid is configured with."""

    kind: Literal["inline"]
    chunk_shapes: tuple[RectilinearDimSpec, ...]

    def problems(self) -> "Iterator[ValidationProblem]":
        """Every extent, and every run's length and count, is at least 1."""
        for axis, spec in enumerate(self.chunk_shapes):
            if isinstance(spec, int):
                if spec < 1:
                    yield _not_positive(("chunk_shapes", axis), spec)
                continue
            for index, entry in enumerate(spec):
                if isinstance(entry, int):
                    if entry < 1:
                        yield _not_positive(("chunk_shapes", axis, index), entry)
                    continue
                for position, value in enumerate(entry):
                    if value < 1:
                        yield _not_positive(("chunk_shapes", axis, index, position), value)

chunk_shapes instance-attribute

chunk_shapes: tuple[RectilinearDimSpec, ...]

kind instance-attribute

kind: Literal['inline']

__init__

__init__(
    kind: Literal["inline"],
    chunk_shapes: tuple[RectilinearDimSpec, ...],
) -> None

__post_init__

__post_init__() -> None

Refuse every member of the wrong type, so GzipOptions(level="high") raises.

Source code in src/zarr_metadata/v3/_entity.py

def __post_init__(self) -> None:
    """Refuse every member of the wrong type, so `GzipOptions(level="high")` raises."""
    found = mistyped(self)
    if len(found) != 0:
        raise MetadataValidationError(found)

create_unchecked classmethod

create_unchecked(**members: object) -> Self

This record with these members, built without the constructor's check.

The one way around the check, for a caller that has just made it: the parser, which type-checked every member against the same annotations before building the record. Every field is given -- the parser gives an absent optional member as UNSET -- since nothing here applies a default. Anything that has not checked the members goes through the constructor.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def create_unchecked(cls, **members: object) -> Self:
    """This record with these members, built without the constructor's check.

    The one way around the check, for a caller that has just made
    it: the parser, which type-checked every member against the same
    annotations before building the record. Every field is given --
    the parser gives an absent optional member as `UNSET` -- since
    nothing here applies a default. Anything that has not checked
    the members goes through the constructor.
    """
    record = object.__new__(cls)
    for name, value in members.items():
        object.__setattr__(record, name, value)
    return record

problems

problems() -> Iterator[ValidationProblem]

Every extent, and every run's length and count, is at least 1.

Source code in src/zarr_metadata/v3/chunk_grid/rectilinear.py

def problems(self) -> "Iterator[ValidationProblem]":
    """Every extent, and every run's length and count, is at least 1."""
    for axis, spec in enumerate(self.chunk_shapes):
        if isinstance(spec, int):
            if spec < 1:
                yield _not_positive(("chunk_shapes", axis), spec)
            continue
        for index, entry in enumerate(spec):
            if isinstance(entry, int):
                if entry < 1:
                    yield _not_positive(("chunk_shapes", axis, index), entry)
                continue
            for position, value in enumerate(entry):
                if value < 1:
                    yield _not_positive(("chunk_shapes", axis, index, position), value)

canonical_chunk_shapes

canonical_chunk_shapes(
    chunk_shapes: tuple[RectilinearDimSpec, ...],
) -> tuple[RectilinearDimSpec, ...]

Every dimension's chunk sizes in their simplest equivalent form.

Source code in src/zarr_metadata/v3/chunk_grid/rectilinear.py

def canonical_chunk_shapes(
    chunk_shapes: tuple[RectilinearDimSpec, ...],
) -> tuple[RectilinearDimSpec, ...]:
    """Every dimension's chunk sizes in their simplest equivalent form."""
    return tuple(canonical_dim_spec(spec) for spec in chunk_shapes)

canonical_dim_spec

canonical_dim_spec(
    spec: RectilinearDimSpec,
) -> RectilinearDimSpec

One dimension's chunk sizes in their simplest equivalent form.

Runs of equal sizes collapse to [size, count] pairs, because that is the spelling that does not grow with the number of chunks: a million equal chunks is two numbers, not a million. A run of one stays a bare size, and [size, 1] collapses to one, since a pair says nothing extra there. Adjacent spellings of the same size merge, which is what makes this idempotent: [[32, 2], 32] and [32, [32, 2]] both become [[32, 3]].

A dimension-level bare integer is left alone. It is a step that repeats until it covers the extent, so it is not equivalent to any fixed list — expanding it would pin a grid that currently adapts, and the two would diverge the moment the array were resized. For the same reason a one-element list is never collapsed to a bare integer: [32] declares exactly one chunk and 32 declares as many as it takes.

Assumes a spec the shape validator has already accepted.

Source code in src/zarr_metadata/v3/chunk_grid/rectilinear.py

def canonical_dim_spec(spec: RectilinearDimSpec) -> RectilinearDimSpec:
    """One dimension's chunk sizes in their simplest equivalent form.

    Runs of equal sizes collapse to `[size, count]` pairs, because that is
    the spelling that does not grow with the number of chunks: a million
    equal chunks is two numbers, not a million. A run of one stays a bare
    size, and `[size, 1]` collapses to one, since a pair says nothing extra
    there. Adjacent spellings of the same size merge, which is what makes
    this idempotent: `[[32, 2], 32]` and `[32, [32, 2]]` both become
    `[[32, 3]]`.

    A dimension-level bare integer is left alone. It is a *step* that
    repeats until it covers the extent, so it is not equivalent to any
    fixed list — expanding it would pin a grid that currently adapts, and
    the two would diverge the moment the array were resized. For the same
    reason a one-element list is never collapsed to a bare integer:
    `[32]` declares exactly one chunk and `32` declares as many as it takes.

    Assumes a spec the shape validator has already accepted.
    """
    if not isinstance(spec, tuple):
        return spec
    runs: list[tuple[int, int]] = []
    for entry in spec:
        size, count = entry if isinstance(entry, tuple) else (entry, 1)
        if len(runs) != 0 and runs[-1][0] == size:
            runs[-1] = (size, runs[-1][1] + count)
        else:
            runs.append((size, count))
    return tuple(size if count == 1 else (size, count) for size, count in runs)