zarr_metadata.v3.codec

zarr_metadata.v3.codec

Zarr v3 codec spec types.

Each codec defined by the spec or by zarr-extensions has its own submodule (blosc, bytes, cast_value, crc32c, gzip, scale_offset, sharding_indexed, transpose, zstd).

The <X>CodecMetadata aliases re-exported here are the canonical type for each codec's permitted JSON shapes (object form plus, where the spec allows, a bare-string short-hand form). For the underlying <X>CodecObject, <X>CodecConfiguration, etc., import directly from the leaf submodule.

For the field-level "any codec entry" alias (used in array metadata's codecs list and in sharding's inner pipelines), import ZarrV3MetadataFieldJSON from zarr_metadata.v3.

Each codec's pipeline position (array -> array, array -> bytes, bytes -> bytes) is the kind class its entity subclasses, in zarr_metadata.v3.entity.

See https://zarr-specs.readthedocs.io/en/latest/v3/codecs/index.html

zarr_metadata.v3.codec.blosc

Blosc codec types.

See https://zarr-specs.readthedocs.io/en/latest/v3/codecs/blosc/index.html

BLOSC_CNAME module-attribute

BLOSC_CNAME: Final = (
    "lz4",
    "lz4hc",
    "blosclz",
    "snappy",
    "zlib",
    "zstd",
)

Tuple of permitted values for the cname field of the blosc codec.

BLOSC_CODEC_NAME module-attribute

BLOSC_CODEC_NAME: Final = 'blosc'

The name field value of the blosc codec.

BLOSC_NO_SHUFFLE module-attribute

BLOSC_NO_SHUFFLE: Final = 'noshuffle'

The shuffle value under which typesize carries no information.

The spec requires typesize "unless shuffle is "noshuffle", in which case the value is ignored", so this is the one value that changes whether another member is required.

BLOSC_SHUFFLE module-attribute

BLOSC_SHUFFLE: Final = (
    "noshuffle",
    "shuffle",
    "bitshuffle",
)

Tuple of permitted values for the shuffle field of the blosc codec.

BloscCName module-attribute

BloscCName = Literal[
    "lz4", "lz4hc", "blosclz", "snappy", "zlib", "zstd"
]

Literal type of blosc compressor identifiers.

BloscCodecMetadata module-attribute

BloscCodecMetadata = BloscCodecObject

Permitted JSON shape for blosc codec metadata.

The configuration has multiple required keys (cname, clevel, shuffle, blocksize), so only the object form is valid; the short-hand-name form is not permitted by the spec for this codec. https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/codecs/blosc/index.rst#L57-L98 (configuration parameters) https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L1562-L1564 (short-hand names only "if no configuration metadata is required")

BloscCodecName module-attribute

BloscCodecName = Literal['blosc']

Literal type of the name field of the blosc codec.

BloscShuffle module-attribute

BloscShuffle = Literal["noshuffle", "shuffle", "bitshuffle"]

Literal type of blosc shuffle mode names.

__all__ module-attribute

__all__ = [
    "BLOSC_CNAME",
    "BLOSC_CODEC_NAME",
    "BLOSC_NO_SHUFFLE",
    "BLOSC_SHUFFLE",
    "BloscCName",
    "BloscCodec",
    "BloscCodecConfiguration",
    "BloscCodecMetadata",
    "BloscCodecName",
    "BloscCodecObject",
    "BloscOptions",
    "BloscShuffle",
]

BloscCodec dataclass

Bases: BytesBytesCodec

The blosc codec, coerced from its metadata.

Everything blosc knows about itself: the shape its metadata takes, the values the spec allows in it, and the simplest spelling of an equivalent document.

Source code in src/zarr_metadata/v3/codec/blosc.py

@dataclass(frozen=True)
class BloscCodec(BytesBytesCodec):
    """The `blosc` codec, coerced from its metadata.

    Everything blosc knows about itself: the shape its metadata takes, the
    values the spec allows in it, and the simplest spelling of an
    equivalent document.
    """

    configuration: BloscOptions

    identifier: ClassVar[str] = BLOSC_CODEC_NAME
    variable_size: ClassVar[bool] = True

    # Every member is required but `typesize`, which only means something
    # when shuffling; `BloscOptions.problems` is where that conditional lives.

    def canonical(self) -> Self:
        """Without a `typesize` that `noshuffle` renders meaningless.

        The spec says of that case that "the value is ignored", so two
        documents differing only there describe the same codec.
        """
        if self.configuration.shuffle != BLOSC_NO_SHUFFLE or self.configuration.typesize is UNSET:
            return self
        return self.with_configuration(typesize=UNSET)

configuration instance-attribute

configuration: BloscOptions

The record of this entity's members.

An entity with members narrows it to its own record, configuration: GzipOptions, its one positional argument. An entity of a bare name defaults it to the empty record -- configuration: Configuration = field(default_factory=Configuration) -- so that Crc32cCodec() builds; coerce passes the record either way.

identifier class-attribute

identifier: str = BLOSC_CODEC_NAME

The name this entity is registered under.

Usually the name the metadata carries. The raw-bytes data types are the exception: every r<N> spelling is one family, so the family gets an invented identifier that no real name can collide with.

variable_size class-attribute

variable_size: bool = True

Whether this codec's output size depends on the bytes it is given.

A compressor's does, so a shard index encoded with one has no size derivable from metadata alone, and the shard cannot be read. Every codec says, because a default in either direction is a verdict.

__init__

__init__(configuration: BloscOptions) -> None

__post_init__

__post_init__() -> None

Refuse a record that is not this entity's own, then the first problem the rules find.

The runtime half of the entity's type, as the record's constructor is of the record's: GzipCodec(BloscOptions(...)) and a family member carrying a name that is not a string are refused before any rule reads them. Then BloscCodec(BloscOptions(clevel=99)) raises on the first problem the rules yield.

Source code in src/zarr_metadata/v3/_entity.py

def __post_init__(self) -> None:
    """Refuse a record that is not this entity's own, then the first problem the rules find.

    The runtime half of the entity's type, as the record's constructor
    is of the record's: `GzipCodec(BloscOptions(...))` and a family
    member carrying a name that is not a string are refused before
    any rule reads them. Then `BloscCodec(BloscOptions(clevel=99))`
    raises on the first problem the rules yield.
    """
    plan = _plan(type(self))
    if not isinstance(self.configuration, plan.record):
        raise MetadataValidationError(
            problem(
                (),
                f"expected a {plan.record.__name__} configuration, got "
                f"{type(self.configuration).__name__}",
            )
        )
    name: object = self.identifier if plan.from_name is None else getattr(self, plan.from_name)
    if not isinstance(name, str):
        raise MetadataValidationError(problem((), f"expected a string name, got {name!r}"))
    first = next(type(self).name_problems(name), None)
    if first is None:
        first = next(self.configuration.problems(), None)
    if first is not None:
        raise MetadataValidationError((first,))

accepts classmethod

accepts(name: str) -> bool

Whether name denotes this entity.

Constant for all but the raw-bytes family, where one class covers every r<N>.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def accepts(cls, name: str) -> bool:
    """Whether `name` denotes this entity.

    Constant for all but the raw-bytes family, where one class covers
    every `r<N>`.
    """
    return name == cls.identifier

canonical

canonical() -> Self

Without a typesize that noshuffle renders meaningless.

The spec says of that case that "the value is ignored", so two documents differing only there describe the same codec.

Source code in src/zarr_metadata/v3/codec/blosc.py

def canonical(self) -> Self:
    """Without a `typesize` that `noshuffle` renders meaningless.

    The spec says of that case that "the value is ignored", so two
    documents differing only there describe the same codec.
    """
    if self.configuration.shuffle != BLOSC_NO_SHUFFLE or self.configuration.typesize is UNSET:
        return self
    return self.with_configuration(typesize=UNSET)

coerce classmethod

coerce(value: JSONValue, context: Context) -> Coerced[Self]

value as this entity, or the reasons it is not one: the class's validation routine.

resolve relates a field's name to this class and hands it the field, refined JSON with arrays as tuples, which is what value is; this is what the class does with it. The configuration is parsed against the record the configuration field names, member by member; a member holding another entity is read in context, the scope this reading is happening in. An optional member the document left out is UNSET in the record, so no field's default decides what a document said. The entity is built only when every member of its own read -- its rules are written over a whole configuration -- and handed back only when everything inside it read too.

The envelope is the field's, not the class's, and resolve judges it: a stray member or a must_understand of false is not reported here. Called on a class no scope has registered, this runs with none of registration's refusals having happened.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def coerce(cls, value: JSONValue, context: Context) -> Coerced[Self]:
    """`value` as this entity, or the reasons it is not one: the class's validation routine.

    `resolve` relates a field's name to this class and hands it the
    field, refined JSON with arrays as tuples, which is what `value`
    is; this is what the class does with it. The configuration is
    parsed against the record the `configuration` field names,
    member by member; a member holding another entity is read in
    `context`, the scope this reading is happening in. An optional
    member the document left out is `UNSET` in the record, so no
    field's default decides what a document said. The entity is
    built only when every member of its own read -- its rules are
    written over a whole configuration -- and handed back only when
    everything inside it read too.

    The envelope is the field's, not the class's, and `resolve`
    judges it: a stray member or a `must_understand` of `false` is
    not reported here. Called on a class no scope has registered,
    this runs with none of registration's refusals having happened.
    """
    name, given, envelope = named_configuration(value)
    if name is None or not cls.accepts(name):
        return None, problem((), f"expected the {cls.identifier!r} entity")
    if len(envelope) != 0:
        return None, envelope
    plan = _plan(cls)
    carried: dict[str, str] = {} if plan.from_name is None else {plan.from_name: name}
    if given is None and plan.requires_configuration:
        return None, problem(
            ("configuration",),
            f"{cls.identifier!r} requires a configuration",
            "missing_key",
        )
    reading = _Reading(context, [])
    # A bare name's record has no members, so any key is an unknown one.
    record, own = plan.read({} if given is None else given, ("configuration",), reading)
    found = (*own, *reading.nested)
    if record is None:
        # An unknown key is survivable; a member that could not be
        # read is a hole, and judging around it would be guessing.
        return None, found
    # The rules, asked of the name and of the record before anything
    # is built: a name problem lands on the entity, a configuration
    # problem under the configuration.
    refused = (*cls.name_problems(name), *within((), tuple(record.problems())))
    if len(refused) != 0:
        # Values the spec disallows: reported rather than raised,
        # every one.
        return None, (*found, *refused)
    if any(entry.kind != "unknown_key" for entry in reading.nested):
        # A contained entity could not be read. This entity's own
        # rules ran -- an invalid inner is an `Opaque`, as an
        # out-of-scope one is -- but what is handed back is not an
        # entity that would be asked composition questions it cannot
        # answer.
        return None, found
    return cls.create_unchecked(configuration=record, **carried), found

create_unchecked classmethod

create_unchecked(**fields: object) -> Self

This entity with these fields, built without the constructor's checks.

The one way around them, for a caller that has just made them: coerce, which type-checked the record and ran the rules before building. Every field is given -- the record, and the carried name for a family -- since nothing here applies a default. Anything that has not checked goes through the constructor.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def create_unchecked(cls, **fields: object) -> Self:
    """This entity with these fields, built without the constructor's checks.

    The one way around them, for a caller that has just made them:
    `coerce`, which type-checked the record and ran the rules before
    building. Every field is given -- the record, and the carried
    name for a family -- since nothing here applies a default.
    Anything that has not checked goes through the constructor.
    """
    entity = object.__new__(cls)
    for name, value in fields.items():
        object.__setattr__(entity, name, value)
    return entity

incoming_problems

incoming_problems(
    incoming: ArrayParts | None,
) -> tuple[ValidationProblem, ...]

Why this codec cannot be applied to the array that reaches it.

incoming is None once the chain can no longer say what reaches here, and the default answer to that is nothing: declining beats guessing. Locations are relative to this codec's configuration; an empty one lands on the codec itself.

Source code in src/zarr_metadata/v3/_entity.py

def incoming_problems(self, incoming: ArrayParts | None) -> tuple[ValidationProblem, ...]:
    """Why this codec cannot be applied to the array that reaches it.

    `incoming` is None once the chain can no longer say what reaches
    here, and the default answer to that is nothing: declining beats
    guessing. Locations are relative to this codec's `configuration`;
    an empty one lands on the codec itself.
    """
    return ()

inner_pipelines

inner_pipelines(
    incoming: ArrayParts | None,
) -> Mapping[
    str,
    tuple[
        Sequence[CodecEntity | Opaque], ArrayParts | None
    ],
]

The pipelines this codec holds, by the member holding each, with what each is handed.

A shard holds two: its codecs, handed its inner chunk, and its index_codecs, handed the shard index. Refinement walks them as it walks the pipeline this codec stands in, locating what it finds under the member, so a codec that holds pipelines says which and what they receive, and judges nothing inside them itself. Default: none.

Source code in src/zarr_metadata/v3/_entity.py

def inner_pipelines(
    self, incoming: ArrayParts | None
) -> Mapping[str, tuple[Sequence[CodecEntity | Opaque], ArrayParts | None]]:
    """The pipelines this codec holds, by the member holding each, with what each is handed.

    A shard holds two: its `codecs`, handed its inner chunk, and its
    `index_codecs`, handed the shard index. Refinement walks them as
    it walks the pipeline this codec stands in, locating what it
    finds under the member, so a codec that holds pipelines says
    which and what they receive, and judges nothing inside them
    itself. Default: none.
    """
    return {}

name_problems classmethod

name_problems(name: str) -> Iterator[ValidationProblem]

Why name, which accepts claimed, is not a well-formed name of this family.

For a family, whose names carry data -- r<N> -- and which claims a malformed member so that it is reported rather than waved through as an unknown extension. Locations are relative to the entity: (). Default: none, for an entity of one name.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def name_problems(cls, name: str) -> Iterator[ValidationProblem]:
    """Why `name`, which `accepts` claimed, is not a well-formed name of this family.

    For a family, whose names carry data -- `r<N>` -- and which claims
    a malformed member so that it is reported rather than waved
    through as an unknown extension. Locations are relative to the
    entity: `()`. Default: none, for an entity of one name.
    """
    yield from ()

to_json

to_json() -> ZarrV3MetadataFieldJSON

This entity as a document would write it.

Written from the configuration record by the same declaration coerce reads it by, each member by the writer its annotation implies: the bare name when every member it holds is absent, the object otherwise, a contained entity through its own to_json, a JSON-valued member copied so the document is not a handle on the entity. Faithful to every member: read a document, write it back, and those come out as they went in. The envelope is written the entity's way -- the bare name when nothing is configured, the object otherwise, no must_understand, which means what absence means -- because an entity alone has no document to be faithful to; ArrayDocumentV3.to_json puts back the spelling the document used. Ask canonical first if you want the simplest equivalent spelling.

An entity whose JSON is not its fields overrides this; none in the package does.

Source code in src/zarr_metadata/v3/_entity.py

def to_json(self) -> ZarrV3MetadataFieldJSON:
    """This entity as a document would write it.

    Written from the configuration record by the same declaration
    `coerce` reads it by, each member by the writer its annotation
    implies: the bare name when every member it holds is absent, the
    object otherwise, a contained entity through its own `to_json`,
    a JSON-valued member copied so the document is not a handle on
    the entity. Faithful to every member: read a document, write it
    back, and those come out as they went in. The envelope is
    written the entity's way -- the bare name when nothing is
    configured, the object otherwise, no `must_understand`, which
    means what absence means -- because an entity alone has no
    document to be faithful to; `ArrayDocumentV3.to_json` puts back
    the spelling the document used. Ask `canonical` first if you
    want the simplest equivalent spelling.

    An entity whose JSON is not its fields overrides this; none in
    the package does.
    """
    plan = _plan(type(self))
    name = self.identifier
    if plan.from_name is not None:
        carried = getattr(self, plan.from_name)
        name = carried if isinstance(carried, str) else name
    configuration = plan.write(self)
    if len(configuration) == 0:
        return name
    return {"name": name, "configuration": configuration}

with_configuration

with_configuration(**changes: object) -> Self

This entity with these configuration members changed.

codec.with_configuration(typesize=UNSET) is the record rebuilt through its constructor, which refuses a member of the wrong type, and the entity rebuilt through its own, which refuses a value the rules disallow -- the same checks as any construction, since pyright cannot see the members through **changes. A name that is not a member is refused the way replace refuses it.

Source code in src/zarr_metadata/v3/_entity.py

def with_configuration(self, **changes: object) -> Self:
    """This entity with these configuration members changed.

    `codec.with_configuration(typesize=UNSET)` is the record rebuilt
    through its constructor, which refuses a member of the wrong
    type, and the entity rebuilt through its own, which refuses a
    value the rules disallow -- the same checks as any construction,
    since pyright cannot see the members through `**changes`. A
    name that is not a member is refused the way `replace` refuses
    it.
    """
    return replace(self, configuration=replace(self.configuration, **changes))

BloscCodecConfiguration

Bases: TypedDict

Configuration for the Zarr v3 blosc codec.

Source code in src/zarr_metadata/v3/codec/blosc.py

class BloscCodecConfiguration(TypedDict, closed=True):
    """Configuration for the Zarr v3 `blosc` codec."""

    cname: BloscCName
    clevel: int
    shuffle: BloscShuffle
    blocksize: int
    typesize: NotRequired[int]

blocksize instance-attribute

blocksize: int

clevel instance-attribute

clevel: int

cname instance-attribute

cname: BloscCName

shuffle instance-attribute

shuffle: BloscShuffle

typesize instance-attribute

typesize: NotRequired[int]

BloscCodecObject

Bases: TypedDict

blosc codec metadata in object form.

Source code in src/zarr_metadata/v3/codec/blosc.py

class BloscCodecObject(TypedDict, closed=True):
    """`blosc` codec metadata in object form."""

    name: BloscCodecName
    configuration: BloscCodecConfiguration
    must_understand: NotRequired[bool]

configuration instance-attribute

configuration: BloscCodecConfiguration

must_understand instance-attribute

must_understand: NotRequired[bool]

name instance-attribute

name: BloscCodecName

BloscOptions dataclass

Bases: Configuration

What blosc is configured with.

Source code in src/zarr_metadata/v3/codec/blosc.py

@dataclass(frozen=True)
class BloscOptions(Configuration):
    """What `blosc` is configured with."""

    cname: BloscCName
    clevel: int
    shuffle: BloscShuffle
    blocksize: int
    typesize: int | UNSET = UNSET

    def problems(self) -> "Iterator[ValidationProblem]":
        """Bounds on `clevel` and `blocksize`; `typesize` against `shuffle`.

        Under `noshuffle` the spec says of `typesize` that "the value is
        ignored", and `canonical` drops it; under either shuffle it is
        required, and positive.
        """
        if not 0 <= self.clevel <= 9:
            yield ValidationProblem(
                ("clevel",), f"expected an integer in [0, 9], got {self.clevel}", "invalid_value"
            )
        if self.blocksize < 0:
            yield ValidationProblem(
                ("blocksize",), f"expected an integer >= 0, got {self.blocksize}", "invalid_value"
            )
        if self.shuffle != BLOSC_NO_SHUFFLE:
            if self.typesize is UNSET:
                yield ValidationProblem(
                    ("typesize",),
                    f"typesize is required when shuffle is {self.shuffle!r}",
                    "missing_key",
                )
            elif self.typesize < 1:
                yield ValidationProblem(
                    ("typesize",),
                    f"expected a positive integer, got {self.typesize}",
                    "invalid_value",
                )

blocksize instance-attribute

blocksize: int

clevel instance-attribute

clevel: int

cname instance-attribute

cname: BloscCName

shuffle instance-attribute

shuffle: BloscShuffle

typesize class-attribute instance-attribute

typesize: int | UNSET = UNSET

__init__

__init__(
    cname: BloscCName,
    clevel: int,
    shuffle: BloscShuffle,
    blocksize: int,
    typesize: int | UNSET = UNSET,
) -> None

__post_init__

__post_init__() -> None

Refuse every member of the wrong type, so GzipOptions(level="high") raises.

Source code in src/zarr_metadata/v3/_entity.py

def __post_init__(self) -> None:
    """Refuse every member of the wrong type, so `GzipOptions(level="high")` raises."""
    found = mistyped(self)
    if len(found) != 0:
        raise MetadataValidationError(found)

create_unchecked classmethod

create_unchecked(**members: object) -> Self

This record with these members, built without the constructor's check.

The one way around the check, for a caller that has just made it: the parser, which type-checked every member against the same annotations before building the record. Every field is given -- the parser gives an absent optional member as UNSET -- since nothing here applies a default. Anything that has not checked the members goes through the constructor.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def create_unchecked(cls, **members: object) -> Self:
    """This record with these members, built without the constructor's check.

    The one way around the check, for a caller that has just made
    it: the parser, which type-checked every member against the same
    annotations before building the record. Every field is given --
    the parser gives an absent optional member as `UNSET` -- since
    nothing here applies a default. Anything that has not checked
    the members goes through the constructor.
    """
    record = object.__new__(cls)
    for name, value in members.items():
        object.__setattr__(record, name, value)
    return record

problems

problems() -> Iterator[ValidationProblem]

Bounds on clevel and blocksize; typesize against shuffle.

Under noshuffle the spec says of typesize that "the value is ignored", and canonical drops it; under either shuffle it is required, and positive.

Source code in src/zarr_metadata/v3/codec/blosc.py

def problems(self) -> "Iterator[ValidationProblem]":
    """Bounds on `clevel` and `blocksize`; `typesize` against `shuffle`.

    Under `noshuffle` the spec says of `typesize` that "the value is
    ignored", and `canonical` drops it; under either shuffle it is
    required, and positive.
    """
    if not 0 <= self.clevel <= 9:
        yield ValidationProblem(
            ("clevel",), f"expected an integer in [0, 9], got {self.clevel}", "invalid_value"
        )
    if self.blocksize < 0:
        yield ValidationProblem(
            ("blocksize",), f"expected an integer >= 0, got {self.blocksize}", "invalid_value"
        )
    if self.shuffle != BLOSC_NO_SHUFFLE:
        if self.typesize is UNSET:
            yield ValidationProblem(
                ("typesize",),
                f"typesize is required when shuffle is {self.shuffle!r}",
                "missing_key",
            )
        elif self.typesize < 1:
            yield ValidationProblem(
                ("typesize",),
                f"expected a positive integer, got {self.typesize}",
                "invalid_value",
            )

zarr_metadata.v3.codec.bytes

Bytes codec types.

See https://zarr-specs.readthedocs.io/en/latest/v3/codecs/bytes/index.html

BYTES_CODEC_NAME module-attribute

BYTES_CODEC_NAME: Final = 'bytes'

The name field value of the bytes codec.

BytesCodecMetadata module-attribute

BytesCodecMetadata = BytesCodecObject | BytesCodecName

Permitted JSON shapes for bytes codec metadata.

The configuration has no required keys (endian is conditionally required at runtime based on data type), so the spec's short-hand-name form is permitted in addition to the object form, and the object form may itself omit configuration entirely. https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/codecs/bytes/index.rst#L64-L69 ("endian: Required for data types for which endianness is applicable") https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L1562-L1564

BytesCodecName module-attribute

BytesCodecName = Literal['bytes']

Literal type of the name field of the bytes codec.

ENDIANNESS module-attribute

ENDIANNESS: Final = ('little', 'big')

Tuple of permitted values for the endian field of the bytes codec.

Endianness module-attribute

Endianness = Literal['little', 'big']

Literal type of byte order of multi-byte numeric data.

__all__ module-attribute

__all__ = [
    "BYTES_CODEC_NAME",
    "ENDIANNESS",
    "BytesCodec",
    "BytesCodecConfiguration",
    "BytesCodecMetadata",
    "BytesCodecName",
    "BytesCodecObject",
    "BytesOptions",
    "Endianness",
]

BytesCodec dataclass

Bases: ArrayBytesCodec

The bytes codec, coerced from its metadata.

endian is optional and absent means something: a one-byte data type has no byte order to state, and the spec lets such an array omit it.

Source code in src/zarr_metadata/v3/codec/bytes.py

@dataclass(frozen=True)
class BytesCodec(ArrayBytesCodec):
    """The `bytes` codec, coerced from its metadata.

    `endian` is optional and absent means something: a one-byte data type
    has no byte order to state, and the spec lets such an array omit it.
    """

    configuration: BytesOptions

    identifier: ClassVar[str] = BYTES_CODEC_NAME
    variable_size: ClassVar[bool] = False

    def incoming_problems(self, incoming: ArrayParts | None) -> tuple[ValidationProblem, ...]:
        """The data type reaching here must have a raw byte representation.

        A variable-length type has no fixed one, so this codec cannot
        encode it. A multi-byte one has several orderings, so `endian` is
        required -- and the message names the type, because inside a
        shard's `index_codecs` the array is the shard index, whose
        `uint64` type appears nowhere in the document.
        """
        data_type = incoming.data_type if incoming is not None else None
        if not isinstance(data_type, DataTypeEntity):
            return ()
        storage = data_type.storage_class()
        name = type(data_type).identifier
        if storage == "variable_length":
            return problem(
                (),
                f"bytes codec is not compatible with variable-length data_type {name!r}",
                "invalid_value",
            )
        if storage == "multi_byte" and self.configuration.endian is UNSET:
            return problem(
                ("endian",),
                f"endian is required for data type {name!r}, which contains multi-byte values",
                "missing_key",
            )
        return ()

configuration instance-attribute

configuration: BytesOptions

The record of this entity's members.

An entity with members narrows it to its own record, configuration: GzipOptions, its one positional argument. An entity of a bare name defaults it to the empty record -- configuration: Configuration = field(default_factory=Configuration) -- so that Crc32cCodec() builds; coerce passes the record either way.

identifier class-attribute

identifier: str = BYTES_CODEC_NAME

The name this entity is registered under.

Usually the name the metadata carries. The raw-bytes data types are the exception: every r<N> spelling is one family, so the family gets an invented identifier that no real name can collide with.

variable_size class-attribute

variable_size: bool = False

Whether this codec's output size depends on the bytes it is given.

A compressor's does, so a shard index encoded with one has no size derivable from metadata alone, and the shard cannot be read. Every codec says, because a default in either direction is a verdict.

__init__

__init__(configuration: BytesOptions) -> None

__post_init__

__post_init__() -> None

Refuse a record that is not this entity's own, then the first problem the rules find.

The runtime half of the entity's type, as the record's constructor is of the record's: GzipCodec(BloscOptions(...)) and a family member carrying a name that is not a string are refused before any rule reads them. Then BloscCodec(BloscOptions(clevel=99)) raises on the first problem the rules yield.

Source code in src/zarr_metadata/v3/_entity.py

def __post_init__(self) -> None:
    """Refuse a record that is not this entity's own, then the first problem the rules find.

    The runtime half of the entity's type, as the record's constructor
    is of the record's: `GzipCodec(BloscOptions(...))` and a family
    member carrying a name that is not a string are refused before
    any rule reads them. Then `BloscCodec(BloscOptions(clevel=99))`
    raises on the first problem the rules yield.
    """
    plan = _plan(type(self))
    if not isinstance(self.configuration, plan.record):
        raise MetadataValidationError(
            problem(
                (),
                f"expected a {plan.record.__name__} configuration, got "
                f"{type(self.configuration).__name__}",
            )
        )
    name: object = self.identifier if plan.from_name is None else getattr(self, plan.from_name)
    if not isinstance(name, str):
        raise MetadataValidationError(problem((), f"expected a string name, got {name!r}"))
    first = next(type(self).name_problems(name), None)
    if first is None:
        first = next(self.configuration.problems(), None)
    if first is not None:
        raise MetadataValidationError((first,))

accepts classmethod

accepts(name: str) -> bool

Whether name denotes this entity.

Constant for all but the raw-bytes family, where one class covers every r<N>.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def accepts(cls, name: str) -> bool:
    """Whether `name` denotes this entity.

    Constant for all but the raw-bytes family, where one class covers
    every `r<N>`.
    """
    return name == cls.identifier

canonical

canonical() -> Self

This entity in the simplest form that means the same thing.

A transformation, asked for by canonicalize_array_metadata_v3 and by nothing else. to_json does not apply it, because writing a document back is not the same as asking for it to be rewritten: a reader that reads and writes should not change bytes it was not asked to change.

The default is the entity itself. Override it where two spellings of the entity's members mean the same -- a rectilinear dimension's run-length encoding, a typesize that noshuffle ignores -- and, in an entity that contains entities, to put those in canonical form: self.with_configuration(inner=self.inner.canonical()).

Source code in src/zarr_metadata/v3/_entity.py

def canonical(self) -> Self:
    """This entity in the simplest form that means the same thing.

    A *transformation*, asked for by `canonicalize_array_metadata_v3`
    and by nothing else. `to_json` does not apply it, because writing
    a document back is not the same as asking for it to be rewritten:
    a reader that reads and writes should not change bytes it was not
    asked to change.

    The default is the entity itself. Override it where two spellings
    of the entity's members mean the same -- a rectilinear
    dimension's run-length encoding, a `typesize` that `noshuffle`
    ignores -- and, in an entity that contains entities, to put those
    in canonical form: `self.with_configuration(inner=self.inner.canonical())`.
    """
    return self

coerce classmethod

coerce(value: JSONValue, context: Context) -> Coerced[Self]

value as this entity, or the reasons it is not one: the class's validation routine.

resolve relates a field's name to this class and hands it the field, refined JSON with arrays as tuples, which is what value is; this is what the class does with it. The configuration is parsed against the record the configuration field names, member by member; a member holding another entity is read in context, the scope this reading is happening in. An optional member the document left out is UNSET in the record, so no field's default decides what a document said. The entity is built only when every member of its own read -- its rules are written over a whole configuration -- and handed back only when everything inside it read too.

The envelope is the field's, not the class's, and resolve judges it: a stray member or a must_understand of false is not reported here. Called on a class no scope has registered, this runs with none of registration's refusals having happened.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def coerce(cls, value: JSONValue, context: Context) -> Coerced[Self]:
    """`value` as this entity, or the reasons it is not one: the class's validation routine.

    `resolve` relates a field's name to this class and hands it the
    field, refined JSON with arrays as tuples, which is what `value`
    is; this is what the class does with it. The configuration is
    parsed against the record the `configuration` field names,
    member by member; a member holding another entity is read in
    `context`, the scope this reading is happening in. An optional
    member the document left out is `UNSET` in the record, so no
    field's default decides what a document said. The entity is
    built only when every member of its own read -- its rules are
    written over a whole configuration -- and handed back only when
    everything inside it read too.

    The envelope is the field's, not the class's, and `resolve`
    judges it: a stray member or a `must_understand` of `false` is
    not reported here. Called on a class no scope has registered,
    this runs with none of registration's refusals having happened.
    """
    name, given, envelope = named_configuration(value)
    if name is None or not cls.accepts(name):
        return None, problem((), f"expected the {cls.identifier!r} entity")
    if len(envelope) != 0:
        return None, envelope
    plan = _plan(cls)
    carried: dict[str, str] = {} if plan.from_name is None else {plan.from_name: name}
    if given is None and plan.requires_configuration:
        return None, problem(
            ("configuration",),
            f"{cls.identifier!r} requires a configuration",
            "missing_key",
        )
    reading = _Reading(context, [])
    # A bare name's record has no members, so any key is an unknown one.
    record, own = plan.read({} if given is None else given, ("configuration",), reading)
    found = (*own, *reading.nested)
    if record is None:
        # An unknown key is survivable; a member that could not be
        # read is a hole, and judging around it would be guessing.
        return None, found
    # The rules, asked of the name and of the record before anything
    # is built: a name problem lands on the entity, a configuration
    # problem under the configuration.
    refused = (*cls.name_problems(name), *within((), tuple(record.problems())))
    if len(refused) != 0:
        # Values the spec disallows: reported rather than raised,
        # every one.
        return None, (*found, *refused)
    if any(entry.kind != "unknown_key" for entry in reading.nested):
        # A contained entity could not be read. This entity's own
        # rules ran -- an invalid inner is an `Opaque`, as an
        # out-of-scope one is -- but what is handed back is not an
        # entity that would be asked composition questions it cannot
        # answer.
        return None, found
    return cls.create_unchecked(configuration=record, **carried), found

create_unchecked classmethod

create_unchecked(**fields: object) -> Self

This entity with these fields, built without the constructor's checks.

The one way around them, for a caller that has just made them: coerce, which type-checked the record and ran the rules before building. Every field is given -- the record, and the carried name for a family -- since nothing here applies a default. Anything that has not checked goes through the constructor.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def create_unchecked(cls, **fields: object) -> Self:
    """This entity with these fields, built without the constructor's checks.

    The one way around them, for a caller that has just made them:
    `coerce`, which type-checked the record and ran the rules before
    building. Every field is given -- the record, and the carried
    name for a family -- since nothing here applies a default.
    Anything that has not checked goes through the constructor.
    """
    entity = object.__new__(cls)
    for name, value in fields.items():
        object.__setattr__(entity, name, value)
    return entity

incoming_problems

incoming_problems(
    incoming: ArrayParts | None,
) -> tuple[ValidationProblem, ...]

The data type reaching here must have a raw byte representation.

A variable-length type has no fixed one, so this codec cannot encode it. A multi-byte one has several orderings, so endian is required -- and the message names the type, because inside a shard's index_codecs the array is the shard index, whose uint64 type appears nowhere in the document.

Source code in src/zarr_metadata/v3/codec/bytes.py

def incoming_problems(self, incoming: ArrayParts | None) -> tuple[ValidationProblem, ...]:
    """The data type reaching here must have a raw byte representation.

    A variable-length type has no fixed one, so this codec cannot
    encode it. A multi-byte one has several orderings, so `endian` is
    required -- and the message names the type, because inside a
    shard's `index_codecs` the array is the shard index, whose
    `uint64` type appears nowhere in the document.
    """
    data_type = incoming.data_type if incoming is not None else None
    if not isinstance(data_type, DataTypeEntity):
        return ()
    storage = data_type.storage_class()
    name = type(data_type).identifier
    if storage == "variable_length":
        return problem(
            (),
            f"bytes codec is not compatible with variable-length data_type {name!r}",
            "invalid_value",
        )
    if storage == "multi_byte" and self.configuration.endian is UNSET:
        return problem(
            ("endian",),
            f"endian is required for data type {name!r}, which contains multi-byte values",
            "missing_key",
        )
    return ()

inner_pipelines

inner_pipelines(
    incoming: ArrayParts | None,
) -> Mapping[
    str,
    tuple[
        Sequence[CodecEntity | Opaque], ArrayParts | None
    ],
]

The pipelines this codec holds, by the member holding each, with what each is handed.

A shard holds two: its codecs, handed its inner chunk, and its index_codecs, handed the shard index. Refinement walks them as it walks the pipeline this codec stands in, locating what it finds under the member, so a codec that holds pipelines says which and what they receive, and judges nothing inside them itself. Default: none.

Source code in src/zarr_metadata/v3/_entity.py

def inner_pipelines(
    self, incoming: ArrayParts | None
) -> Mapping[str, tuple[Sequence[CodecEntity | Opaque], ArrayParts | None]]:
    """The pipelines this codec holds, by the member holding each, with what each is handed.

    A shard holds two: its `codecs`, handed its inner chunk, and its
    `index_codecs`, handed the shard index. Refinement walks them as
    it walks the pipeline this codec stands in, locating what it
    finds under the member, so a codec that holds pipelines says
    which and what they receive, and judges nothing inside them
    itself. Default: none.
    """
    return {}

name_problems classmethod

name_problems(name: str) -> Iterator[ValidationProblem]

Why name, which accepts claimed, is not a well-formed name of this family.

For a family, whose names carry data -- r<N> -- and which claims a malformed member so that it is reported rather than waved through as an unknown extension. Locations are relative to the entity: (). Default: none, for an entity of one name.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def name_problems(cls, name: str) -> Iterator[ValidationProblem]:
    """Why `name`, which `accepts` claimed, is not a well-formed name of this family.

    For a family, whose names carry data -- `r<N>` -- and which claims
    a malformed member so that it is reported rather than waved
    through as an unknown extension. Locations are relative to the
    entity: `()`. Default: none, for an entity of one name.
    """
    yield from ()

to_json

to_json() -> ZarrV3MetadataFieldJSON

This entity as a document would write it.

Written from the configuration record by the same declaration coerce reads it by, each member by the writer its annotation implies: the bare name when every member it holds is absent, the object otherwise, a contained entity through its own to_json, a JSON-valued member copied so the document is not a handle on the entity. Faithful to every member: read a document, write it back, and those come out as they went in. The envelope is written the entity's way -- the bare name when nothing is configured, the object otherwise, no must_understand, which means what absence means -- because an entity alone has no document to be faithful to; ArrayDocumentV3.to_json puts back the spelling the document used. Ask canonical first if you want the simplest equivalent spelling.

An entity whose JSON is not its fields overrides this; none in the package does.

Source code in src/zarr_metadata/v3/_entity.py

def to_json(self) -> ZarrV3MetadataFieldJSON:
    """This entity as a document would write it.

    Written from the configuration record by the same declaration
    `coerce` reads it by, each member by the writer its annotation
    implies: the bare name when every member it holds is absent, the
    object otherwise, a contained entity through its own `to_json`,
    a JSON-valued member copied so the document is not a handle on
    the entity. Faithful to every member: read a document, write it
    back, and those come out as they went in. The envelope is
    written the entity's way -- the bare name when nothing is
    configured, the object otherwise, no `must_understand`, which
    means what absence means -- because an entity alone has no
    document to be faithful to; `ArrayDocumentV3.to_json` puts back
    the spelling the document used. Ask `canonical` first if you
    want the simplest equivalent spelling.

    An entity whose JSON is not its fields overrides this; none in
    the package does.
    """
    plan = _plan(type(self))
    name = self.identifier
    if plan.from_name is not None:
        carried = getattr(self, plan.from_name)
        name = carried if isinstance(carried, str) else name
    configuration = plan.write(self)
    if len(configuration) == 0:
        return name
    return {"name": name, "configuration": configuration}

with_configuration

with_configuration(**changes: object) -> Self

This entity with these configuration members changed.

codec.with_configuration(typesize=UNSET) is the record rebuilt through its constructor, which refuses a member of the wrong type, and the entity rebuilt through its own, which refuses a value the rules disallow -- the same checks as any construction, since pyright cannot see the members through **changes. A name that is not a member is refused the way replace refuses it.

Source code in src/zarr_metadata/v3/_entity.py

def with_configuration(self, **changes: object) -> Self:
    """This entity with these configuration members changed.

    `codec.with_configuration(typesize=UNSET)` is the record rebuilt
    through its constructor, which refuses a member of the wrong
    type, and the entity rebuilt through its own, which refuses a
    value the rules disallow -- the same checks as any construction,
    since pyright cannot see the members through `**changes`. A
    name that is not a member is refused the way `replace` refuses
    it.
    """
    return replace(self, configuration=replace(self.configuration, **changes))

BytesCodecConfiguration

Bases: TypedDict

Configuration for the Zarr v3 bytes codec.

The endian field is required for multi-byte data types.

Source code in src/zarr_metadata/v3/codec/bytes.py

class BytesCodecConfiguration(TypedDict, closed=True):
    """
    Configuration for the Zarr v3 `bytes` codec.

    The `endian` field is required for multi-byte data types.
    """

    endian: NotRequired[Endianness]

endian instance-attribute

endian: NotRequired[Endianness]

BytesCodecObject

Bases: TypedDict

bytes codec metadata in object form.

configuration is itself optional — when no configuration fields are set, the entire configuration key may be omitted. This matches the bare-string short-hand form (BytesCodecName) at the canonical data level; both encodings describe a bytes codec with default settings.

Source code in src/zarr_metadata/v3/codec/bytes.py

class BytesCodecObject(TypedDict, closed=True):
    """`bytes` codec metadata in object form.

    `configuration` is itself optional — when no configuration fields are
    set, the entire `configuration` key may be omitted. This matches the
    bare-string short-hand form (`BytesCodecName`) at the canonical data
    level; both encodings describe a `bytes` codec with default settings.
    """

    name: BytesCodecName
    configuration: NotRequired[BytesCodecConfiguration]
    must_understand: NotRequired[bool]

configuration instance-attribute

configuration: NotRequired[BytesCodecConfiguration]

must_understand instance-attribute

must_understand: NotRequired[bool]

name instance-attribute

name: BytesCodecName

BytesOptions dataclass

Bases: Configuration

What bytes is configured with.

Source code in src/zarr_metadata/v3/codec/bytes.py

@dataclass(frozen=True)
class BytesOptions(Configuration):
    """What `bytes` is configured with."""

    endian: Endianness | UNSET = UNSET

endian class-attribute instance-attribute

endian: Endianness | UNSET = UNSET

__init__

__init__(endian: Endianness | UNSET = UNSET) -> None

__post_init__

__post_init__() -> None

Refuse every member of the wrong type, so GzipOptions(level="high") raises.

Source code in src/zarr_metadata/v3/_entity.py

def __post_init__(self) -> None:
    """Refuse every member of the wrong type, so `GzipOptions(level="high")` raises."""
    found = mistyped(self)
    if len(found) != 0:
        raise MetadataValidationError(found)

create_unchecked classmethod

create_unchecked(**members: object) -> Self

This record with these members, built without the constructor's check.

The one way around the check, for a caller that has just made it: the parser, which type-checked every member against the same annotations before building the record. Every field is given -- the parser gives an absent optional member as UNSET -- since nothing here applies a default. Anything that has not checked the members goes through the constructor.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def create_unchecked(cls, **members: object) -> Self:
    """This record with these members, built without the constructor's check.

    The one way around the check, for a caller that has just made
    it: the parser, which type-checked every member against the same
    annotations before building the record. Every field is given --
    the parser gives an absent optional member as `UNSET` -- since
    nothing here applies a default. Anything that has not checked
    the members goes through the constructor.
    """
    record = object.__new__(cls)
    for name, value in members.items():
        object.__setattr__(record, name, value)
    return record

problems

problems() -> Iterator[ValidationProblem]

Every reason these values are not allowed, yielded as found. Default: none.

Source code in src/zarr_metadata/v3/_entity.py

def problems(self) -> Iterator[ValidationProblem]:
    """Every reason these values are not allowed, yielded as found. Default: none."""
    yield from ()

zarr_metadata.v3.codec.cast_value

Cast-value codec types.

See https://github.com/zarr-developers/zarr-extensions/blob/4da7b37a84f76e660902f6d3de3eaef0e0febae6/codecs/cast_value/README.md

CAST_OUT_OF_RANGE_MODE module-attribute

CAST_OUT_OF_RANGE_MODE: Final = ('clamp', 'wrap')

Tuple of permitted values for the out_of_range field of the cast_value codec.

CAST_ROUNDING_MODE module-attribute

CAST_ROUNDING_MODE: Final = (
    "nearest-even",
    "towards-zero",
    "towards-positive",
    "towards-negative",
    "nearest-away",
)

Tuple of permitted values for the rounding field of the cast_value codec.

CAST_VALUE_CODEC_NAME module-attribute

CAST_VALUE_CODEC_NAME: Final = 'cast_value'

The name field value of the cast_value codec.

CastOutOfRangeMode module-attribute

CastOutOfRangeMode = Literal['clamp', 'wrap']

Literal type of permitted values for the out_of_range configuration field.

If absent, out-of-range values are an encoding/decoding error.

CastRoundingMode module-attribute

CastRoundingMode = Literal[
    "nearest-even",
    "towards-zero",
    "towards-positive",
    "towards-negative",
    "nearest-away",
]

Literal type of permitted values for the rounding configuration field.

Defaults to "nearest-even" if absent.

CastValueCodecMetadata module-attribute

CastValueCodecMetadata = CastValueCodecObject

Permitted JSON shape for cast_value codec metadata.

configuration.data_type is required, so only the object form is valid; the short-hand-name form is not permitted by the spec for this codec. https://github.com/zarr-developers/zarr-extensions/blob/4da7b37a84f76e660902f6d3de3eaef0e0febae6/codecs/cast_value/README.md#L33-L36 and #L46-L48 (required fields) https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L1562-L1564 (short-hand names only "if no configuration metadata is required")

CastValueCodecName module-attribute

CastValueCodecName = Literal['cast_value']

Literal type of the name field of the cast_value codec.

SCALAR_MAP_KEYS module-attribute

SCALAR_MAP_KEYS: Final = ('encode', 'decode')

The two directions a scalar_map can override, both optional.

ScalarMapEntry module-attribute

ScalarMapEntry = tuple[JSONValue, JSONValue]

A single [input, output] mapping in a scalar_map direction.

Each scalar is JSON-encoded per its data type's fill-value rules (so e.g. "NaN" and "+Infinity" are permitted).

__all__ module-attribute

__all__ = [
    "CAST_OUT_OF_RANGE_MODE",
    "CAST_ROUNDING_MODE",
    "CAST_VALUE_CODEC_NAME",
    "SCALAR_MAP_KEYS",
    "CastOutOfRangeMode",
    "CastRoundingMode",
    "CastValueCodec",
    "CastValueCodecConfiguration",
    "CastValueCodecMetadata",
    "CastValueCodecName",
    "CastValueCodecObject",
    "CastValueOptions",
    "ScalarMap",
    "ScalarMapEntry",
]

CastValueCodec dataclass

Bases: ArrayArrayCodec

The cast_value codec, coerced from its metadata.

Holds the data type it casts to, so like sharding_indexed it is read in a scope rather than on its own.

Source code in src/zarr_metadata/v3/codec/cast_value.py

@dataclass(frozen=True)
class CastValueCodec(ArrayArrayCodec):
    """The `cast_value` codec, coerced from its metadata.

    Holds the data type it casts to, so like `sharding_indexed` it is
    read in a scope rather than on its own.
    """

    configuration: CastValueOptions

    identifier: ClassVar[str] = CAST_VALUE_CODEC_NAME
    variable_size: ClassVar[bool] = False

    def canonical(self) -> Self:
        """The target data type in its own canonical form."""
        return self.with_configuration(data_type=self.configuration.data_type.canonical())

    def transition(self, incoming: ArrayParts) -> ArrayParts | None:
        """The same parts, holding the type this codec casts to."""
        data_type = self.configuration.data_type
        return incoming.with_data_type(data_type if isinstance(data_type, DataTypeEntity) else None)

configuration instance-attribute

configuration: CastValueOptions

The record of this entity's members.

An entity with members narrows it to its own record, configuration: GzipOptions, its one positional argument. An entity of a bare name defaults it to the empty record -- configuration: Configuration = field(default_factory=Configuration) -- so that Crc32cCodec() builds; coerce passes the record either way.

identifier class-attribute

identifier: str = CAST_VALUE_CODEC_NAME

The name this entity is registered under.

Usually the name the metadata carries. The raw-bytes data types are the exception: every r<N> spelling is one family, so the family gets an invented identifier that no real name can collide with.

variable_size class-attribute

variable_size: bool = False

Whether this codec's output size depends on the bytes it is given.

A compressor's does, so a shard index encoded with one has no size derivable from metadata alone, and the shard cannot be read. Every codec says, because a default in either direction is a verdict.

__init__

__init__(configuration: CastValueOptions) -> None

__post_init__

__post_init__() -> None

Refuse a record that is not this entity's own, then the first problem the rules find.

The runtime half of the entity's type, as the record's constructor is of the record's: GzipCodec(BloscOptions(...)) and a family member carrying a name that is not a string are refused before any rule reads them. Then BloscCodec(BloscOptions(clevel=99)) raises on the first problem the rules yield.

Source code in src/zarr_metadata/v3/_entity.py

def __post_init__(self) -> None:
    """Refuse a record that is not this entity's own, then the first problem the rules find.

    The runtime half of the entity's type, as the record's constructor
    is of the record's: `GzipCodec(BloscOptions(...))` and a family
    member carrying a name that is not a string are refused before
    any rule reads them. Then `BloscCodec(BloscOptions(clevel=99))`
    raises on the first problem the rules yield.
    """
    plan = _plan(type(self))
    if not isinstance(self.configuration, plan.record):
        raise MetadataValidationError(
            problem(
                (),
                f"expected a {plan.record.__name__} configuration, got "
                f"{type(self.configuration).__name__}",
            )
        )
    name: object = self.identifier if plan.from_name is None else getattr(self, plan.from_name)
    if not isinstance(name, str):
        raise MetadataValidationError(problem((), f"expected a string name, got {name!r}"))
    first = next(type(self).name_problems(name), None)
    if first is None:
        first = next(self.configuration.problems(), None)
    if first is not None:
        raise MetadataValidationError((first,))

accepts classmethod

accepts(name: str) -> bool

Whether name denotes this entity.

Constant for all but the raw-bytes family, where one class covers every r<N>.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def accepts(cls, name: str) -> bool:
    """Whether `name` denotes this entity.

    Constant for all but the raw-bytes family, where one class covers
    every `r<N>`.
    """
    return name == cls.identifier

canonical

canonical() -> Self

The target data type in its own canonical form.

Source code in src/zarr_metadata/v3/codec/cast_value.py

def canonical(self) -> Self:
    """The target data type in its own canonical form."""
    return self.with_configuration(data_type=self.configuration.data_type.canonical())

coerce classmethod

coerce(value: JSONValue, context: Context) -> Coerced[Self]

value as this entity, or the reasons it is not one: the class's validation routine.

resolve relates a field's name to this class and hands it the field, refined JSON with arrays as tuples, which is what value is; this is what the class does with it. The configuration is parsed against the record the configuration field names, member by member; a member holding another entity is read in context, the scope this reading is happening in. An optional member the document left out is UNSET in the record, so no field's default decides what a document said. The entity is built only when every member of its own read -- its rules are written over a whole configuration -- and handed back only when everything inside it read too.

The envelope is the field's, not the class's, and resolve judges it: a stray member or a must_understand of false is not reported here. Called on a class no scope has registered, this runs with none of registration's refusals having happened.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def coerce(cls, value: JSONValue, context: Context) -> Coerced[Self]:
    """`value` as this entity, or the reasons it is not one: the class's validation routine.

    `resolve` relates a field's name to this class and hands it the
    field, refined JSON with arrays as tuples, which is what `value`
    is; this is what the class does with it. The configuration is
    parsed against the record the `configuration` field names,
    member by member; a member holding another entity is read in
    `context`, the scope this reading is happening in. An optional
    member the document left out is `UNSET` in the record, so no
    field's default decides what a document said. The entity is
    built only when every member of its own read -- its rules are
    written over a whole configuration -- and handed back only when
    everything inside it read too.

    The envelope is the field's, not the class's, and `resolve`
    judges it: a stray member or a `must_understand` of `false` is
    not reported here. Called on a class no scope has registered,
    this runs with none of registration's refusals having happened.
    """
    name, given, envelope = named_configuration(value)
    if name is None or not cls.accepts(name):
        return None, problem((), f"expected the {cls.identifier!r} entity")
    if len(envelope) != 0:
        return None, envelope
    plan = _plan(cls)
    carried: dict[str, str] = {} if plan.from_name is None else {plan.from_name: name}
    if given is None and plan.requires_configuration:
        return None, problem(
            ("configuration",),
            f"{cls.identifier!r} requires a configuration",
            "missing_key",
        )
    reading = _Reading(context, [])
    # A bare name's record has no members, so any key is an unknown one.
    record, own = plan.read({} if given is None else given, ("configuration",), reading)
    found = (*own, *reading.nested)
    if record is None:
        # An unknown key is survivable; a member that could not be
        # read is a hole, and judging around it would be guessing.
        return None, found
    # The rules, asked of the name and of the record before anything
    # is built: a name problem lands on the entity, a configuration
    # problem under the configuration.
    refused = (*cls.name_problems(name), *within((), tuple(record.problems())))
    if len(refused) != 0:
        # Values the spec disallows: reported rather than raised,
        # every one.
        return None, (*found, *refused)
    if any(entry.kind != "unknown_key" for entry in reading.nested):
        # A contained entity could not be read. This entity's own
        # rules ran -- an invalid inner is an `Opaque`, as an
        # out-of-scope one is -- but what is handed back is not an
        # entity that would be asked composition questions it cannot
        # answer.
        return None, found
    return cls.create_unchecked(configuration=record, **carried), found

create_unchecked classmethod

create_unchecked(**fields: object) -> Self

This entity with these fields, built without the constructor's checks.

The one way around them, for a caller that has just made them: coerce, which type-checked the record and ran the rules before building. Every field is given -- the record, and the carried name for a family -- since nothing here applies a default. Anything that has not checked goes through the constructor.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def create_unchecked(cls, **fields: object) -> Self:
    """This entity with these fields, built without the constructor's checks.

    The one way around them, for a caller that has just made them:
    `coerce`, which type-checked the record and ran the rules before
    building. Every field is given -- the record, and the carried
    name for a family -- since nothing here applies a default.
    Anything that has not checked goes through the constructor.
    """
    entity = object.__new__(cls)
    for name, value in fields.items():
        object.__setattr__(entity, name, value)
    return entity

incoming_problems

incoming_problems(
    incoming: ArrayParts | None,
) -> tuple[ValidationProblem, ...]

Why this codec cannot be applied to the array that reaches it.

incoming is None once the chain can no longer say what reaches here, and the default answer to that is nothing: declining beats guessing. Locations are relative to this codec's configuration; an empty one lands on the codec itself.

Source code in src/zarr_metadata/v3/_entity.py

def incoming_problems(self, incoming: ArrayParts | None) -> tuple[ValidationProblem, ...]:
    """Why this codec cannot be applied to the array that reaches it.

    `incoming` is None once the chain can no longer say what reaches
    here, and the default answer to that is nothing: declining beats
    guessing. Locations are relative to this codec's `configuration`;
    an empty one lands on the codec itself.
    """
    return ()

inner_pipelines

inner_pipelines(
    incoming: ArrayParts | None,
) -> Mapping[
    str,
    tuple[
        Sequence[CodecEntity | Opaque], ArrayParts | None
    ],
]

The pipelines this codec holds, by the member holding each, with what each is handed.

A shard holds two: its codecs, handed its inner chunk, and its index_codecs, handed the shard index. Refinement walks them as it walks the pipeline this codec stands in, locating what it finds under the member, so a codec that holds pipelines says which and what they receive, and judges nothing inside them itself. Default: none.

Source code in src/zarr_metadata/v3/_entity.py

def inner_pipelines(
    self, incoming: ArrayParts | None
) -> Mapping[str, tuple[Sequence[CodecEntity | Opaque], ArrayParts | None]]:
    """The pipelines this codec holds, by the member holding each, with what each is handed.

    A shard holds two: its `codecs`, handed its inner chunk, and its
    `index_codecs`, handed the shard index. Refinement walks them as
    it walks the pipeline this codec stands in, locating what it
    finds under the member, so a codec that holds pipelines says
    which and what they receive, and judges nothing inside them
    itself. Default: none.
    """
    return {}

name_problems classmethod

name_problems(name: str) -> Iterator[ValidationProblem]

Why name, which accepts claimed, is not a well-formed name of this family.

For a family, whose names carry data -- r<N> -- and which claims a malformed member so that it is reported rather than waved through as an unknown extension. Locations are relative to the entity: (). Default: none, for an entity of one name.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def name_problems(cls, name: str) -> Iterator[ValidationProblem]:
    """Why `name`, which `accepts` claimed, is not a well-formed name of this family.

    For a family, whose names carry data -- `r<N>` -- and which claims
    a malformed member so that it is reported rather than waved
    through as an unknown extension. Locations are relative to the
    entity: `()`. Default: none, for an entity of one name.
    """
    yield from ()

to_json

to_json() -> ZarrV3MetadataFieldJSON

This entity as a document would write it.

Written from the configuration record by the same declaration coerce reads it by, each member by the writer its annotation implies: the bare name when every member it holds is absent, the object otherwise, a contained entity through its own to_json, a JSON-valued member copied so the document is not a handle on the entity. Faithful to every member: read a document, write it back, and those come out as they went in. The envelope is written the entity's way -- the bare name when nothing is configured, the object otherwise, no must_understand, which means what absence means -- because an entity alone has no document to be faithful to; ArrayDocumentV3.to_json puts back the spelling the document used. Ask canonical first if you want the simplest equivalent spelling.

An entity whose JSON is not its fields overrides this; none in the package does.

Source code in src/zarr_metadata/v3/_entity.py

def to_json(self) -> ZarrV3MetadataFieldJSON:
    """This entity as a document would write it.

    Written from the configuration record by the same declaration
    `coerce` reads it by, each member by the writer its annotation
    implies: the bare name when every member it holds is absent, the
    object otherwise, a contained entity through its own `to_json`,
    a JSON-valued member copied so the document is not a handle on
    the entity. Faithful to every member: read a document, write it
    back, and those come out as they went in. The envelope is
    written the entity's way -- the bare name when nothing is
    configured, the object otherwise, no `must_understand`, which
    means what absence means -- because an entity alone has no
    document to be faithful to; `ArrayDocumentV3.to_json` puts back
    the spelling the document used. Ask `canonical` first if you
    want the simplest equivalent spelling.

    An entity whose JSON is not its fields overrides this; none in
    the package does.
    """
    plan = _plan(type(self))
    name = self.identifier
    if plan.from_name is not None:
        carried = getattr(self, plan.from_name)
        name = carried if isinstance(carried, str) else name
    configuration = plan.write(self)
    if len(configuration) == 0:
        return name
    return {"name": name, "configuration": configuration}

transition

transition(incoming: ArrayParts) -> ArrayParts | None

The same parts, holding the type this codec casts to.

Source code in src/zarr_metadata/v3/codec/cast_value.py

def transition(self, incoming: ArrayParts) -> ArrayParts | None:
    """The same parts, holding the type this codec casts to."""
    data_type = self.configuration.data_type
    return incoming.with_data_type(data_type if isinstance(data_type, DataTypeEntity) else None)

with_configuration

with_configuration(**changes: object) -> Self

This entity with these configuration members changed.

codec.with_configuration(typesize=UNSET) is the record rebuilt through its constructor, which refuses a member of the wrong type, and the entity rebuilt through its own, which refuses a value the rules disallow -- the same checks as any construction, since pyright cannot see the members through **changes. A name that is not a member is refused the way replace refuses it.

Source code in src/zarr_metadata/v3/_entity.py

def with_configuration(self, **changes: object) -> Self:
    """This entity with these configuration members changed.

    `codec.with_configuration(typesize=UNSET)` is the record rebuilt
    through its constructor, which refuses a member of the wrong
    type, and the entity rebuilt through its own, which refuses a
    value the rules disallow -- the same checks as any construction,
    since pyright cannot see the members through `**changes`. A
    name that is not a member is refused the way `replace` refuses
    it.
    """
    return replace(self, configuration=replace(self.configuration, **changes))

CastValueCodecConfiguration

Bases: TypedDict

Configuration for the Zarr v3 cast_value codec.

data_type is the target data type that input values are cast to. It is the same shape as the top-level array data_type field: either a bare-string primitive name or a {name, configuration} envelope.

Source code in src/zarr_metadata/v3/codec/cast_value.py

class CastValueCodecConfiguration(TypedDict, closed=True):
    """
    Configuration for the Zarr v3 `cast_value` codec.

    `data_type` is the target data type that input values are cast to. It
    is the same shape as the top-level array `data_type` field: either a
    bare-string primitive name or a `{name, configuration}` envelope.
    """

    data_type: ZarrV3MetadataFieldJSON
    rounding: NotRequired[CastRoundingMode]
    out_of_range: NotRequired[CastOutOfRangeMode]
    scalar_map: NotRequired[ScalarMap]

data_type instance-attribute

data_type: ZarrV3MetadataFieldJSON

out_of_range instance-attribute

out_of_range: NotRequired[CastOutOfRangeMode]

rounding instance-attribute

rounding: NotRequired[CastRoundingMode]

scalar_map instance-attribute

scalar_map: NotRequired[ScalarMap]

CastValueCodecObject

Bases: TypedDict

cast_value codec metadata in object form.

Source code in src/zarr_metadata/v3/codec/cast_value.py

class CastValueCodecObject(TypedDict, closed=True):
    """`cast_value` codec metadata in object form."""

    name: CastValueCodecName
    configuration: CastValueCodecConfiguration
    must_understand: NotRequired[bool]

configuration instance-attribute

configuration: CastValueCodecConfiguration

must_understand instance-attribute

must_understand: NotRequired[bool]

name instance-attribute

name: CastValueCodecName

CastValueOptions dataclass

Bases: Configuration

What cast_value is configured with.

Source code in src/zarr_metadata/v3/codec/cast_value.py

@dataclass(frozen=True)
class CastValueOptions(Configuration):
    """What `cast_value` is configured with."""

    data_type: DataTypeEntity | Opaque
    rounding: CastRoundingMode | UNSET = UNSET
    out_of_range: CastOutOfRangeMode | UNSET = UNSET
    scalar_map: ScalarMap | UNSET = UNSET

data_type instance-attribute

data_type: DataTypeEntity | Opaque

out_of_range class-attribute instance-attribute

out_of_range: CastOutOfRangeMode | UNSET = UNSET

rounding class-attribute instance-attribute

rounding: CastRoundingMode | UNSET = UNSET

scalar_map class-attribute instance-attribute

scalar_map: ScalarMap | UNSET = UNSET

__init__

__init__(
    data_type: DataTypeEntity | Opaque,
    rounding: CastRoundingMode | UNSET = UNSET,
    out_of_range: CastOutOfRangeMode | UNSET = UNSET,
    scalar_map: ScalarMap | UNSET = UNSET,
) -> None

__post_init__

__post_init__() -> None

Refuse every member of the wrong type, so GzipOptions(level="high") raises.

Source code in src/zarr_metadata/v3/_entity.py

def __post_init__(self) -> None:
    """Refuse every member of the wrong type, so `GzipOptions(level="high")` raises."""
    found = mistyped(self)
    if len(found) != 0:
        raise MetadataValidationError(found)

create_unchecked classmethod

create_unchecked(**members: object) -> Self

This record with these members, built without the constructor's check.

The one way around the check, for a caller that has just made it: the parser, which type-checked every member against the same annotations before building the record. Every field is given -- the parser gives an absent optional member as UNSET -- since nothing here applies a default. Anything that has not checked the members goes through the constructor.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def create_unchecked(cls, **members: object) -> Self:
    """This record with these members, built without the constructor's check.

    The one way around the check, for a caller that has just made
    it: the parser, which type-checked every member against the same
    annotations before building the record. Every field is given --
    the parser gives an absent optional member as `UNSET` -- since
    nothing here applies a default. Anything that has not checked
    the members goes through the constructor.
    """
    record = object.__new__(cls)
    for name, value in members.items():
        object.__setattr__(record, name, value)
    return record

problems

problems() -> Iterator[ValidationProblem]

Every reason these values are not allowed, yielded as found. Default: none.

Source code in src/zarr_metadata/v3/_entity.py

def problems(self) -> Iterator[ValidationProblem]:
    """Every reason these values are not allowed, yielded as found. Default: none."""
    yield from ()

ScalarMap

Bases: TypedDict

Optional encode/decode scalar overrides for the cast_value codec.

Source code in src/zarr_metadata/v3/codec/cast_value.py

class ScalarMap(TypedDict, closed=True):
    """Optional encode/decode scalar overrides for the cast_value codec."""

    encode: NotRequired[tuple[ScalarMapEntry, ...]]
    decode: NotRequired[tuple[ScalarMapEntry, ...]]

decode instance-attribute

decode: NotRequired[tuple[ScalarMapEntry, ...]]

encode instance-attribute

encode: NotRequired[tuple[ScalarMapEntry, ...]]

zarr_metadata.v3.codec.crc32c

CRC32C codec types.

See https://zarr-specs.readthedocs.io/en/latest/v3/codecs/crc32c/index.html

The CRC32C codec has no configuration fields, so the configuration key is absent from the metadata.

CRC32C_CODEC_NAME module-attribute

CRC32C_CODEC_NAME: Final = 'crc32c'

The name field value of the crc32c codec.

Crc32cCodecMetadata module-attribute

Crc32cCodecMetadata = Crc32cCodecObject | Crc32cCodecName

Permitted JSON shapes for crc32c codec metadata.

The spec's Extension definition allows extensions with no required configuration to be encoded as a bare short-hand name. CRC32C has no configuration, so both forms are valid. https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L1562-L1564

Crc32cCodecName module-attribute

Crc32cCodecName = Literal['crc32c']

Literal type of the name field of the crc32c codec.

__all__ module-attribute

__all__ = [
    "CRC32C_CODEC_NAME",
    "Crc32cCodec",
    "Crc32cCodecMetadata",
    "Crc32cCodecName",
    "Crc32cCodecObject",
]

Crc32cCodec dataclass

Bases: BytesBytesCodec

The crc32c codec, coerced from its metadata.

The name says everything: a checksum has nothing to configure.

Source code in src/zarr_metadata/v3/codec/crc32c.py

@dataclass(frozen=True)
class Crc32cCodec(BytesBytesCodec):
    """The `crc32c` codec, coerced from its metadata.

    The name says everything: a checksum has nothing to configure.
    """

    configuration: Configuration = field(default_factory=Configuration)

    identifier: ClassVar[str] = CRC32C_CODEC_NAME
    variable_size: ClassVar[bool] = False

configuration class-attribute instance-attribute

configuration: Configuration = field(
    default_factory=Configuration
)

The record of this entity's members.

An entity with members narrows it to its own record, configuration: GzipOptions, its one positional argument. An entity of a bare name defaults it to the empty record -- configuration: Configuration = field(default_factory=Configuration) -- so that Crc32cCodec() builds; coerce passes the record either way.

identifier class-attribute

identifier: str = CRC32C_CODEC_NAME

The name this entity is registered under.

Usually the name the metadata carries. The raw-bytes data types are the exception: every r<N> spelling is one family, so the family gets an invented identifier that no real name can collide with.

variable_size class-attribute

variable_size: bool = False

Whether this codec's output size depends on the bytes it is given.

A compressor's does, so a shard index encoded with one has no size derivable from metadata alone, and the shard cannot be read. Every codec says, because a default in either direction is a verdict.

__init__

__init__(
    configuration: Configuration = Configuration(),
) -> None

__post_init__

__post_init__() -> None

Refuse a record that is not this entity's own, then the first problem the rules find.

The runtime half of the entity's type, as the record's constructor is of the record's: GzipCodec(BloscOptions(...)) and a family member carrying a name that is not a string are refused before any rule reads them. Then BloscCodec(BloscOptions(clevel=99)) raises on the first problem the rules yield.

Source code in src/zarr_metadata/v3/_entity.py

def __post_init__(self) -> None:
    """Refuse a record that is not this entity's own, then the first problem the rules find.

    The runtime half of the entity's type, as the record's constructor
    is of the record's: `GzipCodec(BloscOptions(...))` and a family
    member carrying a name that is not a string are refused before
    any rule reads them. Then `BloscCodec(BloscOptions(clevel=99))`
    raises on the first problem the rules yield.
    """
    plan = _plan(type(self))
    if not isinstance(self.configuration, plan.record):
        raise MetadataValidationError(
            problem(
                (),
                f"expected a {plan.record.__name__} configuration, got "
                f"{type(self.configuration).__name__}",
            )
        )
    name: object = self.identifier if plan.from_name is None else getattr(self, plan.from_name)
    if not isinstance(name, str):
        raise MetadataValidationError(problem((), f"expected a string name, got {name!r}"))
    first = next(type(self).name_problems(name), None)
    if first is None:
        first = next(self.configuration.problems(), None)
    if first is not None:
        raise MetadataValidationError((first,))

accepts classmethod

accepts(name: str) -> bool

Whether name denotes this entity.

Constant for all but the raw-bytes family, where one class covers every r<N>.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def accepts(cls, name: str) -> bool:
    """Whether `name` denotes this entity.

    Constant for all but the raw-bytes family, where one class covers
    every `r<N>`.
    """
    return name == cls.identifier

canonical

canonical() -> Self

This entity in the simplest form that means the same thing.

A transformation, asked for by canonicalize_array_metadata_v3 and by nothing else. to_json does not apply it, because writing a document back is not the same as asking for it to be rewritten: a reader that reads and writes should not change bytes it was not asked to change.

The default is the entity itself. Override it where two spellings of the entity's members mean the same -- a rectilinear dimension's run-length encoding, a typesize that noshuffle ignores -- and, in an entity that contains entities, to put those in canonical form: self.with_configuration(inner=self.inner.canonical()).

Source code in src/zarr_metadata/v3/_entity.py

def canonical(self) -> Self:
    """This entity in the simplest form that means the same thing.

    A *transformation*, asked for by `canonicalize_array_metadata_v3`
    and by nothing else. `to_json` does not apply it, because writing
    a document back is not the same as asking for it to be rewritten:
    a reader that reads and writes should not change bytes it was not
    asked to change.

    The default is the entity itself. Override it where two spellings
    of the entity's members mean the same -- a rectilinear
    dimension's run-length encoding, a `typesize` that `noshuffle`
    ignores -- and, in an entity that contains entities, to put those
    in canonical form: `self.with_configuration(inner=self.inner.canonical())`.
    """
    return self

coerce classmethod

coerce(value: JSONValue, context: Context) -> Coerced[Self]

value as this entity, or the reasons it is not one: the class's validation routine.

resolve relates a field's name to this class and hands it the field, refined JSON with arrays as tuples, which is what value is; this is what the class does with it. The configuration is parsed against the record the configuration field names, member by member; a member holding another entity is read in context, the scope this reading is happening in. An optional member the document left out is UNSET in the record, so no field's default decides what a document said. The entity is built only when every member of its own read -- its rules are written over a whole configuration -- and handed back only when everything inside it read too.

The envelope is the field's, not the class's, and resolve judges it: a stray member or a must_understand of false is not reported here. Called on a class no scope has registered, this runs with none of registration's refusals having happened.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def coerce(cls, value: JSONValue, context: Context) -> Coerced[Self]:
    """`value` as this entity, or the reasons it is not one: the class's validation routine.

    `resolve` relates a field's name to this class and hands it the
    field, refined JSON with arrays as tuples, which is what `value`
    is; this is what the class does with it. The configuration is
    parsed against the record the `configuration` field names,
    member by member; a member holding another entity is read in
    `context`, the scope this reading is happening in. An optional
    member the document left out is `UNSET` in the record, so no
    field's default decides what a document said. The entity is
    built only when every member of its own read -- its rules are
    written over a whole configuration -- and handed back only when
    everything inside it read too.

    The envelope is the field's, not the class's, and `resolve`
    judges it: a stray member or a `must_understand` of `false` is
    not reported here. Called on a class no scope has registered,
    this runs with none of registration's refusals having happened.
    """
    name, given, envelope = named_configuration(value)
    if name is None or not cls.accepts(name):
        return None, problem((), f"expected the {cls.identifier!r} entity")
    if len(envelope) != 0:
        return None, envelope
    plan = _plan(cls)
    carried: dict[str, str] = {} if plan.from_name is None else {plan.from_name: name}
    if given is None and plan.requires_configuration:
        return None, problem(
            ("configuration",),
            f"{cls.identifier!r} requires a configuration",
            "missing_key",
        )
    reading = _Reading(context, [])
    # A bare name's record has no members, so any key is an unknown one.
    record, own = plan.read({} if given is None else given, ("configuration",), reading)
    found = (*own, *reading.nested)
    if record is None:
        # An unknown key is survivable; a member that could not be
        # read is a hole, and judging around it would be guessing.
        return None, found
    # The rules, asked of the name and of the record before anything
    # is built: a name problem lands on the entity, a configuration
    # problem under the configuration.
    refused = (*cls.name_problems(name), *within((), tuple(record.problems())))
    if len(refused) != 0:
        # Values the spec disallows: reported rather than raised,
        # every one.
        return None, (*found, *refused)
    if any(entry.kind != "unknown_key" for entry in reading.nested):
        # A contained entity could not be read. This entity's own
        # rules ran -- an invalid inner is an `Opaque`, as an
        # out-of-scope one is -- but what is handed back is not an
        # entity that would be asked composition questions it cannot
        # answer.
        return None, found
    return cls.create_unchecked(configuration=record, **carried), found

create_unchecked classmethod

create_unchecked(**fields: object) -> Self

This entity with these fields, built without the constructor's checks.

The one way around them, for a caller that has just made them: coerce, which type-checked the record and ran the rules before building. Every field is given -- the record, and the carried name for a family -- since nothing here applies a default. Anything that has not checked goes through the constructor.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def create_unchecked(cls, **fields: object) -> Self:
    """This entity with these fields, built without the constructor's checks.

    The one way around them, for a caller that has just made them:
    `coerce`, which type-checked the record and ran the rules before
    building. Every field is given -- the record, and the carried
    name for a family -- since nothing here applies a default.
    Anything that has not checked goes through the constructor.
    """
    entity = object.__new__(cls)
    for name, value in fields.items():
        object.__setattr__(entity, name, value)
    return entity

incoming_problems

incoming_problems(
    incoming: ArrayParts | None,
) -> tuple[ValidationProblem, ...]

Why this codec cannot be applied to the array that reaches it.

incoming is None once the chain can no longer say what reaches here, and the default answer to that is nothing: declining beats guessing. Locations are relative to this codec's configuration; an empty one lands on the codec itself.

Source code in src/zarr_metadata/v3/_entity.py

def incoming_problems(self, incoming: ArrayParts | None) -> tuple[ValidationProblem, ...]:
    """Why this codec cannot be applied to the array that reaches it.

    `incoming` is None once the chain can no longer say what reaches
    here, and the default answer to that is nothing: declining beats
    guessing. Locations are relative to this codec's `configuration`;
    an empty one lands on the codec itself.
    """
    return ()

inner_pipelines

inner_pipelines(
    incoming: ArrayParts | None,
) -> Mapping[
    str,
    tuple[
        Sequence[CodecEntity | Opaque], ArrayParts | None
    ],
]

The pipelines this codec holds, by the member holding each, with what each is handed.

A shard holds two: its codecs, handed its inner chunk, and its index_codecs, handed the shard index. Refinement walks them as it walks the pipeline this codec stands in, locating what it finds under the member, so a codec that holds pipelines says which and what they receive, and judges nothing inside them itself. Default: none.

Source code in src/zarr_metadata/v3/_entity.py

def inner_pipelines(
    self, incoming: ArrayParts | None
) -> Mapping[str, tuple[Sequence[CodecEntity | Opaque], ArrayParts | None]]:
    """The pipelines this codec holds, by the member holding each, with what each is handed.

    A shard holds two: its `codecs`, handed its inner chunk, and its
    `index_codecs`, handed the shard index. Refinement walks them as
    it walks the pipeline this codec stands in, locating what it
    finds under the member, so a codec that holds pipelines says
    which and what they receive, and judges nothing inside them
    itself. Default: none.
    """
    return {}

name_problems classmethod

name_problems(name: str) -> Iterator[ValidationProblem]

Why name, which accepts claimed, is not a well-formed name of this family.

For a family, whose names carry data -- r<N> -- and which claims a malformed member so that it is reported rather than waved through as an unknown extension. Locations are relative to the entity: (). Default: none, for an entity of one name.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def name_problems(cls, name: str) -> Iterator[ValidationProblem]:
    """Why `name`, which `accepts` claimed, is not a well-formed name of this family.

    For a family, whose names carry data -- `r<N>` -- and which claims
    a malformed member so that it is reported rather than waved
    through as an unknown extension. Locations are relative to the
    entity: `()`. Default: none, for an entity of one name.
    """
    yield from ()

to_json

to_json() -> ZarrV3MetadataFieldJSON

This entity as a document would write it.

Written from the configuration record by the same declaration coerce reads it by, each member by the writer its annotation implies: the bare name when every member it holds is absent, the object otherwise, a contained entity through its own to_json, a JSON-valued member copied so the document is not a handle on the entity. Faithful to every member: read a document, write it back, and those come out as they went in. The envelope is written the entity's way -- the bare name when nothing is configured, the object otherwise, no must_understand, which means what absence means -- because an entity alone has no document to be faithful to; ArrayDocumentV3.to_json puts back the spelling the document used. Ask canonical first if you want the simplest equivalent spelling.

An entity whose JSON is not its fields overrides this; none in the package does.

Source code in src/zarr_metadata/v3/_entity.py

def to_json(self) -> ZarrV3MetadataFieldJSON:
    """This entity as a document would write it.

    Written from the configuration record by the same declaration
    `coerce` reads it by, each member by the writer its annotation
    implies: the bare name when every member it holds is absent, the
    object otherwise, a contained entity through its own `to_json`,
    a JSON-valued member copied so the document is not a handle on
    the entity. Faithful to every member: read a document, write it
    back, and those come out as they went in. The envelope is
    written the entity's way -- the bare name when nothing is
    configured, the object otherwise, no `must_understand`, which
    means what absence means -- because an entity alone has no
    document to be faithful to; `ArrayDocumentV3.to_json` puts back
    the spelling the document used. Ask `canonical` first if you
    want the simplest equivalent spelling.

    An entity whose JSON is not its fields overrides this; none in
    the package does.
    """
    plan = _plan(type(self))
    name = self.identifier
    if plan.from_name is not None:
        carried = getattr(self, plan.from_name)
        name = carried if isinstance(carried, str) else name
    configuration = plan.write(self)
    if len(configuration) == 0:
        return name
    return {"name": name, "configuration": configuration}

with_configuration

with_configuration(**changes: object) -> Self

This entity with these configuration members changed.

codec.with_configuration(typesize=UNSET) is the record rebuilt through its constructor, which refuses a member of the wrong type, and the entity rebuilt through its own, which refuses a value the rules disallow -- the same checks as any construction, since pyright cannot see the members through **changes. A name that is not a member is refused the way replace refuses it.

Source code in src/zarr_metadata/v3/_entity.py

def with_configuration(self, **changes: object) -> Self:
    """This entity with these configuration members changed.

    `codec.with_configuration(typesize=UNSET)` is the record rebuilt
    through its constructor, which refuses a member of the wrong
    type, and the entity rebuilt through its own, which refuses a
    value the rules disallow -- the same checks as any construction,
    since pyright cannot see the members through `**changes`. A
    name that is not a member is refused the way `replace` refuses
    it.
    """
    return replace(self, configuration=replace(self.configuration, **changes))

Crc32cCodecObject

Bases: TypedDict

crc32c codec metadata in object form.

Per spec the codec has no configuration fields. configuration is optional and, if present, should be an empty mapping. https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/codecs/crc32c/index.rst#L63-L66

Source code in src/zarr_metadata/v3/codec/crc32c.py

class Crc32cCodecObject(TypedDict, closed=True):
    """`crc32c` codec metadata in object form.

    Per spec the codec has no configuration fields. `configuration` is
    optional and, if present, should be an empty mapping.
      https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/codecs/crc32c/index.rst#L63-L66
    """

    name: Crc32cCodecName
    configuration: NotRequired[Empty]
    must_understand: NotRequired[bool]

configuration instance-attribute

configuration: NotRequired[Empty]

must_understand instance-attribute

must_understand: NotRequired[bool]

name instance-attribute

name: Crc32cCodecName

Empty

Bases: TypedDict

An empty mapping

Source code in src/zarr_metadata/v3/codec/crc32c.py

class Empty(TypedDict, closed=True):
    """An empty mapping"""

zarr_metadata.v3.codec.gzip

Gzip codec types.

See https://zarr-specs.readthedocs.io/en/latest/v3/codecs/gzip/index.html

GZIP_CODEC_NAME module-attribute

GZIP_CODEC_NAME: Final = 'gzip'

The name field value of the gzip codec.

GzipCodecMetadata module-attribute

GzipCodecMetadata = GzipCodecObject

Permitted JSON shape for gzip codec metadata.

configuration.level is required (it determines the codec's output bytes and is therefore part of the metadata's reproducibility contract), so only the object form is valid; the short-hand-name form is not permitted.

GzipCodecName module-attribute

GzipCodecName = Literal['gzip']

Literal type of the name field of the gzip codec.

__all__ module-attribute

__all__ = [
    "GZIP_CODEC_NAME",
    "GzipCodec",
    "GzipCodecConfiguration",
    "GzipCodecMetadata",
    "GzipCodecName",
    "GzipCodecObject",
    "GzipOptions",
]

GzipCodec dataclass

Bases: BytesBytesCodec

The gzip codec, coerced from its metadata.

Source code in src/zarr_metadata/v3/codec/gzip.py

@dataclass(frozen=True)
class GzipCodec(BytesBytesCodec):
    """The `gzip` codec, coerced from its metadata."""

    configuration: GzipOptions

    identifier: ClassVar[str] = GZIP_CODEC_NAME
    variable_size: ClassVar[bool] = True

configuration instance-attribute

configuration: GzipOptions

The record of this entity's members.

An entity with members narrows it to its own record, configuration: GzipOptions, its one positional argument. An entity of a bare name defaults it to the empty record -- configuration: Configuration = field(default_factory=Configuration) -- so that Crc32cCodec() builds; coerce passes the record either way.

identifier class-attribute

identifier: str = GZIP_CODEC_NAME

The name this entity is registered under.

Usually the name the metadata carries. The raw-bytes data types are the exception: every r<N> spelling is one family, so the family gets an invented identifier that no real name can collide with.

variable_size class-attribute

variable_size: bool = True

Whether this codec's output size depends on the bytes it is given.

A compressor's does, so a shard index encoded with one has no size derivable from metadata alone, and the shard cannot be read. Every codec says, because a default in either direction is a verdict.

__init__

__init__(configuration: GzipOptions) -> None

__post_init__

__post_init__() -> None

Refuse a record that is not this entity's own, then the first problem the rules find.

The runtime half of the entity's type, as the record's constructor is of the record's: GzipCodec(BloscOptions(...)) and a family member carrying a name that is not a string are refused before any rule reads them. Then BloscCodec(BloscOptions(clevel=99)) raises on the first problem the rules yield.

Source code in src/zarr_metadata/v3/_entity.py

def __post_init__(self) -> None:
    """Refuse a record that is not this entity's own, then the first problem the rules find.

    The runtime half of the entity's type, as the record's constructor
    is of the record's: `GzipCodec(BloscOptions(...))` and a family
    member carrying a name that is not a string are refused before
    any rule reads them. Then `BloscCodec(BloscOptions(clevel=99))`
    raises on the first problem the rules yield.
    """
    plan = _plan(type(self))
    if not isinstance(self.configuration, plan.record):
        raise MetadataValidationError(
            problem(
                (),
                f"expected a {plan.record.__name__} configuration, got "
                f"{type(self.configuration).__name__}",
            )
        )
    name: object = self.identifier if plan.from_name is None else getattr(self, plan.from_name)
    if not isinstance(name, str):
        raise MetadataValidationError(problem((), f"expected a string name, got {name!r}"))
    first = next(type(self).name_problems(name), None)
    if first is None:
        first = next(self.configuration.problems(), None)
    if first is not None:
        raise MetadataValidationError((first,))

accepts classmethod

accepts(name: str) -> bool

Whether name denotes this entity.

Constant for all but the raw-bytes family, where one class covers every r<N>.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def accepts(cls, name: str) -> bool:
    """Whether `name` denotes this entity.

    Constant for all but the raw-bytes family, where one class covers
    every `r<N>`.
    """
    return name == cls.identifier

canonical

canonical() -> Self

This entity in the simplest form that means the same thing.

A transformation, asked for by canonicalize_array_metadata_v3 and by nothing else. to_json does not apply it, because writing a document back is not the same as asking for it to be rewritten: a reader that reads and writes should not change bytes it was not asked to change.

The default is the entity itself. Override it where two spellings of the entity's members mean the same -- a rectilinear dimension's run-length encoding, a typesize that noshuffle ignores -- and, in an entity that contains entities, to put those in canonical form: self.with_configuration(inner=self.inner.canonical()).

Source code in src/zarr_metadata/v3/_entity.py

def canonical(self) -> Self:
    """This entity in the simplest form that means the same thing.

    A *transformation*, asked for by `canonicalize_array_metadata_v3`
    and by nothing else. `to_json` does not apply it, because writing
    a document back is not the same as asking for it to be rewritten:
    a reader that reads and writes should not change bytes it was not
    asked to change.

    The default is the entity itself. Override it where two spellings
    of the entity's members mean the same -- a rectilinear
    dimension's run-length encoding, a `typesize` that `noshuffle`
    ignores -- and, in an entity that contains entities, to put those
    in canonical form: `self.with_configuration(inner=self.inner.canonical())`.
    """
    return self

coerce classmethod

coerce(value: JSONValue, context: Context) -> Coerced[Self]

value as this entity, or the reasons it is not one: the class's validation routine.

resolve relates a field's name to this class and hands it the field, refined JSON with arrays as tuples, which is what value is; this is what the class does with it. The configuration is parsed against the record the configuration field names, member by member; a member holding another entity is read in context, the scope this reading is happening in. An optional member the document left out is UNSET in the record, so no field's default decides what a document said. The entity is built only when every member of its own read -- its rules are written over a whole configuration -- and handed back only when everything inside it read too.

The envelope is the field's, not the class's, and resolve judges it: a stray member or a must_understand of false is not reported here. Called on a class no scope has registered, this runs with none of registration's refusals having happened.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def coerce(cls, value: JSONValue, context: Context) -> Coerced[Self]:
    """`value` as this entity, or the reasons it is not one: the class's validation routine.

    `resolve` relates a field's name to this class and hands it the
    field, refined JSON with arrays as tuples, which is what `value`
    is; this is what the class does with it. The configuration is
    parsed against the record the `configuration` field names,
    member by member; a member holding another entity is read in
    `context`, the scope this reading is happening in. An optional
    member the document left out is `UNSET` in the record, so no
    field's default decides what a document said. The entity is
    built only when every member of its own read -- its rules are
    written over a whole configuration -- and handed back only when
    everything inside it read too.

    The envelope is the field's, not the class's, and `resolve`
    judges it: a stray member or a `must_understand` of `false` is
    not reported here. Called on a class no scope has registered,
    this runs with none of registration's refusals having happened.
    """
    name, given, envelope = named_configuration(value)
    if name is None or not cls.accepts(name):
        return None, problem((), f"expected the {cls.identifier!r} entity")
    if len(envelope) != 0:
        return None, envelope
    plan = _plan(cls)
    carried: dict[str, str] = {} if plan.from_name is None else {plan.from_name: name}
    if given is None and plan.requires_configuration:
        return None, problem(
            ("configuration",),
            f"{cls.identifier!r} requires a configuration",
            "missing_key",
        )
    reading = _Reading(context, [])
    # A bare name's record has no members, so any key is an unknown one.
    record, own = plan.read({} if given is None else given, ("configuration",), reading)
    found = (*own, *reading.nested)
    if record is None:
        # An unknown key is survivable; a member that could not be
        # read is a hole, and judging around it would be guessing.
        return None, found
    # The rules, asked of the name and of the record before anything
    # is built: a name problem lands on the entity, a configuration
    # problem under the configuration.
    refused = (*cls.name_problems(name), *within((), tuple(record.problems())))
    if len(refused) != 0:
        # Values the spec disallows: reported rather than raised,
        # every one.
        return None, (*found, *refused)
    if any(entry.kind != "unknown_key" for entry in reading.nested):
        # A contained entity could not be read. This entity's own
        # rules ran -- an invalid inner is an `Opaque`, as an
        # out-of-scope one is -- but what is handed back is not an
        # entity that would be asked composition questions it cannot
        # answer.
        return None, found
    return cls.create_unchecked(configuration=record, **carried), found

create_unchecked classmethod

create_unchecked(**fields: object) -> Self

This entity with these fields, built without the constructor's checks.

The one way around them, for a caller that has just made them: coerce, which type-checked the record and ran the rules before building. Every field is given -- the record, and the carried name for a family -- since nothing here applies a default. Anything that has not checked goes through the constructor.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def create_unchecked(cls, **fields: object) -> Self:
    """This entity with these fields, built without the constructor's checks.

    The one way around them, for a caller that has just made them:
    `coerce`, which type-checked the record and ran the rules before
    building. Every field is given -- the record, and the carried
    name for a family -- since nothing here applies a default.
    Anything that has not checked goes through the constructor.
    """
    entity = object.__new__(cls)
    for name, value in fields.items():
        object.__setattr__(entity, name, value)
    return entity

incoming_problems

incoming_problems(
    incoming: ArrayParts | None,
) -> tuple[ValidationProblem, ...]

Why this codec cannot be applied to the array that reaches it.

incoming is None once the chain can no longer say what reaches here, and the default answer to that is nothing: declining beats guessing. Locations are relative to this codec's configuration; an empty one lands on the codec itself.

Source code in src/zarr_metadata/v3/_entity.py

def incoming_problems(self, incoming: ArrayParts | None) -> tuple[ValidationProblem, ...]:
    """Why this codec cannot be applied to the array that reaches it.

    `incoming` is None once the chain can no longer say what reaches
    here, and the default answer to that is nothing: declining beats
    guessing. Locations are relative to this codec's `configuration`;
    an empty one lands on the codec itself.
    """
    return ()

inner_pipelines

inner_pipelines(
    incoming: ArrayParts | None,
) -> Mapping[
    str,
    tuple[
        Sequence[CodecEntity | Opaque], ArrayParts | None
    ],
]

The pipelines this codec holds, by the member holding each, with what each is handed.

A shard holds two: its codecs, handed its inner chunk, and its index_codecs, handed the shard index. Refinement walks them as it walks the pipeline this codec stands in, locating what it finds under the member, so a codec that holds pipelines says which and what they receive, and judges nothing inside them itself. Default: none.

Source code in src/zarr_metadata/v3/_entity.py

def inner_pipelines(
    self, incoming: ArrayParts | None
) -> Mapping[str, tuple[Sequence[CodecEntity | Opaque], ArrayParts | None]]:
    """The pipelines this codec holds, by the member holding each, with what each is handed.

    A shard holds two: its `codecs`, handed its inner chunk, and its
    `index_codecs`, handed the shard index. Refinement walks them as
    it walks the pipeline this codec stands in, locating what it
    finds under the member, so a codec that holds pipelines says
    which and what they receive, and judges nothing inside them
    itself. Default: none.
    """
    return {}

name_problems classmethod

name_problems(name: str) -> Iterator[ValidationProblem]

Why name, which accepts claimed, is not a well-formed name of this family.

For a family, whose names carry data -- r<N> -- and which claims a malformed member so that it is reported rather than waved through as an unknown extension. Locations are relative to the entity: (). Default: none, for an entity of one name.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def name_problems(cls, name: str) -> Iterator[ValidationProblem]:
    """Why `name`, which `accepts` claimed, is not a well-formed name of this family.

    For a family, whose names carry data -- `r<N>` -- and which claims
    a malformed member so that it is reported rather than waved
    through as an unknown extension. Locations are relative to the
    entity: `()`. Default: none, for an entity of one name.
    """
    yield from ()

to_json

to_json() -> ZarrV3MetadataFieldJSON

This entity as a document would write it.

Written from the configuration record by the same declaration coerce reads it by, each member by the writer its annotation implies: the bare name when every member it holds is absent, the object otherwise, a contained entity through its own to_json, a JSON-valued member copied so the document is not a handle on the entity. Faithful to every member: read a document, write it back, and those come out as they went in. The envelope is written the entity's way -- the bare name when nothing is configured, the object otherwise, no must_understand, which means what absence means -- because an entity alone has no document to be faithful to; ArrayDocumentV3.to_json puts back the spelling the document used. Ask canonical first if you want the simplest equivalent spelling.

An entity whose JSON is not its fields overrides this; none in the package does.

Source code in src/zarr_metadata/v3/_entity.py

def to_json(self) -> ZarrV3MetadataFieldJSON:
    """This entity as a document would write it.

    Written from the configuration record by the same declaration
    `coerce` reads it by, each member by the writer its annotation
    implies: the bare name when every member it holds is absent, the
    object otherwise, a contained entity through its own `to_json`,
    a JSON-valued member copied so the document is not a handle on
    the entity. Faithful to every member: read a document, write it
    back, and those come out as they went in. The envelope is
    written the entity's way -- the bare name when nothing is
    configured, the object otherwise, no `must_understand`, which
    means what absence means -- because an entity alone has no
    document to be faithful to; `ArrayDocumentV3.to_json` puts back
    the spelling the document used. Ask `canonical` first if you
    want the simplest equivalent spelling.

    An entity whose JSON is not its fields overrides this; none in
    the package does.
    """
    plan = _plan(type(self))
    name = self.identifier
    if plan.from_name is not None:
        carried = getattr(self, plan.from_name)
        name = carried if isinstance(carried, str) else name
    configuration = plan.write(self)
    if len(configuration) == 0:
        return name
    return {"name": name, "configuration": configuration}

with_configuration

with_configuration(**changes: object) -> Self

This entity with these configuration members changed.

codec.with_configuration(typesize=UNSET) is the record rebuilt through its constructor, which refuses a member of the wrong type, and the entity rebuilt through its own, which refuses a value the rules disallow -- the same checks as any construction, since pyright cannot see the members through **changes. A name that is not a member is refused the way replace refuses it.

Source code in src/zarr_metadata/v3/_entity.py

def with_configuration(self, **changes: object) -> Self:
    """This entity with these configuration members changed.

    `codec.with_configuration(typesize=UNSET)` is the record rebuilt
    through its constructor, which refuses a member of the wrong
    type, and the entity rebuilt through its own, which refuses a
    value the rules disallow -- the same checks as any construction,
    since pyright cannot see the members through `**changes`. A
    name that is not a member is refused the way `replace` refuses
    it.
    """
    return replace(self, configuration=replace(self.configuration, **changes))

GzipCodecConfiguration

Bases: TypedDict

Configuration for the Zarr v3 gzip codec.

level is an integer in the range 0-9; 0 disables compression and 9 is slowest with the best compression ratio. The codec's compressed output depends on level, so metadata that omits it cannot reproducibly identify the chunk bytes produced by a writer — level is required for the metadata to fulfill its reproducibility role, even though the spec text does not mark it required with RFC 2119 keywords. https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/codecs/gzip/index.rst#L57-L66

Source code in src/zarr_metadata/v3/codec/gzip.py

class GzipCodecConfiguration(TypedDict, closed=True):
    """
    Configuration for the Zarr v3 `gzip` codec.

    `level` is an integer in the range 0-9; 0 disables compression and 9
    is slowest with the best compression ratio. The codec's compressed
    output depends on `level`, so metadata that omits it cannot
    reproducibly identify the chunk bytes produced by a writer — `level`
    is required for the metadata to fulfill its reproducibility role,
    even though the spec text does not mark it required with RFC 2119
    keywords.
      https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/codecs/gzip/index.rst#L57-L66
    """

    level: int

level instance-attribute

level: int

GzipCodecObject

Bases: TypedDict

gzip codec metadata in object form.

Source code in src/zarr_metadata/v3/codec/gzip.py

class GzipCodecObject(TypedDict, closed=True):
    """`gzip` codec metadata in object form."""

    name: GzipCodecName
    configuration: GzipCodecConfiguration
    must_understand: NotRequired[bool]

configuration instance-attribute

configuration: GzipCodecConfiguration

must_understand instance-attribute

must_understand: NotRequired[bool]

name instance-attribute

name: GzipCodecName

GzipOptions dataclass

Bases: Configuration

What gzip is configured with.

Source code in src/zarr_metadata/v3/codec/gzip.py

@dataclass(frozen=True)
class GzipOptions(Configuration):
    """What `gzip` is configured with."""

    level: int

    def problems(self) -> "Iterator[ValidationProblem]":
        if not 0 <= self.level <= 9:
            yield ValidationProblem(
                ("level",), f"expected an integer in [0, 9], got {self.level}", "invalid_value"
            )

level instance-attribute

level: int

__init__

__init__(level: int) -> None

__post_init__

__post_init__() -> None

Refuse every member of the wrong type, so GzipOptions(level="high") raises.

Source code in src/zarr_metadata/v3/_entity.py

def __post_init__(self) -> None:
    """Refuse every member of the wrong type, so `GzipOptions(level="high")` raises."""
    found = mistyped(self)
    if len(found) != 0:
        raise MetadataValidationError(found)

create_unchecked classmethod

create_unchecked(**members: object) -> Self

This record with these members, built without the constructor's check.

The one way around the check, for a caller that has just made it: the parser, which type-checked every member against the same annotations before building the record. Every field is given -- the parser gives an absent optional member as UNSET -- since nothing here applies a default. Anything that has not checked the members goes through the constructor.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def create_unchecked(cls, **members: object) -> Self:
    """This record with these members, built without the constructor's check.

    The one way around the check, for a caller that has just made
    it: the parser, which type-checked every member against the same
    annotations before building the record. Every field is given --
    the parser gives an absent optional member as `UNSET` -- since
    nothing here applies a default. Anything that has not checked
    the members goes through the constructor.
    """
    record = object.__new__(cls)
    for name, value in members.items():
        object.__setattr__(record, name, value)
    return record

problems

problems() -> Iterator[ValidationProblem]

Every reason these values are not allowed, yielded as found. Default: none.

Source code in src/zarr_metadata/v3/codec/gzip.py

def problems(self) -> "Iterator[ValidationProblem]":
    if not 0 <= self.level <= 9:
        yield ValidationProblem(
            ("level",), f"expected an integer in [0, 9], got {self.level}", "invalid_value"
        )

zarr_metadata.v3.codec.scale_offset

Scale-offset codec types.

See https://github.com/zarr-developers/zarr-extensions/blob/4da7b37a84f76e660902f6d3de3eaef0e0febae6/codecs/scale_offset/README.md

SCALE_OFFSET_CODEC_NAME module-attribute

SCALE_OFFSET_CODEC_NAME: Final = 'scale_offset'

The name field value of the scale_offset codec.

ScaleOffsetCodecMetadata module-attribute

ScaleOffsetCodecMetadata = (
    ScaleOffsetCodecObject | ScaleOffsetCodecName
)

Permitted JSON shapes for scale_offset codec metadata.

The configuration has no required keys (both offset and scale are optional, and the configuration itself is optional), so the short-hand-name form is permitted in addition to the object form.

ScaleOffsetCodecName module-attribute

ScaleOffsetCodecName = Literal['scale_offset']

Literal type of the name field of the scale_offset codec.

__all__ module-attribute

__all__ = [
    "SCALE_OFFSET_CODEC_NAME",
    "ScaleOffsetCodec",
    "ScaleOffsetCodecConfiguration",
    "ScaleOffsetCodecMetadata",
    "ScaleOffsetCodecName",
    "ScaleOffsetCodecObject",
    "ScaleOffsetOptions",
]

ScaleOffsetCodec dataclass

Bases: ArrayArrayCodec

The scale_offset codec, coerced from its metadata.

Both members are optional and any JSON scalar is well-typed here; what a given value means depends on the data type it is applied to, which incoming_problems asks of the type that reaches the codec.

Source code in src/zarr_metadata/v3/codec/scale_offset.py

@dataclass(frozen=True)
class ScaleOffsetCodec(ArrayArrayCodec):
    """The `scale_offset` codec, coerced from its metadata.

    Both members are optional and any JSON scalar is well-typed here; what
    a given value means depends on the data type it is applied to, which
    `incoming_problems` asks of the type that reaches the codec.
    """

    configuration: ScaleOffsetOptions

    identifier: ClassVar[str] = SCALE_OFFSET_CODEC_NAME
    variable_size: ClassVar[bool] = False

    def incoming_problems(self, incoming: ArrayParts | None) -> tuple[ValidationProblem, ...]:
        """What the array handed to this codec must be, and what its members must be for it.

        The registry defines the codec for data types with arithmetic and
        lists the integer and floating-point ones. `offset` and `scale`
        are each "encoded to JSON using the Zarr V3 fill value encoding
        for the input array's data type" -- the type that reaches this
        codec, which after a `cast_value` is not the array's own -- so
        each is a fill value of that type, and that type judges it: the
        string `"0"` is no `float32` and no `int32`.
        """
        data_type = incoming.data_type if incoming is not None else None
        if not isinstance(data_type, DataTypeEntity):
            return ()
        if not isinstance(data_type, (IntegerDataType, FloatDataType)):
            return problem(
                (),
                "scale_offset is defined for integer and floating-point data types, not "
                f"{type(data_type).identifier!r}",
                "invalid_value",
            )
        return tuple(
            found
            for member, value in (
                ("offset", self.configuration.offset),
                ("scale", self.configuration.scale),
            )
            if value is not UNSET
            for found in data_type.fill_value_problems(value, (member,))
        )

    def transition(self, incoming: ArrayParts) -> ArrayParts | None:
        """The same array, element for element.

        The registry entry removed the `astype` field, so this codec no
        longer changes the element type -- only the values.
        """
        return incoming

configuration instance-attribute

configuration: ScaleOffsetOptions

The record of this entity's members.

An entity with members narrows it to its own record, configuration: GzipOptions, its one positional argument. An entity of a bare name defaults it to the empty record -- configuration: Configuration = field(default_factory=Configuration) -- so that Crc32cCodec() builds; coerce passes the record either way.

identifier class-attribute

identifier: str = SCALE_OFFSET_CODEC_NAME

The name this entity is registered under.

Usually the name the metadata carries. The raw-bytes data types are the exception: every r<N> spelling is one family, so the family gets an invented identifier that no real name can collide with.

variable_size class-attribute

variable_size: bool = False

Whether this codec's output size depends on the bytes it is given.

A compressor's does, so a shard index encoded with one has no size derivable from metadata alone, and the shard cannot be read. Every codec says, because a default in either direction is a verdict.

__init__

__init__(configuration: ScaleOffsetOptions) -> None

__post_init__

__post_init__() -> None

Refuse a record that is not this entity's own, then the first problem the rules find.

The runtime half of the entity's type, as the record's constructor is of the record's: GzipCodec(BloscOptions(...)) and a family member carrying a name that is not a string are refused before any rule reads them. Then BloscCodec(BloscOptions(clevel=99)) raises on the first problem the rules yield.

Source code in src/zarr_metadata/v3/_entity.py

def __post_init__(self) -> None:
    """Refuse a record that is not this entity's own, then the first problem the rules find.

    The runtime half of the entity's type, as the record's constructor
    is of the record's: `GzipCodec(BloscOptions(...))` and a family
    member carrying a name that is not a string are refused before
    any rule reads them. Then `BloscCodec(BloscOptions(clevel=99))`
    raises on the first problem the rules yield.
    """
    plan = _plan(type(self))
    if not isinstance(self.configuration, plan.record):
        raise MetadataValidationError(
            problem(
                (),
                f"expected a {plan.record.__name__} configuration, got "
                f"{type(self.configuration).__name__}",
            )
        )
    name: object = self.identifier if plan.from_name is None else getattr(self, plan.from_name)
    if not isinstance(name, str):
        raise MetadataValidationError(problem((), f"expected a string name, got {name!r}"))
    first = next(type(self).name_problems(name), None)
    if first is None:
        first = next(self.configuration.problems(), None)
    if first is not None:
        raise MetadataValidationError((first,))

accepts classmethod

accepts(name: str) -> bool

Whether name denotes this entity.

Constant for all but the raw-bytes family, where one class covers every r<N>.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def accepts(cls, name: str) -> bool:
    """Whether `name` denotes this entity.

    Constant for all but the raw-bytes family, where one class covers
    every `r<N>`.
    """
    return name == cls.identifier

canonical

canonical() -> Self

This entity in the simplest form that means the same thing.

A transformation, asked for by canonicalize_array_metadata_v3 and by nothing else. to_json does not apply it, because writing a document back is not the same as asking for it to be rewritten: a reader that reads and writes should not change bytes it was not asked to change.

The default is the entity itself. Override it where two spellings of the entity's members mean the same -- a rectilinear dimension's run-length encoding, a typesize that noshuffle ignores -- and, in an entity that contains entities, to put those in canonical form: self.with_configuration(inner=self.inner.canonical()).

Source code in src/zarr_metadata/v3/_entity.py

def canonical(self) -> Self:
    """This entity in the simplest form that means the same thing.

    A *transformation*, asked for by `canonicalize_array_metadata_v3`
    and by nothing else. `to_json` does not apply it, because writing
    a document back is not the same as asking for it to be rewritten:
    a reader that reads and writes should not change bytes it was not
    asked to change.

    The default is the entity itself. Override it where two spellings
    of the entity's members mean the same -- a rectilinear
    dimension's run-length encoding, a `typesize` that `noshuffle`
    ignores -- and, in an entity that contains entities, to put those
    in canonical form: `self.with_configuration(inner=self.inner.canonical())`.
    """
    return self

coerce classmethod

coerce(value: JSONValue, context: Context) -> Coerced[Self]

value as this entity, or the reasons it is not one: the class's validation routine.

resolve relates a field's name to this class and hands it the field, refined JSON with arrays as tuples, which is what value is; this is what the class does with it. The configuration is parsed against the record the configuration field names, member by member; a member holding another entity is read in context, the scope this reading is happening in. An optional member the document left out is UNSET in the record, so no field's default decides what a document said. The entity is built only when every member of its own read -- its rules are written over a whole configuration -- and handed back only when everything inside it read too.

The envelope is the field's, not the class's, and resolve judges it: a stray member or a must_understand of false is not reported here. Called on a class no scope has registered, this runs with none of registration's refusals having happened.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def coerce(cls, value: JSONValue, context: Context) -> Coerced[Self]:
    """`value` as this entity, or the reasons it is not one: the class's validation routine.

    `resolve` relates a field's name to this class and hands it the
    field, refined JSON with arrays as tuples, which is what `value`
    is; this is what the class does with it. The configuration is
    parsed against the record the `configuration` field names,
    member by member; a member holding another entity is read in
    `context`, the scope this reading is happening in. An optional
    member the document left out is `UNSET` in the record, so no
    field's default decides what a document said. The entity is
    built only when every member of its own read -- its rules are
    written over a whole configuration -- and handed back only when
    everything inside it read too.

    The envelope is the field's, not the class's, and `resolve`
    judges it: a stray member or a `must_understand` of `false` is
    not reported here. Called on a class no scope has registered,
    this runs with none of registration's refusals having happened.
    """
    name, given, envelope = named_configuration(value)
    if name is None or not cls.accepts(name):
        return None, problem((), f"expected the {cls.identifier!r} entity")
    if len(envelope) != 0:
        return None, envelope
    plan = _plan(cls)
    carried: dict[str, str] = {} if plan.from_name is None else {plan.from_name: name}
    if given is None and plan.requires_configuration:
        return None, problem(
            ("configuration",),
            f"{cls.identifier!r} requires a configuration",
            "missing_key",
        )
    reading = _Reading(context, [])
    # A bare name's record has no members, so any key is an unknown one.
    record, own = plan.read({} if given is None else given, ("configuration",), reading)
    found = (*own, *reading.nested)
    if record is None:
        # An unknown key is survivable; a member that could not be
        # read is a hole, and judging around it would be guessing.
        return None, found
    # The rules, asked of the name and of the record before anything
    # is built: a name problem lands on the entity, a configuration
    # problem under the configuration.
    refused = (*cls.name_problems(name), *within((), tuple(record.problems())))
    if len(refused) != 0:
        # Values the spec disallows: reported rather than raised,
        # every one.
        return None, (*found, *refused)
    if any(entry.kind != "unknown_key" for entry in reading.nested):
        # A contained entity could not be read. This entity's own
        # rules ran -- an invalid inner is an `Opaque`, as an
        # out-of-scope one is -- but what is handed back is not an
        # entity that would be asked composition questions it cannot
        # answer.
        return None, found
    return cls.create_unchecked(configuration=record, **carried), found

create_unchecked classmethod

create_unchecked(**fields: object) -> Self

This entity with these fields, built without the constructor's checks.

The one way around them, for a caller that has just made them: coerce, which type-checked the record and ran the rules before building. Every field is given -- the record, and the carried name for a family -- since nothing here applies a default. Anything that has not checked goes through the constructor.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def create_unchecked(cls, **fields: object) -> Self:
    """This entity with these fields, built without the constructor's checks.

    The one way around them, for a caller that has just made them:
    `coerce`, which type-checked the record and ran the rules before
    building. Every field is given -- the record, and the carried
    name for a family -- since nothing here applies a default.
    Anything that has not checked goes through the constructor.
    """
    entity = object.__new__(cls)
    for name, value in fields.items():
        object.__setattr__(entity, name, value)
    return entity

incoming_problems

incoming_problems(
    incoming: ArrayParts | None,
) -> tuple[ValidationProblem, ...]

What the array handed to this codec must be, and what its members must be for it.

The registry defines the codec for data types with arithmetic and lists the integer and floating-point ones. offset and scale are each "encoded to JSON using the Zarr V3 fill value encoding for the input array's data type" -- the type that reaches this codec, which after a cast_value is not the array's own -- so each is a fill value of that type, and that type judges it: the string "0" is no float32 and no int32.

Source code in src/zarr_metadata/v3/codec/scale_offset.py

def incoming_problems(self, incoming: ArrayParts | None) -> tuple[ValidationProblem, ...]:
    """What the array handed to this codec must be, and what its members must be for it.

    The registry defines the codec for data types with arithmetic and
    lists the integer and floating-point ones. `offset` and `scale`
    are each "encoded to JSON using the Zarr V3 fill value encoding
    for the input array's data type" -- the type that reaches this
    codec, which after a `cast_value` is not the array's own -- so
    each is a fill value of that type, and that type judges it: the
    string `"0"` is no `float32` and no `int32`.
    """
    data_type = incoming.data_type if incoming is not None else None
    if not isinstance(data_type, DataTypeEntity):
        return ()
    if not isinstance(data_type, (IntegerDataType, FloatDataType)):
        return problem(
            (),
            "scale_offset is defined for integer and floating-point data types, not "
            f"{type(data_type).identifier!r}",
            "invalid_value",
        )
    return tuple(
        found
        for member, value in (
            ("offset", self.configuration.offset),
            ("scale", self.configuration.scale),
        )
        if value is not UNSET
        for found in data_type.fill_value_problems(value, (member,))
    )

inner_pipelines

inner_pipelines(
    incoming: ArrayParts | None,
) -> Mapping[
    str,
    tuple[
        Sequence[CodecEntity | Opaque], ArrayParts | None
    ],
]

The pipelines this codec holds, by the member holding each, with what each is handed.

A shard holds two: its codecs, handed its inner chunk, and its index_codecs, handed the shard index. Refinement walks them as it walks the pipeline this codec stands in, locating what it finds under the member, so a codec that holds pipelines says which and what they receive, and judges nothing inside them itself. Default: none.

Source code in src/zarr_metadata/v3/_entity.py

def inner_pipelines(
    self, incoming: ArrayParts | None
) -> Mapping[str, tuple[Sequence[CodecEntity | Opaque], ArrayParts | None]]:
    """The pipelines this codec holds, by the member holding each, with what each is handed.

    A shard holds two: its `codecs`, handed its inner chunk, and its
    `index_codecs`, handed the shard index. Refinement walks them as
    it walks the pipeline this codec stands in, locating what it
    finds under the member, so a codec that holds pipelines says
    which and what they receive, and judges nothing inside them
    itself. Default: none.
    """
    return {}

name_problems classmethod

name_problems(name: str) -> Iterator[ValidationProblem]

Why name, which accepts claimed, is not a well-formed name of this family.

For a family, whose names carry data -- r<N> -- and which claims a malformed member so that it is reported rather than waved through as an unknown extension. Locations are relative to the entity: (). Default: none, for an entity of one name.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def name_problems(cls, name: str) -> Iterator[ValidationProblem]:
    """Why `name`, which `accepts` claimed, is not a well-formed name of this family.

    For a family, whose names carry data -- `r<N>` -- and which claims
    a malformed member so that it is reported rather than waved
    through as an unknown extension. Locations are relative to the
    entity: `()`. Default: none, for an entity of one name.
    """
    yield from ()

to_json

to_json() -> ZarrV3MetadataFieldJSON

This entity as a document would write it.

Written from the configuration record by the same declaration coerce reads it by, each member by the writer its annotation implies: the bare name when every member it holds is absent, the object otherwise, a contained entity through its own to_json, a JSON-valued member copied so the document is not a handle on the entity. Faithful to every member: read a document, write it back, and those come out as they went in. The envelope is written the entity's way -- the bare name when nothing is configured, the object otherwise, no must_understand, which means what absence means -- because an entity alone has no document to be faithful to; ArrayDocumentV3.to_json puts back the spelling the document used. Ask canonical first if you want the simplest equivalent spelling.

An entity whose JSON is not its fields overrides this; none in the package does.

Source code in src/zarr_metadata/v3/_entity.py

def to_json(self) -> ZarrV3MetadataFieldJSON:
    """This entity as a document would write it.

    Written from the configuration record by the same declaration
    `coerce` reads it by, each member by the writer its annotation
    implies: the bare name when every member it holds is absent, the
    object otherwise, a contained entity through its own `to_json`,
    a JSON-valued member copied so the document is not a handle on
    the entity. Faithful to every member: read a document, write it
    back, and those come out as they went in. The envelope is
    written the entity's way -- the bare name when nothing is
    configured, the object otherwise, no `must_understand`, which
    means what absence means -- because an entity alone has no
    document to be faithful to; `ArrayDocumentV3.to_json` puts back
    the spelling the document used. Ask `canonical` first if you
    want the simplest equivalent spelling.

    An entity whose JSON is not its fields overrides this; none in
    the package does.
    """
    plan = _plan(type(self))
    name = self.identifier
    if plan.from_name is not None:
        carried = getattr(self, plan.from_name)
        name = carried if isinstance(carried, str) else name
    configuration = plan.write(self)
    if len(configuration) == 0:
        return name
    return {"name": name, "configuration": configuration}

transition

transition(incoming: ArrayParts) -> ArrayParts | None

The same array, element for element.

The registry entry removed the astype field, so this codec no longer changes the element type -- only the values.

Source code in src/zarr_metadata/v3/codec/scale_offset.py

def transition(self, incoming: ArrayParts) -> ArrayParts | None:
    """The same array, element for element.

    The registry entry removed the `astype` field, so this codec no
    longer changes the element type -- only the values.
    """
    return incoming

with_configuration

with_configuration(**changes: object) -> Self

This entity with these configuration members changed.

codec.with_configuration(typesize=UNSET) is the record rebuilt through its constructor, which refuses a member of the wrong type, and the entity rebuilt through its own, which refuses a value the rules disallow -- the same checks as any construction, since pyright cannot see the members through **changes. A name that is not a member is refused the way replace refuses it.

Source code in src/zarr_metadata/v3/_entity.py

def with_configuration(self, **changes: object) -> Self:
    """This entity with these configuration members changed.

    `codec.with_configuration(typesize=UNSET)` is the record rebuilt
    through its constructor, which refuses a member of the wrong
    type, and the entity rebuilt through its own, which refuses a
    value the rules disallow -- the same checks as any construction,
    since pyright cannot see the members through `**changes`. A
    name that is not a member is refused the way `replace` refuses
    it.
    """
    return replace(self, configuration=replace(self.configuration, **changes))

ScaleOffsetCodecConfiguration

Bases: TypedDict

Configuration for the Zarr v3 scale_offset codec.

Both fields are optional. A missing offset is the additive identity (e.g. 0 for numeric types); a missing scale is the multiplicative identity (e.g. 1). Each scalar is JSON-encoded per the input array's fill-value rules, so "NaN" and "+Infinity" style strings are permitted in addition to numbers.

Source code in src/zarr_metadata/v3/codec/scale_offset.py

class ScaleOffsetCodecConfiguration(TypedDict, closed=True):
    """
    Configuration for the Zarr v3 `scale_offset` codec.

    Both fields are optional. A missing `offset` is the additive identity
    (e.g. 0 for numeric types); a missing `scale` is the multiplicative
    identity (e.g. 1). Each scalar is JSON-encoded per the input array's
    fill-value rules, so `"NaN"` and `"+Infinity"` style strings are
    permitted in addition to numbers.
    """

    offset: NotRequired[JSONValue]
    scale: NotRequired[JSONValue]

offset instance-attribute

offset: NotRequired[JSONValue]

scale instance-attribute

scale: NotRequired[JSONValue]

ScaleOffsetCodecObject

Bases: TypedDict

scale_offset codec metadata in object form.

configuration is itself optional per spec — when both offset and scale are at their identity defaults, the codec is a no-op and the entire configuration field may be omitted. https://github.com/zarr-developers/zarr-extensions/blob/4da7b37a84f76e660902f6d3de3eaef0e0febae6/codecs/scale_offset/README.md#L18 and #L35

Source code in src/zarr_metadata/v3/codec/scale_offset.py

class ScaleOffsetCodecObject(TypedDict, closed=True):
    """`scale_offset` codec metadata in object form.

    `configuration` is itself optional per spec — when both `offset` and
    `scale` are at their identity defaults, the codec is a no-op and the
    entire `configuration` field may be omitted.
      https://github.com/zarr-developers/zarr-extensions/blob/4da7b37a84f76e660902f6d3de3eaef0e0febae6/codecs/scale_offset/README.md#L18 and #L35
    """

    name: ScaleOffsetCodecName
    configuration: NotRequired[ScaleOffsetCodecConfiguration]
    must_understand: NotRequired[bool]

configuration instance-attribute

configuration: NotRequired[ScaleOffsetCodecConfiguration]

must_understand instance-attribute

must_understand: NotRequired[bool]

name instance-attribute

name: ScaleOffsetCodecName

ScaleOffsetOptions dataclass

Bases: Configuration

What scale_offset is configured with.

Source code in src/zarr_metadata/v3/codec/scale_offset.py

@dataclass(frozen=True)
class ScaleOffsetOptions(Configuration):
    """What `scale_offset` is configured with."""

    offset: JSONValue | UNSET = UNSET
    scale: JSONValue | UNSET = UNSET

    def problems(self) -> "Iterator[ValidationProblem]":
        """Each value is a scalar of the array's type, so neither is null.

        The registry says each is "JSON-encoded per the input array's
        fill-value rules", and no data type admits `null` as a fill value.
        Which scalar it should be needs the data type, so that part is the
        document's question, not this codec's.
        """
        if self.offset is None:
            yield ValidationProblem(("offset",), "expected a scalar, got null", "invalid_value")
        if self.scale is None:
            yield ValidationProblem(("scale",), "expected a scalar, got null", "invalid_value")

offset class-attribute instance-attribute

offset: JSONValue | UNSET = UNSET

scale class-attribute instance-attribute

scale: JSONValue | UNSET = UNSET

__init__

__init__(
    offset: JSONValue | UNSET = UNSET,
    scale: JSONValue | UNSET = UNSET,
) -> None

__post_init__

__post_init__() -> None

Refuse every member of the wrong type, so GzipOptions(level="high") raises.

Source code in src/zarr_metadata/v3/_entity.py

def __post_init__(self) -> None:
    """Refuse every member of the wrong type, so `GzipOptions(level="high")` raises."""
    found = mistyped(self)
    if len(found) != 0:
        raise MetadataValidationError(found)

create_unchecked classmethod

create_unchecked(**members: object) -> Self

This record with these members, built without the constructor's check.

The one way around the check, for a caller that has just made it: the parser, which type-checked every member against the same annotations before building the record. Every field is given -- the parser gives an absent optional member as UNSET -- since nothing here applies a default. Anything that has not checked the members goes through the constructor.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def create_unchecked(cls, **members: object) -> Self:
    """This record with these members, built without the constructor's check.

    The one way around the check, for a caller that has just made
    it: the parser, which type-checked every member against the same
    annotations before building the record. Every field is given --
    the parser gives an absent optional member as `UNSET` -- since
    nothing here applies a default. Anything that has not checked
    the members goes through the constructor.
    """
    record = object.__new__(cls)
    for name, value in members.items():
        object.__setattr__(record, name, value)
    return record

problems

problems() -> Iterator[ValidationProblem]

Each value is a scalar of the array's type, so neither is null.

The registry says each is "JSON-encoded per the input array's fill-value rules", and no data type admits null as a fill value. Which scalar it should be needs the data type, so that part is the document's question, not this codec's.

Source code in src/zarr_metadata/v3/codec/scale_offset.py

def problems(self) -> "Iterator[ValidationProblem]":
    """Each value is a scalar of the array's type, so neither is null.

    The registry says each is "JSON-encoded per the input array's
    fill-value rules", and no data type admits `null` as a fill value.
    Which scalar it should be needs the data type, so that part is the
    document's question, not this codec's.
    """
    if self.offset is None:
        yield ValidationProblem(("offset",), "expected a scalar, got null", "invalid_value")
    if self.scale is None:
        yield ValidationProblem(("scale",), "expected a scalar, got null", "invalid_value")

zarr_metadata.v3.codec.sharding_indexed

Sharding-indexed codec types.

See https://zarr-specs.readthedocs.io/en/latest/v3/codecs/sharding-indexed/index.html

SHARDING_INDEXED_CODEC_NAME module-attribute

SHARDING_INDEXED_CODEC_NAME: Final = 'sharding_indexed'

The name field value of the sharding_indexed codec.

SHARDING_INDEX_LOCATION module-attribute

SHARDING_INDEX_LOCATION: Final = ('start', 'end')

Tuple of permitted values for the index_location field of the sharding_indexed codec.

ShardingIndexLocation module-attribute

ShardingIndexLocation = Literal['start', 'end']

Literal type of the position of the shard index within the encoded shard.

ShardingIndexedCodecMetadata module-attribute

ShardingIndexedCodecMetadata = ShardingIndexedCodecObject

Permitted JSON shape for sharding_indexed codec metadata.

The configuration has multiple required keys (chunk_shape, codecs, index_codecs), so only the object form is valid; the short-hand-name form is not permitted by the spec for this codec. https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/codecs/sharding-indexed/index.rst#L141-L155 (required members) https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L1562-L1564 (short-hand names only "if no configuration metadata is required")

ShardingIndexedCodecName module-attribute

ShardingIndexedCodecName = Literal['sharding_indexed']

Literal type of the name field of the sharding_indexed codec.

__all__ module-attribute

__all__ = [
    "SHARDING_INDEXED_CODEC_NAME",
    "SHARDING_INDEX_LOCATION",
    "ShardingIndexLocation",
    "ShardingIndexedCodec",
    "ShardingIndexedCodecConfiguration",
    "ShardingIndexedCodecMetadata",
    "ShardingIndexedCodecName",
    "ShardingIndexedCodecObject",
    "ShardingIndexedOptions",
]

ShardingIndexedCodec dataclass

Bases: ArrayBytesCodec

The sharding_indexed codec, coerced from its metadata.

Holds two codec pipelines, so it is one of the few entities that needs the scope it is being read in: an entry of either pipeline is itself an entity, read the same way this one was.

Source code in src/zarr_metadata/v3/codec/sharding_indexed.py

@dataclass(frozen=True)
class ShardingIndexedCodec(ArrayBytesCodec):
    """The `sharding_indexed` codec, coerced from its metadata.

    Holds two codec pipelines, so it is one of the few entities that
    needs the scope it is being read in: an entry of either pipeline is
    itself an entity, read the same way this one was.
    """

    configuration: ShardingIndexedOptions

    identifier: ClassVar[str] = SHARDING_INDEXED_CODEC_NAME
    variable_size: ClassVar[bool] = True

    def canonical(self) -> Self:
        """Each pipeline's codecs in their own canonical form."""
        return self.with_configuration(
            codecs=tuple(codec.canonical() for codec in self.configuration.codecs),
            index_codecs=tuple(codec.canonical() for codec in self.configuration.index_codecs),
        )

    def incoming_problems(self, incoming: ArrayParts | None) -> tuple[ValidationProblem, ...]:
        """This shard against the array reaching it.

        One sharding configuration encodes every chunk, so its inner
        shape has to divide all of them. Under a rectilinear grid an axis
        has several lengths and the inner extent must divide each; an axis
        whose lengths are unknown declines while the others are judged.
        The index must be readable from metadata alone, so no codec of
        variable output size may encode it. The two pipelines are
        `inner_pipelines`, refined by the walk.
        """
        return (
            *self._inner_chunk_problems(incoming),
            *(
                ValidationProblem(
                    ("index_codecs", index),
                    f"{type(codec).identifier!r} produces variable-size output; "
                    "index_codecs must be fixed-size",
                    "invalid_value",
                )
                for index, codec in enumerate(self.configuration.index_codecs)
                if isinstance(codec, CodecEntity) and type(codec).variable_size
            ),
        )

    def inner_pipelines(
        self, incoming: ArrayParts | None
    ) -> "Mapping[str, tuple[Sequence[CodecEntity | Opaque], ArrayParts | None]]":
        """The inner chunk pipeline and the index pipeline, with what each is handed.

        Both start from this codec's own configuration and from the
        spec, so neither waits on what reached the codec. An unreadable
        codec upstream costs the element type and the enclosing extents;
        it does not make the inner chunk shape unknown, and the index is
        a `uint64` array whatever precedes it.
        """
        outer = incoming.grid if incoming is not None else UNKNOWN_GRID
        return {
            "codecs": (
                self.configuration.codecs,
                ArrayParts(
                    ChunkGrid.regular(self.configuration.chunk_shape),
                    incoming.data_type if incoming is not None else None,
                ),
            ),
            "index_codecs": (
                self.configuration.index_codecs,
                ArrayParts(
                    shard_index_grid(outer, self.configuration.chunk_shape), Uint64DataType()
                ),
            ),
        }

    def _inner_chunk_problems(self, incoming: ArrayParts | None) -> tuple[ValidationProblem, ...]:
        """Whether the inner chunk divides every chunk this shard receives."""
        if incoming is None or incoming.grid.rank is None:
            return ()
        if len(self.configuration.chunk_shape) != incoming.grid.rank:
            return problem(
                ("chunk_shape",),
                f"chunk_shape has {len(self.configuration.chunk_shape)} entries but the incoming array "
                f"has {incoming.grid.rank} dimensions",
                "invalid_value",
            )
        found: list[ValidationProblem] = []
        for position, extent in enumerate(self.configuration.chunk_shape):
            lengths = incoming.grid.axis(position)
            if lengths is None or extent < 1:
                continue
            indivisible = sorted(length for length in lengths if length % extent != 0)
            if len(indivisible) != 0:
                found.extend(
                    problem(
                        ("chunk_shape", position),
                        f"inner chunk extent {extent} does not evenly divide the incoming "
                        f"extent {indivisible[0]}",
                        "invalid_value",
                    )
                )
        return tuple(found)

configuration instance-attribute

configuration: ShardingIndexedOptions

The record of this entity's members.

An entity with members narrows it to its own record, configuration: GzipOptions, its one positional argument. An entity of a bare name defaults it to the empty record -- configuration: Configuration = field(default_factory=Configuration) -- so that Crc32cCodec() builds; coerce passes the record either way.

identifier class-attribute

identifier: str = SHARDING_INDEXED_CODEC_NAME

The name this entity is registered under.

Usually the name the metadata carries. The raw-bytes data types are the exception: every r<N> spelling is one family, so the family gets an invented identifier that no real name can collide with.

variable_size class-attribute

variable_size: bool = True

Whether this codec's output size depends on the bytes it is given.

A compressor's does, so a shard index encoded with one has no size derivable from metadata alone, and the shard cannot be read. Every codec says, because a default in either direction is a verdict.

__init__

__init__(configuration: ShardingIndexedOptions) -> None

__post_init__

__post_init__() -> None

Refuse a record that is not this entity's own, then the first problem the rules find.

The runtime half of the entity's type, as the record's constructor is of the record's: GzipCodec(BloscOptions(...)) and a family member carrying a name that is not a string are refused before any rule reads them. Then BloscCodec(BloscOptions(clevel=99)) raises on the first problem the rules yield.

Source code in src/zarr_metadata/v3/_entity.py

def __post_init__(self) -> None:
    """Refuse a record that is not this entity's own, then the first problem the rules find.

    The runtime half of the entity's type, as the record's constructor
    is of the record's: `GzipCodec(BloscOptions(...))` and a family
    member carrying a name that is not a string are refused before
    any rule reads them. Then `BloscCodec(BloscOptions(clevel=99))`
    raises on the first problem the rules yield.
    """
    plan = _plan(type(self))
    if not isinstance(self.configuration, plan.record):
        raise MetadataValidationError(
            problem(
                (),
                f"expected a {plan.record.__name__} configuration, got "
                f"{type(self.configuration).__name__}",
            )
        )
    name: object = self.identifier if plan.from_name is None else getattr(self, plan.from_name)
    if not isinstance(name, str):
        raise MetadataValidationError(problem((), f"expected a string name, got {name!r}"))
    first = next(type(self).name_problems(name), None)
    if first is None:
        first = next(self.configuration.problems(), None)
    if first is not None:
        raise MetadataValidationError((first,))

accepts classmethod

accepts(name: str) -> bool

Whether name denotes this entity.

Constant for all but the raw-bytes family, where one class covers every r<N>.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def accepts(cls, name: str) -> bool:
    """Whether `name` denotes this entity.

    Constant for all but the raw-bytes family, where one class covers
    every `r<N>`.
    """
    return name == cls.identifier

canonical

canonical() -> Self

Each pipeline's codecs in their own canonical form.

Source code in src/zarr_metadata/v3/codec/sharding_indexed.py

def canonical(self) -> Self:
    """Each pipeline's codecs in their own canonical form."""
    return self.with_configuration(
        codecs=tuple(codec.canonical() for codec in self.configuration.codecs),
        index_codecs=tuple(codec.canonical() for codec in self.configuration.index_codecs),
    )

coerce classmethod

coerce(value: JSONValue, context: Context) -> Coerced[Self]

value as this entity, or the reasons it is not one: the class's validation routine.

resolve relates a field's name to this class and hands it the field, refined JSON with arrays as tuples, which is what value is; this is what the class does with it. The configuration is parsed against the record the configuration field names, member by member; a member holding another entity is read in context, the scope this reading is happening in. An optional member the document left out is UNSET in the record, so no field's default decides what a document said. The entity is built only when every member of its own read -- its rules are written over a whole configuration -- and handed back only when everything inside it read too.

The envelope is the field's, not the class's, and resolve judges it: a stray member or a must_understand of false is not reported here. Called on a class no scope has registered, this runs with none of registration's refusals having happened.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def coerce(cls, value: JSONValue, context: Context) -> Coerced[Self]:
    """`value` as this entity, or the reasons it is not one: the class's validation routine.

    `resolve` relates a field's name to this class and hands it the
    field, refined JSON with arrays as tuples, which is what `value`
    is; this is what the class does with it. The configuration is
    parsed against the record the `configuration` field names,
    member by member; a member holding another entity is read in
    `context`, the scope this reading is happening in. An optional
    member the document left out is `UNSET` in the record, so no
    field's default decides what a document said. The entity is
    built only when every member of its own read -- its rules are
    written over a whole configuration -- and handed back only when
    everything inside it read too.

    The envelope is the field's, not the class's, and `resolve`
    judges it: a stray member or a `must_understand` of `false` is
    not reported here. Called on a class no scope has registered,
    this runs with none of registration's refusals having happened.
    """
    name, given, envelope = named_configuration(value)
    if name is None or not cls.accepts(name):
        return None, problem((), f"expected the {cls.identifier!r} entity")
    if len(envelope) != 0:
        return None, envelope
    plan = _plan(cls)
    carried: dict[str, str] = {} if plan.from_name is None else {plan.from_name: name}
    if given is None and plan.requires_configuration:
        return None, problem(
            ("configuration",),
            f"{cls.identifier!r} requires a configuration",
            "missing_key",
        )
    reading = _Reading(context, [])
    # A bare name's record has no members, so any key is an unknown one.
    record, own = plan.read({} if given is None else given, ("configuration",), reading)
    found = (*own, *reading.nested)
    if record is None:
        # An unknown key is survivable; a member that could not be
        # read is a hole, and judging around it would be guessing.
        return None, found
    # The rules, asked of the name and of the record before anything
    # is built: a name problem lands on the entity, a configuration
    # problem under the configuration.
    refused = (*cls.name_problems(name), *within((), tuple(record.problems())))
    if len(refused) != 0:
        # Values the spec disallows: reported rather than raised,
        # every one.
        return None, (*found, *refused)
    if any(entry.kind != "unknown_key" for entry in reading.nested):
        # A contained entity could not be read. This entity's own
        # rules ran -- an invalid inner is an `Opaque`, as an
        # out-of-scope one is -- but what is handed back is not an
        # entity that would be asked composition questions it cannot
        # answer.
        return None, found
    return cls.create_unchecked(configuration=record, **carried), found

create_unchecked classmethod

create_unchecked(**fields: object) -> Self

This entity with these fields, built without the constructor's checks.

The one way around them, for a caller that has just made them: coerce, which type-checked the record and ran the rules before building. Every field is given -- the record, and the carried name for a family -- since nothing here applies a default. Anything that has not checked goes through the constructor.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def create_unchecked(cls, **fields: object) -> Self:
    """This entity with these fields, built without the constructor's checks.

    The one way around them, for a caller that has just made them:
    `coerce`, which type-checked the record and ran the rules before
    building. Every field is given -- the record, and the carried
    name for a family -- since nothing here applies a default.
    Anything that has not checked goes through the constructor.
    """
    entity = object.__new__(cls)
    for name, value in fields.items():
        object.__setattr__(entity, name, value)
    return entity

incoming_problems

incoming_problems(
    incoming: ArrayParts | None,
) -> tuple[ValidationProblem, ...]

This shard against the array reaching it.

One sharding configuration encodes every chunk, so its inner shape has to divide all of them. Under a rectilinear grid an axis has several lengths and the inner extent must divide each; an axis whose lengths are unknown declines while the others are judged. The index must be readable from metadata alone, so no codec of variable output size may encode it. The two pipelines are inner_pipelines, refined by the walk.

Source code in src/zarr_metadata/v3/codec/sharding_indexed.py

def incoming_problems(self, incoming: ArrayParts | None) -> tuple[ValidationProblem, ...]:
    """This shard against the array reaching it.

    One sharding configuration encodes every chunk, so its inner
    shape has to divide all of them. Under a rectilinear grid an axis
    has several lengths and the inner extent must divide each; an axis
    whose lengths are unknown declines while the others are judged.
    The index must be readable from metadata alone, so no codec of
    variable output size may encode it. The two pipelines are
    `inner_pipelines`, refined by the walk.
    """
    return (
        *self._inner_chunk_problems(incoming),
        *(
            ValidationProblem(
                ("index_codecs", index),
                f"{type(codec).identifier!r} produces variable-size output; "
                "index_codecs must be fixed-size",
                "invalid_value",
            )
            for index, codec in enumerate(self.configuration.index_codecs)
            if isinstance(codec, CodecEntity) and type(codec).variable_size
        ),
    )

inner_pipelines

inner_pipelines(
    incoming: ArrayParts | None,
) -> Mapping[
    str,
    tuple[
        Sequence[CodecEntity | Opaque], ArrayParts | None
    ],
]

The inner chunk pipeline and the index pipeline, with what each is handed.

Both start from this codec's own configuration and from the spec, so neither waits on what reached the codec. An unreadable codec upstream costs the element type and the enclosing extents; it does not make the inner chunk shape unknown, and the index is a uint64 array whatever precedes it.

Source code in src/zarr_metadata/v3/codec/sharding_indexed.py

def inner_pipelines(
    self, incoming: ArrayParts | None
) -> "Mapping[str, tuple[Sequence[CodecEntity | Opaque], ArrayParts | None]]":
    """The inner chunk pipeline and the index pipeline, with what each is handed.

    Both start from this codec's own configuration and from the
    spec, so neither waits on what reached the codec. An unreadable
    codec upstream costs the element type and the enclosing extents;
    it does not make the inner chunk shape unknown, and the index is
    a `uint64` array whatever precedes it.
    """
    outer = incoming.grid if incoming is not None else UNKNOWN_GRID
    return {
        "codecs": (
            self.configuration.codecs,
            ArrayParts(
                ChunkGrid.regular(self.configuration.chunk_shape),
                incoming.data_type if incoming is not None else None,
            ),
        ),
        "index_codecs": (
            self.configuration.index_codecs,
            ArrayParts(
                shard_index_grid(outer, self.configuration.chunk_shape), Uint64DataType()
            ),
        ),
    }

name_problems classmethod

name_problems(name: str) -> Iterator[ValidationProblem]

Why name, which accepts claimed, is not a well-formed name of this family.

For a family, whose names carry data -- r<N> -- and which claims a malformed member so that it is reported rather than waved through as an unknown extension. Locations are relative to the entity: (). Default: none, for an entity of one name.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def name_problems(cls, name: str) -> Iterator[ValidationProblem]:
    """Why `name`, which `accepts` claimed, is not a well-formed name of this family.

    For a family, whose names carry data -- `r<N>` -- and which claims
    a malformed member so that it is reported rather than waved
    through as an unknown extension. Locations are relative to the
    entity: `()`. Default: none, for an entity of one name.
    """
    yield from ()

to_json

to_json() -> ZarrV3MetadataFieldJSON

This entity as a document would write it.

Written from the configuration record by the same declaration coerce reads it by, each member by the writer its annotation implies: the bare name when every member it holds is absent, the object otherwise, a contained entity through its own to_json, a JSON-valued member copied so the document is not a handle on the entity. Faithful to every member: read a document, write it back, and those come out as they went in. The envelope is written the entity's way -- the bare name when nothing is configured, the object otherwise, no must_understand, which means what absence means -- because an entity alone has no document to be faithful to; ArrayDocumentV3.to_json puts back the spelling the document used. Ask canonical first if you want the simplest equivalent spelling.

An entity whose JSON is not its fields overrides this; none in the package does.

Source code in src/zarr_metadata/v3/_entity.py

def to_json(self) -> ZarrV3MetadataFieldJSON:
    """This entity as a document would write it.

    Written from the configuration record by the same declaration
    `coerce` reads it by, each member by the writer its annotation
    implies: the bare name when every member it holds is absent, the
    object otherwise, a contained entity through its own `to_json`,
    a JSON-valued member copied so the document is not a handle on
    the entity. Faithful to every member: read a document, write it
    back, and those come out as they went in. The envelope is
    written the entity's way -- the bare name when nothing is
    configured, the object otherwise, no `must_understand`, which
    means what absence means -- because an entity alone has no
    document to be faithful to; `ArrayDocumentV3.to_json` puts back
    the spelling the document used. Ask `canonical` first if you
    want the simplest equivalent spelling.

    An entity whose JSON is not its fields overrides this; none in
    the package does.
    """
    plan = _plan(type(self))
    name = self.identifier
    if plan.from_name is not None:
        carried = getattr(self, plan.from_name)
        name = carried if isinstance(carried, str) else name
    configuration = plan.write(self)
    if len(configuration) == 0:
        return name
    return {"name": name, "configuration": configuration}

with_configuration

with_configuration(**changes: object) -> Self

This entity with these configuration members changed.

codec.with_configuration(typesize=UNSET) is the record rebuilt through its constructor, which refuses a member of the wrong type, and the entity rebuilt through its own, which refuses a value the rules disallow -- the same checks as any construction, since pyright cannot see the members through **changes. A name that is not a member is refused the way replace refuses it.

Source code in src/zarr_metadata/v3/_entity.py

def with_configuration(self, **changes: object) -> Self:
    """This entity with these configuration members changed.

    `codec.with_configuration(typesize=UNSET)` is the record rebuilt
    through its constructor, which refuses a member of the wrong
    type, and the entity rebuilt through its own, which refuses a
    value the rules disallow -- the same checks as any construction,
    since pyright cannot see the members through `**changes`. A
    name that is not a member is refused the way `replace` refuses
    it.
    """
    return replace(self, configuration=replace(self.configuration, **changes))

ShardingIndexedCodecConfiguration

Bases: TypedDict

Configuration for the Zarr v3 sharding_indexed codec.

chunk_shape is the shape of inner chunks along each dimension; it must evenly divide the shard shape.

codecs is the codec pipeline applied to each inner chunk; exactly one array-to-bytes codec is required.

index_codecs is the codec pipeline applied to the shard index; it must be deterministic (no variable-size compression). https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/codecs/sharding-indexed/index.rst#L147-L155

index_location defaults to "end" per the spec. https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/codecs/sharding-indexed/index.rst#L157-L161

Source code in src/zarr_metadata/v3/codec/sharding_indexed.py

class ShardingIndexedCodecConfiguration(TypedDict, closed=True):
    """
    Configuration for the Zarr v3 `sharding_indexed` codec.

    `chunk_shape` is the shape of inner chunks along each dimension;
    it must evenly divide the shard shape.

    `codecs` is the codec pipeline applied to each inner chunk; exactly
    one array-to-bytes codec is required.

    `index_codecs` is the codec pipeline applied to the shard index;
    it must be deterministic (no variable-size compression).
      https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/codecs/sharding-indexed/index.rst#L147-L155

    `index_location` defaults to `"end"` per the spec.
      https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/codecs/sharding-indexed/index.rst#L157-L161
    """

    chunk_shape: tuple[int, ...]
    codecs: tuple[ZarrV3MetadataFieldJSON, ...]
    index_codecs: tuple[ZarrV3MetadataFieldJSON, ...]
    index_location: NotRequired[ShardingIndexLocation]

chunk_shape instance-attribute

chunk_shape: tuple[int, ...]

codecs instance-attribute

codecs: tuple[ZarrV3MetadataFieldJSON, ...]

index_codecs instance-attribute

index_codecs: tuple[ZarrV3MetadataFieldJSON, ...]

index_location instance-attribute

index_location: NotRequired[ShardingIndexLocation]

ShardingIndexedCodecObject

Bases: TypedDict

sharding_indexed codec metadata in object form.

Source code in src/zarr_metadata/v3/codec/sharding_indexed.py

class ShardingIndexedCodecObject(TypedDict, closed=True):
    """`sharding_indexed` codec metadata in object form."""

    name: ShardingIndexedCodecName
    configuration: ShardingIndexedCodecConfiguration
    must_understand: NotRequired[bool]

configuration instance-attribute

configuration: ShardingIndexedCodecConfiguration

must_understand instance-attribute

must_understand: NotRequired[bool]

name instance-attribute

name: ShardingIndexedCodecName

ShardingIndexedOptions dataclass

Bases: Configuration

What sharding_indexed is configured with.

Source code in src/zarr_metadata/v3/codec/sharding_indexed.py

@dataclass(frozen=True)
class ShardingIndexedOptions(Configuration):
    """What `sharding_indexed` is configured with."""

    chunk_shape: tuple[int, ...]
    codecs: tuple[CodecEntity | Opaque, ...]
    index_codecs: tuple[CodecEntity | Opaque, ...]
    index_location: ShardingIndexLocation | UNSET = UNSET

    def problems(self) -> "Iterator[ValidationProblem]":
        for index, extent in enumerate(self.chunk_shape):
            if extent < 1:
                yield ValidationProblem(
                    ("chunk_shape", index),
                    f"expected an integer >= 1, got {extent}",
                    "invalid_value",
                )

chunk_shape instance-attribute

chunk_shape: tuple[int, ...]

codecs instance-attribute

codecs: tuple[CodecEntity | Opaque, ...]

index_codecs instance-attribute

index_codecs: tuple[CodecEntity | Opaque, ...]

index_location class-attribute instance-attribute

index_location: ShardingIndexLocation | UNSET = UNSET

__init__

__init__(
    chunk_shape: tuple[int, ...],
    codecs: tuple[CodecEntity | Opaque, ...],
    index_codecs: tuple[CodecEntity | Opaque, ...],
    index_location: ShardingIndexLocation | UNSET = UNSET,
) -> None

__post_init__

__post_init__() -> None

Refuse every member of the wrong type, so GzipOptions(level="high") raises.

Source code in src/zarr_metadata/v3/_entity.py

def __post_init__(self) -> None:
    """Refuse every member of the wrong type, so `GzipOptions(level="high")` raises."""
    found = mistyped(self)
    if len(found) != 0:
        raise MetadataValidationError(found)

create_unchecked classmethod

create_unchecked(**members: object) -> Self

This record with these members, built without the constructor's check.

The one way around the check, for a caller that has just made it: the parser, which type-checked every member against the same annotations before building the record. Every field is given -- the parser gives an absent optional member as UNSET -- since nothing here applies a default. Anything that has not checked the members goes through the constructor.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def create_unchecked(cls, **members: object) -> Self:
    """This record with these members, built without the constructor's check.

    The one way around the check, for a caller that has just made
    it: the parser, which type-checked every member against the same
    annotations before building the record. Every field is given --
    the parser gives an absent optional member as `UNSET` -- since
    nothing here applies a default. Anything that has not checked
    the members goes through the constructor.
    """
    record = object.__new__(cls)
    for name, value in members.items():
        object.__setattr__(record, name, value)
    return record

problems

problems() -> Iterator[ValidationProblem]

Every reason these values are not allowed, yielded as found. Default: none.

Source code in src/zarr_metadata/v3/codec/sharding_indexed.py

def problems(self) -> "Iterator[ValidationProblem]":
    for index, extent in enumerate(self.chunk_shape):
        if extent < 1:
            yield ValidationProblem(
                ("chunk_shape", index),
                f"expected an integer >= 1, got {extent}",
                "invalid_value",
            )

zarr_metadata.v3.codec.transpose

Transpose codec types.

See https://zarr-specs.readthedocs.io/en/latest/v3/codecs/transpose/index.html

TRANSPOSE_CODEC_NAME module-attribute

TRANSPOSE_CODEC_NAME: Final = 'transpose'

The name field value of the transpose codec.

TransposeCodecMetadata module-attribute

TransposeCodecMetadata = TransposeCodecObject

Permitted JSON shape for transpose codec metadata.

order is required, so only the object form is valid; the short-hand-name form is not permitted by the spec for this codec. https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/codecs/transpose/index.rst#L60-L66 ("order: Required") https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L1562-L1564 (short-hand names only "if no configuration metadata is required")

TransposeCodecName module-attribute

TransposeCodecName = Literal['transpose']

Literal type of the name field of the transpose codec.

__all__ module-attribute

__all__ = [
    "TRANSPOSE_CODEC_NAME",
    "TransposeCodec",
    "TransposeCodecConfiguration",
    "TransposeCodecMetadata",
    "TransposeCodecName",
    "TransposeCodecObject",
    "TransposeOptions",
]

TransposeCodec dataclass

Bases: ArrayArrayCodec

The transpose codec, coerced from its metadata.

Source code in src/zarr_metadata/v3/codec/transpose.py

@dataclass(frozen=True)
class TransposeCodec(ArrayArrayCodec):
    """The `transpose` codec, coerced from its metadata."""

    configuration: TransposeOptions

    identifier: ClassVar[str] = TRANSPOSE_CODEC_NAME
    variable_size: ClassVar[bool] = False

    def incoming_problems(self, incoming: ArrayParts | None) -> tuple[ValidationProblem, ...]:
        """A transpose permutes the array it receives, so ranks must agree.

        Judged against what actually reaches this codec: inside a shard
        that is the inner chunk, and after another transpose it is that
        transpose's output.
        """
        rank = incoming.grid.rank if incoming is not None else None
        if rank is None or len(self.configuration.order) == rank:
            return ()
        return problem(
            ("order",),
            f"order has {len(self.configuration.order)} entries but the incoming array has {rank} dimensions",
            "invalid_value",
        )

    def transition(self, incoming: ArrayParts) -> ArrayParts | None:
        """The same array with its axes reordered.

        A transposed regular grid is still a regular grid, so the parts
        survive the trip; the grid metadata does not, because it is no
        longer the grid the document wrote.
        """
        return incoming.with_grid(incoming.grid.permuted(self.configuration.order))

configuration instance-attribute

configuration: TransposeOptions

The record of this entity's members.

An entity with members narrows it to its own record, configuration: GzipOptions, its one positional argument. An entity of a bare name defaults it to the empty record -- configuration: Configuration = field(default_factory=Configuration) -- so that Crc32cCodec() builds; coerce passes the record either way.

identifier class-attribute

identifier: str = TRANSPOSE_CODEC_NAME

The name this entity is registered under.

Usually the name the metadata carries. The raw-bytes data types are the exception: every r<N> spelling is one family, so the family gets an invented identifier that no real name can collide with.

variable_size class-attribute

variable_size: bool = False

Whether this codec's output size depends on the bytes it is given.

A compressor's does, so a shard index encoded with one has no size derivable from metadata alone, and the shard cannot be read. Every codec says, because a default in either direction is a verdict.

__init__

__init__(configuration: TransposeOptions) -> None

__post_init__

__post_init__() -> None

Refuse a record that is not this entity's own, then the first problem the rules find.

The runtime half of the entity's type, as the record's constructor is of the record's: GzipCodec(BloscOptions(...)) and a family member carrying a name that is not a string are refused before any rule reads them. Then BloscCodec(BloscOptions(clevel=99)) raises on the first problem the rules yield.

Source code in src/zarr_metadata/v3/_entity.py

def __post_init__(self) -> None:
    """Refuse a record that is not this entity's own, then the first problem the rules find.

    The runtime half of the entity's type, as the record's constructor
    is of the record's: `GzipCodec(BloscOptions(...))` and a family
    member carrying a name that is not a string are refused before
    any rule reads them. Then `BloscCodec(BloscOptions(clevel=99))`
    raises on the first problem the rules yield.
    """
    plan = _plan(type(self))
    if not isinstance(self.configuration, plan.record):
        raise MetadataValidationError(
            problem(
                (),
                f"expected a {plan.record.__name__} configuration, got "
                f"{type(self.configuration).__name__}",
            )
        )
    name: object = self.identifier if plan.from_name is None else getattr(self, plan.from_name)
    if not isinstance(name, str):
        raise MetadataValidationError(problem((), f"expected a string name, got {name!r}"))
    first = next(type(self).name_problems(name), None)
    if first is None:
        first = next(self.configuration.problems(), None)
    if first is not None:
        raise MetadataValidationError((first,))

accepts classmethod

accepts(name: str) -> bool

Whether name denotes this entity.

Constant for all but the raw-bytes family, where one class covers every r<N>.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def accepts(cls, name: str) -> bool:
    """Whether `name` denotes this entity.

    Constant for all but the raw-bytes family, where one class covers
    every `r<N>`.
    """
    return name == cls.identifier

canonical

canonical() -> Self

This entity in the simplest form that means the same thing.

A transformation, asked for by canonicalize_array_metadata_v3 and by nothing else. to_json does not apply it, because writing a document back is not the same as asking for it to be rewritten: a reader that reads and writes should not change bytes it was not asked to change.

The default is the entity itself. Override it where two spellings of the entity's members mean the same -- a rectilinear dimension's run-length encoding, a typesize that noshuffle ignores -- and, in an entity that contains entities, to put those in canonical form: self.with_configuration(inner=self.inner.canonical()).

Source code in src/zarr_metadata/v3/_entity.py

def canonical(self) -> Self:
    """This entity in the simplest form that means the same thing.

    A *transformation*, asked for by `canonicalize_array_metadata_v3`
    and by nothing else. `to_json` does not apply it, because writing
    a document back is not the same as asking for it to be rewritten:
    a reader that reads and writes should not change bytes it was not
    asked to change.

    The default is the entity itself. Override it where two spellings
    of the entity's members mean the same -- a rectilinear
    dimension's run-length encoding, a `typesize` that `noshuffle`
    ignores -- and, in an entity that contains entities, to put those
    in canonical form: `self.with_configuration(inner=self.inner.canonical())`.
    """
    return self

coerce classmethod

coerce(value: JSONValue, context: Context) -> Coerced[Self]

value as this entity, or the reasons it is not one: the class's validation routine.

resolve relates a field's name to this class and hands it the field, refined JSON with arrays as tuples, which is what value is; this is what the class does with it. The configuration is parsed against the record the configuration field names, member by member; a member holding another entity is read in context, the scope this reading is happening in. An optional member the document left out is UNSET in the record, so no field's default decides what a document said. The entity is built only when every member of its own read -- its rules are written over a whole configuration -- and handed back only when everything inside it read too.

The envelope is the field's, not the class's, and resolve judges it: a stray member or a must_understand of false is not reported here. Called on a class no scope has registered, this runs with none of registration's refusals having happened.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def coerce(cls, value: JSONValue, context: Context) -> Coerced[Self]:
    """`value` as this entity, or the reasons it is not one: the class's validation routine.

    `resolve` relates a field's name to this class and hands it the
    field, refined JSON with arrays as tuples, which is what `value`
    is; this is what the class does with it. The configuration is
    parsed against the record the `configuration` field names,
    member by member; a member holding another entity is read in
    `context`, the scope this reading is happening in. An optional
    member the document left out is `UNSET` in the record, so no
    field's default decides what a document said. The entity is
    built only when every member of its own read -- its rules are
    written over a whole configuration -- and handed back only when
    everything inside it read too.

    The envelope is the field's, not the class's, and `resolve`
    judges it: a stray member or a `must_understand` of `false` is
    not reported here. Called on a class no scope has registered,
    this runs with none of registration's refusals having happened.
    """
    name, given, envelope = named_configuration(value)
    if name is None or not cls.accepts(name):
        return None, problem((), f"expected the {cls.identifier!r} entity")
    if len(envelope) != 0:
        return None, envelope
    plan = _plan(cls)
    carried: dict[str, str] = {} if plan.from_name is None else {plan.from_name: name}
    if given is None and plan.requires_configuration:
        return None, problem(
            ("configuration",),
            f"{cls.identifier!r} requires a configuration",
            "missing_key",
        )
    reading = _Reading(context, [])
    # A bare name's record has no members, so any key is an unknown one.
    record, own = plan.read({} if given is None else given, ("configuration",), reading)
    found = (*own, *reading.nested)
    if record is None:
        # An unknown key is survivable; a member that could not be
        # read is a hole, and judging around it would be guessing.
        return None, found
    # The rules, asked of the name and of the record before anything
    # is built: a name problem lands on the entity, a configuration
    # problem under the configuration.
    refused = (*cls.name_problems(name), *within((), tuple(record.problems())))
    if len(refused) != 0:
        # Values the spec disallows: reported rather than raised,
        # every one.
        return None, (*found, *refused)
    if any(entry.kind != "unknown_key" for entry in reading.nested):
        # A contained entity could not be read. This entity's own
        # rules ran -- an invalid inner is an `Opaque`, as an
        # out-of-scope one is -- but what is handed back is not an
        # entity that would be asked composition questions it cannot
        # answer.
        return None, found
    return cls.create_unchecked(configuration=record, **carried), found

create_unchecked classmethod

create_unchecked(**fields: object) -> Self

This entity with these fields, built without the constructor's checks.

The one way around them, for a caller that has just made them: coerce, which type-checked the record and ran the rules before building. Every field is given -- the record, and the carried name for a family -- since nothing here applies a default. Anything that has not checked goes through the constructor.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def create_unchecked(cls, **fields: object) -> Self:
    """This entity with these fields, built without the constructor's checks.

    The one way around them, for a caller that has just made them:
    `coerce`, which type-checked the record and ran the rules before
    building. Every field is given -- the record, and the carried
    name for a family -- since nothing here applies a default.
    Anything that has not checked goes through the constructor.
    """
    entity = object.__new__(cls)
    for name, value in fields.items():
        object.__setattr__(entity, name, value)
    return entity

incoming_problems

incoming_problems(
    incoming: ArrayParts | None,
) -> tuple[ValidationProblem, ...]

A transpose permutes the array it receives, so ranks must agree.

Judged against what actually reaches this codec: inside a shard that is the inner chunk, and after another transpose it is that transpose's output.

Source code in src/zarr_metadata/v3/codec/transpose.py

def incoming_problems(self, incoming: ArrayParts | None) -> tuple[ValidationProblem, ...]:
    """A transpose permutes the array it receives, so ranks must agree.

    Judged against what actually reaches this codec: inside a shard
    that is the inner chunk, and after another transpose it is that
    transpose's output.
    """
    rank = incoming.grid.rank if incoming is not None else None
    if rank is None or len(self.configuration.order) == rank:
        return ()
    return problem(
        ("order",),
        f"order has {len(self.configuration.order)} entries but the incoming array has {rank} dimensions",
        "invalid_value",
    )

inner_pipelines

inner_pipelines(
    incoming: ArrayParts | None,
) -> Mapping[
    str,
    tuple[
        Sequence[CodecEntity | Opaque], ArrayParts | None
    ],
]

The pipelines this codec holds, by the member holding each, with what each is handed.

A shard holds two: its codecs, handed its inner chunk, and its index_codecs, handed the shard index. Refinement walks them as it walks the pipeline this codec stands in, locating what it finds under the member, so a codec that holds pipelines says which and what they receive, and judges nothing inside them itself. Default: none.

Source code in src/zarr_metadata/v3/_entity.py

def inner_pipelines(
    self, incoming: ArrayParts | None
) -> Mapping[str, tuple[Sequence[CodecEntity | Opaque], ArrayParts | None]]:
    """The pipelines this codec holds, by the member holding each, with what each is handed.

    A shard holds two: its `codecs`, handed its inner chunk, and its
    `index_codecs`, handed the shard index. Refinement walks them as
    it walks the pipeline this codec stands in, locating what it
    finds under the member, so a codec that holds pipelines says
    which and what they receive, and judges nothing inside them
    itself. Default: none.
    """
    return {}

name_problems classmethod

name_problems(name: str) -> Iterator[ValidationProblem]

Why name, which accepts claimed, is not a well-formed name of this family.

For a family, whose names carry data -- r<N> -- and which claims a malformed member so that it is reported rather than waved through as an unknown extension. Locations are relative to the entity: (). Default: none, for an entity of one name.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def name_problems(cls, name: str) -> Iterator[ValidationProblem]:
    """Why `name`, which `accepts` claimed, is not a well-formed name of this family.

    For a family, whose names carry data -- `r<N>` -- and which claims
    a malformed member so that it is reported rather than waved
    through as an unknown extension. Locations are relative to the
    entity: `()`. Default: none, for an entity of one name.
    """
    yield from ()

to_json

to_json() -> ZarrV3MetadataFieldJSON

This entity as a document would write it.

Written from the configuration record by the same declaration coerce reads it by, each member by the writer its annotation implies: the bare name when every member it holds is absent, the object otherwise, a contained entity through its own to_json, a JSON-valued member copied so the document is not a handle on the entity. Faithful to every member: read a document, write it back, and those come out as they went in. The envelope is written the entity's way -- the bare name when nothing is configured, the object otherwise, no must_understand, which means what absence means -- because an entity alone has no document to be faithful to; ArrayDocumentV3.to_json puts back the spelling the document used. Ask canonical first if you want the simplest equivalent spelling.

An entity whose JSON is not its fields overrides this; none in the package does.

Source code in src/zarr_metadata/v3/_entity.py

def to_json(self) -> ZarrV3MetadataFieldJSON:
    """This entity as a document would write it.

    Written from the configuration record by the same declaration
    `coerce` reads it by, each member by the writer its annotation
    implies: the bare name when every member it holds is absent, the
    object otherwise, a contained entity through its own `to_json`,
    a JSON-valued member copied so the document is not a handle on
    the entity. Faithful to every member: read a document, write it
    back, and those come out as they went in. The envelope is
    written the entity's way -- the bare name when nothing is
    configured, the object otherwise, no `must_understand`, which
    means what absence means -- because an entity alone has no
    document to be faithful to; `ArrayDocumentV3.to_json` puts back
    the spelling the document used. Ask `canonical` first if you
    want the simplest equivalent spelling.

    An entity whose JSON is not its fields overrides this; none in
    the package does.
    """
    plan = _plan(type(self))
    name = self.identifier
    if plan.from_name is not None:
        carried = getattr(self, plan.from_name)
        name = carried if isinstance(carried, str) else name
    configuration = plan.write(self)
    if len(configuration) == 0:
        return name
    return {"name": name, "configuration": configuration}

transition

transition(incoming: ArrayParts) -> ArrayParts | None

The same array with its axes reordered.

A transposed regular grid is still a regular grid, so the parts survive the trip; the grid metadata does not, because it is no longer the grid the document wrote.

Source code in src/zarr_metadata/v3/codec/transpose.py

def transition(self, incoming: ArrayParts) -> ArrayParts | None:
    """The same array with its axes reordered.

    A transposed regular grid is still a regular grid, so the parts
    survive the trip; the grid metadata does not, because it is no
    longer the grid the document wrote.
    """
    return incoming.with_grid(incoming.grid.permuted(self.configuration.order))

with_configuration

with_configuration(**changes: object) -> Self

This entity with these configuration members changed.

codec.with_configuration(typesize=UNSET) is the record rebuilt through its constructor, which refuses a member of the wrong type, and the entity rebuilt through its own, which refuses a value the rules disallow -- the same checks as any construction, since pyright cannot see the members through **changes. A name that is not a member is refused the way replace refuses it.

Source code in src/zarr_metadata/v3/_entity.py

def with_configuration(self, **changes: object) -> Self:
    """This entity with these configuration members changed.

    `codec.with_configuration(typesize=UNSET)` is the record rebuilt
    through its constructor, which refuses a member of the wrong
    type, and the entity rebuilt through its own, which refuses a
    value the rules disallow -- the same checks as any construction,
    since pyright cannot see the members through `**changes`. A
    name that is not a member is refused the way `replace` refuses
    it.
    """
    return replace(self, configuration=replace(self.configuration, **changes))

TransposeCodecConfiguration

Bases: TypedDict

Configuration for the Zarr v3 transpose codec.

order is a permutation of the dimension indices 0..n-1 that specifies the dimension reordering applied during encoding.

Source code in src/zarr_metadata/v3/codec/transpose.py

class TransposeCodecConfiguration(TypedDict, closed=True):
    """
    Configuration for the Zarr v3 `transpose` codec.

    `order` is a permutation of the dimension indices 0..n-1 that
    specifies the dimension reordering applied during encoding.
    """

    order: tuple[int, ...]

order instance-attribute

order: tuple[int, ...]

TransposeCodecObject

Bases: TypedDict

transpose codec metadata in object form.

Source code in src/zarr_metadata/v3/codec/transpose.py

class TransposeCodecObject(TypedDict, closed=True):
    """`transpose` codec metadata in object form."""

    name: TransposeCodecName
    configuration: TransposeCodecConfiguration
    must_understand: NotRequired[bool]

configuration instance-attribute

configuration: TransposeCodecConfiguration

must_understand instance-attribute

must_understand: NotRequired[bool]

name instance-attribute

name: TransposeCodecName

TransposeOptions dataclass

Bases: Configuration

What transpose is configured with.

Source code in src/zarr_metadata/v3/codec/transpose.py

@dataclass(frozen=True)
class TransposeOptions(Configuration):
    """What `transpose` is configured with."""

    order: tuple[int, ...]

    def problems(self) -> "Iterator[ValidationProblem]":
        """`order` must permute its own axes.

        Whether it permutes the *array's* axes is a different question -- it
        needs the array's rank -- and the rules layer asks that one.
        """
        if sorted(self.order) != list(range(len(self.order))):
            yield ValidationProblem(
                ("order",),
                f"expected a permutation of 0..{len(self.order) - 1}, got {self.order!r}",
                "invalid_value",
            )

order instance-attribute

order: tuple[int, ...]

__init__

__init__(order: tuple[int, ...]) -> None

__post_init__

__post_init__() -> None

Refuse every member of the wrong type, so GzipOptions(level="high") raises.

Source code in src/zarr_metadata/v3/_entity.py

def __post_init__(self) -> None:
    """Refuse every member of the wrong type, so `GzipOptions(level="high")` raises."""
    found = mistyped(self)
    if len(found) != 0:
        raise MetadataValidationError(found)

create_unchecked classmethod

create_unchecked(**members: object) -> Self

This record with these members, built without the constructor's check.

The one way around the check, for a caller that has just made it: the parser, which type-checked every member against the same annotations before building the record. Every field is given -- the parser gives an absent optional member as UNSET -- since nothing here applies a default. Anything that has not checked the members goes through the constructor.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def create_unchecked(cls, **members: object) -> Self:
    """This record with these members, built without the constructor's check.

    The one way around the check, for a caller that has just made
    it: the parser, which type-checked every member against the same
    annotations before building the record. Every field is given --
    the parser gives an absent optional member as `UNSET` -- since
    nothing here applies a default. Anything that has not checked
    the members goes through the constructor.
    """
    record = object.__new__(cls)
    for name, value in members.items():
        object.__setattr__(record, name, value)
    return record

problems

problems() -> Iterator[ValidationProblem]

order must permute its own axes.

Whether it permutes the array's axes is a different question -- it needs the array's rank -- and the rules layer asks that one.

Source code in src/zarr_metadata/v3/codec/transpose.py

def problems(self) -> "Iterator[ValidationProblem]":
    """`order` must permute its own axes.

    Whether it permutes the *array's* axes is a different question -- it
    needs the array's rank -- and the rules layer asks that one.
    """
    if sorted(self.order) != list(range(len(self.order))):
        yield ValidationProblem(
            ("order",),
            f"expected a permutation of 0..{len(self.order) - 1}, got {self.order!r}",
            "invalid_value",
        )

zarr_metadata.v3.codec.zstd

Zstandard codec types.

See https://github.com/zarr-developers/zarr-extensions/blob/4da7b37a84f76e660902f6d3de3eaef0e0febae6/codecs/zstd/README.md (the zarr-extensions registry entry; zarr-specs PR #256, which first proposed the codec, was never merged).

ZSTD_CODEC_NAME module-attribute

ZSTD_CODEC_NAME: Final = 'zstd'

The name field value of the zstd codec.

ZSTD_MAX_LEVEL module-attribute

ZSTD_MAX_LEVEL: Final = 22

The highest level zstd accepts: ZSTD_maxCLevel().

ZSTD_MIN_LEVEL module-attribute

ZSTD_MIN_LEVEL: Final = -131072

The lowest level zstd accepts: ZSTD_minCLevel(), -(1 << 17).

ZstdCodecMetadata module-attribute

ZstdCodecMetadata = ZstdCodecObject

Permitted JSON shape for zstd codec metadata.

level is required, so only the object form is valid; the short-hand-name form is not permitted by the spec for this codec. https://github.com/zarr-developers/zarr-extensions/blob/4da7b37a84f76e660902f6d3de3eaef0e0febae6/codecs/zstd/README.md#L9-L19 https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L1562-L1564 (short-hand names only "if no configuration metadata is required")

ZstdCodecName module-attribute

ZstdCodecName = Literal['zstd']

Literal type of the name field of the zstd codec.

__all__ module-attribute

__all__ = [
    "ZSTD_CODEC_NAME",
    "ZSTD_MAX_LEVEL",
    "ZSTD_MIN_LEVEL",
    "ZstdCodec",
    "ZstdCodecConfiguration",
    "ZstdCodecMetadata",
    "ZstdCodecName",
    "ZstdCodecObject",
    "ZstdOptions",
]

ZstdCodec dataclass

Bases: BytesBytesCodec

The zstd codec, coerced from its metadata.

Source code in src/zarr_metadata/v3/codec/zstd.py

@dataclass(frozen=True)
class ZstdCodec(BytesBytesCodec):
    """The `zstd` codec, coerced from its metadata."""

    configuration: ZstdOptions

    identifier: ClassVar[str] = ZSTD_CODEC_NAME
    variable_size: ClassVar[bool] = True

configuration instance-attribute

configuration: ZstdOptions

The record of this entity's members.

An entity with members narrows it to its own record, configuration: GzipOptions, its one positional argument. An entity of a bare name defaults it to the empty record -- configuration: Configuration = field(default_factory=Configuration) -- so that Crc32cCodec() builds; coerce passes the record either way.

identifier class-attribute

identifier: str = ZSTD_CODEC_NAME

The name this entity is registered under.

Usually the name the metadata carries. The raw-bytes data types are the exception: every r<N> spelling is one family, so the family gets an invented identifier that no real name can collide with.

variable_size class-attribute

variable_size: bool = True

Whether this codec's output size depends on the bytes it is given.

A compressor's does, so a shard index encoded with one has no size derivable from metadata alone, and the shard cannot be read. Every codec says, because a default in either direction is a verdict.

__init__

__init__(configuration: ZstdOptions) -> None

__post_init__

__post_init__() -> None

Refuse a record that is not this entity's own, then the first problem the rules find.

The runtime half of the entity's type, as the record's constructor is of the record's: GzipCodec(BloscOptions(...)) and a family member carrying a name that is not a string are refused before any rule reads them. Then BloscCodec(BloscOptions(clevel=99)) raises on the first problem the rules yield.

Source code in src/zarr_metadata/v3/_entity.py

def __post_init__(self) -> None:
    """Refuse a record that is not this entity's own, then the first problem the rules find.

    The runtime half of the entity's type, as the record's constructor
    is of the record's: `GzipCodec(BloscOptions(...))` and a family
    member carrying a name that is not a string are refused before
    any rule reads them. Then `BloscCodec(BloscOptions(clevel=99))`
    raises on the first problem the rules yield.
    """
    plan = _plan(type(self))
    if not isinstance(self.configuration, plan.record):
        raise MetadataValidationError(
            problem(
                (),
                f"expected a {plan.record.__name__} configuration, got "
                f"{type(self.configuration).__name__}",
            )
        )
    name: object = self.identifier if plan.from_name is None else getattr(self, plan.from_name)
    if not isinstance(name, str):
        raise MetadataValidationError(problem((), f"expected a string name, got {name!r}"))
    first = next(type(self).name_problems(name), None)
    if first is None:
        first = next(self.configuration.problems(), None)
    if first is not None:
        raise MetadataValidationError((first,))

accepts classmethod

accepts(name: str) -> bool

Whether name denotes this entity.

Constant for all but the raw-bytes family, where one class covers every r<N>.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def accepts(cls, name: str) -> bool:
    """Whether `name` denotes this entity.

    Constant for all but the raw-bytes family, where one class covers
    every `r<N>`.
    """
    return name == cls.identifier

canonical

canonical() -> Self

This entity in the simplest form that means the same thing.

A transformation, asked for by canonicalize_array_metadata_v3 and by nothing else. to_json does not apply it, because writing a document back is not the same as asking for it to be rewritten: a reader that reads and writes should not change bytes it was not asked to change.

The default is the entity itself. Override it where two spellings of the entity's members mean the same -- a rectilinear dimension's run-length encoding, a typesize that noshuffle ignores -- and, in an entity that contains entities, to put those in canonical form: self.with_configuration(inner=self.inner.canonical()).

Source code in src/zarr_metadata/v3/_entity.py

def canonical(self) -> Self:
    """This entity in the simplest form that means the same thing.

    A *transformation*, asked for by `canonicalize_array_metadata_v3`
    and by nothing else. `to_json` does not apply it, because writing
    a document back is not the same as asking for it to be rewritten:
    a reader that reads and writes should not change bytes it was not
    asked to change.

    The default is the entity itself. Override it where two spellings
    of the entity's members mean the same -- a rectilinear
    dimension's run-length encoding, a `typesize` that `noshuffle`
    ignores -- and, in an entity that contains entities, to put those
    in canonical form: `self.with_configuration(inner=self.inner.canonical())`.
    """
    return self

coerce classmethod

coerce(value: JSONValue, context: Context) -> Coerced[Self]

value as this entity, or the reasons it is not one: the class's validation routine.

resolve relates a field's name to this class and hands it the field, refined JSON with arrays as tuples, which is what value is; this is what the class does with it. The configuration is parsed against the record the configuration field names, member by member; a member holding another entity is read in context, the scope this reading is happening in. An optional member the document left out is UNSET in the record, so no field's default decides what a document said. The entity is built only when every member of its own read -- its rules are written over a whole configuration -- and handed back only when everything inside it read too.

The envelope is the field's, not the class's, and resolve judges it: a stray member or a must_understand of false is not reported here. Called on a class no scope has registered, this runs with none of registration's refusals having happened.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def coerce(cls, value: JSONValue, context: Context) -> Coerced[Self]:
    """`value` as this entity, or the reasons it is not one: the class's validation routine.

    `resolve` relates a field's name to this class and hands it the
    field, refined JSON with arrays as tuples, which is what `value`
    is; this is what the class does with it. The configuration is
    parsed against the record the `configuration` field names,
    member by member; a member holding another entity is read in
    `context`, the scope this reading is happening in. An optional
    member the document left out is `UNSET` in the record, so no
    field's default decides what a document said. The entity is
    built only when every member of its own read -- its rules are
    written over a whole configuration -- and handed back only when
    everything inside it read too.

    The envelope is the field's, not the class's, and `resolve`
    judges it: a stray member or a `must_understand` of `false` is
    not reported here. Called on a class no scope has registered,
    this runs with none of registration's refusals having happened.
    """
    name, given, envelope = named_configuration(value)
    if name is None or not cls.accepts(name):
        return None, problem((), f"expected the {cls.identifier!r} entity")
    if len(envelope) != 0:
        return None, envelope
    plan = _plan(cls)
    carried: dict[str, str] = {} if plan.from_name is None else {plan.from_name: name}
    if given is None and plan.requires_configuration:
        return None, problem(
            ("configuration",),
            f"{cls.identifier!r} requires a configuration",
            "missing_key",
        )
    reading = _Reading(context, [])
    # A bare name's record has no members, so any key is an unknown one.
    record, own = plan.read({} if given is None else given, ("configuration",), reading)
    found = (*own, *reading.nested)
    if record is None:
        # An unknown key is survivable; a member that could not be
        # read is a hole, and judging around it would be guessing.
        return None, found
    # The rules, asked of the name and of the record before anything
    # is built: a name problem lands on the entity, a configuration
    # problem under the configuration.
    refused = (*cls.name_problems(name), *within((), tuple(record.problems())))
    if len(refused) != 0:
        # Values the spec disallows: reported rather than raised,
        # every one.
        return None, (*found, *refused)
    if any(entry.kind != "unknown_key" for entry in reading.nested):
        # A contained entity could not be read. This entity's own
        # rules ran -- an invalid inner is an `Opaque`, as an
        # out-of-scope one is -- but what is handed back is not an
        # entity that would be asked composition questions it cannot
        # answer.
        return None, found
    return cls.create_unchecked(configuration=record, **carried), found

create_unchecked classmethod

create_unchecked(**fields: object) -> Self

This entity with these fields, built without the constructor's checks.

The one way around them, for a caller that has just made them: coerce, which type-checked the record and ran the rules before building. Every field is given -- the record, and the carried name for a family -- since nothing here applies a default. Anything that has not checked goes through the constructor.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def create_unchecked(cls, **fields: object) -> Self:
    """This entity with these fields, built without the constructor's checks.

    The one way around them, for a caller that has just made them:
    `coerce`, which type-checked the record and ran the rules before
    building. Every field is given -- the record, and the carried
    name for a family -- since nothing here applies a default.
    Anything that has not checked goes through the constructor.
    """
    entity = object.__new__(cls)
    for name, value in fields.items():
        object.__setattr__(entity, name, value)
    return entity

incoming_problems

incoming_problems(
    incoming: ArrayParts | None,
) -> tuple[ValidationProblem, ...]

Why this codec cannot be applied to the array that reaches it.

incoming is None once the chain can no longer say what reaches here, and the default answer to that is nothing: declining beats guessing. Locations are relative to this codec's configuration; an empty one lands on the codec itself.

Source code in src/zarr_metadata/v3/_entity.py

def incoming_problems(self, incoming: ArrayParts | None) -> tuple[ValidationProblem, ...]:
    """Why this codec cannot be applied to the array that reaches it.

    `incoming` is None once the chain can no longer say what reaches
    here, and the default answer to that is nothing: declining beats
    guessing. Locations are relative to this codec's `configuration`;
    an empty one lands on the codec itself.
    """
    return ()

inner_pipelines

inner_pipelines(
    incoming: ArrayParts | None,
) -> Mapping[
    str,
    tuple[
        Sequence[CodecEntity | Opaque], ArrayParts | None
    ],
]

The pipelines this codec holds, by the member holding each, with what each is handed.

A shard holds two: its codecs, handed its inner chunk, and its index_codecs, handed the shard index. Refinement walks them as it walks the pipeline this codec stands in, locating what it finds under the member, so a codec that holds pipelines says which and what they receive, and judges nothing inside them itself. Default: none.

Source code in src/zarr_metadata/v3/_entity.py

def inner_pipelines(
    self, incoming: ArrayParts | None
) -> Mapping[str, tuple[Sequence[CodecEntity | Opaque], ArrayParts | None]]:
    """The pipelines this codec holds, by the member holding each, with what each is handed.

    A shard holds two: its `codecs`, handed its inner chunk, and its
    `index_codecs`, handed the shard index. Refinement walks them as
    it walks the pipeline this codec stands in, locating what it
    finds under the member, so a codec that holds pipelines says
    which and what they receive, and judges nothing inside them
    itself. Default: none.
    """
    return {}

name_problems classmethod

name_problems(name: str) -> Iterator[ValidationProblem]

Why name, which accepts claimed, is not a well-formed name of this family.

For a family, whose names carry data -- r<N> -- and which claims a malformed member so that it is reported rather than waved through as an unknown extension. Locations are relative to the entity: (). Default: none, for an entity of one name.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def name_problems(cls, name: str) -> Iterator[ValidationProblem]:
    """Why `name`, which `accepts` claimed, is not a well-formed name of this family.

    For a family, whose names carry data -- `r<N>` -- and which claims
    a malformed member so that it is reported rather than waved
    through as an unknown extension. Locations are relative to the
    entity: `()`. Default: none, for an entity of one name.
    """
    yield from ()

to_json

to_json() -> ZarrV3MetadataFieldJSON

This entity as a document would write it.

Written from the configuration record by the same declaration coerce reads it by, each member by the writer its annotation implies: the bare name when every member it holds is absent, the object otherwise, a contained entity through its own to_json, a JSON-valued member copied so the document is not a handle on the entity. Faithful to every member: read a document, write it back, and those come out as they went in. The envelope is written the entity's way -- the bare name when nothing is configured, the object otherwise, no must_understand, which means what absence means -- because an entity alone has no document to be faithful to; ArrayDocumentV3.to_json puts back the spelling the document used. Ask canonical first if you want the simplest equivalent spelling.

An entity whose JSON is not its fields overrides this; none in the package does.

Source code in src/zarr_metadata/v3/_entity.py

def to_json(self) -> ZarrV3MetadataFieldJSON:
    """This entity as a document would write it.

    Written from the configuration record by the same declaration
    `coerce` reads it by, each member by the writer its annotation
    implies: the bare name when every member it holds is absent, the
    object otherwise, a contained entity through its own `to_json`,
    a JSON-valued member copied so the document is not a handle on
    the entity. Faithful to every member: read a document, write it
    back, and those come out as they went in. The envelope is
    written the entity's way -- the bare name when nothing is
    configured, the object otherwise, no `must_understand`, which
    means what absence means -- because an entity alone has no
    document to be faithful to; `ArrayDocumentV3.to_json` puts back
    the spelling the document used. Ask `canonical` first if you
    want the simplest equivalent spelling.

    An entity whose JSON is not its fields overrides this; none in
    the package does.
    """
    plan = _plan(type(self))
    name = self.identifier
    if plan.from_name is not None:
        carried = getattr(self, plan.from_name)
        name = carried if isinstance(carried, str) else name
    configuration = plan.write(self)
    if len(configuration) == 0:
        return name
    return {"name": name, "configuration": configuration}

with_configuration

with_configuration(**changes: object) -> Self

This entity with these configuration members changed.

codec.with_configuration(typesize=UNSET) is the record rebuilt through its constructor, which refuses a member of the wrong type, and the entity rebuilt through its own, which refuses a value the rules disallow -- the same checks as any construction, since pyright cannot see the members through **changes. A name that is not a member is refused the way replace refuses it.

Source code in src/zarr_metadata/v3/_entity.py

def with_configuration(self, **changes: object) -> Self:
    """This entity with these configuration members changed.

    `codec.with_configuration(typesize=UNSET)` is the record rebuilt
    through its constructor, which refuses a member of the wrong
    type, and the entity rebuilt through its own, which refuses a
    value the rules disallow -- the same checks as any construction,
    since pyright cannot see the members through `**changes`. A
    name that is not a member is refused the way `replace` refuses
    it.
    """
    return replace(self, configuration=replace(self.configuration, **changes))

ZstdCodecConfiguration

Bases: TypedDict

Configuration for the Zarr v3 zstd codec.

level is required; checksum is optional ("Should be omitted if false"). https://github.com/zarr-developers/zarr-extensions/blob/4da7b37a84f76e660902f6d3de3eaef0e0febae6/codecs/zstd/README.md#L9-L19

Source code in src/zarr_metadata/v3/codec/zstd.py

class ZstdCodecConfiguration(TypedDict, closed=True):
    """
    Configuration for the Zarr v3 `zstd` codec.

    `level` is required; `checksum` is optional ("Should be omitted if
    false").
      https://github.com/zarr-developers/zarr-extensions/blob/4da7b37a84f76e660902f6d3de3eaef0e0febae6/codecs/zstd/README.md#L9-L19
    """

    level: int
    checksum: NotRequired[bool]

checksum instance-attribute

checksum: NotRequired[bool]

level instance-attribute

level: int

ZstdCodecObject

Bases: TypedDict

zstd codec metadata in object form.

Source code in src/zarr_metadata/v3/codec/zstd.py

class ZstdCodecObject(TypedDict, closed=True):
    """`zstd` codec metadata in object form."""

    name: ZstdCodecName
    configuration: ZstdCodecConfiguration
    must_understand: NotRequired[bool]

configuration instance-attribute

configuration: ZstdCodecConfiguration

must_understand instance-attribute

must_understand: NotRequired[bool]

name instance-attribute

name: ZstdCodecName

ZstdOptions dataclass

Bases: Configuration

What zstd is configured with.

Source code in src/zarr_metadata/v3/codec/zstd.py

@dataclass(frozen=True)
class ZstdOptions(Configuration):
    """What `zstd` is configured with."""

    level: int
    checksum: bool | UNSET = UNSET

    def problems(self) -> "Iterator[ValidationProblem]":
        if not ZSTD_MIN_LEVEL <= self.level <= ZSTD_MAX_LEVEL:
            yield ValidationProblem(
                ("level",),
                f"expected an integer in [{ZSTD_MIN_LEVEL}, {ZSTD_MAX_LEVEL}], got {self.level}",
                "invalid_value",
            )

checksum class-attribute instance-attribute

checksum: bool | UNSET = UNSET

level instance-attribute

level: int

__init__

__init__(
    level: int, checksum: bool | UNSET = UNSET
) -> None

__post_init__

__post_init__() -> None

Refuse every member of the wrong type, so GzipOptions(level="high") raises.

Source code in src/zarr_metadata/v3/_entity.py

def __post_init__(self) -> None:
    """Refuse every member of the wrong type, so `GzipOptions(level="high")` raises."""
    found = mistyped(self)
    if len(found) != 0:
        raise MetadataValidationError(found)

create_unchecked classmethod

create_unchecked(**members: object) -> Self

This record with these members, built without the constructor's check.

The one way around the check, for a caller that has just made it: the parser, which type-checked every member against the same annotations before building the record. Every field is given -- the parser gives an absent optional member as UNSET -- since nothing here applies a default. Anything that has not checked the members goes through the constructor.

Source code in src/zarr_metadata/v3/_entity.py

@classmethod
def create_unchecked(cls, **members: object) -> Self:
    """This record with these members, built without the constructor's check.

    The one way around the check, for a caller that has just made
    it: the parser, which type-checked every member against the same
    annotations before building the record. Every field is given --
    the parser gives an absent optional member as `UNSET` -- since
    nothing here applies a default. Anything that has not checked
    the members goes through the constructor.
    """
    record = object.__new__(cls)
    for name, value in members.items():
        object.__setattr__(record, name, value)
    return record

problems

problems() -> Iterator[ValidationProblem]

Every reason these values are not allowed, yielded as found. Default: none.

Source code in src/zarr_metadata/v3/codec/zstd.py

def problems(self) -> "Iterator[ValidationProblem]":
    if not ZSTD_MIN_LEVEL <= self.level <= ZSTD_MAX_LEVEL:
        yield ValidationProblem(
            ("level",),
            f"expected an integer in [{ZSTD_MIN_LEVEL}, {ZSTD_MAX_LEVEL}], got {self.level}",
            "invalid_value",
        )