zarr_metadata.rules

zarr_metadata.rules

Validate structure and composition of Zarr metadata documents.

zarr_metadata.model checks JSON structure. This module also checks cross-field constraints such as fill-value compatibility, codec ordering, and dimension counts. Its validate_* and parse_* functions mirror the model API, and canonicalize_array_metadata_v3 answers with either the document in its simplest equivalent spelling or every reason it is not valid.

Rules target canonical metadata and may be stricter than readers that coerce inputs. Unknown entity names are left unjudged. Known entities must match their modeled shape; extra configuration keys produce an unknown_key problem without suppressing other checks. Model round-trips preserve those unmodeled members.

__all__ module-attribute

__all__ = [
    "Canonical",
    "Invalid",
    "canonicalize_array_metadata_v3",
    "parse_array_metadata_v2",
    "parse_array_metadata_v3",
    "parse_group_metadata_v2",
    "parse_group_metadata_v3",
    "validate_array_metadata_v2",
    "validate_array_metadata_v3",
    "validate_group_metadata_v2",
    "validate_group_metadata_v3",
]

Canonical dataclass

Bases: Generic[DocumentT]

A semantically valid document, in its simplest equivalent spelling.

Source code in src/zarr_metadata/rules/_documents.py

@dataclass(frozen=True, slots=True)
class Canonical(Generic[DocumentT]):
    """A semantically valid document, in its simplest equivalent spelling."""

    document: DocumentT
    valid: Literal[True] = True

document instance-attribute

document: DocumentT

valid class-attribute instance-attribute

valid: Literal[True] = True

__init__

__init__(
    document: DocumentT, valid: Literal[True] = True
) -> None

Invalid dataclass

Every reason a document is not semantically valid; never empty.

Source code in src/zarr_metadata/rules/_documents.py

@dataclass(frozen=True, slots=True)
class Invalid:
    """Every reason a document is not semantically valid; never empty."""

    problems: tuple[ValidationProblem, ...]
    valid: Literal[False] = False

    def __post_init__(self) -> None:
        if len(self.problems) == 0:
            msg = "Invalid requires at least one validation problem"
            raise ValueError(msg)

problems instance-attribute

problems: tuple[ValidationProblem, ...]

valid class-attribute instance-attribute

valid: Literal[False] = False

__init__

__init__(
    problems: tuple[ValidationProblem, ...],
    valid: Literal[False] = False,
) -> None

__post_init__

__post_init__() -> None

Source code in src/zarr_metadata/rules/_documents.py

def __post_init__(self) -> None:
    if len(self.problems) == 0:
        msg = "Invalid requires at least one validation problem"
        raise ValueError(msg)

canonicalize_array_metadata_v3

canonicalize_array_metadata_v3(
    document: object,
    *,
    context: Context = CORE_AND_EXTENSIONS,
) -> Canonical[ZarrV3ArrayMetadataJSON] | Invalid

document in canonical form, or every reason it is not valid.

Canonical means the simplest spelling with the same meaning, and the document decides that for itself in ArrayDocumentV3.canonical: each entity in its own canonical form, and dimension_names of nothing but nulls omitted. Two properties are worth holding on to, and tests/rules/test_canonical.py asserts both: canonicalizing twice changes nothing further, and canonicalizing never changes a verdict.

Takes any value, like validate_array_metadata_v3: a document the model layer has not accepted is not an error -- the structural problems come back with the semantic ones, and the result is Invalid rather than a canonical document. The document is read once: the entities that judge it are the entities that are rewritten.

Tell the two apart with result.valid is True or isinstance(result, Invalid); pyright narrows the literal on a comparison, not on if result.valid.

An entity whose canonical form breaks its own rules raises MetadataValidationError from here, as its constructor does: that is a bug in the entity, not a verdict on the document, which was valid.

Source code in src/zarr_metadata/rules/_documents.py

def canonicalize_array_metadata_v3(
    document: object, *, context: Context = CORE_AND_EXTENSIONS
) -> Canonical[ZarrV3ArrayMetadataJSON] | Invalid:
    """`document` in canonical form, or every reason it is not valid.

    Canonical means the simplest spelling with the same meaning, and the
    document decides that for itself in `ArrayDocumentV3.canonical`: each
    entity in its own canonical form, and `dimension_names` of nothing
    but nulls omitted. Two properties are worth holding on to, and
    `tests/rules/test_canonical.py` asserts both: canonicalizing twice
    changes nothing further, and canonicalizing never changes a verdict.

    Takes any value, like `validate_array_metadata_v3`: a document the
    model layer has not accepted is not an error -- the structural
    problems come back with the semantic ones, and the result is
    `Invalid` rather than a canonical document. The document is read
    once: the entities that judge it are the entities that are rewritten.

    Tell the two apart with `result.valid is True` or
    `isinstance(result, Invalid)`; pyright narrows the literal on a
    comparison, not on `if result.valid`.

    An entity whose canonical form breaks its own rules raises
    `MetadataValidationError` from here, as its constructor does: that
    is a bug in the entity, not a verdict on the document, which was
    valid.
    """
    refined, problems = well_formed_array_v3(document)
    if refined is None:
        return Invalid(problems)
    array, found = read_array_v3(refined, context)
    _, composed = refine_array_v3(array)
    problems = (*problems, *found, *composed)
    if len(problems) != 0:
        return Invalid(problems)
    return Canonical(ZarrV3ArrayMetadata.from_json(array.canonical().to_json()).to_json())

parse_array_metadata_v2

parse_array_metadata_v2(
    value: object,
) -> ZarrV2ArrayMetadataJSON

Return value as a valid ZarrV2ArrayMetadataJSON, or raise.

Normalizes JSON arrays to tuples, then raises a single MetadataValidationError carrying every structural and composition problem found.

Source code in src/zarr_metadata/rules/_documents.py

def parse_array_metadata_v2(value: object) -> ZarrV2ArrayMetadataJSON:
    """Return `value` as a valid `ZarrV2ArrayMetadataJSON`, or raise.

    Normalizes JSON arrays to tuples, then raises a single
    `MetadataValidationError` carrying every structural and composition
    problem found.
    """
    refined, problems = _judged(value, _validate_structure_v2, array_problems_v2)
    if len(problems) != 0:
        raise MetadataValidationError(problems)
    return cast("ZarrV2ArrayMetadataJSON", refined)

parse_array_metadata_v3

parse_array_metadata_v3(
    value: object, *, context: Context = CORE_AND_EXTENSIONS
) -> ZarrV3ArrayMetadataJSON

Return value as a valid ZarrV3ArrayMetadataJSON, or raise.

Normalizes JSON arrays to tuples, then raises a single MetadataValidationError carrying every structural and composition problem found.

Source code in src/zarr_metadata/rules/_documents.py

def parse_array_metadata_v3(
    value: object, *, context: Context = CORE_AND_EXTENSIONS
) -> ZarrV3ArrayMetadataJSON:
    """Return `value` as a valid `ZarrV3ArrayMetadataJSON`, or raise.

    Normalizes JSON arrays to tuples, then raises a single
    `MetadataValidationError` carrying every structural and composition
    problem found.
    """
    document, problems = well_formed_array_v3(value)
    if document is not None:
        array, found = read_array_v3(document, context)
        _, composed = refine_array_v3(array)
        problems = (*problems, *found, *composed)
    if document is None or len(problems) != 0:
        raise MetadataValidationError(problems)
    return cast("ZarrV3ArrayMetadataJSON", document)

parse_group_metadata_v2

parse_group_metadata_v2(
    value: object,
) -> ZarrV2GroupMetadataJSON

Return value as a valid ZarrV2GroupMetadataJSON, or raise.

Source code in src/zarr_metadata/rules/_documents.py

def parse_group_metadata_v2(value: object) -> ZarrV2GroupMetadataJSON:
    """Return `value` as a valid `ZarrV2GroupMetadataJSON`, or raise."""
    refined, problems = _judged(value, _validate_group_structure_v2, _no_semantics)
    if len(problems) != 0:
        raise MetadataValidationError(problems)
    return cast("ZarrV2GroupMetadataJSON", refined)

parse_group_metadata_v3

parse_group_metadata_v3(
    value: object, *, context: Context = CORE_AND_EXTENSIONS
) -> ZarrV3GroupMetadataJSON

Return value as a valid ZarrV3GroupMetadataJSON, or raise.

Source code in src/zarr_metadata/rules/_documents.py

def parse_group_metadata_v3(
    value: object, *, context: Context = CORE_AND_EXTENSIONS
) -> ZarrV3GroupMetadataJSON:
    """Return `value` as a valid `ZarrV3GroupMetadataJSON`, or raise."""
    refined, problems = _judged(value, _validate_group_structure_v3, _group_semantics_v3(context))
    if len(problems) != 0:
        raise MetadataValidationError(problems)
    return cast("ZarrV3GroupMetadataJSON", refined)

validate_array_metadata_v2

validate_array_metadata_v2(
    value: object,
) -> tuple[ValidationProblem, ...]

Every reason value is not a valid v2 array document (merged form).

JSON arrays are normalized to tuples before judgment, as in validate_array_metadata_v3.

Source code in src/zarr_metadata/rules/_documents.py

def validate_array_metadata_v2(value: object) -> tuple[ValidationProblem, ...]:
    """Every reason `value` is not a valid v2 array document (merged form).

    JSON arrays are normalized to tuples before judgment, as in
    `validate_array_metadata_v3`.
    """
    return _judged(value, _validate_structure_v2, array_problems_v2)[1]

validate_array_metadata_v3

validate_array_metadata_v3(
    value: object, *, context: Context = CORE_AND_EXTENSIONS
) -> tuple[ValidationProblem, ...]

Why value is not a valid v3 array document.

Every structural problem, and every semantic problem that can be determined. One member that cannot be read costs the composition judgments about the entity holding it -- whether a shard's inner shape divides the array it is handed cannot be answered by a shard that could not be built -- so a document with two defects in one configuration may need a second pass. The verdict is never affected.

The three layers of reading, in order, each handing the next what it needs: the value refined to JSON and judged for shape, the extension points read in context, the whole refined against the array. Their problems are reported together; a value that is not JSON gets only that verdict. List-spelled documents (fresh json.loads output) are judged at the canonical data level rather than rejected for their spelling.

Source code in src/zarr_metadata/rules/_documents.py

def validate_array_metadata_v3(
    value: object, *, context: Context = CORE_AND_EXTENSIONS
) -> tuple[ValidationProblem, ...]:
    """Why `value` is not a valid v3 array document.

    Every structural problem, and every semantic problem that can be
    determined. One member that cannot be read costs the *composition*
    judgments about the entity holding it -- whether a shard's inner
    shape divides the array it is handed cannot be answered by a shard
    that could not be built -- so a document with two defects in one
    configuration may need a second pass. The verdict is never affected.

    The three layers of reading, in order, each handing the next what it
    needs: the value refined to JSON and judged for shape, the extension
    points read in `context`, the whole refined against the array. Their
    problems are reported together; a value that is not JSON gets only
    that verdict. List-spelled documents (fresh `json.loads` output) are
    judged at the canonical data level rather than rejected for their
    spelling.
    """
    document, problems = well_formed_array_v3(value)
    if document is None:
        return problems
    array, found = read_array_v3(document, context)
    _, composed = refine_array_v3(array)
    return (*problems, *found, *composed)

validate_group_metadata_v2

validate_group_metadata_v2(
    value: object,
) -> tuple[ValidationProblem, ...]

Every reason value is not a valid v2 group document (merged form).

v2 group documents carry no composition constraints today, so this is the structural judgment, offered here for a uniform read-side API.

Source code in src/zarr_metadata/rules/_documents.py

def validate_group_metadata_v2(value: object) -> tuple[ValidationProblem, ...]:
    """Every reason `value` is not a valid v2 group document (merged form).

    v2 group documents carry no composition constraints today, so this is
    the structural judgment, offered here for a uniform read-side API.
    """
    return _judged(value, _validate_group_structure_v2, _no_semantics)[1]

validate_group_metadata_v3

validate_group_metadata_v3(
    value: object, *, context: Context = CORE_AND_EXTENSIONS
) -> tuple[ValidationProblem, ...]

Every reason value is not a valid v3 group document.

Composition rules recurse into inline consolidated metadata, so a consolidated child document invalid under its own rules is reported here, at its path.

Source code in src/zarr_metadata/rules/_documents.py

def validate_group_metadata_v3(
    value: object, *, context: Context = CORE_AND_EXTENSIONS
) -> tuple[ValidationProblem, ...]:
    """Every reason `value` is not a valid v3 group document.

    Composition rules recurse into inline consolidated metadata, so a
    consolidated child document invalid under its own rules is reported
    here, at its path.
    """
    return _judged(value, _validate_group_structure_v3, _group_semantics_v3(context))[1]