eoio.serialisation package

eoio.serialisation package#

eoio.serialisation - attribute (de)serialisation shared by readers and writers

Readers populate attrs with rich Python objects that netCDF and Zarr cannot store. This package converts them on the way out and restores them on the way back:

>>> from eoio.serialisation import encode_attrs, decode_attrs
>>> encode_attrs(ds).to_netcdf("out.nc")
>>> ds = decode_attrs(xr.open_dataset("out.nc"))

Support for a new attribute type is added by registering a codec - see eoio.serialisation.codecs.register_codec() - rather than by modifying this package.

class eoio.serialisation.AttrCodec(*args, **kwargs)[source]#

Bases: Protocol

Protocol implemented by attribute codecs.

applies(value: Any) → bool[source]#

Report whether this codec handles value.

Parameters:

value – Candidate attribute value.

Returns:

True when this codec can encode the value.

decode(encoded: Any) → Any[source]#

Reverse encode().

Parameters:

encoded – Previously encoded value.

Returns:

The restored Python object.

encode(value: Any) → Any[source]#

Convert value to a serialisation-safe representation.

Parameters:

value – Attribute value to encode.

Returns:

A value acceptable as a netCDF/Zarr attribute.

name: str#

Stable identifier recorded in the encoding marker.

eoio.serialisation.decode_attrs(obj: T, *, sniff: bool = True, inplace: bool = False) → T[source]#

Restore attributes previously encoded for serialisation.

Where an eoio:attr_encoding marker is present its codecs are used, giving an exact round trip. Attributes with no marker entry are sniffed instead, recognising only JSON objects/arrays and ISO-8601 datetimes.

Parameters:

  • obj – Dataset, DataArray or DataTree whose attributes should be decoded.

  • sniff – Whether to attempt best-effort decoding of unmarked attributes. Set False to decode only what the marker describes.

  • inplace – Whether to modify obj rather than a copy.

Returns:

The object with decoded attributes, and the marker attribute removed.

eoio.serialisation.encode_attrs(obj: T, *, dict_style: str = 'dot', strict: bool = False, inplace: bool = False) → T[source]#

Convert attributes to values that can be written to netCDF or Zarr.

Applies to dataset-level, variable and coordinate attributes, and to every node of a DataTree. An eoio:attr_encoding marker is added to each scope that needed encoding, so decode_attrs() can reverse it exactly.

Parameters:

  • obj – Dataset, DataArray or DataTree whose attributes should be encoded.

  • dict_style – "dot" to dot-flatten nested dicts, or "json" to encode each as a single JSON blob.

  • strict – If True, raise on a value no codec handles. If False, fall back to str() with a warning.

  • inplace – Whether to modify obj rather than a copy.

Returns:

The object with encoded attributes.

Raises:

  • ValueError – If dict_style is not "dot" or "json".

  • TypeError – If strict is set and a value cannot be encoded.

eoio.serialisation.get_codec(name: str) → AttrCodec[source]#

Look up a codec by name.

Parameters:

name – Registered codec name.

Returns:

The codec.

Raises:

KeyError – If no codec is registered under that name.

eoio.serialisation.is_serialisable(value: Any) → bool[source]#

Report whether a value can be written as an attribute without any encoding.

Grounded in what xarray/netCDF4 actually accepts rather than what looks reasonable: bool and None are rejected outright by the netCDF4 backend, and a mixed-type list such as [1, "a"] is silently coerced to ['1', 'a'], so neither is treated as safe here.

Parameters:

value – Attribute value.

Returns:

True when the value can be stored as-is.

eoio.serialisation.list_codecs() → List[str][source]#

List registered codec names in consultation order.

Returns:

Codec names, lowest priority value first.

eoio.serialisation.register_codec(codec: AttrCodec, priority: int = 100, overwrite: bool = False) → AttrCodec[source]#

Register a codec under its own name.

Parameters:

  • codec – Codec instance implementing AttrCodec.

  • priority – Consultation order; lower values are offered the value first.

  • overwrite – Whether replacing an already-registered name is permitted.

Returns:

The registered codec, so it can be assigned in the same statement.

Raises:

KeyError – If the name is already registered and overwrite is False.

eoio.serialisation.select_codec(value: Any) → AttrCodec | None[source]#

Find the codec responsible for value.

Parameters:

value – Attribute value.

Returns:

The first matching codec in priority order, or None if the value needs no encoding or no codec handles it.

eoio.serialisation.unregister_codec(name: str) → AttrCodec[source]#

Remove a registered codec.

Parameters:

name – Registered codec name.

Returns:

The removed codec.

Raises:

KeyError – If no codec is registered under that name.

Submodules#