eoio.serialisation.attrs module

eoio.serialisation.attrs - encode and decode xarray attributes for serialisation

Readers populate attrs with rich Python objects (nested metadata dicts, datetimes, geometries) that netCDF and Zarr cannot store. encode_attrs() converts them to serialisable values before writing; decode_attrs() restores them after reading.

Two design points are worth knowing before using this module.

Nested dicts are dot-flattened. {"product_metadata": {"sensing_time": ...}} is written as an attribute named product_metadata.sensing_time, so it stays readable under ncdump and matches the dot-path convention already used by eoio.read_multi() and the stack processor. Dicts that cannot be flattened unambiguously - non-string keys, keys containing a dot, or names that would collide with an existing attribute - fall back to a single JSON blob instead.

Decoding works on any file, not just eoio’s own. Sniffing is the primary path, so third-party netCDF products decode on a best-effort basis. Files written by eoio also carry an eoio:attr_encoding marker recording which codec encoded which attribute, and when that marker is present it is preferred, making the round trip exact rather than best-effort.

Sniffing is deliberately conservative: only JSON objects/arrays and ISO-8601 datetimes are recognised. Numeric-looking strings are left alone, because coercing them corrupts legitimate values - the controlled vocabulary’s own product_version = "03.01" would otherwise come back as the float 3.01.

Known limitations, none of which lose information:

• Plain Python scalars widen through netCDF itself - an int attribute is returned as np.int64 and a float as np.float64, whatever this module does. Values compare equal, but type() differs. Dot-flattened dict leaves are affected; dict_style="json" leaves are not, since they travel inside a JSON string.

• np.bool_ is restored as a Python bool.

• Dicts with non-string keys cannot be dot-flattened and fall back to JSON, which returns the keys as strings. This is warned about at encode time.

eoio.serialisation.attrs.decode_attrs(obj: T, *, sniff: bool = True, inplace: bool = False) → T

Restore attributes previously encoded for serialisation.

Where an eoio:attr_encoding marker is present its codecs are used, giving an exact round trip. Attributes with no marker entry are sniffed instead, recognising only JSON objects/arrays and ISO-8601 datetimes.

Parameters:

• obj – Dataset, DataArray or DataTree whose attributes should be decoded.

• sniff – Whether to attempt best-effort decoding of unmarked attributes. Set False to decode only what the marker describes.

• inplace – Whether to modify obj rather than a copy.

Returns:

The object with decoded attributes, and the marker attribute removed.

eoio.serialisation.attrs.encode_attrs(obj: T, *, dict_style: str = 'dot', strict: bool = False, inplace: bool = False) → T

Convert attributes to values that can be written to netCDF or Zarr.

Applies to dataset-level, variable and coordinate attributes, and to every node of a DataTree. An eoio:attr_encoding marker is added to each scope that needed encoding, so decode_attrs() can reverse it exactly.

Parameters:

• obj – Dataset, DataArray or DataTree whose attributes should be encoded.

• dict_style – "dot" to dot-flatten nested dicts, or "json" to encode each as a single JSON blob.

• strict – If True, raise on a value no codec handles. If False, fall back to str() with a warning.

• inplace – Whether to modify obj rather than a copy.

Returns:

The object with encoded attributes.

Raises:

• ValueError – If dict_style is not "dot" or "json".

• TypeError – If strict is set and a value cannot be encoded.