Writers

Writers #

This guide describes how to add a new dataset writer to eoio.

Overview #

Writers are the counterpart to readers: a reader turns a product on disk into an xarray object, a writer turns an xarray object back into something on disk.

Like readers, writers are thin orchestrators. The base class owns everything that is the same for every format — configuration, capability checking, conformance checking, provenance, attribute encoding — and the concrete writer owns only the format itself.

The writing pipeline is:

capability check → conformance check → provenance → attribute encoding → format write

Typical package layout for a writer named myformat:

eoio/writers/myformat/
    writer.py       # orchestrator - thin
    encoding.py     # optional: format-specific encoding/compression defaults
    tests/          # unit tests

Note

Place the writer package in eoio/writers/. Place test files in a tests/ sub-package within the writer directory.

Useful API References #

`writers.base.BaseWriter`	Base class for eoio writers.
`writers.base.WriteCapability`	Whether a writer can express a given object in its format.
`writers.base.WriterConfig`	Normalised, validated configuration for a writer instance.
`writers.factory.WriterFactory`	Writer Factory class
`writers.factory.register_writer`	Register a writer class under a name.

Selecting a Writer #

Writers are selected by name, never inferred from the output path:

from eoio.writers import WriterFactory

writer_cls = WriterFactory.get_writer("netcdf")
writer_cls().write(ds, "output.nc")

This is deliberately unlike ReaderFactory, which matches the product path against regular expressions. On read, the format is a fact about the input that has to be discovered; on write, it is the user’s choice. Inferring it from the output extension is rejected because it silently guesses, and because it does not extend to directory-style products such as .SAFE.

Creating a Writer Class #

Required Contract #

A concrete writer must:

Inherit from BaseWriter.
Implement write_dataset().
Implement the static method get_extension().
Register it with @register_writer("name"), which sets name for you.

Attribute / Method	Purpose
`name`	Identifier the writer is selected under (e.g. `"netcdf"`). Set by `@register_writer`; only set it by hand for an unregistered writer
`default_write_params`	Default write-time parameters, extending the base defaults
`all_write_params`	Available values for each parameter, surfaced through `all_options`
`write_dataset()`	Write the object in this format — the only abstract method
`get_extension()`	Return the filename extension produced (e.g. `".nc"`)
`can_write()`	Optional. Refuse objects the format cannot express
`check_conformance()`	Optional. Report CF/ACDD/vocabulary problems before writing

Public Entrypoint #

Users call writer.write(obj, path) — not write_dataset() directly. write() is provided by the base class and performs the shared orchestration before calling write_dataset().

Unlike a reader, which is constructed around a single product path, a writer instance holds only configuration and can be reused across many outputs:

writer = SomeWriter(write_params={"engine": "netcdf4"})
writer.write(ds_a, "a.nc")
writer.write(ds_b, "b.nc")

Unknown write_params keys raise a ValueError at initialisation, matching the reader’s strict behaviour. Merged configuration is stored in writer.config, a WriterConfig dataclass.

Minimal Example #

from pathlib import Path
import xarray as xr
from eoio.writers.base import BaseWriter, WritableObject


from eoio.writers import register_writer


@register_writer("myformat")
class MyWriter(BaseWriter):

    default_write_params = {
        **BaseWriter.default_write_params,
        "compression": "zlib",
    }
    all_write_params = {
        **BaseWriter.all_write_params,
        "compression": "zlib/none",
    }

    @staticmethod
    def get_extension() -> str:
        return ".myformat"

    def write_dataset(self, obj: WritableObject, path: Path) -> None:
        # attrs are already encoded and provenance already applied
        ...

Base Write Parameters #

Every writer inherits these, and should extend rather than replace them:

Parameter	Default	Description
`encode_attrs`	`True`	Encode attributes for serialisation before writing (see `eoio.serialisation`)
`on_nonconformance`	`"warn"`	What to do when the conformance check reports problems: `warn`/`error`/`ignore`
`update_history`	`True`	Append this write to `history` and `eoio:processing_steps`
`create_parents`	`True`	Create missing parent directories for the output path

Extend them with **BaseWriter.default_write_params so the base options keep working.

Declaring Capability #

can_write() reports whether the writer can express a given object, and why not:

def can_write(self, obj):
    base = super().can_write(obj)
    if not base.ok:
        return base
    if obj.attrs.get("instrument") != "MSI":
        return WriteCapability(False, "requires a Sentinel-2 MSI dataset")
    return WriteCapability(True)

General-purpose writers (netCDF, Zarr) accept any Dataset or DataTree — anything expressible in xarray is expressible in those formats — so they simply inherit the default.

Format-specific writers should refuse anything of the wrong shape. The reason string is shown to the user, so it should say what is missing, not merely that something is.

Attribute Encoding #

Reader output routinely carries attributes that no serialisation format can store: nested metadata dicts, datetime objects, geometries. write() runs eoio.serialisation.encode_attrs() before calling write_dataset(), so writers do not have to think about it.

Writers should not encode attributes themselves. If a format needs different treatment, override write_dataset() and set encode_attrs=False in the write params rather than encoding twice.

Provenance #

write() appends to two attributes before writing, so they appear in the written file:

history — a CF-style line recording the output filename, writer and eoio version
eoio:processing_steps — a step entry in the same shape read_multi uses, so a dataset’s processing history reads consistently from either end

The input object is never modified; write() works on a copy.

Registering a Writer #

Built-in writers are listed in eoio/writers/factory.py as lazy (module, class) entries, so an optional backend is only imported when that writer is used:

_BUILTIN_WRITERS = {
    "netcdf": ("eoio.writers.netcdf.writer", "NetCDFWriter"),
}

Writers defined outside that table register with register_writer, a decorator in the same style as @register_processor:

from eoio.writers import register_writer

@register_writer("myformat")
class MyWriter(BaseWriter):
    ...

It can also be called directly, which is what you want for a class you do not own:

register_writer("geotiff", SomeThirdPartyWriter)

Either form sets the class’s name, so a registered writer always knows what it is called — which is what ends up in the provenance a write records.

Testing #

Writer tests should not need real product files — construct a small Dataset in the test and write it to tmp_path. See eoio/writers/tests/test_base.py, which uses a dummy writer to exercise the shared orchestration without touching a real format.

Run ruff check eoio and pytest before committing.

Writers

Contents