eoio package

eoio package#

eoio - Data readers and preprocessing for satellite data

eoio.product_options(path: str, read_params: Dict[str, Any] | None = None, *args, **kwargs) → dict[source]#

Return dictionary of available meas_vars options for the requested EO data product

Parameters:

path – satellite data product

:return : dictionary of available subsetting parameters

eoio.read(path: str, vars_sel: Dict[str, List[str]] | None = None, subset: Dict[str, Any] | None = None, read_params: Dict[str, Any] | None = None, processors: Dict[str, Any] | List[Dict[str, Any]] | None = None, *args, **kwargs) → Dataset[source]#

Reads an Earth Observation (EO) data product and returns the requested variables, optionally applying spatial subsetting and post-processing.

Parameters:

  • path – Path to the EO data product (file or directory, depending on product type).

  • vars_sel –

    Variable selection dictionary defining which variables to read from the product.

    Supported keys are:

    • "meas" (list[str] | str | None, default: "all") – Measurement variables to read. Options are:

      • "all" – read all available measurement variables

      • list[str] – explicit list of measurement variable names

      • None – do not read any measurement variables

    • "mask" (list[str] | None, default: None) – Mask variable names to read. Use None to disable mask reading.

    • "aux" (list[str] | None, default: None) – Auxiliary data variable names to read. Use None to disable auxiliary data reading.

    Available variable names and supported combinations for a given product can be inspected via eoio.product_options.

  • subset –

    Optional definition of subsetting parameters. If omitted or None, the full data product is read without subsetting.

    Supported keys (availability depends on the product type):

    • "roi" – Spatial region of interest for raster data. Supported forms are:

      • None – no spatial subsetting

      • shapely geometry (interpreted in roi_crs_epsg)

      • Bounding box tuple (xmin, ymin, xmax, ymax) in roi_crs_epsg

      • GeoJSON-like dict with a "type" key

      • List of [x, y] coordinate pairs defining a polygon

      • ((x, y), half_width_m) defining a square region centred on a point

    • "roi_crs_epsg" (str) – EPSG code defining the coordinate reference system of roi (e.g. "EPSG:4326").

    • "angle" – (Not implemented) Observation geometry range of interest

    • "spectral" – (Not implemented) Spectral range of interest

  • read_params –

    Optional parameters controlling how the data are read. If omitted, default behaviour is used.

    Supported options include:

    • "save_extracted" (bool, default: False) – If the data product is extracted or uncompressed during reading, controls whether the extracted files are saved to disk.

    • "metadata_level" (None | bool, default: None) – Level of metadata to read:

      • None – Standard/core metadata only

      • False – Do not read metadata

      • True – Read all available metadata

  • processors –

    Optional definition of post-processing steps to apply after reading (e.g. interpolation, unit conversion, etc.).

    Dictionary keys should be the names of the processors to run. The associated entry should be a subdictionary of parameters for that processor. Required parameters are defined at the processor level.

Returns:

xarray.Dataset containing the requested variables and associated metadata from the EO data product.

Examples

Read all measurement variables from an EO product:

>>> ds = eoio.read("/path/to/product")

Read selected variables over a spatial region of interest:

>>> from shapely.geometry import box
>>> ds = read_product(
...     path="/path/to/product",
...     vars_sel={
...         "meas": ["B02", "B03", "B04"],
...         "mask": ["cloud_mask"],
...     },
...     subset={
...         "roi": box(-2.0, 50.0, 0.0, 52.0),
...         "roi_crs_epsg": "EPSG:4326",
...     },
... )
eoio.read_multi(paths: str | List[str], vars_sel: Dict[str, List[str]] | None = None, subset: Dict[str, Any] | None = None, read_params: Dict[str, Any] | None = None, processors: Dict[str, Any] | List[Dict[str, Any]] | None = None, concat_dim: str = 'time', concat_coord: str | List[Any] | Callable[[str, Dataset], Any] | None = None, concat_processors: Dict[str, Any] | List[Dict[str, Any]] | None = None, on_mismatch: str = 'error', attrs: str = 'reconcile', coord_attrs: List[str] | None = None, *args, **kwargs) → Dataset[source]#

Read several EO product files as one call and concatenate them along a new (or existing) dimension.

Each path is read independently via read() (with identical vars_sel/subset/read_params/processors applied to every file), then the resulting datasets are concatenated along concat_dim.

Typical uses are an in-situ time series (e.g. one Hypernets file per measurement cycle, each already carrying its own time dimension) or a multi-date raster stack (e.g. several Sentinel-2 scenes stacked along a new time dimension).

Parameters:

  • paths – Explicit list of product paths, a directory (all files inside it recognised by a registered reader), or a glob pattern. All entries are expected to resolve to the same reader type.

  • vars_sel – See read(). Applied identically to every file.

  • subset – See read(). Applied identically to every file.

  • read_params – See read(). Applied identically to every file.

  • processors – Post-processing steps run per file, before concatenation. See read().

  • concat_dim –

    Name of the dimension to concatenate along. Default "time".

    If concat_dim is already a dimension on every per-file dataset (e.g. in-situ readers that carry their own time dimension), the datasets are concatenated directly. Otherwise a new dimension is created per file via expand_dims, using the coordinate value resolved from concat_coord (e.g. a multi-date raster stack).

  • concat_coord –

    How to compute the coordinate value for each file along concat_dim when a new dimension needs to be created:

    • str – dot-path into ds.attrs, e.g. "product_metadata.sensing_time".

    • list – explicit values, one per path, same order as the resolved paths.

    • callable(path, ds) -> value – escape hatch, e.g. to parse a timestamp out of the filename.

    • None (default) – if concat_dim already exists on every dataset, used as-is; otherwise falls back to file order as an integer index coordinate, with a warning.

  • concat_processors – Post-processing steps run once on the final concatenated dataset.

  • on_mismatch –

    How to handle datasets whose variable set or per-file shape (other than concat_dim) doesn’t match — this also covers raster grid mismatches (a mismatched x/y shape is just another schema mismatch):

    • "error" (default) – raise, naming the offending files.

    • "skip" – drop mismatched files, with a warning.

    • "union" – outer-join concat; missing values become NaN.

  • attrs –

    How to combine dataset-level and per-variable attrs across files:

    • "reconcile" (default) – attrs identical across every file are kept; attrs that differ are kept as a per-file list attr (or promoted to a coordinate, see coord_attrs) rather than being silently dropped.

    • "first" – cheap legacy behaviour: just keep the first file’s attrs, discarding the rest (xarray’s own xr.concat default).

  • coord_attrs – Dataset-level attribute names to promote to a coordinate on concat_dim instead of being kept as a list attr — e.g. coord_attrs=["product_name"] gives you a per-file product_name coordinate alongside time. Only used when attrs="reconcile".

Returns:

The concatenated xarray.Dataset.

eoio.write(obj: Dataset | DataTree, path: str | Path, writer: str, write_params: Dict[str, Any] | None = None, processors: Dict[str, Any] | List[Dict[str, Any]] | None = None) → str[source]#

Writes an xarray object to disk in the requested format.

Parameters:

  • obj – xarray.Dataset or xarray.DataTree to write.

  • path – Destination path. Missing parent directories are created unless the create_parents write parameter says otherwise.

  • writer –

    Name of the writer to use, e.g. "netcdf" or "zarr". Available names can be listed with eoio.writers.WriterFactory.list_writers().

    Unlike reading, where the format is a property of the file being opened and is detected automatically, the output format is a choice and so must be stated. It is deliberately not guessed from the file extension.

  • write_params –

    Optional parameters controlling how the data are written. Which parameters are accepted depends on the writer; inspect them with eoio.writer_options.

    Supported by every writer:

    • "encode_attrs" (bool, default: True) – Encode attributes that the format cannot store natively (nested dictionaries, datetimes, geometries) so that they survive the round trip.

    • "on_nonconformance" (str, default: "warn") – What to do when the dataset does not conform to CF-1.8 / ACDD / the controlled vocabulary: "warn", "error" or "ignore".

    • "update_history" (bool, default: True) – Record this write in the history and eoio:processing_steps attributes.

    • "create_parents" (bool, default: True) – Create missing parent directories of path.

  • processors –

    Optional definition of processing steps to apply before writing, in the same form as read(). Useful for expressing read-transform-write as two calls.

    Unlike in read(), the processing context’s path is the destination path given here, not a source product - write is not told where obj came from. A processor that needs the source product should be given it in its own parameters.

Returns:

The path written, as a string.

Raises:

  • ValueError – If writer is not a known writer name.

  • TypeError – If the chosen writer cannot express obj.

Examples

Write a dataset to netCDF:

>>> eoio.write(ds, "/path/to/output.nc", writer="netcdf")

Write to Zarr, appending along time as new observations arrive. The first call creates the store and later ones extend it:

>>> for scene in scenes:
...     eoio.write(
...         scene,
...         "/path/to/stack.zarr",
...         writer="zarr",
...         write_params={"append_dim": "time"},
...     )
eoio.writer_options(writer: str) → dict[source]#

Return dictionary of available write_params options for the requested writer.

Parameters:

writer – Name of the writer, e.g. "netcdf".

Returns:

Dictionary of available writing parameters.

Raises:

ValueError – If writer is not a known writer name.

Subpackages#

Submodules#