eoio.processors.processor_pipeline module

eoio.processors.processor_pipeline

returns:

Processed xarray Dataset.

rtype:

xarray.Dataset

raises ProcessorPipelineError:

Raised when invalid processor specified. Raised when error occurred in the specified processors during the run.

class eoio.processors.processor_pipeline.ProcessorPipeline(processor_params: Dict[str, Dict[str, Any]], context: Dict[str, Any] | Context)

Bases: object

Sequential processor pipeline that applies registered EOIO processors to an xarray.Dataset.

The pipeline performs the following steps: - validates that the requested processors exist in the registry - instantiates them with their respective parameters - runs each processor in sequence

:param : Mapping of processor names to their specific parameter dictionaries.

The on_missing parameter can be used to specify how to handle errors for each processor, supported values are: - “error” (default): processor pipeline is stopped returning most recent successful processed dataset - “skip”: processor pipeline omits processor when an error occurs

Keys must exist in PROCESSOR_REGISTRY.

:type : param processor_params: :param : Processing context (reader info, metadata view, logger, etc.). :type : param context:

Example

processor_params = {

“processor_name_0”: {param_0:…, param_1:…., “on_missing”:”skip”}, “processor_name_1”: {param_0:…, param_1:…., “on_missing”:”error”} }

context = {…}

processed_ds = ProcessorPipeline(processor_params: Dict[str, Dict[str, Any]], context: Dict[str, Any]).run(ds: xr.Dataset)

run(ds: Dataset) → Dataset

Run all processors sequentially.

Parameters:

ds – Input dataset.

Returns:

The processed xarray dataset.

Raises:

ProcessorPipelineError – If a processor fails and its on_missing is "error".

exception eoio.processors.processor_pipeline.ProcessorPipelineError

Bases: ValueError

Raised when a processor is invalid or fails during execution.