eoio.processors.processor_pipeline module

eoio.processors.processor_pipeline

returns:

Processed xarray Dataset.

rtype:

xarray.Dataset

raises ProcessorPipelineError:

Raised when invalid processor specified. Raised when error occurred in the specified processors during the run.

class eoio.processors.processor_pipeline.ProcessorPipeline(processor_params: Dict[str, Dict[str, Any]] | List[Dict[str, Any]], context: Dict[str, Any] | Context)

Bases: object

Sequential processor pipeline that applies registered EOIO processors to an xarray.Dataset.

The pipeline performs the following steps: - validates that the requested processors exist in the registry - instantiates them with their respective parameters - runs each processor in sequence

:paramEither a dict or a list of single-key dicts mapping processor names to

their parameter dictionaries. The list form allows the same processor to appear more than once with different parameters.

The on_missing parameter can be used to specify how to handle errors for each processor, supported values are: - “error” (default): processor pipeline is stopped returning most recent successful processed dataset - “skip”: processor pipeline omits processor when an error occurs

Keys must exist in PROCESSOR_REGISTRY.

:type : param processor_params: :param : Processing context (reader info, metadata view, logger, etc.). :type : param context:

Example

# Dict form (each processor once): processor_params = {

“processor_name_0”: {param_0:…, param_1:…., “on_missing”:”skip”}, “processor_name_1”: {param_0:…, param_1:…., “on_missing”:”error”}

}

# List form (allows repeated processors): processor_params = [

{“interpolate”: {param_0:…, “on_missing”: “skip”}}, {“interpolate”: {param_0:…, “on_missing”: “error”}},

]

run(ds: Dataset) → Dataset

Run all processors sequentially.

Parameters:

ds – Input dataset.

Returns:

The processed xarray dataset.

Raises:

ProcessorPipelineError – If a processor fails and its on_missing is "error".

exception eoio.processors.processor_pipeline.ProcessorPipelineError

Bases: ValueError

Raised when a processor is invalid or fails during execution.