eoio.processors.processor_pipeline.ProcessorPipeline#
- class eoio.processors.processor_pipeline.ProcessorPipeline(processor_params: Dict[str, Dict[str, Any]], context: Dict[str, Any] | Context)[source]#
Sequential processor pipeline that applies registered EOIO processors to an xarray.Dataset.
The pipeline performs the following steps: - validates that the requested processors exist in the registry - instantiates them with their respective parameters - runs each processor in sequence
:param : Mapping of processor names to their specific parameter dictionaries.
The on_missing parameter can be used to specify how to handle errors for each processor, supported values are: - “error” (default): processor pipeline is stopped returning most recent successful processed dataset - “skip”: processor pipeline omits processor when an error occurs
Keys must exist in
PROCESSOR_REGISTRY.:type : param processor_params: :param : Processing context (reader info, metadata view, logger, etc.). :type : param context:
Example
- processor_params = {
“processor_name_0”: {param_0:…, param_1:…., “on_missing”:”skip”}, “processor_name_1”: {param_0:…, param_1:…., “on_missing”:”error”} }
context = {…}
processed_ds = ProcessorPipeline(processor_params: Dict[str, Dict[str, Any]], context: Dict[str, Any]).run(ds: xr.Dataset)
- __init__(processor_params: Dict[str, Dict[str, Any]], context: Dict[str, Any] | Context) None[source]#
Methods