Determining and Storing Processing State

Determining and Storing Processing State#

Processor classes can be initialised with a defined context. The context object is a container for storing data defining the processor state/configuration. This can be accessed within the processor through the context attribute.

context may be a simple dict, however processor_tools provides a specific Context object that provides useful extra functionality for handling Processor state/configuration.

This page provides a guide on how to use Context objects.

Building a Context object#

Context objects can be defined with a dictionary of configuration values - as follows:

In [1]: from processor_tools import Context

In [2]: config_vals = {
   ...:     "entry1": "value1",
   ...:     "entry2": "value2"
   ...: }
   ...: 

In [3]: context = Context(config_vals)

They can also be built from an equivalent configuration file (of a format readable by read_config), by defining the file path.

So if you define the configuration file "context_file.yaml", with the content:

entry1: value1
entry2: value2

This can be loaded into a Context object as:

In [4]: path = "context_file.yaml"

In [5]: context = Context(path)

The path can also be to directory containing set of configuration files.

Providing a list of configuration paths or dictionaries loads the values from each of the configuration files/directories/dictionaries.

In the case where multiple input configuration files provide the same value, earlier in the list overwrites later in the list. So in the example,

In [6]: path1 = "context_file1.yaml"

In [7]: path2 = "context_file2.yaml"

In [8]: dict1 = {"entry3": "value3"}

In [9]: context = Context([path1, path2, dict1])

If the same configuration value is defined multiple times, the value in "context_file1.yaml" overwrites that in "context_file2.yaml" which overwrites that in dict1.

The Context class variable default_config enables you to set configuration file(s)/directory(ies)/dictionaries that are loaded every time the class is initialised. These configuration values come lower in the priority list than those defined at initialisation.

Therefore, the following has the same effect as the previous example:

In [10]: Context.default_config = [path2, dict1]

In [11]: context = Context(path1)

Interfacing with the Context object#

The Processor context can be accessed within the processor through the context attribute.

Configuration values are listed in the context keys keys.

In [12]: print(context.keys())
['entry3', 'entry1', 'entry2']

Configuration values can be accessed by indexing:

In [13]: print(context["entry1"])
value1

In [14]: context["entry4"] = "value4"

In [15]: print(context.keys())
['entry3', 'entry1', 'entry2', 'entry4']

The update method allows the updating of multiple items (as a deep update) as follows:

In [16]: print(context.config_values)
{'entry3': 'value3', 'entry1': 'value1', 'entry2': 'value2', 'entry4': 'value4'}

In [17]: context.update({"entry1": "new1", "entry2": "new2"})

In [18]: print(context.config_values)
{'entry3': 'value3', 'entry1': 'new1', 'entry2': 'new2', 'entry4': 'value4'}

The configuration values in a context object can be written to a file with the write_config method, as follows:

In [19]: output_path = "output_config.yaml"

In [20]: context.write_config(output_path)

Inheriting values between context objects#

One Context object can inherit configuration values from another, by setting it’s “supercontext”.

This can be done when a context object is initialised:

In [21]: context1 = Context()

In [22]: context2 = Context(supercontext=context1)

Or when the context object is already initialised with the supercontext property.

In [23]: context1 = Context()

In [24]: context2 = Context()

In [25]: context2.supercontext = context1

Values in a context object are overridden by those in its supercontext.

In [26]: context1 = Context({"val1": 1})

In [27]: context2 = Context({"val1": 2})

In [28]: print(context2["val1"])
2

In [29]: context2.supercontext = context1

In [30]: print(context2["val1"])
1

Inheritance between a context object and its supercontext may be limited to a single section of the supercontext, by defining a tuple of:

supercontext (Context) - supercontext object
section (str) - name of section of supercontext to apply as supercontext

In [31]: context1 = Context({
   ....:     "section1": {
   ....:         "val1": "super"
   ....:     },
   ....:     "section2": {
   ....:         "val2": "super"
   ....:     }
   ....: })
   ....: 

In [32]: context2 = Context({"val1": 1, "val2": 2})

In [33]: context2.supercontext = (context1, "section1")

In [34]: print(context2["val1"], context2["val2"])
super 2

Setting a Global Supercontext#

To inherit configuration values between packages and processes a global supercontext may be set, using the set_global_supercontext function. The configuration values override those set in any instantiated context object.

In [35]: from processor_tools import set_global_supercontext

In [36]: context = Context({"val1": 1})

In [37]: print(context["val1"])
1

In [38]: global_supercontext = Context({"val1": "global"})

In [39]: set_global_supercontext(global_supercontext)
Out[39]: <processor_tools.context.set_global_supercontext at 0x7f44d83d9160>

In [40]: print(context["val1"])
global

You can clear all the set global supercontexts with clear_global_supercontext function.

In [41]: from processor_tools import clear_global_supercontext

In [42]: clear_global_supercontext()

In [43]: print(context["val1"])
1

A convenient way to implement this is with a with statement, as follows

In [44]: from processor_tools import set_global_supercontext

In [45]: print(context["val1"])
1

In [46]: with set_global_supercontext(global_supercontext):
   ....:     print(context["val1"])
   ....: 
global

In [47]: print(context["val1"])
1