config
The Pyomo Configuration System
The Pyomo config system provides a set of three classes
(ConfigDict
, ConfigList
, and
ConfigValue
) for managing and documenting structured
configuration information and user input. The system is based around
the ConfigValue class, which provides storage for a single configuration
entry. ConfigValue objects can be grouped using two containers
(ConfigDict and ConfigList), which provide functionality analogous to
Python’s dict and list classes, respectively.
At its simplest, the Config system allows for developers to specify a dictionary of documented configuration entries:
from pyomo.common.config import (
ConfigDict, ConfigList, ConfigValue
)
config = ConfigDict()
config.declare('filename', ConfigValue(
default=None,
domain=str,
description="Input file name",
))
config.declare("bound tolerance", ConfigValue(
default=1E-5,
domain=float,
description="Bound tolerance",
doc="Relative tolerance for bound feasibility checks"
))
config.declare("iteration limit", ConfigValue(
default=30,
domain=int,
description="Iteration limit",
doc="Number of maximum iterations in the decomposition methods"
))
Users can then provide values for those entries, and retrieve the current values:
>>> config['filename'] = 'tmp.txt'
>>> print(config['filename'])
tmp.txt
>>> print(config['iteration limit'])
30
For convenience, ConfigDict objects support read/write access via attributes (with spaces in the declaration names replaced by underscores):
>>> print(config.filename)
tmp.txt
>>> print(config.iteration_limit)
30
>>> config.iteration_limit = 20
>>> print(config.iteration_limit)
20
Domain validation
All Config objects support a domain
keyword that accepts a callable
object (type, function, or callable instance). The domain callable
should take data and map it onto the desired domain, optionally
performing domain validation (see ConfigValue
,
ConfigDict
, and ConfigList
for more
information). This allows client code to accept a very flexible set of
inputs without “cluttering” the code with input validation:
>>> config.iteration_limit = 35.5
>>> print(config.iteration_limit)
35
>>> print(type(config.iteration_limit).__name__)
int
In addition to common types (like int
, float
, bool
, and
str
), the config system profides a number of custom domain
validators for common use cases:
|
Domain validator for bool-like objects. |
|
Domain validation function admitting integers |
|
Domain validation function admitting strictly positive integers |
|
Domain validation function admitting strictly negative integers |
|
Domain validation function admitting integers >= 0 |
|
Domain validation function admitting integers <= 0 |
|
Domain validation function admitting strictly positive numbers |
|
Domain validation function admitting strictly negative numbers |
|
Domain validation function admitting numbers less than or equal to 0 |
|
Domain validation function admitting numbers greater than or equal to 0 |
|
Domain validation class admitting a Container of possible values |
|
Domain validation class admitting an enum value/name. |
|
Domain validator for type checking. |
|
Domain validator for lists of a specified type |
|
Domain validator for modules. |
|
Domain validator for a path-like object. |
|
Domain validator for a list of path-like objects. |
|
Implicit domain that can return a custom domain based on the key. |
Configuring class hierarchies
A feature of the Config system is that the core classes all implement
__call__
, and can themselves be used as domain
values. Beyond
providing domain verification for complex hierarchical structures, this
feature allows ConfigDicts to cleanly support the configuration of
derived objects. Consider the following example:
>>> class Base(object):
... CONFIG = ConfigDict()
... CONFIG.declare('filename', ConfigValue(
... default='input.txt',
... domain=str,
... ))
... def __init__(self, **kwds):
... c = self.CONFIG(kwds)
... c.display()
...
>>> class Derived(Base):
... CONFIG = Base.CONFIG()
... CONFIG.declare('pattern', ConfigValue(
... default=None,
... domain=str,
... ))
...
>>> tmp = Base(filename='foo.txt')
filename: foo.txt
>>> tmp = Derived(pattern='.*warning')
filename: input.txt
pattern: .*warning
Here, the base class Base
declares a class-level attribute CONFIG as a
ConfigDict containing a single entry (filename
). The derived class
(Derived
) then starts by making a copy of the base class’ CONFIG
,
and then defines an additional entry (pattern). Instances of the base
class will still create c
instances that only have the single
filename
entry, whereas instances of the derived class will have c
instances with two entries: the pattern
entry declared by the derived
class, and the filename
entry “inherited” from the base class.
An extension of this design pattern provides a clean approach for
handling “ephemeral” instance options. Consider an interface to an
external “solver”. Our class implements a solve()
method that takes a
problem and sends it to the solver along with some solver configuration
options. We would like to be able to set those options “persistently”
on instances of the interface class, but still override them
“temporarily” for individual calls to solve()
. We implement this by
creating copies of the class’s configuration for both specific instances
and for use by each solve()
call:
>>> class Solver(object):
... CONFIG = ConfigDict()
... CONFIG.declare('iterlim', ConfigValue(
... default=10,
... domain=int,
... ))
... def __init__(self, **kwds):
... self.config = self.CONFIG(kwds)
... def solve(self, model, **options):
... config = self.config(options)
... # Solve the model with the specified iterlim
... config.display()
...
>>> solver = Solver()
>>> solver.solve(None)
iterlim: 10
>>> solver.config.iterlim = 20
>>> solver.solve(None)
iterlim: 20
>>> solver.solve(None, iterlim=50)
iterlim: 50
>>> solver.solve(None)
iterlim: 20
Interacting with argparse
In addition to basic storage and retrieval, the Config system provides
hooks to the argparse command-line argument parsing system. Individual
Config entries can be declared as argparse arguments using the
declare_as_argument()
method. To make declaration
simpler, the declare()
method returns the declared Config
object so that the argument declaration can be done inline:
import argparse
config = ConfigDict()
config.declare('iterlim', ConfigValue(
domain=int,
default=100,
description="iteration limit",
)).declare_as_argument()
config.declare('lbfgs', ConfigValue(
domain=bool,
description="use limited memory BFGS update",
)).declare_as_argument()
config.declare('linesearch', ConfigValue(
domain=bool,
default=True,
description="use line search",
)).declare_as_argument()
config.declare('relative tolerance', ConfigValue(
domain=float,
description="relative convergence tolerance",
)).declare_as_argument('--reltol', '-r', group='Tolerances')
config.declare('absolute tolerance', ConfigValue(
domain=float,
description="absolute convergence tolerance",
)).declare_as_argument('--abstol', '-a', group='Tolerances')
The ConfigDict can then be used to initialize (or augment) an argparse ArgumentParser object:
parser = argparse.ArgumentParser("tester")
config.initialize_argparse(parser)
Key information from the ConfigDict is automatically transferred over to the ArgumentParser object:
>>> print(parser.format_help())
usage: tester [-h] [--iterlim INT] [--lbfgs] [--disable-linesearch]
[--reltol FLOAT] [--abstol FLOAT]
...
-h, --help show this help message and exit
--iterlim INT iteration limit
--lbfgs use limited memory BFGS update
--disable-linesearch [DON'T] use line search
Tolerances:
--reltol... -r FLOAT relative convergence tolerance
--abstol... -a FLOAT absolute convergence tolerance
Parsed arguments can then be imported back into the ConfigDict:
>>> args=parser.parse_args(['--lbfgs', '--reltol', '0.1', '-a', '0.2'])
>>> args = config.import_argparse(args)
>>> config.display()
iterlim: 100
lbfgs: true
linesearch: true
relative tolerance: 0.1
absolute tolerance: 0.2
Accessing user-specified values
It is frequently useful to know which values a user explicitly set, and
which values a user explicitly set but have never been retrieved. The
configuration system provides two generator methods to return the items
that a user explicitly set (user_values()
) and the items that
were set but never retrieved (unused_user_values()
):
>>> print([val.name() for val in config.user_values()])
['lbfgs', 'relative tolerance', 'absolute tolerance']
>>> print(config.relative_tolerance)
0.1
>>> print([val.name() for val in config.unused_user_values()])
['lbfgs', 'absolute tolerance']
Generating output & documentation
Configuration objects support three methods for generating output and
documentation: display()
,
generate_yaml_template()
, and
generate_documentation()
. The simplest is
display()
, which prints out the current values of the
configuration object (and if it is a container type, all of it’s
children). generate_yaml_template()
is similar to
display()
, but also includes the description fields as
formatted comments.
solver_config = config
config = ConfigDict()
config.declare('output', ConfigValue(
default='results.yml',
domain=str,
description='output results filename'
))
config.declare('verbose', ConfigValue(
default=0,
domain=int,
description='output verbosity',
doc='This sets the system verbosity. The default (0) only logs '
'warnings and errors. Larger integer values will produce '
'additional log messages.',
))
config.declare('solvers', ConfigList(
domain=solver_config,
description='list of solvers to apply',
))
>>> config.display()
output: results.yml
verbose: 0
solvers: []
>>> print(config.generate_yaml_template())
output: results.yml # output results filename
verbose: 0 # output verbosity
solvers: [] # list of solvers to apply
It is important to note that both methods document the current state of the configuration object. So, in the example above, since the solvers list is empty, you will not get any information on the elements in the list. Of course, if you add a value to the list, then the data will be output:
>>> tmp = config()
>>> tmp.solvers.append({})
>>> tmp.display()
output: results.yml
verbose: 0
solvers:
-
iterlim: 100
lbfgs: true
linesearch: true
relative tolerance: 0.1
absolute tolerance: 0.2
>>> print(tmp.generate_yaml_template())
output: results.yml # output results filename
verbose: 0 # output verbosity
solvers: # list of solvers to apply
-
iterlim: 100 # iteration limit
lbfgs: true # use limited memory BFGS update
linesearch: true # use line search
relative tolerance: 0.1 # relative convergence tolerance
absolute tolerance: 0.2 # absolute convergence tolerance
The third method (generate_documentation()
) behaves
differently. This method is designed to generate reference
documentation. For each configuration item, the doc field is output.
If the item has no doc, then the description field is used.
List containers have their domain documented and not their current values. The documentation can be configured through optional arguments. The defaults generate LaTeX documentation:
>>> print(config.generate_documentation())
\begin{description}[topsep=0pt,parsep=0.5em,itemsep=-0.4em]
\item[{output}]\hfill
\\output results filename
\item[{verbose}]\hfill
\\This sets the system verbosity. The default (0) only logs warnings and
errors. Larger integer values will produce additional log messages.
\item[{solvers}]\hfill
\\list of solvers to apply
\begin{description}[topsep=0pt,parsep=0.5em,itemsep=-0.4em]
\item[{iterlim}]\hfill
\\iteration limit
\item[{lbfgs}]\hfill
\\use limited memory BFGS update
\item[{linesearch}]\hfill
\\use line search
\item[{relative tolerance}]\hfill
\\relative convergence tolerance
\item[{absolute tolerance}]\hfill
\\absolute convergence tolerance
\end{description}
\end{description}
Enums
|
DEPRECATED. |
Classes
|
|
alias of |
|
|
Store and manipulate a dictionary of configuration values. |
|
Store and manipulate a list of configuration values. |
|
Store and manipulate a single configuration value. |
|
Implicit domain that can return a custom domain based on the key. |
|
|
|
Domain validation class admitting a Container of possible values |
|
Domain validation class admitting an enum value/name. |
|
Domain validator for type checking. |
|
Domain validator for lists of a specified type |
|
Mark instances of ConfigValue as immutable. |
|
Domain validator for modules. |
|
Domain validator for a path-like object. |
|
Domain validator for a list of path-like objects. |
|
|
Mixin class to support delayed data initialization. |
|
|
Decorator to append the documentation of a ConfigDict to the docstring |
Functions
|
Domain validator for bool-like objects. |
|
Domain validation function admitting integers |
|
Domain validation function admitting strictly negative numbers |
|
Domain validation function admitting strictly negative integers |
|
Domain validation function admitting numbers greater than or equal to 0 |
|
Domain validation function admitting integers >= 0 |
|
Domain validation function admitting numbers less than or equal to 0 |
|
Domain validation function admitting integers <= 0 |
|
Domain validation function admitting strictly positive numbers |
|
Domain validation function admitting strictly positive integers |
|
DEPRECATED. |