Future Solver Interface Changes
Note
The new solver interfaces are still under active development. They are included in the releases as development previews. Please be aware that APIs and functionality may change with no notice.
We welcome any feedback and ideas as we develop this capability. Please post feedback on Issue 1030.
Pyomo offers interfaces into multiple solvers, both commercial and open
source. To support better capabilities for solver interfaces, the Pyomo
team is actively redesigning the existing interfaces to make them more
maintainable and intuitive for use. A preview of the redesigned
interfaces can be found in pyomo.contrib.solver
.
New Interface Usage
The new interfaces are not completely backwards compatible with the
existing Pyomo solver interfaces. However, to aid in testing and
evaluation, we are distributing versions of the new solver interfaces
that are compatible with the existing (“legacy”) solver interface.
These “legacy” interfaces are registered with the current
SolverFactory
using slightly different names (to avoid conflicts
with existing interfaces).
Solver |
Name registered in the |
Name registered in the |
---|---|---|
Ipopt |
|
|
Gurobi (persistent) |
|
|
Gurobi (direct) |
|
|
Using the new interfaces through the legacy interface
Here we use the new interface as exposed through the existing (legacy) solver factory and solver interface wrapper. This provides an API that is compatible with the existing (legacy) Pyomo solver interface and can be used with other Pyomo tools / capabilities.
import pyomo.environ as pyo
from pyomo.contrib.solver.util import assert_optimal_termination
model = pyo.ConcreteModel()
model.x = pyo.Var(initialize=1.5)
model.y = pyo.Var(initialize=1.5)
def rosenbrock(model):
return (1.0 - model.x) ** 2 + 100.0 * (model.y - model.x**2) ** 2
model.obj = pyo.Objective(rule=rosenbrock, sense=pyo.minimize)
status = pyo.SolverFactory('ipopt_v2').solve(model)
assert_optimal_termination(status)
model.pprint()
In keeping with our commitment to backwards compatibility, both the legacy and future methods of specifying solver options are supported:
import pyomo.environ as pyo
model = pyo.ConcreteModel()
model.x = pyo.Var(initialize=1.5)
model.y = pyo.Var(initialize=1.5)
def rosenbrock(model):
return (1.0 - model.x) ** 2 + 100.0 * (model.y - model.x**2) ** 2
model.obj = pyo.Objective(rule=rosenbrock, sense=pyo.minimize)
# Backwards compatible
status = pyo.SolverFactory('ipopt_v2').solve(model, options={'max_iter' : 6})
# Forwards compatible
status = pyo.SolverFactory('ipopt_v2').solve(model, solver_options={'max_iter' : 6})
model.pprint()
Using the new interfaces directly
Here we use the new interface by importing it directly:
# Direct import
import pyomo.environ as pyo
from pyomo.contrib.solver.util import assert_optimal_termination
from pyomo.contrib.solver.ipopt import Ipopt
model = pyo.ConcreteModel()
model.x = pyo.Var(initialize=1.5)
model.y = pyo.Var(initialize=1.5)
def rosenbrock(model):
return (1.0 - model.x) ** 2 + 100.0 * (model.y - model.x**2) ** 2
model.obj = pyo.Objective(rule=rosenbrock, sense=pyo.minimize)
opt = Ipopt()
status = opt.solve(model)
assert_optimal_termination(status)
# Displays important results information; only available through the new interfaces
status.display()
model.pprint()
Using the new interfaces through the “new” SolverFactory
Here we use the new interface by retrieving it from the new SolverFactory
:
# Import through new SolverFactory
import pyomo.environ as pyo
from pyomo.contrib.solver.util import assert_optimal_termination
from pyomo.contrib.solver.factory import SolverFactory
model = pyo.ConcreteModel()
model.x = pyo.Var(initialize=1.5)
model.y = pyo.Var(initialize=1.5)
def rosenbrock(model):
return (1.0 - model.x) ** 2 + 100.0 * (model.y - model.x**2) ** 2
model.obj = pyo.Objective(rule=rosenbrock, sense=pyo.minimize)
opt = SolverFactory('ipopt')
status = opt.solve(model)
assert_optimal_termination(status)
# Displays important results information; only available through the new interfaces
status.display()
model.pprint()
Switching all of Pyomo to use the new interfaces
We also provide a mechanism to get a “preview” of the future where we replace the existing (legacy) SolverFactory and utilities with the new (development) version (see Preview capabilities through pyomo.__future__):
# Change default SolverFactory version
import pyomo.environ as pyo
from pyomo.contrib.solver.util import assert_optimal_termination
from pyomo.__future__ import solver_factory_v3
model = pyo.ConcreteModel()
model.x = pyo.Var(initialize=1.5)
model.y = pyo.Var(initialize=1.5)
def rosenbrock(model):
return (1.0 - model.x) ** 2 + 100.0 * (model.y - model.x**2) ** 2
model.obj = pyo.Objective(rule=rosenbrock, sense=pyo.minimize)
status = pyo.SolverFactory('ipopt').solve(model)
assert_optimal_termination(status)
# Displays important results information; only available through the new interfaces
status.display()
model.pprint()
Linear Presolve and Scaling
The new interface allows access to new capabilities in the various
problem writers, including the linear presolve and scaling options
recently incorporated into the redesigned NL writer. For example, you
can control the NL writer in the new ipopt
interface through the
solver’s writer_config
configuration option:
- class pyomo.contrib.solver.ipopt.Ipopt(**kwds)[source]
- solve(model, **kwds)[source]
- Keyword Arguments:
tee (TextIO_or_Logger, default=False) –
tee
acceptsbool
,io.TextIOBase
, orlogging.Logger
(or a list of these types).True
is mapped tosys.stdout
. The solver log will be printed to each of these streams / destinations.working_dir (Path, optional) – The directory in which generated files should be saved. This replaces the keepfiles option.
load_solutions (Bool, default=True) – If True, the values of the primal variables will be loaded into the model.
raise_exception_on_nonoptimal_result (Bool, default=True) – If False, the solve method will continue processing even if the returned result is nonoptimal.
symbolic_solver_labels (Bool, default=False) – If True, the names given to the solver will reflect the names of the Pyomo components. Cannot be changed after set_instance is called.
timer (optional) – A timer object for recording relevant process timing data.
threads (NonNegativeInt, optional) – Number of threads to be used by a solver.
time_limit (NonNegativeFloat, optional) – Time limit applied to the solver (in seconds).
solver_options (dict, optional) – Options to pass to the solver.
executable (default=<pyomo.common.fileutils.ExecutableData object at 0x7fe15ff31100>) – Preferred executable for ipopt. Defaults to searching the
PATH
for the first availableipopt
.writer_config (dict, optional) –
nlwriter
- show_section_timing: bool, default=False
Print timing after writing each section of the NL file
- skip_trivial_constraints: bool, default=True
Skip writing constraints whose body is constant
- file_determinism: InEnum[FileDeterminism], default=<FileDeterminism.ORDERED: 10>
How much effort do we want to put into ensuring the NL file is written deterministically for a Pyomo model:
NONE (0) : None ORDERED (10): rely on underlying component ordering (default) SORT_INDICES (20) : sort keys of indexed components SORT_SYMBOLS (30) : sort keys AND sort names (not declaration order)
- symbolic_solver_labels: bool, default=False
Write the corresponding .row and .col files
- scale_model: bool, default=True
If True, then the writer will output the model constraints and variables in ‘scaled space’ using the scaling from the ‘scaling_factor’ Suffix, if provided.
- export_nonlinear_variables: list, optional
List of variables to ensure are in the NL file (even if they don’t appear in any constraints).
- row_order: optional
List of constraints in the order that they should appear in the NL file. Note that this is only a suggestion, as the NL writer will move all nonlinear constraints before linear ones (preserving row_order within each group).
- column_order: optional
List of variables in the order that they should appear in the NL file. Note that this is only a suggestion, as the NL writer will move all nonlinear variables before linear ones, and within nonlinear variables, variables appearing in both objectives and constraints before variables appearing only in constraints, which appear before variables appearing only in objectives. Within each group, continuous variables appear before discrete variables. In all cases, column_order is preserved within each group.
- export_defined_variables: bool, default=True
If True, export Expression objects to the NL file as ‘defined variables’.
- linear_presolve: bool, default=True
If True, we will perform a basic linear presolve by performing variable elimination (without fill-in).
from pyomo.contrib.solver.ipopt import Ipopt
opt = Ipopt()
opt.config.writer_config.display()
show_section_timing: false
skip_trivial_constraints: true
file_determinism: FileDeterminism.ORDERED
symbolic_solver_labels: false
scale_model: true
export_nonlinear_variables: None
row_order: None
column_order: None
export_defined_variables: true
linear_presolve: true
Note that, by default, both linear_presolve
and scale_model
are enabled.
Users can manipulate linear_presolve
and scale_model
to their preferred
states by changing their values.
>>> opt.config.writer_config.linear_presolve = False
Interface Implementation
All new interfaces should be built upon one of two classes (currently):
SolverBase
or
PersistentSolverBase
.
All solvers should have the following:
- class pyomo.contrib.solver.base.SolverBase(**kwds)[source]
- This base class defines the methods required for all solvers:
- available: Determines whether the solver is able to be run,
combining both whether it can be found on the system and if the license is valid.
solve: The main method of every solver
version: The version of the solver
is_persistent: Set to false for all non-persistent solvers.
Additionally, solvers should have a
config
attribute that inherits from one ofSolverConfig
,BranchAndBoundConfig
,PersistentSolverConfig
, orPersistentBranchAndBoundConfig
.- enum Availability(value)[source]
Class to capture different statuses in which a solver can exist in order to record its availability for use.
- Member Type:
Valid values are as follows:
- FullLicense = <Availability.FullLicense: 2>
- LimitedLicense = <Availability.LimitedLicense: 1>
- NotFound = <Availability.NotFound: 0>
- BadVersion = <Availability.BadVersion: -1>
- BadLicense = <Availability.BadLicense: -2>
- NeedsCompiledExtension = <Availability.NeedsCompiledExtension: -3>
- abstract available() bool [source]
Test if the solver is available on this system.
Nominally, this will return True if the solver interface is valid and can be used to solve problems and False if it cannot.
Note that for licensed solvers there are a number of “levels” of available: depending on the license, the solver may be available with limitations on problem size or runtime (e.g., ‘demo’ vs. ‘community’ vs. ‘full’). In these cases, the solver may return a subclass of enum.IntEnum, with members that resolve to True if the solver is available (possibly with limitations). The Enum may also have multiple members that all resolve to False indicating the reason why the interface is not available (not found, bad license, unsupported version, etc).
- Returns:
available – An enum that indicates “how available” the solver is. Note that the enum can be cast to bool, which will be True if the solver is runable at all and False otherwise.
- Return type:
- is_persistent() bool [source]
- Returns:
is_persistent – True if the solver is a persistent solver.
- Return type:
- abstract solve(model: BlockData, **kwargs) Results [source]
Solve a Pyomo model.
- Parameters:
model (BlockData) – The Pyomo model to be solved
**kwargs – Additional keyword arguments (including solver_options - passthrough options; delivered directly to the solver (with no validation))
- Returns:
results – A results object
- Return type:
- Keyword Arguments:
tee (TextIO_or_Logger, default=False) –
tee
acceptsbool
,io.TextIOBase
, orlogging.Logger
(or a list of these types).True
is mapped tosys.stdout
. The solver log will be printed to each of these streams / destinations.working_dir (Path, optional) – The directory in which generated files should be saved. This replaces the keepfiles option.
load_solutions (Bool, default=True) – If True, the values of the primal variables will be loaded into the model.
raise_exception_on_nonoptimal_result (Bool, default=True) – If False, the solve method will continue processing even if the returned result is nonoptimal.
symbolic_solver_labels (Bool, default=False) – If True, the names given to the solver will reflect the names of the Pyomo components. Cannot be changed after set_instance is called.
timer (optional) – A timer object for recording relevant process timing data.
threads (NonNegativeInt, optional) – Number of threads to be used by a solver.
time_limit (NonNegativeFloat, optional) – Time limit applied to the solver (in seconds).
solver_options (dict, optional) – Options to pass to the solver.
Persistent solvers include additional members as well as other configuration options:
- class pyomo.contrib.solver.base.PersistentSolverBase(**kwds)[source]
Bases:
SolverBase
Base class upon which persistent solvers can be built. This inherits the methods from the solver base class and adds those methods that are necessary for persistent solvers.
Example usage can be seen in the Gurobi interface.
- is_persistent()[source]
- Returns:
is_persistent – True if the solver is a persistent solver.
- Return type:
- abstract solve(model: BlockData, **kwargs) Results [source]
- Keyword Arguments:
tee (TextIO_or_Logger, default=False) –
tee
acceptsbool
,io.TextIOBase
, orlogging.Logger
(or a list of these types).True
is mapped tosys.stdout
. The solver log will be printed to each of these streams / destinations.working_dir (Path, optional) – The directory in which generated files should be saved. This replaces the keepfiles option.
load_solutions (Bool, default=True) – If True, the values of the primal variables will be loaded into the model.
raise_exception_on_nonoptimal_result (Bool, default=True) – If False, the solve method will continue processing even if the returned result is nonoptimal.
symbolic_solver_labels (Bool, default=False) – If True, the names given to the solver will reflect the names of the Pyomo components. Cannot be changed after set_instance is called.
timer (optional) – A timer object for recording relevant process timing data.
threads (NonNegativeInt, optional) – Number of threads to be used by a solver.
time_limit (NonNegativeFloat, optional) – Time limit applied to the solver (in seconds).
solver_options (dict, optional) – Options to pass to the solver.
auto_updates (dict, optional) –
Configuration options to detect changes in model between solves
- check_for_new_or_removed_constraints: bool, default=True
If False, new/old constraints will not be automatically detected on subsequent solves. Use False only when manually updating the solver with opt.add_constraints() and opt.remove_constraints() or when you are certain constraints are not being added to/removed from the model.
- check_for_new_or_removed_vars: bool, default=True
If False, new/old variables will not be automatically detected on subsequent solves. Use False only when manually updating the solver with opt.add_variables() and opt.remove_variables() or when you are certain variables are not being added to / removed from the model.
- check_for_new_or_removed_params: bool, default=True
If False, new/old parameters will not be automatically detected on subsequent solves. Use False only when manually updating the solver with opt.add_parameters() and opt.remove_parameters() or when you are certain parameters are not being added to / removed from the model.
- check_for_new_objective: bool, default=True
If False, new/old objectives will not be automatically detected on subsequent solves. Use False only when manually updating the solver with opt.set_objective() or when you are certain objectives are not being added to / removed from the model.
- update_constraints: bool, default=True
If False, changes to existing constraints will not be automatically detected on subsequent solves. This includes changes to the lower, body, and upper attributes of constraints. Use False only when manually updating the solver with opt.remove_constraints() and opt.add_constraints() or when you are certain constraints are not being modified.
- update_vars: bool, default=True
If False, changes to existing variables will not be automatically detected on subsequent solves. This includes changes to the lb, ub, domain, and fixed attributes of variables. Use False only when manually updating the solver with opt.update_variables() or when you are certain variables are not being modified.
- update_parameters: bool, default=True
If False, changes to parameter values will not be automatically detected on subsequent solves. Use False only when manually updating the solver with opt.update_parameters() or when you are certain parameters are not being modified.
- update_named_expressions: bool, default=True
If False, changes to Expressions will not be automatically detected on subsequent solves. Use False only when manually updating the solver with opt.remove_constraints() and opt.add_constraints() or when you are certain Expressions are not being modified.
- update_objective: bool, default=True
If False, changes to objectives will not be automatically detected on subsequent solves. This includes the expr and sense attributes of objectives. Use False only when manually updating the solver with opt.set_objective() or when you are certain objectives are not being modified.
- treat_fixed_vars_as_params: bool, default=True
[ADVANCED option]
This is an advanced option that should only be used in special circumstances. With the default setting of True, fixed variables will be treated like parameters. This means that z == x*y will be linear if x or y is fixed and the constraint can be written to an LP file. If the value of the fixed variable gets changed, we have to completely reprocess all constraints using that variable. If treat_fixed_vars_as_params is False, then constraints will be processed as if fixed variables are not fixed, and the solver will be told the variable is fixed. This means z == x*y could not be written to an LP file even if x and/or y is fixed. However, updating the values of fixed variables is much faster this way.
Results
Every solver, at the end of a
solve
call, will
return a Results
object. This object is a pyomo.common.config.ConfigDict
,
which can be manipulated similar to a standard dict
in Python.
- class pyomo.contrib.solver.results.Results(description=None, doc=None, implicit=False, implicit_domain=None, visibility=0)[source]
Bases:
ConfigDict
- solution_loader
Object for loading the solution back into the model.
- Type:
SolutionLoaderBase
- termination_condition
The reason the solver exited. This is a member of the TerminationCondition enum.
- Type:
- solution_status
The result of the solve call. This is a member of the SolutionStatus enum.
- Type:
- incumbent_objective
If a feasible solution was found, this is the objective value of the best solution found. If no feasible solution was found, this is None.
- Type:
- objective_bound
The best objective bound found. For minimization problems, this is the lower bound. For maximization problems, this is the upper bound. For solvers that do not provide an objective bound, this should be -inf (minimization) or inf (maximization)
- Type:
- timing_info
- A ConfigDict containing three pieces of information:
start_timestamp
: UTC timestamp of when run was initiatedwall_time
: elapsed wall clock time for entire processtimer
: a HierarchicalTimer object containing timing data about the solve
Specific solvers may add other relevant timing information, as appropriate.
- Type:
- extra_info
A ConfigDict to store extra information such as solver messages.
- Type:
- solver_configuration
A copy of the SolverConfig ConfigDict, for later inspection/reproducibility.
- Type:
Termination Conditions
Pyomo offers a standard set of termination conditions to map to solver
returns. The intent of
TerminationCondition
is to notify the user of why the solver exited. The user is expected
to inspect the Results
object or any returned solver messages or logs for more information.
- class pyomo.contrib.solver.results.TerminationCondition(value)[source]
Bases:
Enum
An Enum that enumerates all possible exit statuses for a solver call.
- convergenceCriteriaSatisfied
The solver exited because convergence criteria of the problem were satisfied.
- Type:
0
- maxTimeLimit
The solver exited due to reaching a specified time limit.
- Type:
1
- iterationLimit
The solver exited due to reaching a specified iteration limit.
- Type:
2
- objectiveLimit
The solver exited due to reaching an objective limit. For example, in Gurobi, the exit message “Optimal objective for model was proven to be worse than the value specified in the Cutoff parameter” would map to objectiveLimit.
- Type:
3
- minStepLength
The solver exited due to a minimum step length. Minimum step length reached may mean that the problem is infeasible or that the problem is feasible but the solver could not converge.
- Type:
4
- unbounded
The solver exited because the problem has been found to be unbounded.
- Type:
5
- provenInfeasible
The solver exited because the problem has been proven infeasible.
- Type:
6
- locallyInfeasible
The solver exited because no feasible solution was found to the submitted problem, but it could not be proven that no such solution exists.
- Type:
7
- infeasibleOrUnbounded
Some solvers do not specify between infeasibility or unboundedness and instead return that one or the other has occurred. For example, in Gurobi, this may occur because there are some steps in presolve that prevent Gurobi from distinguishing between infeasibility and unboundedness.
- Type:
8
- error
The solver exited with some error. The error message will also be captured and returned.
- Type:
9
- interrupted
The solver was interrupted while running.
- Type:
10
- licensingProblems
The solver experienced issues with licensing. This could be that no license was found, the license is of the wrong type for the problem (e.g., problem is too big for type of license), or there was an issue contacting a licensing server.
- Type:
11
- emptyModel
The model being solved did not have any variables
- Type:
12
- unknown
All other unrecognized exit statuses fall in this category.
- Type:
42
Solution Status
Pyomo offers a standard set of solution statuses to map to solver
output. The intent of
SolutionStatus
is to notify the user of what the solver returned at a high level. The
user is expected to inspect the
Results
object or any
returned solver messages or logs for more information.
- class pyomo.contrib.solver.results.SolutionStatus(value)[source]
Bases:
Enum
An enumeration for interpreting the result of a termination. This describes the designated status by the solver to be loaded back into the model.
- noSolution
No (single) solution was found; possible that a population of solutions was returned.
- Type:
0
- infeasible
Solution point does not satisfy some domains and/or constraints.
- Type:
10
- feasible
A solution for which all of the constraints in the model are satisfied.
- Type:
20
- optimal
A feasible solution where the objective function reaches its specified sense (e.g., maximum, minimum)
- Type:
30
Solution
Solutions can be loaded back into a model using a SolutionLoader
. A specific
loader should be written for each unique case. Several have already been
implemented. For example, for ipopt
:
- class pyomo.contrib.solver.ipopt.IpoptSolutionLoader(sol_data: SolFileData, nl_info: NLWriterInfo)[source]
Bases:
SolSolutionLoader
- get_duals(cons_to_load: Sequence[ConstraintData] | None = None) Dict[ConstraintData, float]
Returns a dictionary mapping constraint to dual value.
- get_primals(vars_to_load: Sequence[VarData] | None = None) Mapping[VarData, float]
Returns a ComponentMap mapping variable to var value.
- Parameters:
vars_to_load (list) – A list of the variables whose solution value should be retrieved. If vars_to_load is None, then the values for all variables will be retrieved.
- Returns:
primals – Maps variables to solution values
- Return type:
ComponentMap
- get_reduced_costs(vars_to_load: Sequence[VarData] | None = None) Mapping[VarData, float] [source]
Returns a ComponentMap mapping variable to reduced cost.
- Parameters:
vars_to_load (list) – A list of the variables whose reduced cost should be retrieved. If vars_to_load is None, then the reduced costs for all variables will be loaded.
- Returns:
reduced_costs – Maps variables to reduced costs
- Return type:
ComponentMap
- load_vars(vars_to_load: Sequence[VarData] | None = None) NoReturn
Load the solution of the primal variables into the value attribute of the variables.
- Parameters:
vars_to_load (list) – The minimum set of variables whose solution should be loaded. If vars_to_load is None, then the solution to all primal variables will be loaded. Even if vars_to_load is specified, the values of other variables may also be loaded depending on the interface.