pyOptSparseDriver#

pyOptSparseDriver wraps the optimizer package pyOptSparse, which provides a common interface for 11 optimizers, some of which are included in the package (e.g., SLSQP and NSGA2), and some of which are commercial products that must be obtained from their respective authors (e.g. SNOPT). The pyOptSparse package is based on pyOpt, but adds support for sparse specification of constraint Jacobians. Most of the sparsity features are only applicable when using the SNOPT optimizer.

Note

The pyOptSparse package does not come included with the OpenMDAO installation. It is a separate optional package that can be obtained from mdolab.

In this simple example, we use the SLSQP optimizer to minimize the objective of a Sellar MDA model.

We will use a SellarMDA class to encapsulate the Sellar model with it’s design variables, objective and constraints:

import numpy as np

import openmdao.api as om

from openmdao.test_suite.components.sellar import SellarDis1withDerivatives, SellarDis2withDerivatives
  
class SellarMDA(om.Group):
    """
    Group containing the Sellar MDA model.
    """

    def setup(self):
        # add the two disciplines in an 'mda' subgroup
        self.mda = mda = self.add_subsystem('mda', om.Group(), promotes=['x', 'z', 'y1', 'y2'])
        mda.add_subsystem('d1', SellarDis1withDerivatives(), promotes=['x', 'z', 'y1', 'y2'])
        mda.add_subsystem('d2', SellarDis2withDerivatives(), promotes=['z', 'y1', 'y2'])

        # add components to calculate objectives and constraints
        self.add_subsystem('obj_cmp', om.ExecComp('obj = x**2 + z[1] + y1 + exp(-y2)',
                                                  z=np.array([0.0, 0.0]), x=0.0, y1=0.0, y2=0.0),
                           promotes=['obj', 'x', 'z', 'y1', 'y2'])

        self.add_subsystem('con_cmp1', om.ExecComp('con1 = 3.16 - y1'), promotes=['con1', 'y1'])
        self.add_subsystem('con_cmp2', om.ExecComp('con2 = y2 - 24.0'), promotes=['con2', 'y2'])

        # set default values for the inputs
        self.set_input_defaults('x', 1.0)
        self.set_input_defaults('z', np.array([5.0, 2.0]))
        
        # add design vars, objective and constraints
        self.add_design_var('z', lower=np.array([-10.0, 0.0]), upper=np.array([10.0, 10.0]))
        self.add_design_var('x', lower=0.0, upper=10.0)
        self.add_objective('obj')
        self.add_constraint('con1', upper=0.0)
        self.add_constraint('con2', upper=0.0)

    def configure(self):
        # set the solvers for the model and cycle groups
        self.nonlinear_solver = om.NonlinearBlockGS()
        self.linear_solver = om.ScipyKrylov()
        self.mda.nonlinear_solver = om.NonlinearBlockGS()
        self.mda.linear_solver = om.ScipyKrylov()

        # default to non-verbose
        self.set_solver_print(0)
import openmdao.api as om

prob = om.Problem(model=SellarMDA())
prob.setup(check=False, mode='rev')

prob.driver = om.pyOptSparseDriver(optimizer='SLSQP')

prob.run_driver()
Optimization Problem -- Optimization using pyOpt_sparse
================================================================================
    Objective Function: _objfunc

    Solution: 
--------------------------------------------------------------------------------
    Total Time:                    0.0336
       User Objective Time :       0.0077
       User Sensitivity Time :     0.0218
       Interface Time :            0.0032
       Opt Solver Time:            0.0009
    Calls to Objective Function :       6
    Calls to Sens Function :            6


   Objectives
      Index  Name            Value
          0  obj     3.183394E+00

   Variables (c - continuous, i - integer, d - discrete)
      Index  Name   Type      Lower Bound            Value      Upper Bound     Status
          0  z_0      c    -1.000000E+01     1.977639E+00     1.000000E+01           
          1  z_1      c     0.000000E+00    -4.753534E-15     1.000000E+01          l
          2  x_0      c     0.000000E+00     4.439131E-15     1.000000E+01          l

   Constraints (i - inequality, e - equality)
      Index  Name Type          Lower           Value           Upper    Status  Lagrange Multiplier (N/A)
          0  con1    i  -1.000000E+30   -8.746648E-11    0.000000E+00         u    9.00000E+100
          1  con2    i  -1.000000E+30   -2.024472E+01    0.000000E+00              9.00000E+100


   Exit Status
      Inform  Description
           0  Optimization terminated successfully.
--------------------------------------------------------------------------------
Problem: problem
Driver:  pyOptSparseDriver
  success     : True
  iterations  : 7
  runtime     : 3.7284E-02 s
  model_evals : 7
  model_time  : 6.3333E-03 s
  deriv_evals : 6
  deriv_time  : 2.1665E-02 s
  exit_status : SUCCESS
print(prob.get_val('z', indices=0))
1.9776388834877525

pyOptSparseDriver Options#

OptionDefaultAcceptable ValuesAcceptable TypesDescription
debug_print[]['desvars', 'nl_cons', 'ln_cons', 'objs', 'totals']['list']List of what type of Driver variables to print at each iteration.
gradient_methodopenmdao['snopt_fd', 'openmdao', 'pyopt_fd']N/AFinite difference implementation to use
hist_fileN/AN/A['str']File location for saving pyopt_sparse optimization history. Default is None for no output.
hotstart_fileN/AN/A['str']File location of a pyopt_sparse optimization history to use to hot start the optimization. Default is None.
invalid_desvar_behaviorwarn['warn', 'raise', 'ignore']N/ABehavior of driver if the initial value of a design variable exceeds its bounds. The default value may beset using the `OPENMDAO_INVALID_DESVAR_BEHAVIOR` environment variable to one of the valid options.
optimizerSLSQP['PSQP', 'SLSQP', 'IPOPT', 'CONMIN', 'ALPSO', 'SNOPT', 'NLPQLP', 'ParOpt', 'NSGA2']N/AName of optimizers to use
output_dirDEFAULT_REPORTS_DIRN/A['str', '_ReprClass']Directory location of pyopt_sparse output files.Default is {prob_name}_out/reports.
print_opt_probFalse[True, False]['bool']Print the opt problem summary before running the optimization
print_resultsTrueN/A['bool', 'str']Print pyOpt results if True
singular_jac_behaviorwarn['error', 'warn', 'ignore']N/ADefines behavior of a zero row/col check after first call tocompute_totals:error - raise an error.warn - raise a warning.ignore - don't perform check.
singular_jac_tol1e-16N/AN/ATolerance for zero row/column check.
titleOptimization using pyOpt_sparseN/AN/ATitle of this optimization run
user_terminate_signalN/AN/AN/AOS signal that triggers a clean user-termination. Only SNOPT supports this option.

pyOptSparseDriver Constructor#

The call signature for the pyOptSparseDriver constructor is:

pyOptSparseDriver.__init__(**kwargs)[source]

Initialize pyopt.

Using pyOptSparseDriver#

pyOptSparseDriver has a small number of unified options that can be specified as keyword arguments when it is instantiated or by using the “options” dictionary. We have already shown how to set the optimizer option. Next we see how the print_results option can be used to turn on or off the echoing of the results when the optimization finishes. The default is True, but here, we turn it off.

import openmdao.api as om

prob = om.Problem(model=SellarMDA())
prob.setup(check=False, mode='rev')

prob.driver = om.pyOptSparseDriver(optimizer='SLSQP')

prob.driver.options['print_results'] = False

prob.run_driver()
Problem: problem2
Driver:  pyOptSparseDriver
  success     : True
  iterations  : 7
  runtime     : 3.5466E-02 s
  model_evals : 7
  model_time  : 6.1055E-03 s
  deriv_evals : 6
  deriv_time  : 2.1002E-02 s
  exit_status : SUCCESS
print(prob.get_val('z', indices=0))
1.9776388834877525

Every optimizer also has its own specialized settings that allow you to fine-tune the algorithm that it uses. You can access these within the opt_setting dictionary. These options are different for each optimizer, so to find out what they are, you need to read your optimizer’s documentation. We present a few common ones here.

SLSQP-Specific Settings#

Here, we set a convergence tolerance for SLSQP:

import openmdao.api as om

prob = om.Problem(model=SellarMDA())
prob.setup(check=False, mode='rev')

prob.driver = om.pyOptSparseDriver(optimizer='SLSQP')

prob.driver.opt_settings['ACC'] = 1e-9

prob.setup(check=False, mode='rev')
prob.run_driver()
Optimization Problem -- Optimization using pyOpt_sparse
================================================================================
    Objective Function: _objfunc

    Solution: 
--------------------------------------------------------------------------------
    Total Time:                    0.0320
       User Objective Time :       0.0073
       User Sensitivity Time :     0.0209
       Interface Time :            0.0030
       Opt Solver Time:            0.0008
    Calls to Objective Function :       6
    Calls to Sens Function :            6


   Objectives
      Index  Name            Value
          0  obj     3.183394E+00

   Variables (c - continuous, i - integer, d - discrete)
      Index  Name   Type      Lower Bound            Value      Upper Bound     Status
          0  z_0      c    -1.000000E+01     1.977639E+00     1.000000E+01           
          1  z_1      c     0.000000E+00    -4.753534E-15     1.000000E+01          l
          2  x_0      c     0.000000E+00     4.439131E-15     1.000000E+01          l

   Constraints (i - inequality, e - equality)
      Index  Name Type          Lower           Value           Upper    Status  Lagrange Multiplier (N/A)
          0  con1    i  -1.000000E+30   -8.746648E-11    0.000000E+00         u    9.00000E+100
          1  con2    i  -1.000000E+30   -2.024472E+01    0.000000E+00              9.00000E+100


   Exit Status
      Inform  Description
           0  Optimization terminated successfully.
--------------------------------------------------------------------------------
Problem: problem3
Driver:  pyOptSparseDriver
  success     : True
  iterations  : 7
  runtime     : 3.5781E-02 s
  model_evals : 7
  model_time  : 6.1536E-03 s
  deriv_evals : 6
  deriv_time  : 2.0833E-02 s
  exit_status : SUCCESS
print(prob.get_val('z', indices=0))
1.9776388834877525

Similarly, we can set an iteration limit. Here, we set it to just a few iterations, and don’t quite reach the optimum.

import openmdao.api as om

prob = om.Problem(model=SellarMDA())
prob.setup(check=False, mode='rev')

prob.driver = om.pyOptSparseDriver(optimizer='SLSQP')

prob.driver.opt_settings['MAXIT'] = 3

prob.run_driver()
Optimization Problem -- Optimization using pyOpt_sparse
================================================================================
    Objective Function: _objfunc

    Solution: 
--------------------------------------------------------------------------------
    Total Time:                    0.0215
       User Objective Time :       0.0048
       User Sensitivity Time :     0.0142
       Interface Time :            0.0020
       Opt Solver Time:            0.0005
    Calls to Objective Function :       4
    Calls to Sens Function :            4


   Objectives
      Index  Name            Value
          0  obj     3.203561E+00

   Variables (c - continuous, i - integer, d - discrete)
      Index  Name   Type      Lower Bound            Value      Upper Bound     Status
          0  z_0      c    -1.000000E+01     1.983377E+00     1.000000E+01           
          1  z_1      c     0.000000E+00    -5.325284E-13     1.000000E+01          l
          2  x_0      c     0.000000E+00    -5.051337E-15     1.000000E+01          l

   Constraints (i - inequality, e - equality)
      Index  Name Type          Lower           Value           Upper    Status  Lagrange Multiplier (N/A)
          0  con1    i  -1.000000E+30   -2.043382E-02    0.000000E+00              9.00000E+100
          1  con2    i  -1.000000E+30   -2.023325E+01    0.000000E+00              9.00000E+100


   Exit Status
      Inform  Description
           9  Iteration limit exceeded
--------------------------------------------------------------------------------
Problem: problem4
Driver:  pyOptSparseDriver
  success     : False
  iterations  : 5
  runtime     : 2.4887E-02 s
  model_evals : 5
  model_time  : 4.3524E-03 s
  deriv_evals : 4
  deriv_time  : 1.4117E-02 s
  exit_status : FAIL
print(prob.get_val('z', indices=0))
1.9833770833092614

SNOPT-Specific Settings#

SNOPT has many customizable settings. Here we show two common ones.

Setting the convergence tolerance:

import openmdao.api as om

prob = om.Problem(model=SellarMDA())
prob.setup(check=False, mode='rev')

prob.driver = om.pyOptSparseDriver(optimizer='SNOPT')

prob.driver.opt_settings['Major feasibility tolerance'] = 1e-9

prob.run_driver()
Optimization Problem -- Optimization using pyOpt_sparse
================================================================================
    Objective Function: _objfunc

    Solution: 
--------------------------------------------------------------------------------
    Total Time:                    0.0384
       User Objective Time :       0.0089
       User Sensitivity Time :     0.0243
       Interface Time :            0.0035
       Opt Solver Time:            0.0017
    Calls to Objective Function :       8
    Calls to Sens Function :            7


   Objectives
      Index  Name            Value
          0  obj     3.183394E+00

   Variables (c - continuous, i - integer, d - discrete)
      Index  Name   Type      Lower Bound            Value      Upper Bound     Status
          0  z_0      c    -1.000000E+01     1.977639E+00     1.000000E+01           
          1  z_1      c     0.000000E+00     0.000000E+00     1.000000E+01          l
          2  x_0      c     0.000000E+00     0.000000E+00     1.000000E+01          l

   Constraints (i - inequality, e - equality)
      Index  Name Type          Lower           Value           Upper    Status  Lagrange Multiplier (N/A)
          0  con1    i  -1.000000E+30   -5.966339E-12    0.000000E+00         u    9.00000E+100
          1  con2    i  -1.000000E+30   -2.024472E+01    0.000000E+00              9.00000E+100


   Exit Status
      Inform  Description
           1  optimality conditions satisfied
--------------------------------------------------------------------------------
Problem: problem5
Driver:  pyOptSparseDriver
  success     : True
  iterations  : 9
  runtime     : 4.2671E-02 s
  model_evals : 9
  model_time  : 7.6306E-03 s
  deriv_evals : 7
  deriv_time  : 2.4206E-02 s
  exit_status : SUCCESS
print(prob.get_val('z', indices=0))
1.9776388834648047

Setting a limit on the number of major iterations. Here, we set it to just a few iterations, and don’t quite reach the optimum.

import openmdao.api as om

prob = om.Problem(model=SellarMDA())
prob.setup(check=False, mode='rev')

prob.driver = om.pyOptSparseDriver(optimizer='SNOPT')

prob.driver.opt_settings['Major iterations limit'] = 5

prob.run_driver()
Optimization Problem -- Optimization using pyOpt_sparse
================================================================================
    Objective Function: _objfunc

    Solution: 
--------------------------------------------------------------------------------
    Total Time:                    0.0330
       User Objective Time :       0.0079
       User Sensitivity Time :     0.0207
       Interface Time :            0.0029
       Opt Solver Time:            0.0015
    Calls to Objective Function :       7
    Calls to Sens Function :            6


   Objectives
      Index  Name            Value
          0  obj     3.183402E+00

   Variables (c - continuous, i - integer, d - discrete)
      Index  Name   Type      Lower Bound            Value      Upper Bound     Status
          0  z_0      c    -1.000000E+01     1.977641E+00     1.000000E+01           
          1  z_1      c     0.000000E+00     0.000000E+00     1.000000E+01          l
          2  x_0      c     0.000000E+00     0.000000E+00     1.000000E+01          l

   Constraints (i - inequality, e - equality)
      Index  Name Type          Lower           Value           Upper    Status  Lagrange Multiplier (N/A)
          0  con1    i  -1.000000E+30   -8.621022E-06    0.000000E+00              9.00000E+100
          1  con2    i  -1.000000E+30   -2.024472E+01    0.000000E+00              9.00000E+100


   Exit Status
      Inform  Description
           3  requested accuracy could not be achieved
--------------------------------------------------------------------------------
Problem: problem6
Driver:  pyOptSparseDriver
  success     : False
  iterations  : 8
  runtime     : 3.8318E-02 s
  model_evals : 8
  model_time  : 6.5654E-03 s
  deriv_evals : 6
  deriv_time  : 2.0654E-02 s
  exit_status : FAIL
print(prob.get_val('z', indices=0))
1.9776413083133966

If you have pyoptsparse 1.1 or greater, then you can send a signal such as SIGUSR1 to a running SNOPT optimization to tell it to terminate cleanly. This is useful if an optimization has gotten close enough to an optimum. How to do this is dependent on your operating system in all cases, on your mpi implementation if you are running mpi, and on your queuing software if you are on a supercomputing cluster. Here is a simple example for unix and mpi.

    ktmoore1$ ps -ef |grep sig
      502 17955   951   0  4:05PM ttys000    0:00.02 mpirun -n 2 python sig_demo.py
      502 17956 17955   0  4:05PM ttys000    0:00.03 python sig_demo.py
      502 17957 17955   0  4:05PM ttys000    0:00.03 python sig_demo.py
      502 17959 17312   0  4:05PM ttys001    0:00.00 grep sig

    ktmoore1$ kill -SIGUSR1 17955

You can enable this feature by setting the “user_terminate_signal” option and giving it a signal (imported from the signal library in Python). By default, user_terminate_signal is None, which disables the feature. Here, we set the signal to SIGUSR1:

import openmdao.api as om
import signal

prob = om.Problem()

prob.driver = om.pyOptSparseDriver()
prob.driver.options['optimizer'] = "SNOPT"
prob.driver.options['user_terminate_signal'] = signal.SIGUSR1

You can learn more about the available options in the SNOPT_Manual.