NonlinearBlockGS

NonlinearBlockGS applies Block Gauss-Seidel (also known as fixed-point iteration) to the components and subsystems in the system. This is mainly used to solve cyclic connections. You should try this solver for systems that satisfy the following conditions:

1. System (or subsystem) contains a cycle, though subsystems may.

2. System does not contain any implicit states, though subsystems may.

NonlinearBlockGS is a block solver, so you can specify different nonlinear solvers in the subsystems and they will be utilized to solve the subsystem nonlinear problem.

Note that you may not know if you satisfy the second condition, so choosing a solver can be a trial-and-error proposition. If NonlinearBlockGS doesn’t work, then you will need to use NewtonSolver.

Here, we choose NonlinearBlockGS to solve the Sellar problem, which has two components with a cyclic dependency, has no implicit states, and works very well with Gauss-Seidel.

SellarDis1withDerivatives class definition

class SellarDis1withDerivatives(SellarDis1):
    """
    Component containing Discipline 1 -- derivatives version.
    """

    def setup_partials(self):
        # Analytic Derivs
        self.declare_partials(of='*', wrt='*')

    def compute_partials(self, inputs, partials):
        """
        Jacobian for Sellar discipline 1.
        """
        partials['y1', 'y2'] = -0.2
        partials['y1', 'z'] = np.array([[2.0 * inputs['z'][0], 1.0]])
        partials['y1', 'x'] = 1.0

SellarDis2withDerivatives class definition

class SellarDis2withDerivatives(SellarDis2):
    """
    Component containing Discipline 2 -- derivatives version.
    """

    def setup_partials(self):
        # Analytic Derivs
        self.declare_partials(of='*', wrt='*')

    def compute_partials(self, inputs, J):
        """
        Jacobian for Sellar discipline 2.
        """
        y1 = inputs['y1']
        if y1.real < 0.0:
            y1 *= -1
        if y1.real < 1e-8:
            y1 = 1e-8

        J['y2', 'y1'] = .5*y1**-.5
        J['y2', 'z'] = np.array([[1.0, 1.0]])

SellarDerivatives class definition

class SellarDerivatives(om.Group):
    """
    Group containing the Sellar MDA. This version uses the disciplines with derivatives.
    """

    def setup(self):
        self.add_subsystem('d1', SellarDis1withDerivatives(), promotes=['x', 'z', 'y1', 'y2'])
        self.add_subsystem('d2', SellarDis2withDerivatives(), promotes=['z', 'y1', 'y2'])

        self.add_subsystem('obj_cmp', om.ExecComp('obj = x**2 + z[1] + y1 + exp(-y2)', obj=0.0,
                                                  x=0.0, z=np.array([0.0, 0.0]), y1=0.0, y2=0.0),
                           promotes=['obj', 'x', 'z', 'y1', 'y2'])

        self.add_subsystem('con_cmp1', om.ExecComp('con1 = 3.16 - y1', con1=0.0, y1=0.0),
                           promotes=['con1', 'y1'])
        self.add_subsystem('con_cmp2', om.ExecComp('con2 = y2 - 24.0', con2=0.0, y2=0.0),
                           promotes=['con2', 'y2'])

        self.set_input_defaults('x', 1.0)
        self.set_input_defaults('z', np.array([5.0, 2.0]))

import numpy as np

import openmdao.api as om
from openmdao.test_suite.components.sellar_feature import SellarDerivatives

prob = om.Problem(model=SellarDerivatives())
prob.setup()

prob.model.nonlinear_solver = om.NonlinearBlockGS()

prob.set_val('x', 1.)
prob.set_val('z', np.array([5.0, 2.0]))

prob.run_model()

print(prob.get_val('y1'))
print(prob.get_val('y2'))

NL: NLBGS Converged in 8 iterations
[25.58830237]
[12.05848815]

This solver runs all of the subsystems each iteration, passing data along all connections including the cyclic ones. After each iteration, the iteration count and the residual norm are checked to see if termination has been satisfied.

You can control the termination criteria for the solver using the following options:

NonlinearBlockGS Options

Option	Default	Acceptable Values	Acceptable Types	Description
aitken_initial_factor	1.0	N/A	N/A	initial value for Aitken relaxation factor
aitken_max_factor	1.5	N/A	N/A	upper limit for Aitken relaxation factor
aitken_min_factor	0.1	N/A	N/A	lower limit for Aitken relaxation factor
atol	1e-10	N/A	N/A	absolute error tolerance
cs_reconverge	True	[True, False]	['bool']	When True, when this driver solves under a complex step, nudge the Solution vector by a small amount so that it reconverges.
debug_print	False	[True, False]	['bool']	If true, the values of input and output variables at the start of iteration are printed and written to a file after a failure to converge.
err_on_non_converge	False	[True, False]	['bool']	When True, AnalysisError will be raised if we don't converge.
iprint	1	N/A	['int']	whether to print output
maxiter	10	N/A	['int']	maximum number of iterations
reraise_child_analysiserror	False	[True, False]	['bool']	When the option is true, a solver will reraise any AnalysisError that arises during subsolve; when false, it will continue solving.
restart_from_successful	False	[True, False]	['bool']	If True, the states are cached after a successful solve and used to restart the solver in the case of a failed solve.
rtol	1e-10	N/A	N/A	relative error tolerance
stall_limit	0	N/A	N/A	Number of iterations after which, if the residual norms are identical within the stall_tol, then terminate as if max iterations were reached. Default is 0, which disables this feature.
stall_tol	1e-12	N/A	N/A	When stall checking is enabled, the threshold below which the residual norm is considered unchanged.
stall_tol_type	rel	['abs', 'rel']	N/A	Specifies whether the absolute or relative norm of the residual is used for stall detection.
use_aitken	False	[True, False]	['bool']	set to True to use Aitken relaxation
use_apply_nonlinear	False	[True, False]	['bool']	Set to True to always call apply_nonlinear on the solver's system after solve_nonlinear has been called.

NonlinearBlockGS Constructor

The call signature for the NonlinearBlockGS constructor is:

NonlinearBlockGS.__init__(**kwargs)

Initialize all attributes.

Aitken relaxation

This solver implements Aitken relaxation, as described in Algorithm 1 of this paper on aerostructual design optimization. The relaxation is turned off by default, but it may help convergence for more tightly coupled models.

Residual Calculation

The Unified Derivatives Equations are formulated so that explicit equations (via ExplicitComponent) are also expressed as implicit relationships, and their residual is also calculated in “apply_nonlinear”, which runs the component a second time and saves the difference in the output vector as the residual. However, this would require an extra call to compute, which is inefficient for slower components. To eliminate the inefficiency of running the model twice every iteration the NonlinearBlockGS driver saves a copy of the output vector and uses that to calculate the residual without rerunning the model. This does require a little more memory, so if you are solving a model where memory is more of a concern than execution time, you can set the “use_apply_nonlinear” option to True to use the original formulation that calls “apply_nonlinear” on the subsystem.

NonlinearBlockGS Option Examples

maxiter

maxiter lets you specify the maximum number of Gauss-Seidel iterations to apply. In this example, we cut it back from the default, ten, down to two, so that it terminates a few iterations earlier and doesn’t reach the specified absolute or relative tolerance.

import openmdao.api as om
from openmdao.test_suite.components.sellar_feature import SellarDerivatives

prob = om.Problem(model=SellarDerivatives())
prob.setup()

nlbgs = prob.model.nonlinear_solver = om.NonlinearBlockGS()

# basic test of number of iterations
nlbgs.options['maxiter'] = 1
prob.run_model()

print(prob.model.nonlinear_solver._iter_count)

NL: NLBGSSolver 'NL: NLBGS' on system '' failed to converge in 1 iterations.
1

nlbgs.options['maxiter'] = 5
prob.run_model()
print(prob.model.nonlinear_solver._iter_count)

NL: NLBGSSolver 'NL: NLBGS' on system '' failed to converge in 5 iterations.
5

# test of number of iterations AND solution after exit at maxiter
prob.set_val('x', 1.)
prob.set_val('z', np.array([5.0, 2.0]))

nlbgs.options['maxiter'] = 3
prob.set_solver_print()
prob.run_model()

print(prob.get_val('y1'))
print(prob.get_val('y2'))
print(prob.model.nonlinear_solver._iter_count)

NL: NLBGS 1 ; 1.15925306e-08 1
NL: NLBGS 2 ; 2.29166181e-10 0.0197684344
NL: NLBGS 3 ; 4.53131952e-12 0.000390882687
NL: NLBGS Converged
[25.58830237]
[12.05848815]
3

atol

Here, we set the absolute tolerance to a looser value that will trigger an earlier termination. After each iteration, the norm of the residuals is calculated one of two ways. If the “use_apply_nonlinear” option is set to False (its default), then the norm is calculated by subtracting a cached previous value of the outputs from the current value. If “use_apply_nonlinear” is True, then the norm is calculated by calling apply_nonlinear on all of the subsystems. In this case, ExplicitComponents are executed a second time. If this norm value is lower than the absolute tolerance atol, the iteration will terminate.

import numpy as np

import openmdao.api as om
from openmdao.test_suite.components.sellar_feature import SellarDerivatives

prob = om.Problem(model=SellarDerivatives())
prob.setup()

nlbgs = prob.model.nonlinear_solver = om.NonlinearBlockGS()
nlbgs.options['atol'] = 1e-4

prob.set_val('x', 1.)
prob.set_val('z', np.array([5.0, 2.0]))

prob.run_model()

print(prob.get_val('y1'))
print(prob.get_val('y2'))

NL: NLBGS Converged in 5 iterations
[25.5883027]
[12.05848818]

rtol

Here, we set the relative tolerance to a looser value that will trigger an earlier termination. After each iteration, the norm of the residuals is calculated one of two ways. If the “use_apply_nonlinear” option is set to False (its default), then the norm is calculated by subtracting a cached previous value of the outputs from the current value. If “use_apply_nonlinear” is True, then the norm is calculated by calling apply_nonlinear on all of the subsystems. In this case, ExplicitComponents are executed a second time. If the ratio of the currently calculated norm to the initial residual norm is lower than the relative tolerance rtol, the iteration will terminate.

import numpy as np

import openmdao.api as om
from openmdao.test_suite.components.sellar_feature import SellarDerivatives

prob = om.Problem(model=SellarDerivatives())
prob.setup()

nlbgs = prob.model.nonlinear_solver = om.NonlinearBlockGS()
nlbgs.options['rtol'] = 1e-3

prob.set_val('x', 1.)
prob.set_val('z', np.array([5.0, 2.0]))

prob.run_model()

print(prob.get_val('y1'), 25.5883027, .00001)
print(prob.get_val('y2'), 12.05848819, .00001)

NL: NLBGS Converged in 4 iterations
[25.58828563] 25.5883027 1e-05
[12.0584865] 12.05848819 1e-05