Approximating Semi-Total Derivatives

Approximating Semi-Total Derivatives#

There are times where it makes sense to approximate the derivatives for an entire group in one shot. You can turn on the approximation by calling approx_totals on any Group.

Group.approx_totals(method='fd', step=None, form=None, step_calc=None)[source]

Approximate derivatives for a Group using the specified approximation method.

Parameters:
methodstr

The type of approximation that should be used. Valid options include: ‘fd’: Finite Difference, ‘cs’: Complex Step.

stepfloat

Step size for approximation. Defaults to None, in which case, the approximation method provides its default value.

formstr

Form for finite difference, can be ‘forward’, ‘backward’, or ‘central’. Defaults to None, in which case, the approximation method provides its default value.

step_calcstr

Step type for computing the size of the finite difference step. It can be ‘abs’ for absolute, ‘rel_avg’ for a size relative to the absolute value of the vector input, or ‘rel_element’ for a size relative to each value in the vector input. In addition, it can be ‘rel_legacy’ for a size relative to the norm of the vector. For backwards compatibilty, it can be ‘rel’, which is now equivalent to ‘rel_avg’. Defaults to None, in which case the approximation method provides its default value.

The default method for approximating semi-total derivatives is the finite difference method. When you call the approx_totals method on a group, OpenMDAO will generate an approximate Jacobian for the entire group during the linearization step before derivatives are calculated. OpenMDAO automatically figures out which inputs and output pairs are needed in this Jacobian. When solve_linear is called from any system that contains this system, the approximated Jacobian is used for the derivatives in this system.

The derivatives approximated in this manner are total derivatives of outputs of the group with respect to inputs. If any components in the group contain implicit states, then you must have an appropriate solver (such as NewtonSolver) inside the group to solve the implicit relationships.

Here is a classic example of where you might use an approximation like finite difference. In this example, we could just approximate the partials on components CompOne and CompTwo separately. However, CompTwo has a vector input that is 25 wide, so it would require 25 separate executions under finite difference. If we instead approximate the total derivatives on the whole group, we only have one input, so just one extra execution.

import numpy as np

import openmdao.api as om

class CompOne(om.ExplicitComponent):

    def setup(self):
        self.add_input('x', val=0.0)
        self.add_output('y', val=np.zeros(25))
        self._exec_count = 0

    def compute(self, inputs, outputs):
        x = inputs['x']
        outputs['y'] = np.arange(25) * x
        self._exec_count += 1

class CompTwo(om.ExplicitComponent):

    def setup(self):
        self.add_input('y', val=np.zeros(25))
        self.add_output('z', val=0.0)
        self._exec_count = 0

    def compute(self, inputs, outputs):
        y = inputs['y']
        outputs['z'] = np.sum(y)
        self._exec_count += 1

prob = om.Problem()
model = prob.model

model.set_input_defaults('x', 0.0)

model.add_subsystem('comp1', CompOne(), promotes=['x', 'y'])
comp2 = model.add_subsystem('comp2', CompTwo(), promotes=['y', 'z'])

model.linear_solver = om.ScipyKrylov()
model.approx_totals()

prob.setup()
prob.run_model()

of = ['z']
wrt = ['x']
derivs = prob.compute_totals(of=of, wrt=wrt)
print(derivs['z', 'x'])
[[300.]]
print(comp2._exec_count)
2

The same arguments are used for both partial and total derivative approximation specifications. Here we set the finite difference step size, the form to central differences, and the step_calc to relative instead of absolute.

import numpy as np

import openmdao.api as om

class CompOne(om.ExplicitComponent):

    def setup(self):
        self.add_input('x', val=1.0)
        self.add_output('y', val=np.zeros(25))
        self._exec_count = 0

    def compute(self, inputs, outputs):
        x = inputs['x']
        outputs['y'] = np.arange(25) * x
        self._exec_count += 1

class CompTwo(om.ExplicitComponent):

    def setup(self):
        self.add_input('y', val=np.zeros(25))
        self.add_output('z', val=0.0)
        self._exec_count = 0

    def compute(self, inputs, outputs):
        y = inputs['y']
        outputs['z'] = np.sum(y)
        self._exec_count += 1

prob = om.Problem()
model = prob.model
model.add_subsystem('comp1', CompOne(), promotes=['x', 'y'])
model.add_subsystem('comp2', CompTwo(), promotes=['y', 'z'])

model.linear_solver = om.ScipyKrylov()
model.approx_totals(method='fd', step=1e-7, form='central', step_calc='rel')

prob.setup()
prob.run_model()

of = ['z']
wrt = ['x']
derivs = prob.compute_totals(of=of, wrt=wrt)
print(derivs['z', 'x'])
[[300.00000048]]

Complex Step#

You can also complex step your model or group, though there are some important restrictions.

All components must support complex calculations in solve_nonlinear:

Under complex step, a component’s inputs are complex, all stages of the calculation will operate on complex inputs to produce complex outputs, and the final value placed into outputs is complex. Most Python functions already support complex numbers, so pure Python components will generally satisfy this requirement. Take care with functions like abs, which effectively squelches the complex part of the argument.

If you complex step around a solver that requires gradients, the solver must not get its gradients from complex step:

When you complex step around a nonlinear solver that requires gradients (like Newton), the nonlinear solver must solve a complex linear system rather than a real one. Most of OpenMDAO’s linear solvers (with the exception of PETScKrylov) support the solution of such a system. However, when you linearize the submodel to determine derivatives around a complex point, the application of complex step loses some of its robust properties when compared to real-valued finite difference (in particular, you get subtractive cancelation which causes increased inaccuracy for smaller step sizes.) When OpenMDAO encounters this situation, it will warn the user, and the inner approximation will automatically switch over to using finite difference with default settings.

Care must be taken with iterative solver tolerances; you may need to adjust the stepsize for complex step:

If you are using an iterative nonlinear solver, and you don’t converge it tightly, then the complex stepped linear system may have trouble converging as well. You may need to tighten the convergence of your solvers and increase the step size used for complex step. To prevent the nonlinear solvers from ignoring a tiny complex step, a tiny offset is added to the states to nudge it off the solution, allowing it to reconverge with the complex step. You can also turn this behavior off by setting the “cs_reconverge” to False.

Similarly, an iterative linear solver may also require adjusting the step size, particularly if you are using the ScipyKrylov solver.

import numpy as np

import openmdao.api as om

class CompOne(om.ExplicitComponent):

    def setup(self):
        self.add_input('x', val=0.0)
        self.add_output('y', val=np.zeros(25))
        self._exec_count = 0

    def compute(self, inputs, outputs):
        x = inputs['x']
        outputs['y'] = np.arange(25) * x
        self._exec_count += 1

class CompTwo(om.ExplicitComponent):

    def setup(self):
        self.add_input('y', val=np.zeros(25))
        self.add_output('z', val=0.0)
        self._exec_count = 0

    def compute(self, inputs, outputs):
        y = inputs['y']
        outputs['z'] = np.sum(y)
        self._exec_count += 1

prob = om.Problem()
model = prob.model
model.set_input_defaults('x', 0.0)

model.add_subsystem('comp1', CompOne(), promotes=['x', 'y'])
model.add_subsystem('comp2', CompTwo(), promotes=['y', 'z'])

model.linear_solver = om.ScipyKrylov()
model.approx_totals(method='cs')

prob.setup()
prob.run_model()

of = ['z']
wrt = ['x']
derivs = prob.compute_totals(of=of, wrt=wrt)
print(derivs['z', 'x'])
[[300.]]

Approx-Totals and Relevance#

OpenMDAO builds a relevancy-graph that helps it be more efficient when executing complex models.

One feature that uses this graph is the relevance check that is performed when a Driver begins to run. The relevance check is used to verify that each constraint can be impacted by a design variable.

However, when approx_totals is used, users often omit the declaration of any partials. In this situation, the relevance check would normally fail, so OpenMDAO skips this check if any Group in the system tree is found to use approximated derivatives.

import numpy as np

import openmdao.api as om

class AComp(om.ExplicitComponent):

    def setup(self):
        self.add_input('x', val=0.0)
        self.add_output('y', val=0.0)

    def compute(self, inputs, outputs):
        outputs['y'] = inputs['x'] - 5.0

class BComp(om.ExplicitComponent):

    def setup(self):
        self.add_input('y', val=0.0)
        self.add_output('z', val=0.0)

    def compute(self, inputs, outputs):
        outputs['z'] = inputs['y'] ** 2

prob = om.Problem()
prob.driver = om.ScipyOptimizeDriver(optimizer='SLSQP')
model = prob.model
model.set_input_defaults('x', 0.0)

model.add_subsystem('a', AComp(), promotes=['x', 'y'])
model.add_subsystem('b', BComp(), promotes=['y', 'z'])

model.add_design_var('x', lower=-100, upper=100)
model.add_objective('z')

model.approx_totals(method='cs')

prob.setup()
prob.run_driver()

of = ['z']
wrt = ['x']
Optimization terminated successfully    (Exit mode 0)
            Current function value: 0.0
            Iterations: 2
            Function evaluations: 3
            Gradient evaluations: 2
Optimization Complete
-----------------------------------