Checking Total Derivatives#
If you want to check the analytic derivatives of your model (or just part of it) against finite difference or complex-step approximations, you can use check_totals(). You should always converge your model before calling this method. By default, this method checks the derivatives of all of the driver responses (objectives, constraints) with respect to the design variables, though you can also specify the variables you want to check. Derivatives are computed and compared in an unscaled form by default, but you can optionally request for them to be computed in scaled form using the ref and ref0 that were declared when adding the constraints, objectives, and design variables.
Note
You should probably not use this method until you’ve used check_partials() to verify the partials for each component in your model. check_totals() is a blunt instrument, since it can only tell you that there is a problem, but will not give you much insight into which component or group is causing the problem.
- Problem.check_totals(of=None, wrt=None, out_stream=DEFAULT_OUT_STREAM, compact_print=False, driver_scaling=False, abs_err_tol=1e-06, rel_err_tol=1e-06, method='fd', step=None, form=None, step_calc='abs', show_progress=False, show_only_incorrect=False, directional=False, sort=False)[source]
Check total derivatives for the model vs. finite difference.
- Parameters:
- oflist of variable name str or None
Variables whose derivatives will be computed. Default is None, which uses the driver’s objectives and constraints.
- wrtlist of variable name str or None
Variables with respect to which the derivatives will be computed. Default is None, which uses the driver’s desvars.
- out_streamfile-like object
Where to send human readable output. By default it goes to stdout. Set to None to suppress.
- compact_printbool
Set to True to just print the essentials, one line per input-output pair.
- driver_scalingbool
When True, return derivatives that are scaled according to either the adder and scaler or the ref and ref0 values that were specified when add_design_var, add_objective, and add_constraint were called on the model. Default is False, which is unscaled.
- abs_err_tolfloat
Threshold value for absolute error. Errors about this value will have a ‘*’ displayed next to them in output, making them easy to search for. Default is 1.0E-6.
- rel_err_tolfloat
Threshold value for relative error. Errors about this value will have a ‘*’ displayed next to them in output, making them easy to search for. Note at times there may be a significant relative error due to a minor absolute error. Default is 1.0E-6.
- methodstr
Method, ‘fd’ for finite difference or ‘cs’ for complex step. Default is ‘fd’.
- stepNone, float, or list/tuple of float
Step size for approximation. Default is None, which means 1e-6 for ‘fd’ and 1e-40 for ‘cs’.
- formstr
Form for finite difference, can be ‘forward’, ‘backward’, or ‘central’. Default None, which defaults to ‘forward’ for FD.
- step_calcstr
Step type for computing the size of the finite difference step. It can be ‘abs’ for absolute, ‘rel_avg’ for a size relative to the absolute value of the vector input, or ‘rel_element’ for a size relative to each value in the vector input. In addition, it can be ‘rel_legacy’ for a size relative to the norm of the vector. For backwards compatibilty, it can be ‘rel’, which is now equivalent to ‘rel_avg’. Defaults to None, in which case the approximation method provides its default value..
- show_progressbool
True to show progress of check_totals.
- show_only_incorrectbool, optional
Set to True if output should print only the subjacs found to be incorrect.
- directionalbool
If True, compute a single directional derivative for each ‘of’ in rev mode or each ‘wrt’ in fwd mode.
- sortbool
If True, sort the subjacobian keys alphabetically.
- Returns:
- Dict of Dicts of Tuples of Floats
- First key:
is the (output, input) tuple of strings;
- Second key:
is one of [‘rel error’, ‘abs error’, ‘magnitude’, ‘fdstep’];
- For ‘rel error’, ‘abs error’, ‘magnitude’ the value is: A tuple containing norms for
forward - fd, adjoint - fd, forward - adjoint.
Please check out the Complex Step Guidelines for more information on how to use it on a model.
Examples#
You can check specific combinations of variables by specifying them manually:
SellarDerivatives class definition
class SellarDerivatives(om.Group):
"""
Group containing the Sellar MDA. This version uses the disciplines with derivatives.
"""
def setup(self):
self.add_subsystem('d1', SellarDis1withDerivatives(), promotes=['x', 'z', 'y1', 'y2'])
self.add_subsystem('d2', SellarDis2withDerivatives(), promotes=['z', 'y1', 'y2'])
self.add_subsystem('obj_cmp', om.ExecComp('obj = x**2 + z[1] + y1 + exp(-y2)', obj=0.0,
x=0.0, z=np.array([0.0, 0.0]), y1=0.0, y2=0.0),
promotes=['obj', 'x', 'z', 'y1', 'y2'])
self.add_subsystem('con_cmp1', om.ExecComp('con1 = 3.16 - y1', con1=0.0, y1=0.0),
promotes=['con1', 'y1'])
self.add_subsystem('con_cmp2', om.ExecComp('con2 = y2 - 24.0', con2=0.0, y2=0.0),
promotes=['con2', 'y2'])
self.set_input_defaults('x', 1.0)
self.set_input_defaults('z', np.array([5.0, 2.0]))
import openmdao.api as om
from openmdao.test_suite.components.sellar_feature import SellarDerivatives
prob = om.Problem()
prob.model = SellarDerivatives()
prob.model.nonlinear_solver = om.NonlinearBlockGS()
prob.model.linear_solver = om.ScipyKrylov()
prob.setup()
prob.run_model()
NL: NLBGS Converged in 8 iterations
# manually specify which derivatives to check
prob.check_totals(of=['obj', 'con1'], wrt=['x', 'z'])
NL: NLBGS Converged in 4 iterations
NL: NLBGS Converged in 5 iterations
NL: NLBGS Converged in 5 iterations
-----------------
Total Derivatives
-----------------
Full Model: 'con1' wrt 'x'
Reverse Magnitude: 9.806145e-01
Fd Magnitude: 9.806169e-01 (fd:forward)
Absolute Error (Jrev - Jfd) : 2.407248e-06 *
Relative Error (Jrev - Jfd) / Jfd : 2.454831e-06 *
Raw Reverse Derivative (Jrev)
[[-0.98061448]]
Raw FD Derivative (Jfd)
[[-0.98061688]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Full Model: 'con1' wrt 'z'
Reverse Magnitude: 9.641989e+00
Fd Magnitude: 9.641993e+00 (fd:forward)
Absolute Error (Jrev - Jfd) : 4.447254e-06 *
Relative Error (Jrev - Jfd) / Jfd : 4.612380e-07
Raw Reverse Derivative (Jrev)
[[-9.61002186 -0.78449158]]
Raw FD Derivative (Jfd)
[[-9.61002547 -0.78449417]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Full Model: 'obj' wrt 'x'
Reverse Magnitude: 2.980614e+00
Fd Magnitude: 2.980617e+00 (fd:forward)
Absolute Error (Jrev - Jfd) : 3.404818e-06 *
Relative Error (Jrev - Jfd) / Jfd : 1.142320e-06 *
Raw Reverse Derivative (Jrev)
[[2.98061391]]
Raw FD Derivative (Jfd)
[[2.98061732]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Full Model: 'obj' wrt 'z'
Reverse Magnitude: 9.774287e+00
Fd Magnitude: 9.774291e+00 (fd:forward)
Absolute Error (Jrev - Jfd) : 4.448707e-06 *
Relative Error (Jrev - Jfd) / Jfd : 4.551437e-07
Raw Reverse Derivative (Jrev)
[[9.61001056 1.78448534]]
Raw FD Derivative (Jfd)
[[9.61001417 1.78448793]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
{('obj', 'x'): {'J_fd': array([[2.98061732]]),
'J_rev': array([[2.98061391]]),
'abs error': ErrorTuple(forward=None, reverse=3.40481810656712e-06, forward_reverse=None),
'rel error': ErrorTuple(forward=None, reverse=1.1423197757256306e-06, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=None, reverse=2.9806139134842864, fd=2.980617318302393)},
('obj', 'z'): {'J_fd': array([[9.61001417, 1.78448793]]),
'J_rev': array([[9.61001056, 1.78448534]]),
'abs error': ErrorTuple(forward=None, reverse=4.4487067021045885e-06, forward_reverse=None),
'rel error': ErrorTuple(forward=None, reverse=4.551436605458813e-07, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=None, reverse=9.774287228158455, fd=9.77429125733397)},
('con1', 'x'): {'J_fd': array([[-0.98061688]]),
'J_rev': array([[-0.98061448]]),
'abs error': ErrorTuple(forward=None, reverse=2.407248432922948e-06, forward_reverse=None),
'rel error': ErrorTuple(forward=None, reverse=2.4548307050606205e-06, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=None, reverse=0.9806144751949951, fd=0.980616882443428)},
('con1', 'z'): {'J_fd': array([[-9.61002547, -0.78449417]]),
'J_rev': array([[-9.61002186, -0.78449158]]),
'abs error': ErrorTuple(forward=None, reverse=4.447253572610684e-06, forward_reverse=None),
'rel error': ErrorTuple(forward=None, reverse=4.61238021186646e-07, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=None, reverse=9.641988753864103, fd=9.641992568542056)}}
Check all the derivatives that the driver will need:
import openmdao.api as om
from openmdao.test_suite.components.sellar_feature import SellarDerivatives
prob = om.Problem()
prob.model = SellarDerivatives()
prob.model.nonlinear_solver = om.NonlinearBlockGS()
prob.model.linear_solver = om.ScipyKrylov()
prob.model.add_design_var('x', lower=-100, upper=100)
prob.model.add_design_var('z', lower=-100, upper=100)
prob.model.add_objective('obj')
prob.model.add_constraint('con1', upper=0.0)
prob.model.add_constraint('con2', upper=0.0)
prob.setup()
# We don't call run_driver() here because we don't
# actually want the optimizer to run
prob.run_model()
NL: NLBGS Converged in 8 iterations
# check derivatives of all obj+constraints w.r.t all design variables
prob.check_totals()
NL: NLBGS Converged in 4 iterations
NL: NLBGS Converged in 5 iterations
NL: NLBGS Converged in 5 iterations
-----------------
Total Derivatives
-----------------
Full Model: 'con1' wrt 'x'
Forward Magnitude: 9.806145e-01
Fd Magnitude: 9.806169e-01 (fd:forward)
Absolute Error (Jfor - Jfd) : 2.407248e-06 *
Relative Error (Jfor - Jfd) / Jfd : 2.454831e-06 *
Raw Forward Derivative (Jfor)
[[-0.98061448]]
Raw FD Derivative (Jfd)
[[-0.98061688]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Full Model: 'con1' wrt 'z'
Forward Magnitude: 9.641989e+00
Fd Magnitude: 9.641993e+00 (fd:forward)
Absolute Error (Jfor - Jfd) : 4.447254e-06 *
Relative Error (Jfor - Jfd) / Jfd : 4.612380e-07
Raw Forward Derivative (Jfor)
[[-9.61002186 -0.78449158]]
Raw FD Derivative (Jfd)
[[-9.61002547 -0.78449417]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Full Model: 'con2' wrt 'x'
Forward Magnitude: 9.692762e-02
Fd Magnitude: 9.692786e-02 (fd:forward)
Absolute Error (Jfor - Jfd) : 2.385894e-07
Relative Error (Jfor - Jfd) / Jfd : 2.461515e-06 *
Raw Forward Derivative (Jfor)
[[0.09692762]]
Raw FD Derivative (Jfd)
[[0.09692786]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Full Model: 'con2' wrt 'z'
Forward Magnitude: 2.227817e+00
Fd Magnitude: 2.227818e+00 (fd:forward)
Absolute Error (Jfor - Jfd) : 3.738610e-07
Relative Error (Jfor - Jfd) / Jfd : 1.678149e-07
Raw Forward Derivative (Jfor)
[[1.94989072 1.0775421 ]]
Raw FD Derivative (Jfd)
[[1.94989099 1.07754236]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Full Model: 'obj' wrt 'x'
Forward Magnitude: 2.980614e+00
Fd Magnitude: 2.980617e+00 (fd:forward)
Absolute Error (Jfor - Jfd) : 3.404818e-06 *
Relative Error (Jfor - Jfd) / Jfd : 1.142320e-06 *
Raw Forward Derivative (Jfor)
[[2.98061391]]
Raw FD Derivative (Jfd)
[[2.98061732]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Full Model: 'obj' wrt 'z'
Forward Magnitude: 9.774287e+00
Fd Magnitude: 9.774291e+00 (fd:forward)
Absolute Error (Jfor - Jfd) : 4.448707e-06 *
Relative Error (Jfor - Jfd) / Jfd : 4.551437e-07
Raw Forward Derivative (Jfor)
[[9.61001056 1.78448534]]
Raw FD Derivative (Jfd)
[[9.61001417 1.78448793]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
{('obj', 'x'): {'J_fd': array([[2.98061732]]),
'J_fwd': array([[2.98061391]]),
'abs error': ErrorTuple(forward=3.4048181056789417e-06, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=1.1423197754276459e-06, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=2.9806139134842873, reverse=None, fd=2.980617318302393)},
('obj', 'z'): {'J_fd': array([[9.61001417, 1.78448793]]),
'J_fwd': array([[9.61001056, 1.78448534]]),
'abs error': ErrorTuple(forward=4.448706699345221e-06, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=4.5514366026357264e-07, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=9.774287228158459, reverse=None, fd=9.77429125733397)},
('con1', 'x'): {'J_fd': array([[-0.98061688]]),
'J_fwd': array([[-0.98061448]]),
'abs error': ErrorTuple(forward=2.4072484330339705e-06, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=2.4548307051738375e-06, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=0.980614475194995, reverse=None, fd=0.980616882443428)},
('con1', 'z'): {'J_fd': array([[-9.61002547, -0.78449417]]),
'J_fwd': array([[-9.61002186, -0.78449158]]),
'abs error': ErrorTuple(forward=4.447253574184286e-06, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=4.6123802134984895e-07, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=9.641988753864103, reverse=None, fd=9.641992568542056)},
('con2', 'x'): {'J_fd': array([[0.09692786]]),
'J_fwd': array([[0.09692762]]),
'abs error': ErrorTuple(forward=2.38589368345421e-07, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=2.461514799873363e-06, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=0.09692762402502489, reverse=None, fd=0.09692786261439323)},
('con2', 'z'): {'J_fd': array([[1.94989099, 1.07754236]]),
'J_fwd': array([[1.94989072, 1.0775421 ]]),
'abs error': ErrorTuple(forward=3.7386098645400555e-07, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=1.6781488030575185e-07, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=2.227817492024661, reverse=None, fd=2.2278178536542534)}}
Use the driver scaled values during the check:
import openmdao.api as om
from openmdao.test_suite.components.sellar_feature import SellarDerivatives
prob = om.Problem()
prob.model = SellarDerivatives()
prob.model.nonlinear_solver = om.NonlinearBlockGS()
prob.model.linear_solver = om.ScipyKrylov()
prob.model.add_design_var('x', lower=-100, upper=100, ref=100.0, ref0=-100.0)
prob.model.add_design_var('z', lower=-100, upper=100)
prob.model.add_objective('obj')
prob.model.add_constraint('con1', upper=0.0, ref=3.0)
prob.model.add_constraint('con2', upper=0.0, ref=20.0)
prob.setup()
# We don't call run_driver() here because we don't
# actually want the optimizer to run
prob.run_model()
NL: NLBGS Converged in 8 iterations
# check derivatives of all driver vars using the declared scaling
prob.check_totals(driver_scaling=True)
NL: NLBGS Converged in 4 iterations
NL: NLBGS Converged in 5 iterations
NL: NLBGS Converged in 5 iterations
-----------------
Total Derivatives
-----------------
Full Model: 'con1' wrt 'x'
Forward Magnitude: 6.537430e+01
Fd Magnitude: 6.537446e+01 (fd:forward)
Absolute Error (Jfor - Jfd) : 1.604832e-04 *
Relative Error (Jfor - Jfd) / Jfd : 2.454831e-06 *
Raw Forward Derivative (Jfor)
[[-65.37429835]]
Raw FD Derivative (Jfd)
[[-65.37445883]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Full Model: 'con1' wrt 'z'
Forward Magnitude: 3.213996e+00
Fd Magnitude: 3.213998e+00 (fd:forward)
Absolute Error (Jfor - Jfd) : 1.482418e-06 *
Relative Error (Jfor - Jfd) / Jfd : 4.612380e-07
Raw Forward Derivative (Jfor)
[[-3.20334062 -0.26149719]]
Raw FD Derivative (Jfd)
[[-3.20334182 -0.26149806]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Full Model: 'con2' wrt 'x'
Forward Magnitude: 9.692762e-01
Fd Magnitude: 9.692786e-01 (fd:forward)
Absolute Error (Jfor - Jfd) : 2.385894e-06 *
Relative Error (Jfor - Jfd) / Jfd : 2.461515e-06 *
Raw Forward Derivative (Jfor)
[[0.96927624]]
Raw FD Derivative (Jfd)
[[0.96927863]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Full Model: 'con2' wrt 'z'
Forward Magnitude: 1.113909e-01
Fd Magnitude: 1.113909e-01 (fd:forward)
Absolute Error (Jfor - Jfd) : 1.869305e-08
Relative Error (Jfor - Jfd) / Jfd : 1.678149e-07
Raw Forward Derivative (Jfor)
[[0.09749454 0.0538771 ]]
Raw FD Derivative (Jfd)
[[0.09749455 0.05387712]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Full Model: 'obj' wrt 'x'
Forward Magnitude: 5.961228e+02
Fd Magnitude: 5.961235e+02 (fd:forward)
Absolute Error (Jfor - Jfd) : 6.809636e-04 *
Relative Error (Jfor - Jfd) / Jfd : 1.142320e-06 *
Raw Forward Derivative (Jfor)
[[596.1227827]]
Raw FD Derivative (Jfd)
[[596.12346366]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Full Model: 'obj' wrt 'z'
Forward Magnitude: 9.774287e+00
Fd Magnitude: 9.774291e+00 (fd:forward)
Absolute Error (Jfor - Jfd) : 4.448707e-06 *
Relative Error (Jfor - Jfd) / Jfd : 4.551437e-07
Raw Forward Derivative (Jfor)
[[9.61001056 1.78448534]]
Raw FD Derivative (Jfd)
[[9.61001417 1.78448793]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
{('obj', 'x'): {'J_fd': array([[596.12346366]]),
'J_fwd': array([[596.1227827]]),
'abs error': ErrorTuple(forward=0.0006809636211073666, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=1.1423197753799683e-06, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=596.1227826968575, reverse=None, fd=596.1234636604786)},
('obj', 'z'): {'J_fd': array([[9.61001417, 1.78448793]]),
'J_fwd': array([[9.61001056, 1.78448534]]),
'abs error': ErrorTuple(forward=4.448706699345221e-06, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=4.5514366026357264e-07, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=9.774287228158459, reverse=None, fd=9.77429125733397)},
('con1', 'x'): {'J_fd': array([[-65.37445883]]),
'J_fwd': array([[-65.37429835]]),
'abs error': ErrorTuple(forward=0.00016048322886774713, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=2.4548307051557233e-06, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=65.37429834633299, reverse=None, fd=65.37445882956186)},
('con1', 'z'): {'J_fd': array([[-3.20334182, -0.26149806]]),
'J_fwd': array([[-3.20334062, -0.26149719]]),
'abs error': ErrorTuple(forward=1.4824178579410668e-06, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=4.612380213123997e-07, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=3.213996251288034, reverse=None, fd=3.213997522847352)},
('con2', 'x'): {'J_fd': array([[0.96927863]]),
'J_fwd': array([[0.96927624]]),
'abs error': ErrorTuple(forward=2.3858936832876765e-06, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=2.4615147997015513e-06, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=0.969276240250249, reverse=None, fd=0.9692786261439323)},
('con2', 'z'): {'J_fd': array([[0.09749455, 0.05387712]]),
'J_fwd': array([[0.09749454, 0.0538771 ]]),
'abs error': ErrorTuple(forward=1.8693049323752086e-08, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=1.6781488031519434e-07, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=0.11139087460123305, reverse=None, fd=0.11139089268271268)}}
Display the results in a compact format:
import openmdao.api as om
from openmdao.test_suite.components.sellar_feature import SellarDerivatives
prob = om.Problem()
prob.model = SellarDerivatives()
prob.model.nonlinear_solver = om.NonlinearBlockGS()
prob.model.linear_solver = om.ScipyKrylov()
prob.model.add_design_var('x', lower=-100, upper=100)
prob.model.add_design_var('z', lower=-100, upper=100)
prob.model.add_objective('obj')
prob.model.add_constraint('con1', upper=0.0)
prob.model.add_constraint('con2', upper=0.0)
prob.setup()
# We don't call run_driver() here because we don't
# actually want the optimizer to run
prob.run_model()
NL: NLBGS Converged in 8 iterations
# check derivatives of all obj+constraints w.r.t all design variables
prob.check_totals(compact_print=True)
NL: NLBGS Converged in 4 iterations
NL: NLBGS Converged in 5 iterations
NL: NLBGS Converged in 5 iterations
-----------------
Total Derivatives
-----------------
+-----------------+------------------+-------------+-------------+-------------+-------------+--------------------+
| of '<variable>' | wrt '<variable>' | calc mag. | check mag. | a(cal-chk) | r(cal-chk) | error desc |
+=================+==================+=============+=============+=============+=============+====================+
| 'con1' | 'x' | 9.8061e-01 | 9.8062e-01 | 2.4072e-06 | 2.4548e-06 | >ABS_TOL >REL_TOL |
+-----------------+------------------+-------------+-------------+-------------+-------------+--------------------+
| 'con1' | 'z' | 9.6420e+00 | 9.6420e+00 | 4.4473e-06 | 4.6124e-07 | >ABS_TOL |
+-----------------+------------------+-------------+-------------+-------------+-------------+--------------------+
| 'con2' | 'x' | 9.6928e-02 | 9.6928e-02 | 2.3859e-07 | 2.4615e-06 | >REL_TOL |
+-----------------+------------------+-------------+-------------+-------------+-------------+--------------------+
| 'con2' | 'z' | 2.2278e+00 | 2.2278e+00 | 3.7386e-07 | 1.6781e-07 | |
+-----------------+------------------+-------------+-------------+-------------+-------------+--------------------+
| 'obj' | 'x' | 2.9806e+00 | 2.9806e+00 | 3.4048e-06 | 1.1423e-06 | >ABS_TOL >REL_TOL |
+-----------------+------------------+-------------+-------------+-------------+-------------+--------------------+
| 'obj' | 'z' | 9.7743e+00 | 9.7743e+00 | 4.4487e-06 | 4.5514e-07 | >ABS_TOL |
+-----------------+------------------+-------------+-------------+-------------+-------------+--------------------+
{('obj', 'x'): {'J_fd': array([[2.98061732]]),
'J_fwd': array([[2.98061391]]),
'abs error': ErrorTuple(forward=3.4048181056789417e-06, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=1.1423197754276459e-06, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=2.9806139134842873, reverse=None, fd=2.980617318302393)},
('obj', 'z'): {'J_fd': array([[9.61001417, 1.78448793]]),
'J_fwd': array([[9.61001056, 1.78448534]]),
'abs error': ErrorTuple(forward=4.448706699345221e-06, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=4.5514366026357264e-07, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=9.774287228158459, reverse=None, fd=9.77429125733397)},
('con1', 'x'): {'J_fd': array([[-0.98061688]]),
'J_fwd': array([[-0.98061448]]),
'abs error': ErrorTuple(forward=2.4072484330339705e-06, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=2.4548307051738375e-06, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=0.980614475194995, reverse=None, fd=0.980616882443428)},
('con1', 'z'): {'J_fd': array([[-9.61002547, -0.78449417]]),
'J_fwd': array([[-9.61002186, -0.78449158]]),
'abs error': ErrorTuple(forward=4.447253574184286e-06, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=4.6123802134984895e-07, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=9.641988753864103, reverse=None, fd=9.641992568542056)},
('con2', 'x'): {'J_fd': array([[0.09692786]]),
'J_fwd': array([[0.09692762]]),
'abs error': ErrorTuple(forward=2.38589368345421e-07, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=2.461514799873363e-06, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=0.09692762402502489, reverse=None, fd=0.09692786261439323)},
('con2', 'z'): {'J_fd': array([[1.94989099, 1.07754236]]),
'J_fwd': array([[1.94989072, 1.0775421 ]]),
'abs error': ErrorTuple(forward=3.7386098645400555e-07, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=1.6781488030575185e-07, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=2.227817492024661, reverse=None, fd=2.2278178536542534)}}
Use complex step instead of finite difference for a more accurate check. We also tighten the tolerance on the nonlinear Gauss-Seidel solver so that we get more accurate converged values.
import openmdao.api as om
from openmdao.test_suite.components.sellar_feature import SellarDerivatives
prob = om.Problem()
prob.model = SellarDerivatives()
prob.model.nonlinear_solver = om.NonlinearBlockGS()
prob.model.linear_solver = om.ScipyKrylov()
prob.model.add_design_var('x', lower=-100, upper=100)
prob.model.add_design_var('z', lower=-100, upper=100)
prob.model.add_objective('obj')
prob.model.add_constraint('con1', upper=0.0)
prob.model.add_constraint('con2', upper=0.0)
prob.setup(force_alloc_complex=True)
# We don't call run_driver() here because we don't
# actually want the optimizer to run
prob.run_model()
NL: NLBGS Converged in 8 iterations
prob.model.nonlinear_solver.options['atol'] = 1e-15
prob.model.nonlinear_solver.options['rtol'] = 1e-15
# check derivatives with complex step
prob.check_totals(method='cs')
NL: NLBGS Converged in 7 iterations
NL: NLBGS Converged in 7 iterations
NL: NLBGS Converged in 7 iterations
-----------------
Total Derivatives
-----------------
Full Model: 'con1' wrt 'x'
Forward Magnitude: 9.806145e-01
Fd Magnitude: 9.806145e-01 (cs:None)
Absolute Error (Jfor - Jfd) : 2.139144e-11
Relative Error (Jfor - Jfd) / Jfd : 2.181433e-11
Raw Forward Derivative (Jfor)
[[-0.98061448]]
Raw CS Derivative (Jfd)
[[-0.98061448]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Full Model: 'con1' wrt 'z'
Forward Magnitude: 9.641989e+00
Fd Magnitude: 9.641989e+00 (cs:None)
Absolute Error (Jfor - Jfd) : 9.613200e-09
Relative Error (Jfor - Jfd) / Jfd : 9.970143e-10
Raw Forward Derivative (Jfor)
[[-9.61002186 -0.78449158]]
Raw CS Derivative (Jfd)
[[-9.61002187 -0.78449158]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Full Model: 'con2' wrt 'x'
Forward Magnitude: 9.692762e-02
Fd Magnitude: 9.692762e-02 (cs:None)
Absolute Error (Jfor - Jfd) : 1.010579e-10
Relative Error (Jfor - Jfd) / Jfd : 1.042612e-09
Raw Forward Derivative (Jfor)
[[0.09692762]]
Raw CS Derivative (Jfd)
[[0.09692762]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Full Model: 'con2' wrt 'z'
Forward Magnitude: 2.227817e+00
Fd Magnitude: 2.227817e+00 (cs:None)
Absolute Error (Jfor - Jfd) : 1.002700e-10
Relative Error (Jfor - Jfd) / Jfd : 4.500819e-11
Raw Forward Derivative (Jfor)
[[1.94989072 1.0775421 ]]
Raw CS Derivative (Jfd)
[[1.94989072 1.0775421 ]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Full Model: 'obj' wrt 'x'
Forward Magnitude: 2.980614e+00
Fd Magnitude: 2.980614e+00 (cs:None)
Absolute Error (Jfor - Jfd) : 9.598640e-09
Relative Error (Jfor - Jfd) / Jfd : 3.220357e-09
Raw Forward Derivative (Jfor)
[[2.98061391]]
Raw CS Derivative (Jfd)
[[2.98061392]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Full Model: 'obj' wrt 'z'
Forward Magnitude: 9.774287e+00
Fd Magnitude: 9.774287e+00 (cs:None)
Absolute Error (Jfor - Jfd) : 9.613369e-09
Relative Error (Jfor - Jfd) / Jfd : 9.835366e-10
Raw Forward Derivative (Jfor)
[[9.61001056 1.78448534]]
Raw CS Derivative (Jfd)
[[9.61001057 1.78448534]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
{('obj', 'x'): {'J_fd': array([[2.98061392]]),
'J_fwd': array([[2.98061391]]),
'abs error': ErrorTuple(forward=9.598640104968581e-09, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=3.220356729408435e-09, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=2.9806139134842873, reverse=None, fd=2.9806139230829274)},
('obj', 'z'): {'J_fd': array([[9.61001057, 1.78448534]]),
'J_fwd': array([[9.61001056, 1.78448534]]),
'abs error': ErrorTuple(forward=9.61336942645711e-09, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=9.835366193721887e-10, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=9.774287228158459, reverse=None, fd=9.774287237615534)},
('con1', 'x'): {'J_fd': array([[-0.98061448]]),
'J_fwd': array([[-0.98061448]]),
'abs error': ErrorTuple(forward=2.139144417157013e-11, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=2.181432633538252e-11, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=0.980614475194995, reverse=None, fd=0.9806144752163864)},
('con1', 'z'): {'J_fd': array([[-9.61002187, -0.78449158]]),
'J_fwd': array([[-9.61002186, -0.78449158]]),
'abs error': ErrorTuple(forward=9.613200388027657e-09, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=9.970142699679106e-10, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=9.641988753864103, reverse=None, fd=9.64198876344775)},
('con2', 'x'): {'J_fd': array([[0.09692762]]),
'J_fwd': array([[0.09692762]]),
'abs error': ErrorTuple(forward=1.0105788428305118e-10, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=1.0426117982869784e-09, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=0.09692762402502489, reverse=None, fd=0.096927623923967)},
('con2', 'z'): {'J_fd': array([[1.94989072, 1.0775421 ]]),
'J_fwd': array([[1.94989072, 1.0775421 ]]),
'abs error': ErrorTuple(forward=1.0027002384850114e-10, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=4.5008185909119373e-11, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=2.227817492024661, reverse=None, fd=2.227817491932836)}}
Note that you need to set the argument force_alloc_complex to True when you call setup. This makes sure that the vectors allocate space to pass complex numbers. All systems also have an attribute force_alloc_complex that can be queried during setup in case you need to set up your Component or Group differently to accomodate complex step.
import openmdao.api as om
class SimpleComp(om.ExplicitComponent):
def setup(self):
self.add_input('x', val=1.0)
self.add_output('y', val=1.0)
self.declare_partials(of='y', wrt='x')
if self._force_alloc_complex:
print("Vectors allocated for complex step.")
def compute(self, inputs, outputs):
outputs['y'] = 3.0*inputs['x']
def compute_partials(self, inputs, partials):
partials['y', 'x'] = 3.
prob = om.Problem()
prob.model.add_subsystem('comp', SimpleComp())
prob.model.add_design_var('comp.x', lower=-100, upper=100)
prob.model.add_objective('comp.y')
prob.setup(force_alloc_complex=True)
prob.run_model()
prob.check_totals(method='cs')
Vectors allocated for complex step.
-----------------
Total Derivatives
-----------------
Full Model: 'comp.y' wrt 'comp.x'
Forward Magnitude: 3.000000e+00
Fd Magnitude: 3.000000e+00 (cs:None)
Absolute Error (Jfor - Jfd) : 0.000000e+00
Relative Error (Jfor - Jfd) / Jfd : 0.000000e+00
Raw Forward Derivative (Jfor)
[[3.]]
Raw CS Derivative (Jfd)
[[3.]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
{('comp.y', 'comp.x'): {'J_fd': array([[3.]]),
'J_fwd': array([[3.]]),
'abs error': ErrorTuple(forward=0.0, reverse=None, forward_reverse=None),
'rel error': ErrorTuple(forward=0.0, reverse=None, forward_reverse=None),
'magnitude': MagnitudeTuple(forward=3.0, reverse=None, fd=3.0)}}
Turn off standard output and just view the derivatives in the return:
import openmdao.api as om
from openmdao.test_suite.components.sellar_feature import SellarDerivatives
prob = om.Problem()
prob.model = SellarDerivatives()
prob.model.nonlinear_solver = om.NonlinearBlockGS()
prob.model.linear_solver = om.ScipyKrylov()
prob.model.add_design_var('x', lower=-100, upper=100)
prob.model.add_design_var('z', lower=-100, upper=100)
prob.model.add_objective('obj')
prob.model.add_constraint('con1', upper=0.0)
prob.model.add_constraint('con2', upper=0.0)
prob.setup()
# We don't call run_driver() here because we don't
# actually want the optimizer to run
prob.run_model()
NL: NLBGS Converged in 8 iterations
# check derivatives of all obj+constraints w.r.t all design variables
totals = prob.check_totals(out_stream=None)
NL: NLBGS Converged in 4 iterations
NL: NLBGS Converged in 5 iterations
NL: NLBGS Converged in 5 iterations
print(totals)
{('obj', 'x'): {'J_fd': array([[2.98061732]]), 'J_fwd': array([[2.98061391]]), 'abs error': ErrorTuple(forward=3.4048181056789417e-06, reverse=None, forward_reverse=None), 'rel error': ErrorTuple(forward=1.1423197754276459e-06, reverse=None, forward_reverse=None), 'magnitude': MagnitudeTuple(forward=2.9806139134842873, reverse=None, fd=2.980617318302393)}, ('obj', 'z'): {'J_fd': array([[9.61001417, 1.78448793]]), 'J_fwd': array([[9.61001056, 1.78448534]]), 'abs error': ErrorTuple(forward=4.448706699345221e-06, reverse=None, forward_reverse=None), 'rel error': ErrorTuple(forward=4.5514366026357264e-07, reverse=None, forward_reverse=None), 'magnitude': MagnitudeTuple(forward=9.774287228158459, reverse=None, fd=9.77429125733397)}, ('con1', 'x'): {'J_fd': array([[-0.98061688]]), 'J_fwd': array([[-0.98061448]]), 'abs error': ErrorTuple(forward=2.4072484330339705e-06, reverse=None, forward_reverse=None), 'rel error': ErrorTuple(forward=2.4548307051738375e-06, reverse=None, forward_reverse=None), 'magnitude': MagnitudeTuple(forward=0.980614475194995, reverse=None, fd=0.980616882443428)}, ('con1', 'z'): {'J_fd': array([[-9.61002547, -0.78449417]]), 'J_fwd': array([[-9.61002186, -0.78449158]]), 'abs error': ErrorTuple(forward=4.447253574184286e-06, reverse=None, forward_reverse=None), 'rel error': ErrorTuple(forward=4.6123802134984895e-07, reverse=None, forward_reverse=None), 'magnitude': MagnitudeTuple(forward=9.641988753864103, reverse=None, fd=9.641992568542056)}, ('con2', 'x'): {'J_fd': array([[0.09692786]]), 'J_fwd': array([[0.09692762]]), 'abs error': ErrorTuple(forward=2.38589368345421e-07, reverse=None, forward_reverse=None), 'rel error': ErrorTuple(forward=2.461514799873363e-06, reverse=None, forward_reverse=None), 'magnitude': MagnitudeTuple(forward=0.09692762402502489, reverse=None, fd=0.09692786261439323)}, ('con2', 'z'): {'J_fd': array([[1.94989099, 1.07754236]]), 'J_fwd': array([[1.94989072, 1.0775421 ]]), 'abs error': ErrorTuple(forward=3.7386098645400555e-07, reverse=None, forward_reverse=None), 'rel error': ErrorTuple(forward=1.6781488030575185e-07, reverse=None, forward_reverse=None), 'magnitude': MagnitudeTuple(forward=2.227817492024661, reverse=None, fd=2.2278178536542534)}}
Evaluate the total derivatives using multiple FD step sizes:
import openmdao.api as om
from openmdao.test_suite.components.sellar_feature import SellarDerivatives
prob = om.Problem()
prob.model = SellarDerivatives()
prob.model.nonlinear_solver = om.NonlinearBlockGS()
prob.model.linear_solver = om.ScipyKrylov()
prob.model.add_design_var('x', lower=-100, upper=100)
prob.model.add_design_var('z', lower=-100, upper=100)
prob.model.add_objective('obj')
prob.model.add_constraint('con1', upper=0.0)
prob.model.add_constraint('con2', upper=0.0)
prob.setup()
# We don't call run_driver() here because we don't
# actually want the optimizer to run
prob.run_model()
NL: NLBGS Converged in 8 iterations
# check derivatives of all obj+constraints w.r.t all design variables for FD steps of 1e-6 and 1e-7
prob.check_totals(step=[1.e7, 1.e8])
NL: NLBGS Converged in 2 iterations
NL: NLBGS Converged in 3 iterations
NL: NLBGS Converged in 5 iterations
NL: NLBGS Converged in 2 iterations
NL: NLBGS Converged in 3 iterations
NL: NLBGS Converged in 4 iterations
-----------------
Total Derivatives
-----------------
Full Model: 'con1' wrt 'x'
Forward Magnitude: 9.806145e-01
Fd Magnitude: 9.999369e-01 (fd:forward, step=10000000.0)
Fd Magnitude: 9.999800e-01 (fd:forward, step=100000000.0)
Absolute Error (Jfor - Jfd), step=10000000.0 : 1.932238e-02 *
Absolute Error (Jfor - Jfd), step=100000000.0 : 1.936553e-02 *
Relative Error (Jfor - Jfd) / Jfd, step=10000000.0 : 1.932360e-02 *
Relative Error (Jfor - Jfd) / Jfd, step=100000000.0 : 1.936592e-02 *
Raw Forward Derivative (Jfor)
[[-0.98061448]]
Raw FD Derivative (Jfd), step=10000000.0
[[-0.99993686]]
Raw FD Derivative (Jfd), step=100000000.0
[[-0.99998001]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Full Model: 'con1' wrt 'z'
Forward Magnitude: 9.641989e+00
Fd Magnitude: 1.000001e+07 (fd:forward, step=10000000.0)
Fd Magnitude: 1.000000e+08 (fd:forward, step=100000000.0)
Absolute Error (Jfor - Jfd), step=10000000.0 : 1.000000e+07 *
Absolute Error (Jfor - Jfd), step=100000000.0 : 1.000000e+08 *
Relative Error (Jfor - Jfd) / Jfd, step=10000000.0 : 9.999990e-01 *
Relative Error (Jfor - Jfd) / Jfd, step=100000000.0 : 9.999999e-01 *
Raw Forward Derivative (Jfor)
[[-9.61002186 -0.78449158]]
Raw FD Derivative (Jfd), step=10000000.0
[[-1.00000096e+07 -7.99943535e-01]]
Raw FD Derivative (Jfd), step=100000000.0
[[-1.00000010e+08 -7.99982122e-01]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Full Model: 'con2' wrt 'x'
Forward Magnitude: 9.692762e-02
Fd Magnitude: 3.157123e-04 (fd:forward, step=10000000.0)
Fd Magnitude: 9.994843e-05 (fd:forward, step=100000000.0)
Absolute Error (Jfor - Jfd), step=10000000.0 : 9.661191e-02 *
Absolute Error (Jfor - Jfd), step=100000000.0 : 9.682768e-02 *
Relative Error (Jfor - Jfd) / Jfd, step=10000000.0 : 3.060125e+02 *
Relative Error (Jfor - Jfd) / Jfd, step=100000000.0 : 9.687764e+02 *
Raw Forward Derivative (Jfor)
[[0.09692762]]
Raw FD Derivative (Jfd), step=10000000.0
[[0.00031571]]
Raw FD Derivative (Jfd), step=100000000.0
[[9.99484284e-05]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Full Model: 'con2' wrt 'z'
Forward Magnitude: 2.227817e+00
Fd Magnitude: 2.236194e+00 (fd:forward, step=10000000.0)
Fd Magnitude: 2.236108e+00 (fd:forward, step=100000000.0)
Absolute Error (Jfor - Jfd), step=10000000.0 : 9.208697e-02 *
Absolute Error (Jfor - Jfd), step=100000000.0 : 9.224891e-02 *
Relative Error (Jfor - Jfd) / Jfd, step=10000000.0 : 4.118022e-02 *
Relative Error (Jfor - Jfd) / Jfd, step=100000000.0 : 4.125423e-02 *
Raw Forward Derivative (Jfor)
[[1.94989072 1.0775421 ]]
Raw FD Derivative (Jfd), step=10000000.0
[[1.99999997 1.00028233]]
Raw FD Derivative (Jfd), step=100000000.0
[[2. 1.00008939]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Full Model: 'obj' wrt 'x'
Forward Magnitude: 2.980614e+00
Fd Magnitude: 1.000000e+07 (fd:forward, step=10000000.0)
Fd Magnitude: 1.000000e+08 (fd:forward, step=100000000.0)
Absolute Error (Jfor - Jfd), step=10000000.0 : 1.000000e+07 *
Absolute Error (Jfor - Jfd), step=100000000.0 : 1.000000e+08 *
Relative Error (Jfor - Jfd) / Jfd, step=10000000.0 : 9.999997e-01 *
Relative Error (Jfor - Jfd) / Jfd, step=100000000.0 : 1.000000e+00 *
Raw Forward Derivative (Jfor)
[[2.98061391]]
Raw FD Derivative (Jfd), step=10000000.0
[[10000002.99993685]]
Raw FD Derivative (Jfd), step=100000000.0
[[1.00000003e+08]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Full Model: 'obj' wrt 'z'
Forward Magnitude: 9.774287e+00
Fd Magnitude: 1.000001e+07 (fd:forward, step=10000000.0)
Fd Magnitude: 1.000000e+08 (fd:forward, step=100000000.0)
Absolute Error (Jfor - Jfd), step=10000000.0 : 1.000000e+07 *
Absolute Error (Jfor - Jfd), step=100000000.0 : 1.000000e+08 *
Relative Error (Jfor - Jfd) / Jfd, step=10000000.0 : 9.999990e-01 *
Relative Error (Jfor - Jfd) / Jfd, step=100000000.0 : 9.999999e-01 *
Raw Forward Derivative (Jfor)
[[9.61001056 1.78448534]]
Raw FD Derivative (Jfd), step=10000000.0
[[1.00000096e+07 1.79994353e+00]]
Raw FD Derivative (Jfd), step=100000000.0
[[1.00000010e+08 1.79998212e+00]]
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
{('obj',
'x'): {'J_fd': [array([[10000002.99993685]]),
array([[1.00000003e+08]])], 'steps': [10000000.0,
100000000.0], 'J_fwd': array([[2.98061391]]), 'abs error': [ErrorTuple(forward=10000000.019322941, reverse=None, forward_reverse=None),
ErrorTuple(forward=100000000.01936609, reverse=None, forward_reverse=None)], 'rel error': [ErrorTuple(forward=0.999999701938698, reverse=None, forward_reverse=None),
ErrorTuple(forward=0.9999999701938618, reverse=None, forward_reverse=None)], 'magnitude': [MagnitudeTuple(forward=2.9806139134842873, reverse=None, fd=10000002.999936854),
MagnitudeTuple(forward=2.9806139134842873, reverse=None, fd=100000002.99998)]},
('obj',
'z'): {'J_fd': [array([[1.00000096e+07, 1.79994353e+00]]),
array([[1.00000010e+08, 1.79998212e+00]])], 'steps': [10000000.0,
100000000.0], 'J_fwd': array([[9.61001056, 1.78448534]]), 'abs error': [ErrorTuple(forward=9999999.989989448, reverse=None, forward_reverse=None),
ErrorTuple(forward=99999999.98998943, reverse=None, forward_reverse=None)], 'rel error': [ErrorTuple(forward=0.9999990389998507, reverse=None, forward_reverse=None),
ErrorTuple(forward=0.9999999038999035, reverse=None, forward_reverse=None)], 'magnitude': [MagnitudeTuple(forward=9.774287228158459, reverse=None, fd=10000009.600000167),
MagnitudeTuple(forward=9.774287228158459, reverse=None, fd=100000009.60000001)]},
('con1', 'x'): {'J_fd': [array([[-0.99993686]]), array([[-0.99998001]])],
'steps': [10000000.0, 100000000.0],
'J_fwd': array([[-0.98061448]]),
'abs error': [ErrorTuple(forward=0.01932238034064737, reverse=None, forward_reverse=None),
ErrorTuple(forward=0.0193655349194225, reverse=None, forward_reverse=None)],
'rel error': [ErrorTuple(forward=0.019323600519051607, reverse=None, forward_reverse=None),
ErrorTuple(forward=0.01936592204198832, reverse=None, forward_reverse=None)],
'magnitude': [MagnitudeTuple(forward=0.980614475194995, reverse=None, fd=0.9999368555356424),
MagnitudeTuple(forward=0.980614475194995, reverse=None, fd=0.9999800101144175)]},
('con1',
'z'): {'J_fd': [array([[-1.00000096e+07, -7.99943535e-01]]),
array([[-1.00000010e+08, -7.99982122e-01]])], 'steps': [10000000.0,
100000000.0], 'J_fwd': array([[-9.61002186, -0.78449158]]), 'abs error': [ErrorTuple(forward=9999999.989978148, reverse=None, forward_reverse=None),
ErrorTuple(forward=99999999.98997813, reverse=None, forward_reverse=None)], 'rel error': [ErrorTuple(forward=0.9999990389987337, reverse=None, forward_reverse=None),
ErrorTuple(forward=0.9999999038997907, reverse=None, forward_reverse=None)], 'magnitude': [MagnitudeTuple(forward=9.641988753864103, reverse=None, fd=10000009.600000037),
MagnitudeTuple(forward=9.641988753864103, reverse=None, fd=100000009.6)]},
('con2', 'x'): {'J_fd': [array([[0.00031571]]), array([[9.99484284e-05]])],
'steps': [10000000.0, 100000000.0],
'J_fwd': array([[0.09692762]]),
'abs error': [ErrorTuple(forward=0.09661191168739808, reverse=None, forward_reverse=None),
ErrorTuple(forward=0.09682767559661139, reverse=None, forward_reverse=None)],
'rel error': [ErrorTuple(forward=306.01246822858707, reverse=None, forward_reverse=None),
ErrorTuple(forward=968.7763693094231, reverse=None, forward_reverse=None)],
'magnitude': [MagnitudeTuple(forward=0.09692762402502489, reverse=None, fd=0.00031571233762681307),
MagnitudeTuple(forward=0.09692762402502489, reverse=None, fd=9.994842841349801e-05)]},
('con2',
'z'): {'J_fd': [array([[1.99999997, 1.00028233]]),
array([[2. , 1.00008939]])], 'steps': [10000000.0,
100000000.0], 'J_fwd': array([[1.94989072, 1.0775421 ]]), 'abs error': [ErrorTuple(forward=0.09208697063065134, reverse=None, forward_reverse=None),
ErrorTuple(forward=0.09224891396139286, reverse=None, forward_reverse=None)], 'rel error': [ErrorTuple(forward=0.04118022013725983, reverse=None, forward_reverse=None),
ErrorTuple(forward=0.041254230957301075, reverse=None, forward_reverse=None)], 'magnitude': [MagnitudeTuple(forward=2.227817492024661, reverse=None, fd=2.2361942292613226),
MagnitudeTuple(forward=2.227817492024661, reverse=None, fd=2.236107953554443)]}}