.. index:: derivatives
.. _`Adding-Derivatives-to-Your-Components`:
Adding Derivatives to Your Components
======================================
Many optimization algorithms make use of gradients. In our simple example problem, the SLSQP driver estimates
the gradient at various times during the solution procedure by performing a
local finite-difference step. Calculating the gradient typically involves one or
more executions of the objective function depending on the finite difference
method that is used. This, of course, means that your model is executed
additional times each iteration.
Sometimes, the solution process can be sped up by having a component supply its own
derivatives. These derivatives may be analytical (as you will see in this example),
or they might be estimated by some other means. Additionally, these derivatives can
be more accurate than those estimated by finite differencing the component, and
they are not dependent on the right choice of a step-size parameter.
.. index:: Finite Difference with Analytical Derivatives (FFAD)
In OpenMDAO, derivatives can be specified in the component API. A component's
set of specified derivatives is used to replace that component's output
with the first-order Taylor series expansion whenever the optimizer initiates
a finite difference estimation of the gradient. This is called *Finite
Difference with Analytical Derivatives* (FFAD). It provides an efficient
way of calculating gradients for mixed models -- models with components
that can provide derivatives and those that cannot. Via FFAD you can specify gradients (first
derivatives) and Hessians (second derivatives) in mixed models. Not all optimizers
will use gradients or Hessians, so they only get calculated if requested by an optimizer.
Four steps are involved in specifying derivatives for a component:
::
1. Inherit from ComponentWithDerivatives instead of Component
2. Declare derivatives in the "__init__" method
3. Calculate the first derivatives in the "calculate_first_derivatives" method
4. Calculate the second derivatives in the "calculate_second_derivatives" method (if needed)
You must declare the derivatives that you want to define so that the component can
be checked for missing derivatives. In declaration, you aren't defining a value
but just declaring that this derivative is provided by the component. In
the general case, you need to have derivatives for all possible permutations
between the inputs and outputs of your component. However, during any specific
optimization, you only need the derivatives for inputs that are connected to
upstream components and outputs that pass info to downstream components. This set
can be reduced further when you consider that you only need the inputs and outputs
that are active in the loop between the optimizer's parameters and its objective and
constraints. Derivatives are valid only for the `Float` variable type.
Derivative declaration is guided by the *sparse matrix* policy: if you don't
declare a derivative, it is assumed to be zero. You don't have to actively
set it to zero, and there is no superfluous multiplication by zero in any part
of the calculation. This philosophy leads to a clean interface and efficient
calculation, but the burden is on the component developer to make sure not
to miss a declaration of a derivative for an important output pair.
You can add analytical derivatives to the Paraboloid component by following
the steps mentioned above. First, we must inherit from ComponentWithDerivatives.
.. testcode:: Paraboloid_derivative
from openmdao.main.api import ComponentWithDerivatives
from openmdao.lib.datatypes.api import Float
class ParaboloidDerivative(ComponentWithDerivatives):
""" Evaluates the equation f(x,y) = (x-3)^2 + xy + (y+4)^2 - 3 """
# set up interface to the framework
# pylint: disable-msg=E1101
x = Float(0.0, iotype='in', desc='The variable x')
y = Float(0.0, iotype='in', desc='The variable y')
f_xy = Float(iotype='out', desc='F(x,y)')
.. testcode:: Paraboloid_derivative
:hide:
from openmdao.examples.simple.paraboloid import Paraboloid
self = Paraboloid()
The ``__init__`` method is a function that every class calls when it is instantiated.
We need to add an ``__init__`` method that defines derivatives between the inputs
(`x, y`) and the output ``f_xy``. Let's add both first and second derivatives.
.. testcode:: Paraboloid_derivative
def __init__(self):
""" declare what derivatives that we can provide"""
super(Paraboloid_Derivative, self).__init__()
self.derivatives.declare_first_derivative('f_xy', 'x')
self.derivatives.declare_first_derivative('f_xy', 'y')
self.derivatives.declare_second_derivative('f_xy', 'x', 'x')
self.derivatives.declare_second_derivative('f_xy', 'x', 'y')
self.derivatives.declare_second_derivative('f_xy', 'y', 'y')
The *super* command executes the parent's ``__init__`` function. **This is
required for the component to behave properly in OpenMDAO, so don't forget to
include it.**
Also, don't forget the cross-variable terms when declaring second derivatives
(in this case, the second derivative of ``f_xy`` with respect to `x` **and** `y`.)
Next, we define the ``calculate_first_derivatives`` and the
``calculate_second_derivatives`` methods.
.. testcode:: Paraboloid_derivative
def calculate_first_derivatives(self):
"""Analytical first derivatives"""
df_dx = 2.0*self.x - 6.0 + self.y
df_dy = 2.0*self.y + 8.0 + self.x
self.derivatives.set_first_derivative('f_xy', 'x', df_dx)
self.derivatives.set_first_derivative('f_xy', 'y', df_dy)
def calculate_second_derivatives(self):
"""Analytical second derivatives"""
df_dxdx = 2.0
df_dxdy = 1.0
df_dydy = 2.0
self.derivatives.set_second_derivative('f_xy', 'x', 'x', df_dxdx)
self.derivatives.set_second_derivative('f_xy', 'x', 'y', df_dxdy)
self.derivatives.set_second_derivative('f_xy', 'y', 'y', df_dydy)
The Hessian matrix is symmetric, so ``df/dxdy`` is the same as
``df/dydx``, and only one of these has to be set.
Note that no changes are required to the OptimizationConstrained or
OptimizationUnconstrained assembly at this point. If the driver uses
gradients (or Hessians) and can take advantage of the analytical ones
you provide, then it will do so. Below is our model, using the new
component with derivatives. We put this model in a file called
:download:`optimization_constrained_derivative.py <../../examples/openmdao.examples.simple/openmdao/examples/simple/optimization_constrained_derivative.py>`.
.. literalinclude:: ../../examples/openmdao.examples.simple/openmdao/examples/simple/optimization_constrained_derivative.py
*Benchmarking*
~~~~~~~~~~~~~~
Sometimes it is useful to know how many times your component executes and
how many times it calculates its derivatives. OpenMDAO provides this
information for every component through a pair of counters: ``exec_count``
is incremented whenever a component executes, and ``derivative_exec_count``
is incremented whenever the derivatives are calculated. The following
example shows how they can be accessed and used.
.. doctest::
:options: +SKIP
>>> # Paraboloid Model
>>>
>>> from openmdao.examples.simple.optimization_constrained import OptimizationConstrained
>>> model = OptimizationConstrained()
>>> model.run()
>>> print model.paraboloid.exec_count
29
>>> print model.paraboloid.derivative_exec_count
0
>>> # Paraboloid Model with analytical derivatives
>>>
>>> from openmdao.examples.simple.optimization_constrained_derivative import OptimizationConstrained
>>> model = OptimizationConstrained()
>>> model.run()
>>> print model.paraboloid.exec_count
17
>>> print model.paraboloid.derivative_exec_count
6
Here, we've printed out the number of function and derivative executions for
the paraboloid examples, both without and with analytical derivatives.
Because this model is a simple equation, the advantage of using the
analytical derivative isn't evident in a comparison of the clock time, but
the number of functional executions is much lower when you have them, at a
cost of a small number of derivative evaluations.
This concludes an introduction to OpenMDAO using a simple problem of component creation and
execution. The next tutorial introduces a problem with more complexity and presents additional
features of the framework.