OpenMDAO `Problem` Functional Interface#

The functional interface lets you wrap an assembled OpenMDAO Problem in a Python callable so that external tools — optimizers, surrogate trainers, uncertainty-quantification frameworks, etc. — can drive it using ordinary NumPy arrays instead of the OpenMDAO variable-name API.

The entry point is Problem.get_callback(), which returns a _FunctionalCallback object. Calling that object

takes a flat input vector as an argument and writes it into the problem,
calls Problem.run_model(), and
returns outputs and/or total derivatives as NumPy arrays, depending on which form was requested.

The three forms#

`form`	Signature	Returns
`'f'`	`cb(x[, y])`	flat output vector `y`
`'dfdx'`	`cb(x[, J])`	Jacobian `J` of shape `(n_outputs, n_inputs)`
`'fdfdx'`	`cb(x[, y[, J]])`	tuple `(y, J)`

All three forms accept pre-allocated output arrays as optional arguments to avoid repeated memory allocation.

Specifying variables#

The input_vars and output_vars arguments to get_callback() control which model variables are included as inputs and outputs of the callback function.

Each entry in either list may be:

a plain string — the variable name; or
a dict mapping an alias to a metadata dict with optional keys:
- 'name' — actual variable name when the alias differs;
- 'indices' — a subset of elements, in any format accepted by Problem.get_val();
- 'units' — the units in which the variable should be expressed. For form='f' this controls the units used when reading back output values. For the derivative forms ('dfdx' and 'fdfdx') this scales the corresponding rows (output variable) or columns (input variable) of the Jacobian so that derivatives are expressed in the requested units.

For the derivative forms ('dfdx' and 'fdfdx'), omitting input_vars causes the callback to use the driver’s registered design variables, and omitting output_vars causes it to use the driver’s registered responses (objectives + constraints).

For form='f', both arguments are required.

Variables are packed into the flat vector in the order they appear in the list, with multi-element variables occupying contiguous slices.

Helper methods on the callback object#

Method	Description
`create_input_vector()`	Allocate a flat input array pre-filled with the current problem values.
`create_output_vector()`	Allocate a flat output array pre-filled with the current problem values.
`create_jacobian_matrix()`	Allocate a zero Jacobian of shape `(n_outputs, n_inputs)`.
`get_input_val(name)`	Return the current flat value of one registered input variable.
`get_output_val(name)`	Return the current flat value of one registered output variable.
`input_var_names`	List of registered input variable names (or aliases).
`output_var_names`	List of registered output variable names (or aliases).
`form`	The form string this callback was created with.

Example 1: evaluating outputs (`form='f'`)#

This example uses everyone’s favorite Paraboloid component, \(f(x, y) = (x-3)^2 + xy + (y+4)^2 - 3\), to show how to evaluate model outputs through the functional interface. In this example the OpenMDAO Model will consist of just this one Component, but the Model could be arbitrarily complex.

Both input_vars and output_vars must be supplied when using form='f'. The callback accepts a flat vector whose elements correspond to the variables listed in input_vars in order, and returns a flat vector whose elements correspond to output_vars in order.

import numpy as np
import openmdao.api as om
from openmdao.test_suite.components.paraboloid import Paraboloid

prob = om.Problem()
prob.model.add_subsystem('comp', Paraboloid(),
                         promotes_inputs=['x', 'y'],
                         promotes_outputs=['f_xy'])
prob.setup()
prob.final_setup()

# Create a callback that maps [x, y] -> [f_xy].
f = prob.get_callback('f', input_vars=['x', 'y'], output_vars=['f_xy'])

print('input variable names :', f.input_var_names)
print('output variable names:', f.output_var_names)

# create_input_vector() returns a 1-D array pre-filled with the current
# problem values.  Modify it in-place before each call.
x = f.create_input_vector()
x[0] = 3.0   # x
x[1] = -4.0  # y

y = f(x)
print('f(3, -4) =', y[0])

x[0] = 6.6667
x[1] = -7.3333
print('f(6.6667, -7.3333) =', f(x)[0])

input variable names : ['x', 'y']
output variable names: ['f_xy']
f(3, -4) = -15.0
f(6.6667, -7.3333) = -27.333333330000006

Passing a pre-allocated output array#

Pass a pre-allocated array as y to avoid a memory allocation on every call. Use create_output_vector() to get a correctly-sized array.

y_buf = f.create_output_vector()  # allocate once

x[0], x[1] = 9.0, 10.0
f(x, y=y_buf)  # result written into y_buf; return value is also y_buf
print('f(9, 10) =', y_buf[0])

f(9, 10) = 319.0

Holding some inputs fixed#

List only the variables you want to vary in input_vars. Any variable omitted from the list keeps whatever value is currently stored in the problem. Set that fixed value with Problem.set_val() before creating the callback.

prob.set_val('y', -4.0)  # fix y; only x will be varied

f_x_only = prob.get_callback('f', input_vars=['x'], output_vars=['f_xy'])

x1 = f_x_only.create_input_vector()  # length 1: only x
x1[0] = 3.0
print('f(x=3, y=-4) =', f_x_only(x1)[0])

f(x=3, y=-4) = -15.0

Example 2: total derivatives (`form='dfdx'`)#

Use form='dfdx' to obtain the Jacobian \(\partial f / \partial x\). The callback returns a 2-D array of shape (n_outputs, n_inputs).

When input_vars and output_vars are both omitted the callback automatically uses the driver’s registered design variables as inputs and its registered responses (objectives + constraints) as outputs.

prob2 = om.Problem()
prob2.model.add_subsystem('comp', Paraboloid(),
                          promotes_inputs=['x', 'y'],
                          promotes_outputs=['f_xy'])
prob2.model.add_design_var('x', lower=-50, upper=50)
prob2.model.add_design_var('y', lower=-50, upper=50)
prob2.model.add_objective('f_xy')
prob2.driver = om.ScipyOptimizeDriver()
prob2.setup()
prob2.final_setup()

# No input_vars / output_vars: falls back to driver design vars and responses.
dfdx = prob2.get_callback('dfdx')

print('input  vars:', dfdx.input_var_names)
print('output vars:', dfdx.output_var_names)

x = dfdx.create_input_vector()
x[0] = 1.5  # x
x[1] = 2.5  # y

J = dfdx(x)
print('J =', J)
print('shape:', J.shape)  # (1, 2): one output, two inputs

input  vars: ['x', 'y']
output vars: ['f_xy']
J = [[-0.5 14.5]]
shape: (1, 2)

Passing a pre-allocated Jacobian#

Use create_jacobian_matrix() to allocate a zero matrix of the right shape, then pass it as the J keyword argument to avoid re-allocation.

J_buf = dfdx.create_jacobian_matrix()  # shape (1, 2), all zeros

x[0], x[1] = 1.6, 2.6
dfdx(x, J=J_buf)
print('J =', J_buf)

J = [[-0.2 14.8]]

Example 3: outputs and derivatives together (`form='fdfdx'`)#

Use form='fdfdx' when you need both the function value and the Jacobian from the same model evaluation. The callback returns the tuple (y, J).

fdfdx = prob2.get_callback('fdfdx', input_vars=['x', 'y'],
                            output_vars=['f_xy'])

x = fdfdx.create_input_vector()
x[0] = 3.0
x[1] = -4.0

y, J = fdfdx(x)
print('f(3, -4)  =', y[0])
print('df/dx     =', J[0, 0])
print('df/dy     =', J[0, 1])

f(3, -4)  = -15.0
df/dx     = -4.0
df/dy     = 3.0

Pre-allocating both output arrays#

Pass pre-allocated y and J buffers together to eliminate all allocation.

y_buf = fdfdx.create_output_vector()
J_buf = fdfdx.create_jacobian_matrix()

x[0], x[1] = 6.6667, -7.3333
fdfdx(x, y=y_buf, J=J_buf)
print('f(6.6667, -7.3333) =', y_buf[0])
print('J =', J_buf)

f(6.6667, -7.3333) = -27.333333330000006
J = [[1.e-04 1.e-04]]

Example 4: selecting variable subsets with `indices`#

When a model variable is an array, you can select only a subset of its elements to be included as either an input or output to the functional interface by supplying an 'indices' key in the variable metadata dict. The indices may be an integer list (flat indexing) or a tuple of arrays (multi-dimensional indexing).

The example below uses a component with two scalar inputs x and y of shape (2,) and one output f of shape (2,). The callback is configured to expose only x[0] and f[0].

class QuadComp(om.ExplicitComponent):
    """f[0] = (x[0]+2)^2 + (x[1]+3)^2 - 4,  f[1] = 2*x[0] + 3*x[1]."""

    def setup(self):
        self.add_input('x', val=0.0, shape=(2,))
        self.add_output('f', val=0.0, shape=(2,))

    def setup_partials(self):
        self.declare_partials('*', '*', method='cs')

    def compute(self, inputs, outputs):
        x = inputs['x']
        outputs['f'][0] = (x[0]+2)**2 + (x[1]+3)**2 - 4
        outputs['f'][1] = 2*x[0] + 3*x[1]


prob3 = om.Problem()
prob3.model.add_subsystem('comp', QuadComp(),
                           promotes_inputs=['x'], promotes_outputs=['f'])
prob3.model.add_design_var('x', lower=-50, upper=50)
prob3.model.add_objective('f', index=0)
prob3.driver = om.ScipyOptimizeDriver()
prob3.setup(force_alloc_complex=True)
prob3.final_setup()
prob3.set_val('x', [-2.0, -3.0])
prob3.run_model()

# Expose only x[0] as input and f[0] as output.
fdfdx3 = prob3.get_callback(
    'fdfdx',
    input_vars=[{'x': {'indices': [0]}}],
    output_vars=[{'f': {'indices': [0]}}],
)

x3 = fdfdx3.create_input_vector()  # length 1
print('x3 =', x3)                  # reflects current prob value of x[0]

f3, J3 = fdfdx3(x3)
print('f[0]         =', f3[0])
print('df[0]/dx[0]  =', J3[0, 0])   # shape (1, 1)

x3 = [-2.]
f[0]         = -4.0
df[0]/dx[0]  = -0.0

Example 5: driver variables used automatically#

When a Problem has design variables and responses registered on the driver, calling get_callback('dfdx') (or 'fdfdx') without input_vars / output_vars automatically selects those driver variables. This is convenient after running an optimization: the callback immediately reflects the driver’s variable set including any index subsets specified on the constraint or objective.

The input_var_names and output_var_names properties contain which variables were selected.

prob4 = om.Problem()
prob4.model.add_subsystem('comp', QuadComp(),
                           promotes_inputs=['x'], promotes_outputs=['f'])
prob4.model.add_design_var('x', lower=-50, upper=50)
prob4.model.add_objective('f', index=0, alias='f_obj')
prob4.model.add_constraint('f', indices=[1], equals=13.0, alias='f_con')
prob4.driver = om.ScipyOptimizeDriver()
prob4.setup(force_alloc_complex=True)
prob4.set_val('x', [0.0, 0.0])
prob4.run_driver()

# Both input_vars and output_vars omitted: uses driver design vars + responses.
fdfdx4 = prob4.get_callback('fdfdx')
print('inputs :', fdfdx4.input_var_names)
print('outputs:', fdfdx4.output_var_names)

x4 = fdfdx4.create_input_vector()
print('x at optimum:', x4)   # pre-filled with the post-optimization values

f4, J4 = fdfdx4(x4)
print('f4 =', f4)
print('J4 =\n', J4)

Optimization terminated successfully    (Exit mode 0)
            Current function value: 47.99999999999999
            Iterations: 3
            Function evaluations: 4
            Gradient evaluations: 3
Optimization Complete
-----------------------------------
inputs : ['x']
outputs: ['f_obj', 'f_con']
x at optimum: [2. 3.]
f4 = [48. 13.]
J4 =
 [[ 8. 12.]
 [ 2.  3.]]

Inspecting current values after a call#

After calling the callback the problem is left in the state corresponding to the last x that was passed in. The helper methods get_input_val() and get_output_val() retrieve the current flat values of any registered variable without another model evaluation.

# Continue from Example 3.
x_in = fdfdx.create_input_vector()
x_in[0] = 3.0
x_in[1] = -4.0
fdfdx(x_in)

print('x from callback  :', fdfdx.get_input_val('x'))
print('y from callback  :', fdfdx.get_output_val('f_xy'))

x from callback  : [3.]
y from callback  : [-15.]

Example 6: unit conversion in Jacobians#

When model variables have physical units the Jacobian entries are expressed in those units. The functional interface lets you request different units for any input or output variable by adding a 'units' key to the variable metadata dict. The Jacobian is then automatically scaled so that derivatives are expressed in the requested units.

Consider a component that computes f = 2 * x where x is in metres and f is in Newtons. The native Jacobian is \(\partial f / \partial x = 2\ \text{N/m}\).

Requesting f in kN scales the output rows by \(10^{-3}\), giving \(0.002\ \text{kN/m}\).
Requesting x in km scales the input columns by \(10^{3}\), giving \(2000\ \text{N/km}\).
Requesting both gives \(2\ \text{kN/km}\) (the two scalings cancel).

Example 7: using `return_index_map` to locate variables in flat vectors#

The create_input_vector(), create_output_vector(), and create_jacobian_matrix() methods all accept a return_index_map keyword argument. When set to True, each method returns a two-element tuple; the second element is a dict that maps variable names to slice objects selecting each variable’s elements within the flat array.

This is useful when a callback covers several variables and you want to read or write a specific variable by name rather than by manually tracking index offsets.

For create_jacobian_matrix(return_index_map=True), the dict keys are (output_name, input_name) tuples and the values are (row_slice, col_slice) tuples that identify the corresponding sub-block of the Jacobian matrix.

Example 8: `AnalysisDriver`-like functionality#

OpenMDAO’s AnalysisDriver is an OpenMDAO Driver that runs an OpenMDAO Model for a range of user-specified inputs. The Problem functional interface described here provides a similar capability that might be preferable. We’ll recreate the Paraboloid example from the AnalysisDriver docs here with the functional interface.

The first step is to create the OpenMDAO Problem, just like usual:

With an AnalysisDriver we’re usually only interested in the outputs, not derivatives, so we’ll use the f version of the functional interface:

We can use the fancy case generating functionality from the AnalysisDriver docs with the functional interface. Here we’ll use the ProductGenerator:

Now we can create an input and output vector from the functional interface, and iterate over the cases like this:

Example 9: Optimization with `scipy.optimize`#

The functional interface can be used with external optimizers such as scipy.optimize.minimize. Using form='fdfdx' the callback returns both the objective value and its gradient in a single model evaluation, which is the signature expected by scipy.optimize.minimize when jac=True.

The example below solves the same unconstrained paraboloid minimization as the Simple Optimization example — finding the minimum of \(f(x,y) = (x-3)^2 + xy + (y+4)^2 - 3\) — but drives it with scipy.optimize.minimize rather than OpenMDAO’s built-in driver.

Limitations#

form='f' requires explicit variable lists: unlike the derivative forms, form='f' cannot fall back to driver variables because no _TotalJacInfo object is created. Both input_vars and output_vars must be provided.
run_model() on every call: each invocation of the callback calls Problem.run_model(). There is no caching; if you need to evaluate outputs and derivatives at the same point prefer form='fdfdx' over separate 'f' and 'dfdx' calls.

OpenMDAO Problem Functional Interface#