DOEDriver

DOEDriver facilitates performing a design of experiments (DOE) with your OpenMDAO model. It will run your model multiple times with different values for the design variables depending on the selected input generator. A number of generators are available, each with its own parameters that can be specified when it is instantiated:

Note

FullFactorialGenerator, PlackettBurmanGenerator, BoxBehnkenGenerator and LatinHypercubeGenerator are provided via the pyDOE2 package, which is an updated version of pyDOE. See the original pyDOE page for information on those algorithms.

The generator instance may be supplied as an argument to the DOEDriver or as an option.

DOEDriver Options

Option Default Acceptable Values Acceptable Types Description
debug_print [] N/A [‘list’] List of what type of Driver variables to print at each iteration. Valid items in list are ‘desvars’, ‘ln_cons’, ‘nl_cons’, ‘objs’, ‘totals’
generator DOEGenerator N/A [‘DOEGenerator’] The case generator. If default, no cases are generated.
procs_per_model 1 N/A N/A Number of processors to give each model under MPI.
run_parallel False N/A N/A Set to True to execute cases in parallel.

Note: Options can be passed as keyword arguments at initialization.

Simple Example

UniformGenerator implements the simplest method and will generate a requested number of samples randomly selected from a uniform distribution across the valid range for each design variable. This example demonstrates its use with a model built on the Paraboloid Component. An SqliteRecorder is used to capture the cases that were generated. We can see that that the model was evaluated at random values of x and y between -10 and 10, per the lower and upper bounds of those design variables.

from openmdao.api import Problem, IndepVarComp
from openmdao.test_suite.components.paraboloid import Paraboloid

from openmdao.api import DOEDriver, UniformGenerator, SqliteRecorder, CaseReader

prob = Problem()
model = prob.model

model.add_subsystem('p1', IndepVarComp('x', 0.), promotes=['*'])
model.add_subsystem('p2', IndepVarComp('y', 0.), promotes=['*'])
model.add_subsystem('comp', Paraboloid(), promotes=['*'])

model.add_design_var('x', lower=-10, upper=10)
model.add_design_var('y', lower=-10, upper=10)
model.add_objective('f_xy')

prob.driver = DOEDriver(UniformGenerator(num_samples=5))
prob.driver.add_recorder(SqliteRecorder("cases.sql"))

prob.setup()
prob.run_driver()
prob.cleanup()

cases = CaseReader("cases.sql").driver_cases

print(cases.num_cases)
5
values = []
for n in range(cases.num_cases):
    outputs = cases.get_case(n).outputs
    values.append((outputs['x'], outputs['y'], outputs['f_xy']))

print("\n".join(["x: %5.2f, y: %5.2f, f_xy: %6.2f" % (x, y, f_xy) for x, y, f_xy in values]))
x:  0.98, y:  4.30, f_xy:  74.25
x:  2.06, y:  0.90, f_xy:  23.72
x: -1.53, y:  2.92, f_xy:  60.89
x: -1.25, y:  7.84, f_xy: 145.35
x:  9.27, y: -2.33, f_xy:  17.52

Running a DOE in Parallel

In a parallel processing environment, it is possible for DOEDriver to run cases concurrently. This is done by setting the run_parallel option to True as shown in the following example.

Here we are using the FullFactorialGenerator with 3 levels to generate inputs for our Paraboloid model. With two inputs, \(3^2=9\) cases have been generated. In this case we are running on two processors and have specified options['run_parallel']=True to run cases on all available processors. The cases have therefore been split with 5 cases run on the first processor and 4 cases on the second.

Note that, when running in parallel, the SqliteRecorder will generate a separate case file for each processor on which cases are recorded. The case files will have a suffix indicating the recording rank and a message will be displayed indicating the file name, as seen in the example.

from openmdao.api import Problem, IndepVarComp
from openmdao.test_suite.components.paraboloid import Paraboloid

from openmdao.api import DOEDriver, FullFactorialGenerator
from openmdao.api import SqliteRecorder, CaseReader

from mpi4py import MPI

prob = Problem()
model = prob.model

model.add_subsystem('p1', IndepVarComp('x', 0.0), promotes=['x'])
model.add_subsystem('p2', IndepVarComp('y', 0.0), promotes=['y'])
model.add_subsystem('comp', Paraboloid(), promotes=['x', 'y', 'f_xy'])

model.add_design_var('x', lower=0.0, upper=1.0)
model.add_design_var('y', lower=0.0, upper=1.0)
model.add_objective('f_xy')

prob.driver = DOEDriver(FullFactorialGenerator(levels=3))
prob.driver.options['run_parallel'] =  True
prob.driver.options['procs_per_model'] =  1

prob.driver.add_recorder(SqliteRecorder("cases.sql"))

prob.setup()
prob.run_driver()
(rank 0) Note: SqliteRecorder is running on multiple processors. Cases from rank 0 are being written to cases.sql_0.
(rank 1) Note: SqliteRecorder is running on multiple processors. Cases from rank 1 are being written to cases.sql_1.
prob.cleanup()

print(MPI.COMM_WORLD.size)
(rank 0) 2
(rank 1) 2
# check recorded cases from each case file
rank = MPI.COMM_WORLD.rank
filename = "cases.sql_%d" % rank
print(filename)
(rank 0) cases.sql_0
(rank 1) cases.sql_1
cases = CaseReader(filename).driver_cases
print(cases.num_cases)
(rank 0) 5
(rank 1) 4
values = []
for n in range(cases.num_cases):
    outputs = cases.get_case(n).outputs
    values.append((outputs['x'], outputs['y'], outputs['f_xy']))

print("\n"+"\n".join(["x: %5.2f, y: %5.2f, f_xy: %6.2f" % (x, y, f_xy) for x, y, f_xy in values]))
(rank 0) 
x:  0.00, y:  0.00, f_xy:  22.00
x:  1.00, y:  0.00, f_xy:  17.00
x:  0.50, y:  0.50, f_xy:  23.75
x:  0.00, y:  1.00, f_xy:  31.00
x:  1.00, y:  1.00, f_xy:  27.00
(rank 1) 
x:  0.50, y:  0.00, f_xy:  19.25
x:  0.00, y:  0.50, f_xy:  26.25
x:  1.00, y:  0.50, f_xy:  21.75
x:  0.50, y:  1.00, f_xy:  28.75

Running a DOE in Parallel with a Parallel Model

If the model that is being subjected to the DOE is also parallel, then the total number of processors should reflect the model size as well as the desired concurrency.

To illustrate this, we will demonstrate performing a DOE on a model based on the ParallelGroup example:

class FanInGrouped(Group):
    """
    Topology where two components in a Group feed a single component
    outside of that Group.
    """

    def __init__(self):
        super(FanInGrouped, self).__init__()

        iv = self.add_subsystem('iv', IndepVarComp())
        iv.add_output('x1', 1.0)
        iv.add_output('x2', 1.0)
        iv.add_output('x3', 1.0)

        self.sub = self.add_subsystem('sub', ParallelGroup())
        self.sub.add_subsystem('c1', ExecComp(['y=-2.0*x']))
        self.sub.add_subsystem('c2', ExecComp(['y=5.0*x']))

        self.add_subsystem('c3', ExecComp(['y=3.0*x1+7.0*x2']))

        self.connect("sub.c1.y", "c3.x1")
        self.connect("sub.c2.y", "c3.x2")

        self.connect("iv.x1", "sub.c1.x")
        self.connect("iv.x2", "sub.c2.x")

In this case, the model itself requires two processors, so in order to run cases concurrently we need to allocate at least four processors in total. We can allocate as many processors as we have available, however the number of processors must be a multiple of the number of processors per model, which is 2 here. Regardless of how many processors we allocate, we need to tell the DOEDriver that the model needs 2 processors, which is done by specifying options['procs_per_model']=2. From this, the driver figures out how many models it can run in parallel, which in this case is also 2.

The SqliteRecorder will record cases on the first two processors, which serve as the “root” processors for the parallel cases.

from openmdao.api import Problem
from openmdao.test_suite.groups.parallel_groups import FanInGrouped

from openmdao.api import DOEDriver, FullFactorialGenerator
from openmdao.api import SqliteRecorder, CaseReader

from mpi4py import MPI

prob = Problem(FanInGrouped())
model = prob.model

model.add_design_var('iv.x1', lower=0.0, upper=1.0)
model.add_design_var('iv.x2', lower=0.0, upper=1.0)

model.add_objective('c3.y')

prob.driver = DOEDriver(FullFactorialGenerator(levels=3))
prob.driver.add_recorder(SqliteRecorder("cases.sql"))
prob.driver.options['run_parallel'] =  True

# run 2 cases at a time, each using 2 of our 4 procs
doe_parallel = prob.driver.options['procs_per_model'] = 2

prob.setup()
prob.run_driver()
prob.cleanup()

rank = MPI.COMM_WORLD.rank

# check recorded cases from each case file
if rank < doe_parallel:
    filename = "cases.sql_%d" % rank

    cases = CaseReader(filename).driver_cases

    values = []
    for n in range(cases.num_cases):
        outputs = cases.get_case(n).outputs
        values.append((outputs['iv.x1'], outputs['iv.x2'], outputs['c3.y']))

    print("\n"+"\n".join(["iv.x1: %5.2f, iv.x2: %5.2f, c3.y: %6.2f" % (x1, x2, y) for x1, x2, y in values]))
Note: SqliteRecorder is running on multiple processors. Cases from rank 0 are being written to cases.sql_0.

iv.x1:  0.00, iv.x2:  0.00, c3.y:   0.00
iv.x1:  1.00, iv.x2:  0.00, c3.y:  -6.00
iv.x1:  0.50, iv.x2:  0.50, c3.y:  14.50
iv.x1:  0.00, iv.x2:  1.00, c3.y:  35.00
iv.x1:  1.00, iv.x2:  1.00, c3.y:  29.00

Note: SqliteRecorder is running on multiple processors. Cases from rank 1 are being written to cases.sql_1.

iv.x1:  0.50, iv.x2:  0.00, c3.y:  -3.00
iv.x1:  0.00, iv.x2:  0.50, c3.y:  17.50
iv.x1:  1.00, iv.x2:  0.50, c3.y:  11.50
iv.x1:  0.50, iv.x2:  1.00, c3.y:  32.00


Using Prepared Cases

If you have a previously generated set of cases that you want to run using DOEDriver, there are a couple of ways to do that. The first is to provide those inputs via an external file in the CSV (comma separated values) format. The file should be organized with one column per design variable, with the first row containing the names of the design variables. The following example demonstrates how to use such a file to run a DOE using the CSVGenerator:

from openmdao.api import Problem, IndepVarComp
from openmdao.test_suite.components.paraboloid import Paraboloid

from openmdao.api import DOEDriver, CSVGenerator, SqliteRecorder, CaseReader

prob = Problem()
model = prob.model

model.add_subsystem('p1', IndepVarComp('x', 0.0), promotes=['x'])
model.add_subsystem('p2', IndepVarComp('y', 0.0), promotes=['y'])
model.add_subsystem('comp', Paraboloid(), promotes=['x', 'y', 'f_xy'])

model.add_design_var('x', lower=0.0, upper=1.0)
model.add_design_var('y', lower=0.0, upper=1.0)
model.add_objective('f_xy')

prob.setup()

# this file contains design variable inputs in CSV format
with open('cases.csv', 'r') as f:
    print(f.read())
 x ,   y
0.0,  0.0
0.5,  0.0
1.0,  0.0
0.0,  0.5
0.5,  0.5
1.0,  0.5
0.0,  1.0
0.5,  1.0
1.0,  1.0
# run problem with DOEDriver using the CSV file
prob.driver = DOEDriver(CSVGenerator('cases.csv'))
prob.driver.add_recorder(SqliteRecorder("cases.sql"))

prob.run_driver()
prob.cleanup()

cases = CaseReader("cases.sql").driver_cases

values = []
for n in range(cases.num_cases):
    outputs = cases.get_case(n).outputs
    values.append((outputs['x'], outputs['y'], outputs['f_xy']))

print("\n".join(["x: %5.2f, y: %5.2f, f_xy: %6.2f" % (x, y, f_xy) for x, y, f_xy in values]))
x:  0.00, y:  0.00, f_xy:  22.00
x:  0.50, y:  0.00, f_xy:  19.25
x:  1.00, y:  0.00, f_xy:  17.00
x:  0.00, y:  0.50, f_xy:  26.25
x:  0.50, y:  0.50, f_xy:  23.75
x:  1.00, y:  0.50, f_xy:  21.75
x:  0.00, y:  1.00, f_xy:  31.00
x:  0.50, y:  1.00, f_xy:  28.75
x:  1.00, y:  1.00, f_xy:  27.00

The second method is to provide the data directly as a list of cases, where each case is a collection of name/value pairs for the design variables. You might use this method if you want to generate the cases programmatically via another algorithm or if the data is available in some format other than a CSV file and you can reformat it into this simple list structure. The DOEGenerator you would use in this case is the ListGenerator, but if you pass a list directly to the DOEDriver it will construct the ListGenerator for you. In the following example, a set of cases has been pre-generated and saved in JSON (JavaScript Object Notation) format. The data is decoded and provided to the DOEDriver as a list:

from openmdao.api import Problem, IndepVarComp
from openmdao.test_suite.components.paraboloid import Paraboloid

from openmdao.api import DOEDriver, ListGenerator, SqliteRecorder, CaseReader

import json

prob = Problem()
model = prob.model

model.add_subsystem('p1', IndepVarComp('x', 0.0), promotes=['x'])
model.add_subsystem('p2', IndepVarComp('y', 0.0), promotes=['y'])
model.add_subsystem('comp', Paraboloid(), promotes=['x', 'y', 'f_xy'])

model.add_design_var('x', lower=0.0, upper=1.0)
model.add_design_var('y', lower=0.0, upper=1.0)
model.add_objective('f_xy')

prob.setup()

# load design variable inputs from JSON file and decode into list
with open('cases.json', 'r') as f:
    json_data = f.read()

print(json_data)
[[["x", [0.0]], ["y", [0.0]]],
 [["x", [0.5]], ["y", [0.0]]],
 [["x", [1.0]], ["y", [0.0]]],
 [["x", [0.0]], ["y", [0.5]]],
 [["x", [0.5]], ["y", [0.5]]],
 [["x", [1.0]], ["y", [0.5]]],
 [["x", [0.0]], ["y", [1.0]]],
 [["x", [0.5]], ["y", [1.0]]],
 [["x", [1.0]], ["y", [1.0]]]]
case_list = json.loads(json_data)

print(case_list)
[[['x', [0.0]], ['y', [0.0]]], [['x', [0.5]], ['y', [0.0]]], [['x', [1.0]], ['y', [0.0]]], [['x', [0.0]], ['y', [0.5]]], [['x', [0.5]], ['y', [0.5]]], [['x', [1.0]], ['y', [0.5]]], [['x', [0.0]], ['y', [1.0]]], [['x', [0.5]], ['y', [1.0]]], [['x', [1.0]], ['y', [1.0]]]]
# create DOEDriver using provided list of cases
prob.driver = DOEDriver(case_list)

# a ListGenerator was created
print(type(prob.driver.options['generator']))
<class 'openmdao.drivers.doe_generators.ListGenerator'>
prob.driver.add_recorder(SqliteRecorder("cases.sql"))

prob.run_driver()
prob.cleanup()

cases = CaseReader("cases.sql").driver_cases

values = []
for n in range(cases.num_cases):
    outputs = cases.get_case(n).outputs
    values.append((outputs['x'], outputs['y'], outputs['f_xy']))

print("\n".join(["x: %5.2f, y: %5.2f, f_xy: %6.2f" % (x, y, f_xy) for x, y, f_xy in values]))
x:  0.00, y:  0.00, f_xy:  22.00
x:  0.50, y:  0.00, f_xy:  19.25
x:  1.00, y:  0.00, f_xy:  17.00
x:  0.00, y:  0.50, f_xy:  26.25
x:  0.50, y:  0.50, f_xy:  23.75
x:  1.00, y:  0.50, f_xy:  21.75
x:  0.00, y:  1.00, f_xy:  31.00
x:  0.50, y:  1.00, f_xy:  28.75
x:  1.00, y:  1.00, f_xy:  27.00

Warning

When using pre-generated cases via CSVGenerator or ListGenerator, there is no enforcement of the declared bounds on a design variable as with the algorithmic generators.

Tags

Driver, DOE