Generally, every sub-package in the OpenMDAO hierarchy has a
tests directory with test modules for the various features of that package. If your change impacts one of those features, then you should add your test(s) to the relevant module. Optionally, you can create a new test module under the appropriate sub-package.
While you can use
python -m unittest to run many of the OpenMDAO unit tests, it is recommended that you use testflo, which is a testing framework that was designed specifically to facilitate OpenMDAO testing.
Per the README,
testflo is a python testing framework that uses a pipeline of iterators to process test specifications, run the tests, and process the results. Some OpenMDAO features require execution under MPI while some others don’t, so
testflo has the basic functionality of other test frameworks but with the additional
ability to run MPI unit tests using
unittest.TestCase in the same way as all of our other tests.
In addition to MPI unit testing, there are a number of other convenient features that
testflo provides over and above other testing frameworks. Some of these are:
can be given a directory, a file, a module path, file:testcase.method, module:testcase.method, or a file containing a list of any of these
pre_announce option to print test name before running in order to quickly identify hanging MPI tests
end of testing summary including failed tests and optionally skipped tests, test coverage, all tests that run for longer than a certain time threshold, and tests that generate deprecation warnings
You can install testflo with several other optional testing dependencies by appending
[test] to the pip install target. For example:
pip install openmdao[test] or
pip install -e .[test]
openmdao.test_suite package provides a number of test models, scripts and utilities that can be useful when writing unit tests. Here you will find multiple variations on the simple Paraboloid and Sellar models, for example, as well as more complex examples like a circuit analysis and a multipoint beam optimization. Using components and models from the test suite can expedite the task of writing new tests and reduce the occurrence of repetitive code in unit tests. When creating new tests, consider whether there is an existing component or model in the test suite that can be used to exercise the feature under test, and if not then adding one for others to use.
openmdao.utils package provides a couple of modules to assist with common needs of unit testing.
testing_utils module has decorators and context managers to control if and how a test is run. Here are a few examples:
@use_tempdirsdecorator will run test functions in a temporary directory, which is useful when a test generates outputs files that need to be cleaned up or can cause collisions with other tests.
pyOptSparse is an optional dependency for OpenMDAO, but it provides some of the most powerful optimizers (IPOPT and SNOPT in particular). If your test makes use of one of these, you should use the
@require_pyoptsparsedecorator so that the test is skipped for users that have not installed
pyOptSparseand the specified optimizer.
For tests that depend on environment variables, the
set_env_varscontext manager is provided to temporarily modify the environment to suit a specific test.
check_partialsfunction will not check the partial derivatives of Components that are part of OpenMDAO itself. The
force_check_partialsfunction provided here will override that behavior if needed in a particular test.
assert_utils module provides helper functions and context managers that are useful for asserting the outcome of tests. Some of these are:
assert_near_equalis a convenient function for comparing values, including a variety of supported data structures, to the values expected by a test. It is more flexible and can be used in place of various other assert functions provided in the Python
assert_no_dict_jacobiansare beneficial for checking that derivatives are calculated correctly for the model under test.
assert_no_warningcontext managers can be used to check that warnings are issued (or not) as expected.