Reports System#

Default Usage#

OpenMDAO has a reports system which will generate reports when you run your model. To see a list of the reports that are run by default, along with a brief description of each, use the command:

openmdao list_reports -d

Running the same command without the -d will list all reports that are available.

Here is a table with more info about these default commands.

Report Name

Description

When Run

Doc link

inputs

Interactive table of model inputs

after Problem.final_setup

n2

Interactive, graphical view of the model

after Problem.final_setup

N2 diagram

optimizer

Summary of results of an optimization run

after Drive._post_run

optimization report

scaling

Information on design variables, objectives, and constraints

after Driver._compute_totals

scaling report

total_coloring

Metadata associated with the creation of the coloring

after Driver._get_coloring

Note that the reports are only run the first time the method is called on a given instance. For example, the scaling report is only run after the first call to Driver._compute_totals.

These reports will, by default, be put into a subfolder of reports named after the Problem name. For example, if your model has a single Problem and its default name, problem1, was not changed by the user, the reports will be placed in a directory named reports/problem1.

The user has control over the reports system by setting some environment variables which are described below.

Controlling which reports are generated#

There are two ways to control which reports are generated on which models.

Controlling reporting using an environment variable#

To turn off OpenMDAO’s default report generation, the user should set the OPENMDAO_REPORTS environment variable to any one of these five case-insensitive values: ‘0’, ‘false’, ‘no’, ‘off’, or ‘none’.

The user can also have more fine-grained control over what reports are generated by setting this environment variable to a string that lists the reports to run. For example

  1. OPENMDAO_REPORTS=n2

  2. OPENMDAO_REPORTS=n2,scaling

  3. OPENMDAO_REPORTS=none

If OPENMDAO_REPORTS is unset, the default reports will be generated.

The user can also explicitly turn on all available reports by setting OPENMDAO_REPORTS to ‘all’.

Controlling reporting using the Problem reports argument#

The user can also control what reports are generated using the reports argument to Problem.

If reports is the default of _UNDEFINED or True, the OPENMDAO_REPORTS environment variable determines reporting.

If the value for reports is None or False, it will disable all reports on that Problem.

Otherwise, reports may be a list of names or a single comma-delimited string of the names of the reports to run for that Problem.

Overall, OPENMDAO_REPORTS represents the global default list of reports, and individual Problems can either use that default list or override it by specifying their own list explicitly.

Setting the directory for the reports#

As of OpenMDAO 3.35.0, reports are placed within a “reports” directory under the problem’s output directory. By default, this directory is f'{problem_name}_out' in the current working directory.

These directories are separated to ensure that data from multiple OpenMDAO executions does not collide. A side-effect of this is that a users disk space can become cluttered with output files over time. There are two ways to address this:

  1. The user has the option of setting a common output directory for ALL problems using the OPENMDAO_WORKDIR environment variable. Thus there is a single directory that contains all OpenMDAO output data.

  2. The openmdao clean or openmdao.api.clean_outputs function can be used to remove these directories.

As an alternative to setting OPENMDAO_WORKDIR to set the top level output directory for all problems, a user can set work_dir as an option of Problem in order to set the top level output directory under which that problem’s '{problem_name}_out' directory will be placed. If provided, the value of the work_dir option takes precedence.

Support for subproblems#

The reporting system supports subproblems. For example, if a model has two Problems and they have the names problem1 and problem2, the reporting system will put the reports into the subfolders, problem1 and problem2 under the top level reports directory, which defaults to ./reports or is specified by the value of OPENMDAO_REPORTS_DIR.

Adding user defined reports#

The user can register a user defined report using the register_report function.

openmdao.utils.reports_system.register_report(name, func, desc, class_name, method, pre_or_post, inst_id=None, predicate=None, **kwargs)[source]

Register a report with the reporting system.

Parameters:
namestr

Name of report. Report names must be unique across all reports.

funcfunction

A function to do the reporting. Expects the first argument to be an instance of class_name.

descstr

A description of the report.

class_namestr

The name of the class owning the method where the report will be run.

methodstr

In which method of class_name should this be run.

pre_or_poststr

Valid values are ‘pre’ and ‘post’. Indicates when to run the report in the method.

inst_idstr or None

Either the instance ID of an OpenMDAO object (e.g. Problem, Driver) or None. If None, then this report will be run for all objects of type class_name.

predicatefunction or None

If not None, this function will be called to determine if the report’s hook function should run. The predicate function should take the class instance as its only argument and return True if the report should run. Note that returning False does not disable the hook, it just prevents the hook from running at that time.

**kwargsdict

Keyword args passed to the report function.

Returns:
Report

The registered report object.

Here is a script to show how this is done. Note that the function that is called to generate the actual report must have as its first argument the OpenMDAO object instance that owns the method where the report will be run. Currently, the only option is a Problem or Driver instance. The full file path of the report will be determined based on OPENMDAO_REPORTS_DIR and the name of the enclosing Problem instance. The directory where the report will be written can be obtained by calling the get_reports_dir method on the instance passed into the reporting function.

Note that register_report only adds a report type to the global reports registry. In order to activate the report so that it actually generates output, it must appear in either OPENMDAO_REPORTS or must be passed in the Problem reports arg as shown below.

After this script is run, there will be a file named user_report.txt in the directory problem1.

import pathlib
import openmdao.api as om
from openmdao.test_suite.components.paraboloid import Paraboloid

user_report_filename = 'user_report.txt'

def user_defined_report(prob):
    path = pathlib.Path(prob.get_reports_dir()).joinpath(user_report_filename)
    with open(path, "w") as f:
        f.write(f"Do some reporting on the Problem, {prob._name}\n")

# Run the report before the call to Problem.setup
om.register_report("user_report", user_defined_report, "This report is user defined",
                   'Problem', 'setup', 'post')

prob = om.Problem(reports='user_report')
model = prob.model

model.add_subsystem('p1', om.IndepVarComp('x', 0.0), promotes=['x'])
model.add_subsystem('p2', om.IndepVarComp('y', 0.0), promotes=['y'])
model.add_subsystem('comp', Paraboloid(), promotes=['x', 'y', 'f_xy'])

model.add_design_var('x', lower=0.0, upper=1.0)
model.add_design_var('y', lower=0.0, upper=1.0)
model.add_objective('f_xy')

prob.driver = om.ScipyOptimizeDriver()

prob.setup()
prob.run_driver()
prob.cleanup()

Optimization terminated successfully    (Exit mode 0)
            Current function value: 17.000000000000348
            Iterations: 2
            Function evaluations: 2
            Gradient evaluations: 2
Optimization Complete
-----------------------------------

There are some advanced cases where it may be necessary to register more then one hook for a report. This happens currently with the ‘n2’ report, which may run after setup() if there are errors before or during setup, or after final_setup() if there are no errors. In cases where multiple hooks are necessary, multiple calls to register_report_hook should be used instead of a single call to register_report. The signature for register_report_hook is:

openmdao.utils.reports_system.register_report_hook(name, fname, class_name, inst_id=None, pre=None, post=None, description='', **kwargs)[source]

Register a hook with a specific report name in the reporting system.

By calling this multiple times, multiple hooks can be registered for the same report. This is sometimes necessary to get the correct behavior when setup errors occur prior to report creation.

Parameters:
namestr

Name of report. Report names must be unique across all reports.

fnamestr

The name of the function where the pre and/or post hook will be applied.

class_namestr

The name of the class owning the method where the hook will be applied.

inst_idstr or None

The name of the instance owning the method where the hook will be applied.

prefunction (None)

If not None, this hook will run before the function named by fname runs.

postfunction (None)

If not None, this hook will run after the function named by fname runs.

descriptionstr

A description of the report.

**kwargsdict of keyword arguments

Keyword arguments that will be passed to the hook function.

Creating a report plugin#

What if you’ve developed an OpenMDAO-related python package and would like anyone using that package to have access to a new report type that you’ve defined within that package? You can do this by creating an openmdao_report plugin using the steps outlined below. This example will use the existing summary report plugin as an example.

  1. Create the function that generates your report. For example:

    def _summary_report(prob):
    path = str(pathlib.Path(prob.get_reports_dir()).joinpath('summary.html'))
    s = StringIO()
    config_summary(prob, s)
    with open(path, 'w') as f:
        f.write(text2html(s.getvalue()))

    Note that your report function must take as an argument an instance of the type that contains the method where your report is registered to run. For example, in the register_report call below, it refers to the final_setup method of Problem, so our instance in this case must be an instance of Problem. Also, note that the function calls prob.get_reports_dir() to get the location of the reports directory where the report shouild be written. In addition, the string generated from calling config_summary is plain text, so it is wrapped in html using the text2html function to make it easily viewable in a browser, and therefore accessible using the openmdao view_reports command.

  2. Create a function that will register your new report type. For example:

    def _summary_report_register():
    register_report('summary', _summary_report, 'Model summary', 'Problem', 'final_setup', 'post')

    The call above will result in a report type called summary that triggers at the end of Problem.final_setup.

  3. Create an entry point in your package’s setup.py file. In this example, our report registry function is found in the openmdao.devtools.debug.py file, so we’ll update the entry_point arg in the call to setup in our setup.py file as follows:

        entry_points = {
        
        # other entry point groups here...
        
        'openmdao_report': [
            
            # other report entry points here...
            
            'summary_report=openmdao.devtools.debug:_summary_report_register',
        ],
    }

    As shown above, report entry points are part of the openmdao_report entry point group.

    Note that the name for your entry point needs to be unique. Here, we use the name “summary_report” because an entry point named “summary” already exists for one of the built-in reports.

Viewing existing reports#

As mentioned earlier, the openmdao view_reports command provides an easy way to view all of the reports that have been generated, either for an entire script or just for a specific Problem instance. To view only reports for a specific problem, use

openmdao view_reports probname

where probname is the name of the problem of interest.

Listing available reports#

As mentioned before, the user can list available reports using the openmdao list_reports command.

It can also be done programmatically, using the list_reports function.

openmdao.utils.reports_system.list_reports(default=False, outfile=None, max_width=80)[source]

Write table of information about reports currently registered in the reporting system.

Parameters:
defaultbool

If True, list only the default reports.

outfilestr or None

Where to send report info. None will result in output to stdout.

max_widthint

Maximum width of the table. Defaults to 80.

om.list_reports()

╔════════════════╤════════════════╤════════════╤═════════════════╤═════════════╗
║ name           ┊ description    ┊ class name ┊ method          ┊ pre or post ║
╠════════════════╪════════════════╪════════════╪═════════════════╪═════════════╣
║ checks         ┊ Config checks  ┊ Problem    ┊ final_setup     ┊ post        ║
╟┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┈╢
║ connections    ┊ Connections    ┊ Problem    ┊ final_setup     ┊ post        ║
║                ┊ viewer         ┊            ┊                 ┊             ║
╟┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┈╢
║ inputs         ┊ Inputs report  ┊ Problem    ┊ final_setup     ┊ post        ║
╟┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┈╢
║ n2             ┊ N2 diagram     ┊ Problem    ┊ final_setup     ┊ post        ║
╟┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┈╢
║ optimizer      ┊ Summary of     ┊ Problem    ┊ run_driver      ┊ post        ║
║                ┊ optimization   ┊            ┊                 ┊             ║
╟┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┈╢
║ scaling        ┊ Driver scaling ┊ Driver     ┊ _compute_totals ┊ post        ║
║                ┊ report         ┊            ┊                 ┊             ║
╟┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┈╢
║ summary        ┊ Model summary  ┊ Problem    ┊ final_setup     ┊ post        ║
╟┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┿┈┈┈┈┈┈┈┈┈┈┈┈┈╢
║ total_coloring ┊ Total coloring ┊ Driver     ┊ _get_coloring   ┊ post        ║
╚════════════════╧════════════════╧════════════╧═════════════════╧═════════════╝