Validate a Model Post Run#
After executing a model, there may be an interest in checking the values of certain inputs or outputs. Although the values may be mathematically correct and converged, the inputs may have been set resulting in an output value that you do not want or that is not physical. If an input / output is found to have an undesired value, this may prompt you to change the value of an input, discretely change the structure and/or connections of the model, or raise a warning if a variable is close to an undesired bound. The validate method of a system in the model hierarchy can be overwritten and used to do validation of any inputs and / or outputs. The validate method on a system takes in the inputs / outputs in a read-only mode and can be overwritten to do whatever post-run checks are necessary in the system.
- System.validate(inputs, outputs, discrete_inputs=None, discrete_outputs=None)[source]
Check any final input / output values after a run.
The model is assumed to be in an unscaled state. An inherited component may choose to either override this function or ignore it. Any errors or warnings raised in this method will be collected and all printed / raised together.
- Parameters:
- inputsVector
Unscaled, dimensional input variables read via inputs[key].
- outputsVector
Unscaled, dimensional output variables read via outputs[key].
- discrete_inputsdict-like or None
If not None, dict-like object containing discrete input values.
- discrete_outputsdict-like or None
If not None, dict-like object containing discrete output values.
If we, for example, take the systems from the Sellar problem, validate methods can be added to the systems to check values after the model has converged:
SellarDis1 class definition
class SellarDis1(om.ExplicitComponent):
"""
Component containing Discipline 1 -- no derivatives version.
"""
def __init__(self, units=None, scaling=None):
super().__init__()
self.execution_count = 0
self._units = units
self._do_scaling = scaling
def setup(self):
if self._units:
units = 'ft'
else:
units = None
if self._do_scaling:
ref = .1
else:
ref = 1.
# Global Design Variable
self.add_input('z', val=np.zeros(2), units=units)
# Local Design Variable
self.add_input('x', val=0., units=units)
# Coupling parameter
self.add_input('y2', val=1.0, units=units)
# Coupling output
self.add_output('y1', val=1.0, lower=0.1, upper=1000., units=units, ref=ref)
def setup_partials(self):
# Finite difference everything
self.declare_partials('*', '*', method='fd')
def compute(self, inputs, outputs):
"""
Evaluates the equation
y1 = z1**2 + z2 + x1 - 0.2*y2
"""
z1 = inputs['z'][0]
z2 = inputs['z'][1]
x1 = inputs['x']
y2 = inputs['y2']
outputs['y1'] = z1**2 + z2 + x1 - 0.2*y2
self.execution_count += 1
SellarDis2 class definition
class SellarDis2(om.ExplicitComponent):
"""
Component containing Discipline 2 -- no derivatives version.
"""
def __init__(self, units=None, scaling=None):
super().__init__()
self.execution_count = 0
self._units = units
self._do_scaling = scaling
def setup(self):
if self._units:
units = 'inch'
else:
units = None
if self._do_scaling:
ref = .18
else:
ref = 1.
# Global Design Variable
self.add_input('z', val=np.zeros(2), units=units)
# Coupling parameter
self.add_input('y1', val=1.0, units=units)
# Coupling output
self.add_output('y2', val=1.0, lower=0.1, upper=1000., units=units, ref=ref)
def setup_partials(self):
# Finite difference everything
self.declare_partials('*', '*', method='fd')
def compute(self, inputs, outputs):
"""
Evaluates the equation
y2 = y1**(.5) + z1 + z2
"""
z1 = inputs['z'][0]
z2 = inputs['z'][1]
y1 = inputs['y1']
# Note: this may cause some issues. However, y1 is constrained to be
# above 3.16, so lets just let it converge, and the optimizer will
# throw it out
if y1.real < 0.0:
y1 *= -1
outputs['y2'] = y1**.5 + z1 + z2
self.execution_count += 1
import warnings
import numpy as np
import openmdao.api as om
from openmdao.test_suite.components.sellar import SellarDis1, SellarDis2
class ValidatedSellar1(SellarDis1):
def validate(self, inputs, outputs):
if outputs['y1'] > 20.0:
raise ValueError('Output "y1" is greater than 20.')
class SellarMDA(om.Group):
def setup(self):
cycle = self.add_subsystem('cycle', om.Group(), promotes=['*'])
cycle.add_subsystem('d1', ValidatedSellar1(), promotes_inputs=['x', 'z', 'y2'],
promotes_outputs=['y1'])
cycle.add_subsystem('d2', SellarDis2(), promotes_inputs=['z', 'y1'],
promotes_outputs=['y2'])
cycle.set_input_defaults('x', 1.0)
cycle.set_input_defaults('z', np.array([5.0, 2.0]))
# Nonlinear Block Gauss Seidel is a gradient free solver
cycle.nonlinear_solver = om.NonlinearBlockGS()
def validate(self, inputs, outputs):
if outputs['y2'] < 100.0:
warnings.warn('Output "y2" is less than 100.')
If any of the systems in a model have a validate method that you would like to run, the hierarchy of validates can be run using the run_validation method. This method can be run from any system and will go down the model hierarchy relative to the system that called it and run the validate method on each system in the hierarchy. Once run_model() or run_driver() is finished the run_validation method may be run.
- System.run_validation()[source]
Run validate method on all systems below this system.
The validate method on each system can be used to check any final input / output values after a run.
Note that when run_validation is run, it will collect all of the errors and warnings raised during the validate methods that it calls. If there are any errors among them, it will show everything collected under a ValidationError.
If the above problem is run, for example:
prob = om.Problem(model=om.Group())
prob.model.add_subsystem('cycle', SellarMDA())
prob.setup()
prob.run_model()
try:
prob.model.run_validation()
except om.ValidationError as e:
# Avoid printing the entire error traceback for easier viewing
print(f'Validation Error: {e}')
===========
cycle.cycle
===========
NL: NLBGS Converged in 8 iterations
Validation Error:
The following errors / warnings were collected during validation:
-----------------------------------------------------------------
UserWarning: 'cycle' <class SellarMDA>: Error calling validate(), Output "y2" is less than 100.
ValueError: 'cycle.cycle.d1' <class ValidatedSellar1>: Error calling validate(), Output "y1" is greater than 20.
-----------------------------------------------------------------
If there are no errors among what is collected from the validate methods, run_validation will print out everything collected without an error.
If the above problem is edited to only have warnings with no errors:
class ValidatedSellar1(SellarDis1):
def validate(self, inputs, outputs):
if outputs['y1'] > 20.0:
warnings.warn('Output "y1" is greater than 20.')
prob = om.Problem(model=om.Group())
prob.model.add_subsystem('cycle', SellarMDA())
prob.setup()
prob.run_model()
prob.model.run_validation()
===========
cycle.cycle
===========
NL: NLBGS Converged in 8 iterations
The following warnings were collected during validation:
-----------------------------------------------------------------
UserWarning: 'cycle' <class SellarMDA>: Error calling validate(), Output "y2" is less than 100.
UserWarning: 'cycle.cycle.d1' <class ValidatedSellar1>: Error calling validate(), Output "y1" is greater than 20.
-----------------------------------------------------------------
And finally, if there are no warnings nor errors collected from the validate methods, the run_validation will print out a simple message confirming that nothing was raised.
If the Sellar problem is run with no validate errors / warnings raised:
from openmdao.test_suite.components.sellar import SellarNoDerivatives
prob = om.Problem(model=om.Group())
prob.model.add_subsystem('cycle', SellarNoDerivatives())
prob.setup()
prob.run_model()
prob.model.run_validation()
No errors / warnings were collected during validation.