coloring.py

Routines to compute coloring for use with simultaneous derivatives.

class openmdao.utils.coloring.Coloring(sparsity, row_vars=None, row_var_sizes=None, col_vars=None, col_var_sizes=None)

Bases: object

Container for all information relevant to a coloring.

Parameters:

sparsityndarray

Full jacobian sparsity matrix (dense bool form).

row_varslist of str or None

Names of variables corresponding to rows.

row_var_sizesndarray or None

Sizes of row variables.

col_varslist of str or None

Names of variables corresponding to columns.

col_var_sizesndarray or None

Sizes of column variables.

Attributes:

_shapetuple of int (nrows, ncols)

Tuple describing the shape of the sparsity matrix.

_nzrowsndarray of int

Row indices of nonzero entries in the full jac sparsity matrix.

_nzcolsndarray of int

Column indices of nonzero entries in the full jac sparsity matrix.

_pct_nonzerofloat

If known, percentage of nonzero vs total array entries.

_fwdtuple (col_lists, row_maps) or None

Contains lists of grouped columns and nonzero rows for each column for forward coloring.

_revtuple (col_lists, row_maps) or None

Contains lists of grouped columns and nonzero rows for each column for reverse coloring.

_col_varslist of str or None

Names of variables corresponding to columns.

_col_var_sizesndarray or None

Sizes of column variables.

_row_varslist of str or None

Names of variables corresponding to rows.

_row_var_sizesndarray or None

Sizes of row variables.

_metadict

Dictionary of metadata used to create the coloring.

_names_arrayndarray or None:

Names of total jacobian rows or columns.

_local_arrayndarray or None:

Indices of total jacobian rows or columns.

_abs2prom{‘input’: dict, ‘output’: dict}

Dictionary mapping absolute names to promoted names.

_subtractionslist

List of subtraction tuples. Only used for the substitution method of bidirectional coloring.

__init__(sparsity, row_vars=None, row_var_sizes=None, col_vars=None, col_var_sizes=None)

Initialize data structures.

color_iter(direction)

Given a direction, yield an iterator over column (or row) groups.

Parameters:

directionstr

Derivative direction (‘fwd’ or ‘rev’).

Yields:

list of int

Lists of column indices (in fwd mode) or row indices (in rev mode).

color_nonzero_iter(direction)

Given a direction, yield an iterator over (columns, nz_rows) or (rows, nz_columns).

Parameters:

directionstr

Indicates which coloring subdict (‘fwd’ or ‘rev’) to use.

Yields:

(column or row groups, nonzero row or column lists)

Yields a list of columns/rows and their associated nonzero rows/columns for each color.

colored_jac_iter(compressed_j, direction, trans=None)

Yield nonzero parts of columns (fwd) or rows (rev) of a colored jacobian.

Parameters:

compressed_jndarray

The compressed jacobian.

directionstr

Derivative computation direction (‘fwd’ or ‘rev’).

transndarray

Index array to translate from compressed jac in function context to openmdao jac.

Yields:

ndarray

Nonzero part of current jacobian column or row.

ndarray

Indices into full jacobian column or row where nonzero values should be placed.

int

Index into the full jacobian of the current column or row.

display(show=True, fname='coloring.png')

Display a plot of the sparsity pattern, showing grouping by color.

Parameters:

showbool

If True, show the plot. Otherwise, just save the plot in a file. Default is True.

fnamestr

Path to the location where the plot file should be saved.

display_bokeh(output_file='total_coloring.html', show=False, max_colors=200, use_prom_names=True)

Display a plot of the sparsity pattern, showing grouping by color.

Parameters:

sourcestr or Coloring Driver

The source for the coloring information, which can either be a string of the filepath to the coloring data file, an instance of Coloring, or the Driver containing the coloring information.

output_filestr or None

The name of the output html file in which the display is written. If None, the resulting plots will not be saved.

showbool

If True, a browswer will be opened to display the generated file.

max_colorsint

Bokeh supports at most 256 colors in a colormap. This function reduces that number to some default length, otherwise both forward and reverse displays may share shades very near white and be difficult to distinguish. Once the number of forward or reverse solves exceeds this threshold, the color pattern restarts.

use_prom_namesbool

If True, display promoted names rather than absolute path names for variables.

display_txt(out_stream=DEFAULT_OUT_STREAM, html=False, summary=True, use_prom_names=True)

Print the structure of a boolean array with coloring info for each nonzero value.

Forward mode colored nonzeros are denoted by ‘f’, reverse mode nonzeros by ‘r’, overlapping nonzeros by ‘O’ and uncolored nonzeros by ‘x’. Zeros are denoted by ‘.’. Note that x’s and O’s should never appear unless there is a bug in the coloring algorithm.

If names and sizes of row and column vars are known, print the name of the row var alongside each row and print the names of the column vars, aligned with each column, at the bottom.

Parameters:

out_streamfile-like or _DEFAULT_OUT_STREAM

The destination stream to which the text representation of coloring is to be written.

htmlbool

If True, the output will be formatted as HTML. If False, the resulting output will be plain text.

summarybool

If True, include the coloring summary.

use_prom_namesbool

If True, display promoted names rather than absolute path names for variables.

expand_jac(compressed_j, direction)

Expand the given compressed jacobian into a full jacobian.

Parameters:

compressed_jndarray

The compressed jacobian.

directionstr

Derivative computation direction (‘fwd’ or ‘rev’).

Returns:

ndarray

The full jacobian.

get_declare_partials_calls()

Return a string containing declare_partials() calls based on the subjac sparsity.

Returns:

str

A string containing a declare_partials() call for each nonzero subjac. This string may be cut and pasted into a component’s setup() method.

get_dense_sparsity(dtype=<class 'numpy.uint8'>)

Return a dense array representing the full sparsity.

Parameters:

dtypeobject

Data type of returned numpy array.

Returns:

ndarray

Dense sparsity matrix.

get_renamed_copy(row_translate, col_translate)

Return a new Coloring object with the variables renamed.

Parameters:

row_translatedict

Dictionary mapping old row names to new row names.

col_translatedict

Dictionary mapping old column names to new column names.

Returns:

Coloring

New Coloring object with the variables renamed.

get_row_col_map(direction)

Return mapping of nonzero rows to each column (fwd) or nonzeros columns to each row (rev).

Parameters:

directionstr

Indicator of forward mode (‘fwd’) or reverse mode (‘rev’).

Returns:

list of lists of int

List where each entry contains list of nonzero rows/cols for the index corresponding to each column/row.

get_row_var_coloring(varname)

Return the number of fwd and rev solves needed for a particular row variable.

Parameters:

varnamestr

Name of the row variable.

Returns:

int

Number of forward solves needed for the given variable.

int

Number of reverse solves needed for the given variable.

get_subjac_sparsity()

Compute the sparsity structure of each subjacobian based on the full jac sparsity.

If row/col variables and sizes are not known, returns None.

Returns:

dict or None

Mapping of (of, wrt) keys to their corresponding (nzrows, nzcols, shape).

static load(fname)

Read the coloring object from the given pickle file.

Parameters:

fnamestr

Name of file to read from.

Returns:

Coloring

See docstring for Coloring class.

modes()

Return a tuple containing the modes included in this coloring.

Returns:

tuple

Tuple containing some subset of (‘fwd’, ‘rev’).

save(fname)

Write the coloring object to the given stream, creating intermediate dirs if needed.

Parameters:

fnamestr or pathlib.Path

File to save to.

property sparsity

Return the sparsity matrix as a COO sparse matrix.

Returns:

coo_matrix

The sparsity matrix.

summary(out_stream=DEFAULT_OUT_STREAM)

Print a summary of this coloring.

Parameters:

out_streamfile-like or _DEFAULT_OUT_STREAM

The destination stream to which the text representation of coloring is to be written.

tangent_iter(direction, arr=None, trans=None)

Given a direction, return input (fwd) or output (rev) tangent arrays.

Each array will contain multiple columns/rows of the identity matrix that share the same color.

Parameters:

directionstr

Indicates which coloring subdict (‘fwd’ or ‘rev’) to use.

arrndarray or None

Storage for the current array value.

transndarray or None

Index translation array.

Yields:

ndarray

tangent array for inputs (fwd) or outputs (rev)

tangent_matrix(direction, trans=None)

Return a tangent or cotangent matrix for use with jax.

Parameters:

directionstr

Derivative computation direction (‘fwd’ or ‘rev’).

transndarray or None

Index translation array.

Returns:

ndarray

The tangent or cotangent matrix.

total_solves(fwd=True, rev=True)

Return total number of solves required based on the given coloring info.

Parameters:

fwdbool

If True, add fwd colors to total.

revbool

If True, add rev colors to total.

Returns:

int

Total number of solves required to compute the jacobian.

class openmdao.utils.coloring.ColoringMeta(num_full_jacs=3, tol=1e-25, orders=None, min_improve_pct=5.0, show_summary=True, show_sparsity=False, dynamic=False, static=None, perturb_size=1e-09, use_scaling=False, msginfo='', direct=True)

Bases: object

Container for all metadata relevant to a coloring.

Parameters:

num_full_jacsint

Number of full jacobians to generate while computing sparsity.

tolfloat

Use this tolerance to determine what’s a zero when determining sparsity.

ordersint or None

Number of orders += around ‘tol’ for the tolerance sweep when determining sparsity. If None, no tolerance sweep will be performed and whatever ‘tol’ is specified will be used.

min_improve_pctfloat

Don’t use coloring unless at least min_improve_pct percentage decrease in number of solves.

show_summarybool

If True, print a short summary of the coloring. Defaults to True.

show_sparsitybool

If True, show a plot of the sparsity. Defaults to False.

dynamicbool

True if dynamic coloring is being used.

staticColoring, str, or None

If a Coloring object, just use that. If a filename, load the coloring from that file. If None, do not attempt to use a static coloring.

perturb_sizefloat

Size of input/output perturbation during generation of sparsity.

use_scalingbool

If True, use driver scaling when computing sparsity.

msginfostr

Prefix for warning/error messages.

directbool

If doing bidirectional coloring, use the direct method for assembling the column adjacency matrix of partitions, else use the substitution method.

Attributes:

num_full_jacsint

Number of full jacobians to generate while computing sparsity.

tolfloat

Use this tolerance to determine what’s a zero when determining sparsity.

ordersint or None

Number of orders += around ‘tol’ for the tolerance sweep when determining sparsity. If None, no tolerance sweep will be performed and whatever ‘tol’ is specified will be used.

min_improve_pctfloat

Don’t use coloring unless at least min_improve_pct percentage decrease in number of solves.

show_summarybool

If True, print a short summary of the coloring. Defaults to True.

show_sparsitybool

If True, show a plot of the sparsity. Defaults to False.

dynamicbool

True if dynamic coloring is being used.

staticColoring, str, or None

If a Coloring object, just use that. If a filename, load the coloring from that file. If None, do not attempt to use a static coloring.

perturb_sizefloat

Size of input/output perturbation during generation of sparsity.

use_scalingbool

If True, use driver scaling when computing sparsity.

msginfostr

Prefix for warning/error messages.

_coloringColoring or None

The coloring object.

_failedbool

If True, coloring was already generated but failed.

_approxbool

If True, this is an approx coloring.

randomize_subjacsbool

If True, use random subjacs when computing sparsity.

randomize_seedsbool

If True, use random seeds when computing sparsity.

directbool

If doing bidirectional coloring, use the direct method for assembling the column adjacency matrix of partitions, else use the substitution method.

__getitem__(name)

Get the value of the named metadata.

Parameters:

namestr

Name of the metadata.

Returns:

object

Value of the named metadata.

__init__(num_full_jacs=3, tol=1e-25, orders=None, min_improve_pct=5.0, show_summary=True, show_sparsity=False, dynamic=False, static=None, perturb_size=1e-09, use_scaling=False, msginfo='', direct=True)

Initialize data structures.

__iter__()

Iterate over the metadata.

Yields:

(str, object)

Tuple containing the name and value of each metadata item.

__setitem__(name, value)

Set the value of the named metadata.

Parameters:

namestr

Name of the metadata.

valueobject

Value of the metadata.

property coloring

Return the coloring.

Returns:

Coloring or None

The coloring.

copy()

Return a new object with metadata copied from this object.

Returns:

ColoringMeta

Copy of the metadata.

display()

Display information about the coloring.

do_compute_coloring()

Return True if coloring should be computed.

Returns:

bool

True if coloring should be computed.

get(name, default=None)

Get the value of the named metadata.

Parameters:

namestr

Name of the metadata.

defaultobject or None

The value to return if the named metadata is not found.

Returns:

object

Value of the named metadata.

reset_coloring()

Reset the coloring to None.

set_coloring(coloring, msginfo='')

Set the coloring.

Parameters:

coloringColoring or None

The coloring.

msginfostr

Prefix for warning/error messages.

update(dct)

Update the metadata.

Parameters:

dctdict

Dictionary of metadata.

exception openmdao.utils.coloring.InvalidColoringError

Bases: Exception

A custom error class that is raised in the event of an invalid coloring.

__init__(*args, **kwargs)

add_note()

Exception.add_note(note) – add a note to the exception

args

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

openmdao.utils.coloring.MNCO_bidir(J, direct=True)

Compute bidirectional coloring using Minimum Nonzero Count Order (MNCO).

Based on the algorithm found in Coleman, T.F., Verma, A. (1998) The efficient Computation of Sparse Jacobian Matrices Using Automatic Differentiation. SIAM Journal on Scientific Computing, 19(4), 1210-1233.

Parameters:

Jcoo_matrix

Jacobian sparsity matrix (boolean).

directbool

If True, use the direct method to compute the column adjacency matrix of partitions. Otherwise, use the substitution method. See the paper for descriptions of both methods.

Returns:

Coloring

See docstring for Coloring class.

class openmdao.utils.coloring.Partial_ColoringMeta(wrt_patterns=('*',), method='fd', form=None, step=None, per_instance=True, perturb_size=1e-09, num_full_jacs=3, tol=1e-25, orders=None, min_improve_pct=5.0, show_summary=True, show_sparsity=False, dynamic=False, static=None, msginfo='')

Bases: ColoringMeta

Container for all metadata relevant to a partial coloring.

Parameters:

wrt_patternslist/tuple of str or str

Patterns used to match wrt variables.

methodstr

Finite differencing method (‘fd’ or ‘cs’).

formstr

Form of the derivatives (‘forward’, ‘backward’, or ‘central’). Only used if method is ‘fd’.

stepfloat

Step size for ‘fd’, or ‘cs’.

per_instancebool

Assume each instance can have a different coloring, so coloring will not be saved as a class coloring.

perturb_sizefloat

Size of input/output perturbation during generation of sparsity.

num_full_jacsint

Number of full jacobians to generate while computing sparsity.

tolfloat

Use this tolerance to determine what’s a zero when determining sparsity.

ordersint or None

Number of orders += around ‘tol’ for the tolerance sweep when determining sparsity. If None, no tolerance sweep will be performed and whatever ‘tol’ is specified will be used.

min_improve_pctfloat

Don’t use coloring unless at least min_improve_pct percentage decrease in number of solves.

show_summarybool

If True, print a short summary of the coloring. Defaults to True.

show_sparsitybool

If True, show a plot of the sparsity. Defaults to False.

dynamicbool

True if dynamic coloring is being used.

staticColoring, str, or None

If a Coloring object, just use that. If a filename, load the coloring from that file. If None, do not attempt to use a static coloring.

msginfostr

Prefix for warning/error messages.

Attributes:

wrt_patternslist/tuple of str or str

Return the wrt patterns.

methodstr

Finite differencing method (‘fd’ or ‘cs’).

formstr

Form of the derivatives (‘forward’, ‘backward’, or ‘central’). Only used if method is ‘fd’.

stepfloat

Step size for ‘fd’, or ‘cs’.

per_instancebool

Assume each instance can have a different coloring, so coloring will not be saved as a class coloring.

fnamestr or None

Filename where coloring is stored.

wrt_matchesset of str or None

Where matched wrt names are stored.

__getitem__(name)

Get the value of the named metadata.

Parameters:

namestr

Name of the metadata.

Returns:

object

Value of the named metadata.

__init__(wrt_patterns=('*',), method='fd', form=None, step=None, per_instance=True, perturb_size=1e-09, num_full_jacs=3, tol=1e-25, orders=None, min_improve_pct=5.0, show_summary=True, show_sparsity=False, dynamic=False, static=None, msginfo='')

Initialize data structures.

__iter__()

Iterate over the metadata.

Yields:

(str, object)

Tuple containing the name and value of each metadata item.

__setitem__(name, value)

Set the value of the named metadata.

Parameters:

namestr

Name of the metadata.

valueobject

Value of the metadata.

property coloring

Return the coloring.

Returns:

Coloring or None

The coloring.

copy()

Return a new object with metadata copied from this object.

Returns:

ColoringMeta

Copy of the metadata.

display()

Display information about the coloring.

do_compute_coloring()

Return True if coloring should be computed.

Returns:

bool

True if coloring should be computed.

get(name, default=None)

Get the value of the named metadata.

Parameters:

namestr

Name of the metadata.

defaultobject or None

The value to return if the named metadata is not found.

Returns:

object

Value of the named metadata.

reset_coloring()

Reset coloring and fname metadata.

set_coloring(coloring, msginfo='')

Set the coloring.

Parameters:

coloringColoring or None

The coloring.

msginfostr

Prefix for warning/error messages.

update(dct)

Update the metadata.

Parameters:

dctdict

Dictionary of metadata.

property wrt_patterns

Return the wrt patterns.

Returns:

list of tuple or None

Patterns used to match wrt variables.

openmdao.utils.coloring.compute_total_coloring(problem, mode=None, of=None, wrt=None, num_full_jacs=3, tol=1e-25, orders=None, setup=False, run_model=False, fname=None, driver=None)

Compute simultaneous derivative colorings for the total jacobian of the given problem.

Parameters:

problemProblem

The Problem being analyzed.

modestr or None

The direction for computing derivatives. If None, use problem._mode.

ofiter of str or None

Names of the ‘response’ variables.

wrtiter of str or None

Names of the ‘design’ variables.

num_full_jacsint

Number of times to repeat total jacobian computation.

tolfloat

Tolerance used to determine if an array entry is nonzero.

ordersint

Number of orders above and below the tolerance to check during the tolerance sweep.

setupbool

If True, run setup before calling compute_totals.

run_modelbool

If True, run run_model before calling compute_totals.

fnamefilename or None

File where output coloring info will be written. If None, no info will be written.

driver<Driver>, None, or False

The driver associated with the coloring. If None, use problem.driver. If False, no driver will be used.

Returns:

Coloring

See docstring for Coloring class.

openmdao.utils.coloring.display_coloring(source, output_file='total_coloring.html', as_text=False, show=True, max_colors=200)

Display the coloring information from source to html format.

Parameters:

sourcestr or Coloring or Driver

The source of the coloring information. If given as a string, source should be a valid coloring file path. If given as a Driver, display_colroing will attempt to obtain coloring information from the Driver.

output_filestr or Path or None

The output file to which the coloring display should be sent. If as_text is True and output_file ends with .html, then the coloring will be sent to that file as html, otherwise it will the html file will be saved in a temporary file.

as_textbool

If True, render the coloring information using plain text.

showbool

If True, open the resulting html file in the system browser.

max_colorsint

Bokeh colormaps support at most 256 colors. Near the upper end of this interval, the colors are nearly white and may be difficult to distinguish. This setting sets the upper limit for the color index before the pattern repeats.

openmdao.utils.coloring.dynamic_total_coloring(driver, run_model=True, fname=None, of=None, wrt=None)

Compute simultaneous deriv coloring during runtime.

Parameters:

driver<Driver>

The driver performing the optimization.

run_modelbool

If True, call run_model before computing coloring.

fnamestr or None

Name of file where coloring will be saved.

ofiter of str or None

Names of the ‘response’ variables.

wrtiter of str or None

Names of the ‘design’ variables.

Returns:

Coloring or None

The computed coloring.