Setting Up Project Documentation in Sphinx

Importing Tools from OpenMDAO

During this process, to get your docs to build properly, you may need access to a couple of things from within OpenMDAO:

openmdao.docutils will get you things like our sourcedoc-building script, generate_docs, which will be called from conf.py, to create an organized set of source documentation. This package will also get you access to our powerful custom extensions, such as our Sphinx embedding library, including embed_code, and embed_options. Our code-embedding tool will help you to include things into your documentation that will stay dynamically updated with the code in your project and/or in the OpenMDAO project. To get access to these items, both in your local install and on CI, you can just import them from openmdao.docutils.

Here’s how you might bring in an OpenMDAO extension, by importing it, and then adding it to your other extensions in conf.py:

from openmdao.docutils import embed_code, embed_options

extensions = [
    'sphinx.ext.autodoc',
    'sphinx.ext.autosummary',
    'sphinx.ext.doctest',
    'sphinx.ext.todo',
    'sphinx.ext.coverage',
    'sphinx.ext.mathjax',
    'sphinx.ext.viewcode',
    'numpydoc',
    'embed_code',
    'embed_options',
]

Note

For more information on some of OpenMDAO’s custom embed extensions, please see Custom Directives in OpenMDAO

General Docs Settings

Your Sphinx documentation will need its own docs/conf.py, theme directory, and style.css so that you may customize the docs into something that will make them their own. You can use OpenMDAO’s docs/conf.py, docs/_theme/theme.conf and docs/_theme/static/style.css as a starting point.

OpenMDAO’s docs/conf.py file looks like this:

# -*- coding: utf-8 -*-
# This file is execfile()d with the current directory set to its
# containing dir.
import sys
import os
import importlib

from mock import Mock

from openmdao.docs.config_params import MOCK_MODULES
from openmdao.docs._utils.patch import do_monkeypatch
from openmdao.docs._utils.upload_doc_version import get_doc_version

# Only mock the ones that don't import.
for mod_name in MOCK_MODULES:
    try:
        importlib.import_module(mod_name)
    except ImportError:
        sys.modules[mod_name] = Mock()

# start off running the monkeypatch to keep options/parameters
# usable in docstring for autodoc.
do_monkeypatch()

# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
sys.path.insert(0, os.path.abspath('..'))
sys.path.insert(0, os.path.abspath('.'))
sys.path.insert(0, os.path.abspath('./_exts'))

# -- General configuration ------------------------------------------------

# If your documentation needs a minimal Sphinx version, state it here.
needs_sphinx = '1.5'

# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
    'sphinx.ext.autodoc',
    'sphinx.ext.autosummary',
    'sphinx.ext.doctest',
    'sphinx.ext.todo',
    'sphinx.ext.coverage',
    'sphinx.ext.mathjax',
    'sphinx.ext.viewcode',
    'numpydoc',
    'embed_code',
    'embed_options',
    'embed_compare',
    'embed_shell_cmd',
    'embed_bibtex',
    'embed_n2',
    'tags'
]

numpydoc_show_class_members = False

# autodoc_default_flags = ['members']

# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']

# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
# source_suffix = ['.rst', '.md']
source_suffix = '.rst'

# The master toctree document.
master_doc = 'index'

# General information about the project.
project = u'OpenMDAO'
copyright = u'2016, openmdao.org'
author = u'openmdao.org'

# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#

# The full version, including alpha/beta/rc tags.
import openmdao
release = openmdao.__version__ + ' Beta'
print('Release: %s' % release)

# Will be the Release version (x.y.z) or the commit ID if not a release build.
name, rel = get_doc_version()
if rel:
    version = 'Version: %s' % name
else:
    version = 'Commit ID: %s' % name
print(version)


# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = None


# exclude_patterns is a list of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = ['_build', '_srcdocs/dev']
absp = os.path.join('.', '_srcdocs')
sys.path.insert(0, os.path.abspath(absp))

packages = [
    'approximation_schemes',
    'core',
    'components',
    'drivers',
    'error_checking',
    'jacobians',
    'matrices',
    'proc_allocators',
    'recorders',
    'solvers',
    'surrogate_models',
    'solvers.linear',
    'solvers.nonlinear',
    'solvers.linesearch',
    'test_suite.components',
    'test_suite.scripts',
    'vectors',
    'utils',
    'visualization',
]

if os.path.isfile("make_sourcedocs"):
    from openmdao.docs._utils.generate_sourcedocs import generate_docs
    generate_docs("..", "../..", packages)

# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'

# If true, `todo` and `todoList` produce output, else they produce nothing.
todo_include_todos = False

# -- Options for HTML output ----------------------------------------------

# The theme to use for HTML and HTML Help pages.  See the documentation for
# a list of builtin themes.
html_theme = '_theme'

# Add any paths that contain custom themes here, relative to this directory.
html_theme_path = ['.']

# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
html_logo = '_static/OpenMDAO_Logo.png'

# The name of an image file (within the static path) to use as favicon of the
# docs.  This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
html_favicon = '_static/OpenMDAO_Favicon.ico'

# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
html_last_updated_fmt = '%b %d, %Y'

# Output file base name for HTML help builder.
htmlhelp_basename = 'OpenMDAOdoc'

#Customize sidebar
html_sidebars = {
   '**': ['globaltoc.html', 'searchbox.html']
}
# -- Options for manual page output ---------------------------------------

# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
    (master_doc, 'openmdao', u'OpenMDAO Documentation',
     [author], 1)
]

OpenMDAO numpydoc monkeypatch

OpenMDAO uses a monkeypatch to the numpydoc standard that allows extra fields in docstrings such as Options and Parameters. It also removes private Attributes (those beginning with an underscore) from the auto-documentation pages. To import this:

from openmdao.docutils import do_monkeypatch

Then simply calling the do_monkeypatch() from within your conf.py would set your docstrings up to behave similarly to OpenMDAO’s.

OpenMDAO docs Makefile

The OpenMDAO docs/Makefile can’t be imported, per se, but can be used as a template for your own Sphinx docs Makefile, and can accomplish several things that the default Sphinx Makefile falls short of (e.g. our Makefile can build only files that have recently changed, rather than the whole project; e.g. our makefile can rebuild an .rst file whose embed-code dependency has changed, though the .rst file hasn’t) Here are the commands from OpenMDAO’s Makefile:

###################################
# COMMANDS

# to remake only recently-changed files, not the entire document base.
#  note - first item in makefile is default action of "make"
html-update: mock redbaron matplotlib build

# to force the rebuild a file when its dependecy (e.g. code-embed) changes, not its rst file
single: mock redbaron matplotlib touch build

# build it all over again (note: make all == make html)
all html: make_srcdocs mock redbaron matplotlib tagg buildall post_remove

# build it all on CI machines; all warnings are raised as errors.
travis: make_srcdocs mock redbaron matplotlib tagg buildalltravis post_remove

clean:
    rm -rf $(BUILDDIR)/*
    rm -rf _srcdocs/*
    rm -rf tags
    rm -rf tmp
    rm -rf make_sourcedocs
    rm -rf doc_plot_*.png

OpenMDAO Auto-documentation Generator

OpenMDAO’s docs have a custom script, generate_sourcedocs, that creates an organized subdirectory of source documentation that is sorted by subpackage. To import this tool:

from openmdao.docutils import generate_docs

Then, from your docs/conf.py, invoke it with arguments of:

  1. Where to find packages (relative to where it’s being called).

  2. Root of the project (relative to where it’s being called).

  3. Which packages to include–omit things like “test” that don’t make sense to document.

packages = [
'subpackage1',
'subpackage2',
]

from openmdao.docutils import generate_docs
generate_docs("..", "../..", packages, project_name='yourproject')

OpenMDAO Tagging Tool

OpenMDAO’s docs have a custom script that pre-processes all the .rst files found within a set of Sphinx documentation, and creates a custom blog-like tagging system that helps organize and cross-reference docs.

The script finds occurrences of the .. tags:: directive and sets up the structure of the tags directory. One file is created for each subject tag, and that file contains links to each instance of the tag throughout the docs.

from openmdao.docutils import preprocess_tags

In OpenMDAO, we run preprocess_tags.py, (which calls the tag() function) from our docs/Makefile, prior to the building of the docs, so that all the proper files and links can be set up in advance of the actual Sphinx docbuild. Your project could benefit from a similar procedure. Use of tags is a completely optional step, your docs will work with or without tags.

Getting Docs For Your Plugin Transferred to github-pages

Once you have your documents organized and building locally, and building without errors on Travis CI, then we can explore transferring those built docs from Travis to github-pages. This is discussed in detail in the next doc on github-pages setup.