Using Pixi for reproducable environments

Using Pixi for reproducable environments#

OpenMDAO requires a number of dependencies to fully take advantage of its capabilities. Sometimes finding versions of dependencies which are compatible and are known to function correctly is difficult.

OpenMDAO uses Pixi for dependency management. Pixi is especially useful because some OpenMDAO dependencies (like MPI, PETSc) are not available on PyPI but are needed for parallel computing features.

Using Pixi allows users and developers to get consistent, reproducable environments.

Installing Pixi#

Before users can utilize Pixi, it needs to be installed:

curl -fsSL https://pixi.sh/install.sh | bash

Getting the Pixi environments#

Pixi stores information about OpenMDAO environments in a manifest file (pixi.toml) and a lock file (pixi.lock) in the root directory of the OpenMDAO git repository.

The intent is that the lock file will be tested in our CI machinery, and should provide a consistent environment for as many of our users as possible.

Option 1: Using the environment from the OpenMDAO git repo#

Users who want to develop OpenMDAO itself can setup the Pixi environments from the cloned repository.

# Clone the repository
git clone https://github.com/OpenMDAO/OpenMDAO.git

# Enter the directory with the pixi.toml and pixi.lock files
cd OpenMDAO

Option 2: Downloading the Pixi manifest and lock files#

Other users who just want to pip install without cloning the repo will need to dowload the manifest and lock files. These should be placed in the top-level of the users desired working directory/project directory.

# Download the latest Pixi configuration...
curl -O https://raw.githubusercontent.com/OpenMDAO/OpenMDAO/master/pixi.toml
curl -O https://raw.githubusercontent.com/OpenMDAO/OpenMDAO/master/pixi.lock

# Alternatively, for a specific release...
curl -O https://raw.githubusercontent.com/OpenMDAO/OpenMDAO/3.42.0/pixi.toml
curl -O https://raw.githubusercontent.com/OpenMDAO/OpenMDAO/3.42.0/pixi.lock

Pixi files are only available as of OpenMDAO version 3.42.0.

Installing the desired environment#

From the directory with the manifest and lock files:

# Install the default environment (no mpi)
pixi install --frozen

# Or, install the python 3.13 environment with all dependencies
pixi install -e py313 --frozen

Using –frozen causes Pixi to use the exact versions for the environment as defined in the pixi.lock file. This is what we test against on CI and should be the most reliable environment.

You can remove --frozen and Pixi will attempt to install the latest possible dependencies for most environments.

Using the given environment via pixi shell#

You can then either shell into the environment (more akin to activating a conda environment), or you can run a command using that environment.

# Shell into the all environment (if installed)...
pixi shell -e all

# Install OpenMDAO in developer mode (if in the OpenMDAO git repo)...
python -m pip install -e .

# OR, Install OpenMDAO from pypi...
python -m pip install openmdao

That last step may seem a bit odd, but OpenMDAO is not a dependency of itself. You will need to install OpenMDAO into whatever pixi environment you are using. You only need to do this once after a given environment is installed.

Now the environment is ready to be used

# Run a script in the environment that we have shelled into...
python my_openmdao_script.py

More information about our Pixi implementation, including available environments and other ways of using it, are avaiable in the developer section of our documentation.

Using the given environment via pixi run#

Alternatively, users can run commands given a desired Pixi environment without shelling into it.

As before, make sure OpenMDAO has been installed in the environment:

# Install OpenMDAO from pypi into the all environment
pixi run -e py313 python -m pip install openmdao

Or, install a local OpenMDAO git repository in developer mode into the all environment:

# Install OpenMDAO from a local git repository
cd OpenMDAO
pixi run python -m pip install -e .

And now, run an OpenMDAO python script using the given environment

pixi run python path/to/my_openmdao_script.py

Available Environments#

default#

General development environment with JAX, numba, visualization, notebooks, and testing support, but no MPI. Uses the latest solvable environment.

Best for most users and everyday development work. Usage: pixi shell (or pixi shell -e default)

minimal#

Default dependencies only plus testing related dependencies, with latest solvable environment.

Used for testing in the absence of dependencies. Usage: pixi shell -e minimal

py311, py312, py313#

Comprehensive environment with all optional dependencies except documenation and testing-related ones. Pinned to a specific python version.

This will give most users the “full” OpenMDAO capability. Usage:pixi shell -e all

dev#

The latest solvable environment with the addition of documentation and packaging-related dependencies.

Usage: pixi shell -e docs

oldest#

Includes oldest supported versions of all dependencies except packaging-related ones.

Usage: pixi shell -e oldest

What if the latest solvable environment has known bugs?#

If a dependency is released with a change that breaks our functionality due to a bug, we will pin that dependency to the latest known working version on our master repository (and in any releases) until it is fixed.

Reach out to the team with an issue or discussion on our Github repostiory

Using OpenMDAO’s environments for Github actions in dependent projects#

OpenMDAO’s repository contains an action definition that can be used for Github actions by other projects. This might be convenient for dependent projects to streamline their CI process and ensure they’re using the same repos that we’ve tested against.

Adding the following step to your workflow/action will install the desired OpenMDAO pixi environement with the given version of OpenMDAO.

      - name: 'Setup OpenMDAO pixi Environment'
        id: setup_pixi_environment
        uses: OpenMDAO/OpenMDAO/.github/actions/setup_openmdao_pixi@master
        with:
          environment: 'py312'
          openmdao_install_from: 'pypi'
          openmdao_version: ''

Options:

The with: section allows you to specify the following arguments:

  • environment

    Pixi environment(s) to install

    • Required: false

    • Default: 'py313'

  • frozen

    Use frozen lock file (recommended for CI). Setting this to false will attempt to find newer versions of dependencies.

    • Required: false

    • Default: 'true'

  • openmdao_install_from

    How to install OpenMDAO: pypi, github, wheel, local, local_developer, or none. Generally, other repositories should only be using the pypi or github options here, as the others assume the action is operating on the OpenMDAO repo itself.

    • Required: false

    • Default: 'pypi'

  • openmdao_version

    OpenMDAO version (for PyPI and Github installs only): “” (latest) or str such as “3.42.0”.

    • Required: false

    • Default: ''

    To use the environment in a given step of a test, you’ll need to use the pixi run or the pixi shell-hook command. For example:

      - name: Perform linting with Ruff
        if: ${{ matrix.NAME == 'Ubuntu Baseline' }}
        run: |
          eval "$(pixi shell-hook -e ${{ matrix.PIXI_ENVIRONMENT }})"
          ruff check . --config pyproject.toml
Contents