OpenMDAO 2 represents a total re-write and significant update of the framework with much cleaner APIs and performance improvements. It is also a backwards-incompatible release, and does not yet (as of November 2017) include all of the features that were in version 1. Given that, when and how should you go about updating your code? While each group will have to make this choice for itself, we want to give our perspective on the issue in the hope that it will help make the decision easier for you.
Why a total rewrite?
OpenMDAO 1.0 offered two key features that made it unique:
- Tight integration with high-fidelity, parallel-analysis tools.
- Automatic analytic derivatives.
The high fidelity HPC capability was achieved by integrating a novel computational architecture that supported distributed memory parallelism via MPI. The analytic derivatives features were enabled by the development of a sparse, matrix-free linear solver sub-system. The first feature worked extremely well, but the second was only partially successful because it suffered some performance limitations.
The challenge we faced was that we had designed a system that was efficient for large sparse analyses (e.g. problems including CFD models), but that had come at the cost of performance for smaller more dense analyses (e.g. low fidelity serial propulsion models).
Once we identified the performance issue, the fix was clear. We needed to use a dense linear solver system for smaller problems. The challenge was that 1.0 was designed around the sparse, matrix-free assumption, and it wasn’t clear that we could easily refactor the code around that. So we decided that a re-write was necessary.
So version 2 includes a hybrid linear solver architecture that supports a mixture of sparse and dense operations. Our testing on our own internal applications has shown an 10x reduction in compute cost for smaller problems while maintaining efficiency for larger scale problems.
These massive performance gains make a compelling reason to upgrade to v2.0, and we do expect that the user base will all make the switch eventually! The time may not be quite right for everyone to upgrade, because we haven’t just re-written the underlying code. We’ve also made several backwards-incompatible changes to the user-facing APIs.
Why make it backwards incompatible?
In making the transition to the hybrid linear solver architecture, it became clear that some new component level APIs were going to be needed, and that it would not be easy to make them backwards incompatible. Also, in a number of spots we felt that the version 1 APIs were both a little inconsistent and somewhat confusing. So we decided to free ourselves to make backwards-incompatible changes to the API. We feel that these updates are for the best in the long term.
However, writing a clean and fully self-consistent API is easier said than done. As we’ve developed version 2, we’ve had to repeatedly re-evaluate our API choices as we implement new features and build new applications that stress the framework. As such, we’re still making tweaks to the APIs and likely will be for through summer 2018. At this point (November 2017), the API changes are fairly small and mostly consist of “find and replace” type operations, or represent expansions via new API methods, but changes are still being made.
This API-changing issue was the primary reason we decided to release version 2 when we did. We decided that it was important to let the community know about the new APIs and start getting some outside feedback. It also gave users the ability to fix their work to a specific version and guarantee a stable API for themselves. To that end, our version numbering system uses a three-digit convention: 2.x.y. The ‘x’ is for releases that include API changes. All new APIs will be listed in the release notes, and any backwards incompatible changes will be called out. The ‘y’ is for minor releases (e.g. bug fixes and docs updates) and won’t represent new APIs. In other words: changing from 2.1 vs 2.2 will mean an API change, changing from 2.2.0 to 2.2.1 will not.
So which version should you use?
If you’re starting a new project from scratch, you should start from version 2! The only exception would be if you immediately need some of the features we have not yet implemented. As of November 2017, the most significant things on the list of missing features are the DOEDriver and Pass-By-Object variables. If you need either of those features, you’ll have to wait at least a few more months.
If you have a significant code base already implemented in 1.0, the decision is a little harder. Although the APIs are different, they are dauntingly so. As a point of reference to the difficulty of translating from 1 to 2: an internal application built on OpenMDAO that included over 100 files and 10000 lines of code was updated from version 1 to version 2 by one person in less than 3 days. However, if your application is currently working fine in version 1 and you’d rather wait a few more months till we’ve finished porting all the major features and settled down on backwards incompatible API changes, thats a reasonable choice.
By August 2018, things will have settled down considerably, and at that point we’ll start recommending that everyone make the update. Again, new codes should definitely start out in version 2! For the time being, though, good arguments can be made for using either version 1 or version 2 with existing codes, and the choice is yours.
