Twenty years ago, version 1.0 of the Conservation LAWs PACKage (CLAWPACK, now Clawpack was first released by Randy LeVeque. It seems fitting to take the occasion to look back on the intervening years. What follows are my thoughts on some of the great things that have resulted.
As far as I can tell, this item in the NA-Digest is the first public announcement of its existence. It was also announced more verbosely the same year in this conference paper, from the proceedings of the 5th HYP conference. Reading that conference paper now, I am struck by how it incorporated many of the ideals of scientific software development that we now discuss as if they were new ideas. For instance,
In this day of so much ado about credit for software, it’s also interesting to view this paper as an early example of a mathematical publication that is all about software.
Looking through the code snippets in the paper, I was astonished to recognize how much of the original Fortran 77 code remains virtually unaltered – including many variable names, function interfaces, and overall design. This is a testament to the quality of the original code design.
The central algorithms in Clawpack have also stood the test of time. The 80’s saw the heyday of research into second-order TVD methods for conservation laws, and Clawpack was released just as that era came to a close. Since then, research has gone in other directions – high-order methods, well-balancing, and positivity preservation, to name a few. While these new directions have provided additions to Clawpack, the “classic” algorithms have not changed and are still hard to beat as a robust general-purpose tool.
Of course, much has happened in the intervening twenty years. The original library handled 1- and 2-dimensional problems on regular cartesian grids. In the next few years, subsequent versions added algorithms for 3D, mapped grids, and adaptively refined meshes.
Additional algorithmic innovations are too numerous to try to list, but one that has had a lot of impact is the f-wave technique.
The problems to which Clawpack has been applied are certainly much too numerous to list. But you can start to get an idea by looking at citations of major Clawpack papers like this, this, this, this, and this. Perhaps the heaviest use in recent years has involved geophysical flows such as tsunamis and storm surges, in GeoClaw.
Clawpack has spawned numerous offshoots and extensions, including (but not limited to) AMRClaw, BearClaw, ZPLClaw, SharpClaw, CUDAClaw, ForestClaw, ManyClaw, PyClaw, and GeoClaw. Some of these have become part of the Clawpack-5 suite while others have forked and gone in other directions.
Nowadays, the term Clawpack refers to a collection of interrelated packages that are maintained and developed at github.com/clawpack. They include:
The Github organization also includes repositories for the docs and for contributed applications.
As far as I know, the original release was a one-person effort. But like most open-source projects, Clawpack quickly became a broader collaboration. I won’t attempt to credit everyone here; you can see some of the major contributors here, and many more by looking at the contributors pages on Github.
I was surprised to realize that I’ve now been involved with Clawpack
for half of its existence – ten years! During those years I’ve gotten to
work with an group of exceptional researchers who are also just
outstanding people. They say that the culture of an open-source software
community is shaped strongly by its founder, and I think Clawpack is no
exception.
It seems to me that its creator is not only a great applied
mathematician, but also somene who consistently leads the way in terms
of improving the way we do science. Clawpack exemplifies his commitment
to reproducibility and sustainable scientific software development, long
before those words came into scientific vogue. He was an advocate for
publishing in journals with low subscription prices, long before open
access became a movement.
Most significantly, he has always been interested first in finding and
solving interesting problems, and only secondarily in publishing papers.
Both through his personal influence and as chair of the SIAM Journals
Committee, he has been influential in making progress in these
directions, including the establishment of a Software section in SISC,
the acceptance by SIAM journals of supplementary materials (including
code), and a new policy allowing authors to post published articles on
their own or institutional websites.
As a result, the culture surrounding Clawpack has always encouraged openness and a willingness to accept new contributions. Furthermore, I think that the Clawpack developers have maintained a healthy skepticism toward our own algorithms and code. Although we try to make our code useful to as many people as possible, there has never been any attempt to evangelize the community in order to increase use of a particular set of algorithms or to increase metrics like citation counts. Because of this attitude, the code is continually improved through incorporation of new algorithmic innovations.
Of course, it would be wrong to say that Clawpack has been a perfect model for scientific code development. There are plenty of things we’ve done wrong or could learn to improve.
The original announcement says that “contributions from other users will be gratefully accepted,” and that has always been true. Nevertheless, the widely accepted development model for a long time was that most users would take the code, fork it, and make their own enhancements that would never get back to the main codebase. While this prevented feature bloat, it also meant that a great wealth of knowledge – largely in the form of sophisticated approximate Riemann solvers – will perish on some dusty hard drive rather than benefitting the larger community. We’re trying to change that now by encouraging users to submit pull requests for Riemann solvers and for entire applications.
Another example of where I see room for improvement is in output and visualization, where we have, to some degree, reinvented the wheel. Clawpack has long used custom ASCII and binary file formats, that can only be read in by Clawpack (or by reverse-engineering code for the relatively simple formats). We are now pushing to move to a more standard default format (probably HDF5), which would allow easier integration with standard visualization and post-processing libraries.
On the visualization side, the Clawpack developers have created some extremely useful tools for plotting time-dependent data on structured grids (including block-structured AMR). These tools sit on top of MATLAB and matplotlib. A large amount of work has gone into these “in-house” tools rather than into leveraging and contributing to dedicated visualization tools. Meanwhile, individual users have occasionally connected Clawpack to powerful visualization tools, but their custom code never got back to the main codebase. The limited capabilities of matplotlib in 3D seem to finally be providing sufficient impetus to force us to integrate with a sophisticated visualization library. I have been working lately on integration with yt.
It may come as a surprise for a code that’s so long in the tooth, but I think Clawpack development at present is more vibrant than ever. Since 2011, we’ve held annual developer workshops, the latest of which took place last week here at KAUST. The pictures on this page are from those workshops (the cake in the first photo, which shows the Clawpack logo, was made by my wife, and is a fondant version of a fluid-dynamical shockwave hitting a low-density bubble).
As for the future, I won’t claim enough clairvoyance to see 20 years ahead. But here are some things I hope we can accomplish in the next few years:
Are you excited yet? I certainly am. Come join the fun!
