Welcome to pySDC!

The pySDC project is a Python implementation of the spectral deferred correction (SDC) approach and its flavors, esp. the multilevel extension MLSDC and PFASST. It is intended for rapid prototyping and educational purposes. New ideas like e.g. sweepers or predictors can be tested and first toy problems can be easily implemented.

Features

  • Variants of SDC: explicit, implicit, IMEX, multi-implicit, Verlet, multi-level, diagonal, multi-step

  • Variants of PFASST: virtual parallel or MPI-based parallel, classical of multigrid perspective

  • 8 tutorials: from setting up a first collocation problem to SDC, PFASST and advanced topics

  • Projects: many documented projects with defined and tested outcomes

  • Many different examples, collocation types, data types already implemented

  • Works with FEniCS, mpi4py-fft and PETSc (through petsc4py)

  • Continuous integration via Github Actions

  • Fully compatible with Python 3.6 - 3.9

Getting started

The code is hosted on GitHub, see https://github.com/Parallel-in-Time/pySDC, and PyPI, see https://pypi.python.org/pypi/pySDC. Use

pip install pySDC

to get the latest stable release including the core dependencies. Note that this will omit some of the more complex packages not required for the core functionality of pySDC, e.g. mpi4py, fenics and petsc4py (see below). All requirements are listed in the files requirements.txt . To work with the source files, checkout the code from Github and install the dependencies e.g. by using a conda environment and

conda install -c conda-forge --file requirements.txt

To check your installation, run

pytest pySDC/tests

You may need to update your PYTHONPATH by running

export PYTHONPATH=$PYTHONPATH:/path/to/pySDC/root/folder

in particular if you want to run any of the playgrounds, projects or tutorials. All import statements there assume that the pySDC’s base directory is part of PYTHONPATH.

Note: When installing mpi4py, fenics and petsc4py, make sure they use the same MPI installation (e.g. MPICH3). You can achieve this e.g. by

conda install -c conda-forge mpich petsc4py mpi4py fenics

Most of the code is tested automatically using Github Actions, so a working version of the installation process can always be found in the ci_pipeline.yml file. This workflow can be run locally using act by

act --env CONDA=/usr/share/miniconda -j ci

See also this page for details on how to run workflows depending on miniconda locally.

For many examples, LaTeX is used for the plots, i.e. a decent installation of this is needed in order to run the tests. When using fenics or petsc4py, a C++ compiler is required (although installation may go through at first).

For more details on pySDC, check out http://www.parallel-in-time.org/pySDC.

How to cite

If you use pySDC or parts of it for your work, great! Let us know if we can help you with this. Also, we would greatly appreciate a citation of this paper:

Robert Speck, Algorithm 997: pySDC - Prototyping Spectral Deferred Corrections, ACM Transactions on Mathematical Software (TOMS), Volume 45 Issue 3, August 2019, https://doi.org/10.1145/3310410

The current software release can be cited using Zenodo: zenodo

Acknowledgements

This project has received funding from the European High-Performance Computing Joint Undertaking (JU) under grant agreement No 955701 (TIME-X). The JU receives support from the European Union’s Horizon 2020 research and innovation programme and Belgium, France, Germany, and Switzerland. This project also received funding from the German Federal Ministry of Education and Research (BMBF) grant 16HPC047.

docs/source/logo/Time-X.png docs/source/logo/EU.png docs/source/logo/BMBF.jpg

Changelog

  • December 13, 2021: Version 4.2 brings compatibility with Python 3.9, including some code cleanup. The CI test suite seems to run faster now, since sorting out the dependencies is faster. Tested mamba, which for now makes the CI pipeline much faster. Also, the CI workflow can now run locally using act. We introduced markers for soem of the tests in preparation of distributed tests on different platforms. And finally, a LaTeX installation is no longer needed use plotting (but recommended).

  • August 11, 2021: Version 4.1 has some more changes under the hood, most of them with no significant impact to users. The CI pipeline has been completely rewritten, porting the code to Github Actions (away from Travis CI), to flake8 and to pytest (away from nose). One thing that may have an impact on users is that following the changes made in Version 4.0, the PETSc data structures are now much easier, removing a lot of unnecessary boilerplate code.

  • May 4, 2021: Long time, no see, but this major release 4.0 marks some improvements under the hood:

    • Rewrote ``mesh`` and ``particle`` data type: Creation of new arrays for each operation is now avoided by directly subclassing Numpy’s ndarray. Somewhat faster, definitively better, less code, future-proof, but also breaking the API. If you use pySDC for your project, make sure you adapt to the new data types (or don’t upgrade).

    • Faster quadrature: Thanks to tlunet the computation of the weights is now faster and (even) more reliable. No breaking of any API here…

    • Bugfixing and version pushes: The code should run without (many) complaints with Python 3.6, 3.7 and potentially above. Also, the plotting routines have been adapted to work with recent versions of matplotlib.

    This is not much (yet) and if it were not for the API changes, this would have been a minor release.

  • August 30, 2019: Version 3.1 adds many more examples like the nonlinear Schrödinger equation, more on Gray-Scott and in particular Allen-Cahn. Those are many implemented using the parallel FFT library mpi4pi-fft, which can now be used with pySDC. There are now 8 tutorials, where step 7 shows the usage of three external libraries with pySDC: mpi4py, FEniCS and petsc4py. The MPI controller has been improved after performaning a detailed performance analysis using Score-P and Extrae. Finally: first steps towards error/iteration estimators are taken, too.

  • February 14, 2019: Released version 3 of pySDC. This release is accompanied by the ACM TOMS paper “pySDC – Prototyping spectral deferred corrections”. It release contains some breaking changes to the API. In detail:

    • Dropped Python 2 support: Starting with this version, pySDC relies on Python 3. Various incompabilities led to inconsistent treatment of dependencies, so that parts of the code had to use Python 2 while other relied on Python 3 or could do both. We follow A pledge to migrate to Python 3 with this decision, as most prominent dependencies of pySDC already do.

    • Unified controllers: Instead of providing (and maintaining) four different controllers, this release only has one for emulated and one for MPI-based time-parallelization (controller_nonMPI and controller_MPI). This should avoid further confusion and makes the code easier to maintain. Both controllers use the multigrid perspective for the algorithm (first exchange data, than compute updates), but the classical way of determining when to stop locally (each time-step is stopped when ready, if the previous one is ready, too). The complete multigrid behavior can be restored using a flag. All included projects and tutorials have been adapted to this.

    • No more data types in the front-ends: The redundant use of data type specifications in the description dictionaries has been removed. Data types are now declared within each problem class (more precisely, in the header of the __init__-method to allow inhertiance). All included projects and tutorials have been adapted to this.

    • Renewed FEniCS support: This release revives the deprecated FEniCS support, now requiring at least FEniCS 2018.1. The integration is tested using Travis-CI.

    • More consistent handling of local initial conditions: The treatment of u[0] and f[0] has been fixed and made consistent throughout the code.

    • As usual, many bugs have been discovered and fixed.

  • May 23, 3018: Version 2.4 adds support for petsc4py! You can now use PETSc data types (pySDC ships with DMDA for distributed structured grids) and parallel solvers right from your examples and problem classes. There is also a new tutorial (7.C) showing this in a bit more detail, including communicator splitting for parallelization in space and time. Warning: in order to get this to work you need to install petsc4py and mpi4py first! Make sure both use MPICH3 bindings. Downloading pySDC from PyPI does not include these packages.

  • February 8, 2018: Ever got annoyed at pySDC’s incredibly slow setup phase when multiple time-steps are used? Version 2.3 changes this by copying the data structure of the first step to all other steps using the dill Package. Setup times could be reduced by 90% and more for certain problems. We also increase the speed for certain calculations, in particular for the Penning trap example.

  • November 7, 2017: Version 2.2 contains matrix-based versions of PFASST within the project matrixPFASST. This involved quite a few changes in more or less unexpected places, e.g. in the multigrid controller and the transfer base class. The impact of these changes on other projects should be negligible, though.

  • October 25, 2017: For the 6th Workshop on Parallel-in-Time Integration pySDC has been updated to version 2.1. It is now available on PyPI - the Python Package Index, see https://pypi.python.org/pypi/pySDC and can be installed simply by using pip install pySDC. Naturally, this release contains a lot of bugfixes and minor improvements. Most notably, the file structure has been changed again to meet the standards for Python packaging (at least a bit).

  • November 24, 2016: Released version 2 of pySDC. This release contains major changes to the code and its structure:

    • Complete redesign of code structure: The core part of pySDC only contains the core modules and classes, while implementations contains the actual implementations necessary to run something. This now includes separate files for all collocation classes, as well as a collection of problems, transfer classes and so on. Most examples have been ported to either tutorials, playgrounds or projects.

    • Introduction of tutorials: We added a tutorial (see below) to explain many of pySDC’s features in a step-by-step fashion. We start with a simple spatial discretization and collocation formulations and move step by step to SDC, MLSDC and PFASST. All tutorials are accompanied by tests.

    • New all-inclusive controllers: Instead of having two PFASST controllers which could also do SDC and MLSDC (and more), we now have four generic controllers which can do all these methods, depending on the input. They are split into two by two class: MPI and NonMPI for real or virtual parallelisim as well as classic and multigrid for the standard and multigrid-like implementation of PFASST and the likes. Initialization has been simplified a lot, too.

    • Collocation-based coarsening As the standard PFASST libraries libpfasst and PFASST++ pySDC now offers collocation-based coarsening, i.e. the number of collocation nodes can be reduced during coarsening. Also, time-step coarsening is in preparation, but not implemented yet.

    • Testing and documentation The core, implementations and plugin packages and their subpackages are fully documented using sphinx-apidoc, see below. This documentation as well as this website are generated automatically using Travis-CI. Most of the code is supported by tests, mainly realized by using the tutorial as the test routines with clearly defined results. Also, projects are accompanied by tests.

    • Further, minor changes:

      • Switched to more stable barycentric interpolation for the quadrature weights

      • New collocation class: EquidistantSpline_Right for spline-based quadrature

      • Collocation tests are realized by generators and not by classes

      • Multi-step SDC (aka single-level PFASST) now works as expected

      • Reworked many of the internal structures for consistency and simplicity

Tutorial

To flatten the steep learning curve of SDC and the likes, this tutorial contains eight steps to introduce, demonstrate and briefly discuss the main features of pySDC. We start with a simple problem setup, add the collocation problem, solve this via SDC, MLSDC and finally PFASST and discuss further extensions. Along the way we highlight special features of pySDC and explain how different features or implementations can be used (or disabled).

All tutorials are supported by tests and are run automatically after pushing to master. The results are shown within the description and can be found here (yes, yes, Jupyter notebooks would be awesome, just need to find someone to do it):

Playgrounds

This folder gathers various small to medium experiments done with pySDC. They are for example grouped by application (e.g. Boris) or by problem type (e.g. ODEs). Also, the deprecated folder lists a few old applications which have not been ported to the current version of pySDC. Experiments which become more complex or which eventually evolve into a publication of any form should go to the projects folder and should receive proper documentation and description. The playground codes are supposed to be cleaned up, but there is no detailed documentation available.

Projects

In the projects folder we gather focussed experiments with pySDC which go beyond simple toy problems or feature tests. In particular, projects are structured and documented codes which form the basis for publications or presentations. All projects are expected to contain a README file describing the intention and content of the work. Parts of the Python scripts are supported by tests and checked for PEP8 conformity.

Tests

Most of the code is supported by tests, mainly realized by using the tutorial as the test routines with clearly defined results. Also, projects are accompanied by tests.

Github Action reports: badge, test coverage results can be found here.

API documentation

The Python files contained in core, implementations and helpers are fully documented and their API is generated automatically with sphinx-apidoc after each update:

Indices and tables