Part A: The non-MPI controller#
pySDC comes with (at least) two controllers: the standard, non-MPI controller we have used so far, and the MPI-parallel one. The non-MPI controller runs simulations without having to worry about parallelization and MPI installations. By monitoring the convergence, it can already give a detailed idea of how PFASST will work for a given problem.
We run an unforced heat equation, on a smaller grid than in Step 5, once on a
single level, as SDC, and then with two levels on 1, 2, 4 and 8 processes: MLSDC on one, PFASST on more. The setup is in pfasst_setup.py next to this tutorial, as
Parts B and C use it as well. The parameters of the
multi-level runs:
def set_parameters_ml():
"""
Helper routine to set parameters for the following multi-level runs
Returns:
dict: dictionary containing the simulation parameters
dict: dictionary containing the controller parameters
float: starting time
float: end time
"""
# initialize level parameters
level_params = {}
level_params['restol'] = 5e-10
level_params['dt'] = 0.125
# initialize sweeper parameters
sweeper_params = {}
sweeper_params['quad_type'] = 'RADAU-RIGHT'
sweeper_params['num_nodes'] = [3]
sweeper_params['QI'] = 'LU'
# initialize problem parameters
problem_params = {}
problem_params['nu'] = 0.1 # diffusion coefficient
problem_params['freq'] = 2 # frequency for the test value
problem_params['nvars'] = [63, 31] # number of degrees of freedom for each level
problem_params['bc'] = 'dirichlet-zero' # boundary conditions
# initialize step parameters
step_params = {}
step_params['maxiter'] = 50
step_params['errtol'] = 1e-05
# initialize space transfer parameters
space_transfer_params = {}
space_transfer_params['rorder'] = 2
space_transfer_params['iorder'] = 6
# initialize controller parameters
controller_params = {}
controller_params['logger_level'] = 30
controller_params['all_to_done'] = True # can ask the controller to keep iterating all steps until the end
controller_params['predict_type'] = 'pfasst_burnin' # PFASST's coarse-level predictor
# fill description dictionary for easy step instantiation
description = {}
description['problem_class'] = heatNd_unforced
description['problem_params'] = problem_params
description['sweeper_class'] = generic_implicit
description['sweeper_params'] = sweeper_params
description['level_params'] = level_params
description['step_params'] = step_params
description['space_transfer_class'] = mesh_to_mesh
description['space_transfer_params'] = space_transfer_params
# set time parameters
t0 = 0.0
Tend = 1.0
return description, controller_params, t0, Tend
The single-level ones, set_parameters_sl, are the same problem and sweeper on one level, without the transfer
class and without the controller options above. And this is the loop that runs the controller for each number of
processes, printing the error and the iterations of each step:
def run_pfasst(num_proc_list=None, fname=None, multi_level=True):
"""
Run PFASST with the non-MPI controller, for each number of processes in the list, and write what it prints to
data/<fname> as well, for the comparison with the MPI controller in part C
Args:
num_proc_list: list of number of processes to test with
fname: filename/path for output
multi_level (bool): do multi-level run or single-level
Returns:
dict: the iteration counts of each time step, for each number of processes
"""
if multi_level:
description, controller_params, t0, Tend = set_parameters_ml()
else:
assert all(num_proc == 1 for num_proc in num_proc_list), (
'ERROR: single-level run can only use 1 processor, got %s' % num_proc_list
)
description, controller_params, t0, Tend = set_parameters_sl()
Path("data").mkdir(parents=True, exist_ok=True)
f = open('data/' + fname, 'w')
iterations = {}
# loop over different numbers of processes
for num_proc in num_proc_list:
out = 'Working with %2i processes...' % num_proc
f.write(out + '\n')
print(out)
# instantiate controllers
controller = controller_nonMPI(num_procs=num_proc, controller_params=controller_params, description=description)
# get initial values on finest level
P = controller.MS[0].levels[0].prob
uinit = P.u_exact(t0)
# call main functions to get things done...
uend, stats = controller.run(u0=uinit, t0=t0, Tend=Tend)
# compute exact solution and compare with both results
uex = P.u_exact(Tend)
err = abs(uex - uend)
out = 'Error vs. exact solution: %12.8e' % err
f.write(out + '\n')
print(out)
# filter statistics by type (number of iterations)
iter_counts = get_sorted(stats, type='niter', sortby='time')
# compute and print statistics
for item in iter_counts:
out = 'Number of iterations for time %4.2f: %1i ' % (item[0], item[1])
f.write(out + '\n')
print(out)
f.write('\n')
print()
assert all(item[1] <= 8 for item in iter_counts), "ERROR: weird iteration counts, got %s" % iter_counts
iterations[num_proc] = [item[1] for item in iter_counts]
f.close()
return iterations
The output also goes to a file in data/, for the comparison with the MPI controller in Part C.
import matplotlib.pyplot as plt
from pySDC.tutorial.step_6.pfasst_setup import run_pfasst
iterations_sl = run_pfasst(num_proc_list=[1], fname='step_6_A_sl_out.txt', multi_level=False)
Working with 1 processes...
Error vs. exact solution: 2.87627033e-07
Number of iterations for time 0.00: 8
Number of iterations for time 0.12: 8
Number of iterations for time 0.25: 8
Number of iterations for time 0.38: 8
Number of iterations for time 0.50: 7
Number of iterations for time 0.62: 7
Number of iterations for time 0.75: 7
Number of iterations for time 0.88: 7
And with two levels, PFASST:
iterations_ml = run_pfasst(num_proc_list=[1, 2, 4, 8], fname='step_6_A_ml_out.txt', multi_level=True)
Working with 1 processes...
Error vs. exact solution: 2.87300679e-07
Number of iterations for time 0.00: 4
Number of iterations for time 0.12: 4
Number of iterations for time 0.25: 3
Number of iterations for time 0.38: 3
Number of iterations for time 0.50: 3
Number of iterations for time 0.62: 3
Number of iterations for time 0.75: 3
Number of iterations for time 0.88: 3
Working with 2 processes...
Error vs. exact solution: 2.87272106e-07
Number of iterations for time 0.00: 4
Number of iterations for time 0.12: 4
Number of iterations for time 0.25: 4
Number of iterations for time 0.38: 4
Number of iterations for time 0.50: 4
Number of iterations for time 0.62: 4
Number of iterations for time 0.75: 4
Number of iterations for time 0.88: 4
Working with 4 processes...
Error vs. exact solution: 2.87294206e-07
Number of iterations for time 0.00: 5
Number of iterations for time 0.12: 5
Number of iterations for time 0.25: 5
Number of iterations for time 0.38: 5
Number of iterations for time 0.50: 5
Number of iterations for time 0.62: 5
Number of iterations for time 0.75: 5
Number of iterations for time 0.88: 5
Working with 8 processes...
Error vs. exact solution: 2.87290945e-07
Number of iterations for time 0.00: 7
Number of iterations for time 0.12: 7
Number of iterations for time 0.25: 7
Number of iterations for time 0.38: 7
Number of iterations for time 0.50: 7
Number of iterations for time 0.62: 7
Number of iterations for time 0.75: 7
Number of iterations for time 0.88: 7
<matplotlib.colorbar.Colorbar at 0x7f3d0fae4050>
Important things to note
If you don’t want to deal with parallelization and/or are only interested in SDC, MLSDC or the convergence of PFASST, use the non-MPI controller.
If you care about parallelization, use the MPI controller, see Part C.
The check the tests run is inside run_pfasst: no step needs more than 8 iterations.