Coverage for pySDC/tutorial/step_6/C_MPI_parallelization.py: 100%

1# --- 

2# jupyter: 

3# jupytext: 

4# formats: py:percent 

5# kernelspec: 

6# display_name: Python 3 

7# name: python3 

8# language_info: 

9# name: python 

10# --- 

11 

12# %% [markdown] 

13# # Part C: MPI parallelization 

14# 

15# Since PFASST is actually a parallel algorithm, executing it in parallel, e.g. with MPI, might be an interesting 

16# exercise. For this, pySDC comes with the MPI-parallel controller `controller_MPI`. It is supposed to yield the same 

17# results as its non-MPI counterpart, and this is what we demonstrate here, for one particular example: the code is the 

18# same as in [Parts A](A_run_non_MPI_controller) and [B](B_odd_temporal_distribution), with the parameters from 

19# `pfasst_setup.py`, but with `controller_MPI` instead of the non-MPI controller. 

20# 

21# Run it as you would run any MPI program, with one rank per parallel step: 

22# 

23# ```bash 

24# mpirun -np 4 python C_MPI_parallelization.py 

25# ``` 

26# 

27# The number of parallel steps is simply the size of `MPI.COMM_WORLD`, so there is nothing to configure: run it on 4 

28# ranks for 4 parallel steps, on 3 for 3, and so on. 

29 

30# %% 

31from pathlib import Path 

32 

33from mpi4py import MPI 

34 

35from pySDC.helpers.stats_helper import get_sorted 

36from pySDC.implementations.controller_classes.controller_MPI import controller_MPI 

37from pySDC.tutorial.step_6.pfasst_setup import set_parameters_ml 

38 

39# set MPI communicator 

40comm = MPI.COMM_WORLD 

41 

42# get the parameters of Parts A and B 

43description, controller_params, t0, Tend = set_parameters_ml() 

44 

45# instantiate the controller, which holds just one step: the one of this rank 

46controller = controller_MPI(controller_params=controller_params, description=description, comm=comm) 

47 

48# get initial values on finest level 

49P = controller.S.levels[0].prob 

50uinit = P.u_exact(t0) 

51 

52# call main functions to get things done... 

53uend, stats = controller.run(u0=uinit, t0=t0, Tend=Tend) 

54 

55# %% [markdown] 

56# Each rank only has the statistics of its own steps, so they are gathered on rank 0, which prints them, in the same 

57# format as Parts A and B, and writes them to a file for the comparison. 

58 

59# %% 

60# filter statistics by type (number of iterations) 

61iter_counts = get_sorted(stats, type='niter', sortby='time') 

62 

63# combine statistics into list of statistics 

64iter_counts_list = comm.gather(iter_counts, root=0) 

65 

66rank = comm.Get_rank() 

67size = comm.Get_size() 

68 

69if rank == 0: 

70 Path("data").mkdir(parents=True, exist_ok=True) 

71 f = open(f'data/step_6_C_np{size}.txt', 'w') 

72 out = 'Working with %2i processes...' % size 

73 f.write(out + '\n') 

74 print(out) 

75 

76 # compute exact solutions and compare with both results 

77 uex = P.u_exact(Tend) 

78 err = abs(uex - uend) 

79 

80 out = 'Error vs. exact solution: %12.8e' % err 

81 f.write(out + '\n') 

82 print(out) 

83 

84 # build one list of statistics instead of list of lists, the sort by time 

85 iter_counts_gather = [item for sublist in iter_counts_list for item in sublist] 

86 iter_counts = sorted(iter_counts_gather, key=lambda tup: tup[0]) 

87 

88 # compute and print statistics 

89 for item in iter_counts: 

90 out = 'Number of iterations for time %4.2f: %1i ' % (item[0], item[1]) 

91 f.write(out + '\n') 

92 print(out) 

93 

94 f.write('\n') 

95 print() 

96 f.close() 

97 

98 assert all(item[1] <= 8 for item in iter_counts), "ERROR: weird iteration counts, got %s" % iter_counts 

99 

100# %% [markdown] 

101# ## Results 

102# 

103# Our CI runs this on 1, 2, 4 and 8 ranks, as in Part A, and on 3, 5, 7 and 9 ranks, as in Part B, and checks that 

104# the MPI controller needs the same iterations and reaches the same accuracy as the non-MPI one, up to machine 

105# precision. These are the results of the CI run that built this page, first for the even distributions: 

106# 

107# :::{literalinclude} /../../data/step_6_C1_out.txt 

108# :language: text 

109# ::: 

110# 

111# And for the odd ones: 

112# 

113# :::{literalinclude} /../../data/step_6_C2_out.txt 

114# :language: text 

115# ::: 

116# 

117# :::{admonition} Important things to note 

118# - This example also shows how the statistics of multiple MPI processes can be gathered and processed by rank 0. 

119# - The controller needs a working installation of `mpi4py`. Since this is not always easy to achieve, and since 

120# debugging a parallel program can cause a lot of headaches, the non-MPI controller performs the same operations in 

121# serial. 

122# - The test that covers this part runs the same file on 1, 2, 3, 4, 5, 7, 8 and 9 ranks through 

123# [mpi-pytest](https://github.com/firedrakeproject/mpi-pytest), which is also how the rest of pySDC's MPI tests are 

124# run. 

125# :::