Coverage for pySDC/tutorial/step_9/E_paradiag_MPI.py: 100%

1# --- 

2# jupyter: 

3# jupytext: 

4# formats: py:percent 

5# kernelspec: 

6# display_name: Python 3 

7# name: python3 

8# language_info: 

9# name: python 

10# --- 

11 

12# %% [markdown] 

13# # Part E: MPI-parallel ParaDiag 

14# 

15# Parts A to D all ran ParaDiag with the "virtually parallel" controller, which keeps every step in a single process. 

16# That is what you want while developing, but it does not actually run in parallel. This part uses 

17# `controller_ParaDiag_MPI` instead, with one time step per rank and the communicator spanning the block that is 

18# diagonalized. 

19# 

20# Nothing about the method changes: the description, the controller parameters and the $\alpha$ settings are the 

21# ones [Part D](D_adaptive_alpha) already used. In fact `run` there takes a communicator, and passing one is the 

22# entire difference, so you can develop a setup serially and then run it in parallel without touching it. 

23# 

24# Run it the way you would run any MPI program, with one rank per time step: 

25# 

26# ```bash 

27# mpirun -np 4 python E_paradiag_MPI.py 

28# ``` 

29# 

30# We always integrate the same total number of time steps and only vary how many of them run in parallel, so the 

31# number of ranks is the block size. With four steps in total and a block size of one, two or four, the controller 

32# windows through four, two or one block respectively. 

33 

34# %% 

35from pathlib import Path 

36 

37from mpi4py import MPI 

38 

39from pySDC.tutorial.step_9.paradiag_setup import alpha_settings, format_result, run 

40 

41comm = MPI.COMM_WORLD 

42 

43lines = [] 

44for alpha in alpha_settings: 

45 uend, niter, error, final_alpha = run(alpha, comm.size, comm=comm) 

46 # the block size goes in the label, so that the results of all block sizes can be put side by side 

47 lines.append(format_result(f'MPI on {comm.size}', alpha, niter, error, final_alpha)) 

48 

49# only the last rank holds the end point of the block, so only it writes the output 

50if comm.rank == comm.size - 1: 

51 Path("data").mkdir(parents=True, exist_ok=True) 

52 with open(f'data/step_9_E_np{comm.size}.txt', 'w') as f: 

53 for line in lines: 

54 f.write(line + '\n') 

55 print(line) 

56 

57# %% [markdown] 

58# ## Results 

59# 

60# Our CI runs this on one, two and four ranks. These are the results of the CI run that built this page: 

61# 

62# :::{literalinclude} /../../data/step_9_E_out.txt 

63# :language: text 

64# ::: 

65# 

66# And these are the ones of [Part D](D_adaptive_alpha), with the virtually parallel controller on one block of four 

67# steps: 

68# 

69# :::{literalinclude} /../../data/step_9_D_out.txt 

70# :language: text 

71# ::: 

72# 

73# :::{admonition} Important things to note 

74# - All steps of a block iterate together. In PFASST an early step can converge and drop out, which is what makes it 

75# pipelined; ParaDiag cannot do that, because the transform in time needs every step. A step that stopped early 

76# would leave the others waiting. 

77# - Consequently the block is always full. If the end time does not divide into whole blocks, ParaDiag solves past 

78# it rather than truncating, and says so. 

79# - At a given block size the MPI and the virtually parallel controllers must agree exactly, which is the comparison 

80# against Part D. 

81# - Windowing does not change what is being solved. Where the iteration count comes out the same the answers agree 

82# to about $10^{-11}$; where it does not (a fixed $\alpha$ of $10^{-2}$ is loose enough that a larger block costs 

83# one extra iteration) the two runs stop at slightly different residuals, and their errors differ by about 

84# $2 \cdot 10^{-8}$, three orders inside the discretisation error. 

85# - Adaptive $\alpha$ does not notice the block size either, but for a more interesting reason: it picks a 

86# *different* $\alpha$ for each one, because $\gamma$ scales with the number of steps in the block, and still 

87# converges in the same number of iterations. 

88# :::