helpers.fieldsIO module

Generic utility class to write and read cartesian grid field solutions into binary files. It implements the base file handler class FieldsIO, that is specialized into :

• Scalar : for 0D fields (scalar) with a given number of variables

• Rectilinear : for fields on N-dimensional rectilinear grids

While each file handler need to be setup with specific parameters (grid, …), each written file can be read using the same interface implemented in the base abstract class.

Example

>>> import numpy as np
>>> from pySDC.helpers.fieldsIO import Rectilinear
>>>
>>> # Write some fields in files
>>> x = np.linspace(0, 1, 128)
>>> y = np.linspace(0, 1, 64)
>>> fOut = Rectilinear(np.float64, "file.pysdc")
>>> fOut.setHeader(nVar=2, coords=[x, y])
>>> fOut.initialize()
>>> times = [0, 1, 2]
>>> xGrid, yGrid = np.meshgrid(x, y, indexing="ij")
>>> u0 = np.array([-1, 1]).reshape((-1, 1, 1))*xGrid*yGrid
>>> # u0 has shape [2, nX, nY]
>>> for t in times:
>>>    fOut.addField(t, t*u0)
>>>
>>> # Read the file using the generic interface
>>> from pySDC.helpers.fieldsIO import FieldsIO
>>> fIn = FieldsIO.fromFile("file.pysdc")
>>> times = fIn.times
>>> assert len(times) == fIn.nFields
>>> tEnd, uEnd = fIn.readField(-1)
>>> assert tEnd == times[-1]

Notes

🚀 Rectilinear is compatible with a MPI-based cartesian decomposition. See pySDC.helpers.fieldsIO.writeFields_MPI for an illustrative example.

Warning

To use MPI collective writing, you need to call first the class methods Rectilinear.setupMPI (cf their docstring). Also, Rectilinear.setHeader must be given the global grids coordinates, whether the code is run in parallel or not.

class FieldsIO(dtype, fileName)

Bases: object

Abstract IO file handler

ALLOW_OVERWRITE = False

STRUCTS = {0: <class 'helpers.fieldsIO.Scalar'>, 1: <class 'helpers.fieldsIO.Rectilinear'>}

addField(time, field)

Append one field solution at the end of the file with one given time.

Parameters:

• time (float-like) – The associated time of the field solution.

• field (np.ndarray) – The field values.

property fSize

Full size of a field (in bytes)

property fileSize

Current size of the file (in bytes)

formatIndex(idx)

Utility method to format a fields index to a positional integer (negative starts from last field index, like python lists)

classmethod fromFile(fileName)

Read a file storing fields, and return the FieldsIO of the appropriate field type (structure).

Parameters:

fileName (str) – Name of the binary file.

Returns:

fieldsIO – The specialized FieldsIO adapted to the file.

Return type:

FieldsIO

property hBase: ndarray

Base header into numpy array format

property hInfos: list[ndarray]

(Abstract) Array representing the grid structure to be written in the binary file.

property hSize

Size of the full header (in bytes)

initialize()

Initialize the file handler : create the file with header, removing any existing file with the same name

property itemSize

Size of one field value (in bytes)

property nFields

Number of fields currently stored in the binary file

readField(idx)

Read one field stored in the binary file, corresponding to the given time index.

Parameters:

idx (int) – Positional index of the field.

Returns:

• t (float) – Stored time for this field.

• field (np.ndarray) – Read fields in a numpy array.

readHeader(f)

(Abstract) Read the header from the file storing the fields.

Parameters:

f (_io.TextIOWrapper) – File to read the header from.

classmethod register(sID)

Decorator used to register a new class FieldsIO specialized class

Parameters:

sID (int) – Unique identifyer for the file, used in the binary file. Since it’s encoded on a 8-bytes signed integer, it must be between -128 and 127

Example

>>> # New specialized FieldsIO class
>>> @FieldsIO.register(sID=31)
>>> class HexaMesh2D(FieldsIO):
>>>     pass # ... implementation

reshape(field)

Eventually reshape the field to correspond to the grid structure

sID = None

setHeader(**params)

(Abstract) Set the header before creating a new file to store the fields

tSize = 8

time(idx)

Time stored at a given field index

property times

Vector of all times stored in the binary file

class Rectilinear(dtype, fileName)

Bases: Scalar

FieldsIO handler storing a given number of scalar variables on a N-dimensional rectilinear grid

MPI_FILE_CLOSE()

Close the binary file in MPI mode

MPI_FILE_OPEN(mode)

Open the binary file in MPI mode

property MPI_ON

Wether or not MPI is activated

MPI_READ_AT_ALL(offset, data: ndarray)

Read data from the binary file in MPI mode, with a given offset relative to the beginning of the file.

Parameters:

• offset (int) – Offset to read at, relative to the beginning of the file, in bytes.

• data (np.ndarray) – Array on which to read the data from the binary file.

property MPI_ROOT

Wether or not the process is MPI Root

MPI_SETUP_FILETYPE()

Setup subarray masks for each processes

MPI_WRITE(data)

Write data (np.ndarray) in the binary file in MPI mode, at the current file cursor position.

MPI_WRITE_AT_ALL(offset, data: ndarray)

Write data in the binary file in MPI mode, with a given offset relative to the beginning of the file.

Parameters:

• offset (int) – Offset to write at, relative to the beginning of the file, in bytes.

• data (np.ndarray) – Data to be written in the binary file.

addField(time, field)

Append one field solution at the end of the file with one given time, possibly using MPI.

Parameters:

• time (float-like) – The associated time of the field solution.

• field (np.ndarray) – The (local) field values.

Note

If a MPI decomposition is used, field must be the local field values.

comm: mpi4py.MPI.Intracomm = None

property dim

Number of grid dimensions

property gridSizes

Number of points in y direction

property hInfos

Array representing the grid structure to be written in the binary file.

initialize()

Initialize the binary file (write header) in MPI mode

property nDoF

Number of degrees of freedom for one variable

readField(idx)

Read one field stored in the binary file, corresponding to the given time index, using MPI in the eventuality of space parallel decomposition.

Parameters:

idx (int) – Positional index of the field.

Returns:

• t (float) – Stored time for this field.

• field (np.ndarray) – Read (local) fields in a numpy array.

Note

If a MPI decomposition is used, it reads and returns the local fields values only.

readHeader(f)

Read the header from the binary file storing the fields.

Parameters:

f (_io.TextIOWrapper) – File to read the header from.

reshape(fields: ndarray)

Reshape the fields to a N-d array (inplace operation)

sID = 1

setHeader(nVar, coords)

Set the descriptive grid structure to be stored in the file header.

Parameters:

• nVar (int) – Number of 1D variables stored.

• coords (np.1darray or list[np.1darray]) – The grid coordinates in each dimensions.

Note

When used in MPI decomposition mode, all coordinate must be the global grid.

static setupCoords(*coords)

Utility function to setup grids in multiple dimensions, given the keyword arguments

classmethod setupMPI(comm: mpi4py.MPI.Intracomm, iLoc, nLoc)

Setup the MPI mode for the files IO, considering a decomposition of the 1D grid into contiguous subintervals.

Parameters:

• comm (MPI.Intracomm) – The space decomposition communicator.

• iLoc (list[int]) – Starting index of the local sub-domain in the global coordinates.

• nLoc (list[int]) – Number of points in the local sub-domain.

toVTR(baseName, varNames, idxFormat='{:06d}')

Convert all 3D fields stored in binary format (FieldsIO) into a list of VTR files, that can be read later with Paraview or equivalent to make videos.

Parameters:

• baseName (str) – Base name of the VTR file.

• varNames (list[str]) – Variable names of the fields.

• idxFormat (str, optional) – Formatting string for the index of the VTR file. The default is “{:06d}”.

Example

>>> # Suppose the FieldsIO object is already written into outputs.pysdc
>>> import os
>>> from pySDC.utils.fieldsIO import Rectilinear
>>> os.makedirs("vtrFiles")  # to store all VTR files into a subfolder
>>> Rectilinear.fromFile("outputs.pysdc").toVTR(
>>>    baseName="vtrFiles/field", varNames=["u", "v", "w", "T", "p"])

class Scalar(dtype, fileName)

Bases: FieldsIO

FieldsIO handler storing a given number of scalar

property hInfos

Array representing the grid structure to be written in the binary file.

property nVar

Number of variables in a fields, as described in the header

readHeader(f)

Read the header from the binary file storing the fields.

Parameters:

f (_io.TextIOWrapper) – File to read the header from.

sID = 0

setHeader(nVar)

Set the descriptive grid structure to be stored in the file header.

Parameters:

nVar (int) – Number of scalar variable stored.

compareFields_MPI(fileName, u0, nSteps)

initGrid(nVar, gridSizes)

writeFields_MPI(fileName, dtypeIdx, algo, nSteps, nVar, gridSizes)