Parallel filesystems and parallel IO libraries @MdS

Feb 13, 2014, 9:00 AM → Feb 14, 2014, 5:30 PM CET

Maison de la Simulation

The increase in computational power goes hand in hand with an increase in the size of the data to be managed, both on the input and the output sides. IO can easily become a bottleneck for large scale architectures. 

Even to make efficient of the existing high level libraires (like Netcdf or HDF), users need to be aware of how the underlying parallel file system works, and of the interactions between the different layers of the IO system.

This course is an introoduction to the issues that must be taken i nto account when doing I/O on high end parallel systems. A practical example of a library used in a large climate simulation code will be presented.

Topics:

• MPI-IO
• Best practices for parallel IO, MPI-IO hints
• Parallel file systems: Lustre, GPFS
• High level IO libraries (NetCDF, HDF5, XIOS)

Instructors: J.Gaidamour (Idris), Ph. Wautelet (Idris), Ph. Deniel (TGCC), Y. Meurdesoif (LSCE), F. Bothorel (IBM), M. Haefele (HLST)

Learning outcomes: After this course, participants should understand the tradeoffs implied by using a parallel file-system, and know how to efficiently use parallel IO libraries.

Prerequisites: Knowledge of C or Fortran programming language, parallel programing with MPI