Parallel I/O and Portable Data Formats @ JSC

CET
Computer Lab 1, building 16.3, r. 213a (Jülich Supercomputing Centre)

Computer Lab 1, building 16.3, r. 213a

Jülich Supercomputing Centre

Forschungszentrum Jülich Jülich Supercomputing Centre 52425 Jülich Germany
Description

Numerical simulations conducted on current high-performance computing (HPC) systems face an ever growing need for scalability. Larger HPC platforms provide opportunities to push the limitations on size and properties of what can be accurately simulated. Therefore, it is needed to process larger data sets, be it reading input data or writing results. Serial approaches on handling I/O in a parallel application will dominate the performance on massively parallel systems, leaving a lot of computing resources idle during those serial application phases.

In addition to the need for parallel I/O, input and output data is often processed on different platforms. Heterogeneity of platforms can impose a high level of maintenance, when different data representations are needed. Portable, selfdescribing data formats such as HDF5 and netCDF are examples of already widely used data formats within certain communities.

This course will start with an introduction to the basics of I/O, including basic I/O-relevant terms, an overview over parallel file systems with a focus on GPFS, and the HPC hardware available at JSC. Different I/O strategies will be presented. The course will introduce the use of the HDF5, the NetCDF and the SIONlib library interfaces as well as MPI-I/O. Optimization potential and best practices are discussed.

Instructors: Sebastian Lührs, Benedikt Steinbusch, Jülich Supercomputing Centre

Contact
For any questions concerning the course please send an e-mail to s.luehrs@fz-juelich.de.

    • 09:00 10:30
      Introduction 1h 30m Computer Lab 1, building 16.3, r. 213a

      Computer Lab 1, building 16.3, r. 213a

      Jülich Supercomputing Centre

      Forschungszentrum Jülich Jülich Supercomputing Centre 52425 Jülich Germany
    • 10:30 11:00
      Break 30m Computer Lab 1, building 16.3, r. 213a

      Computer Lab 1, building 16.3, r. 213a

      Jülich Supercomputing Centre

      Forschungszentrum Jülich Jülich Supercomputing Centre 52425 Jülich Germany
    • 11:00 12:00
      Introduction 1h Computer Lab 1, building 16.3, r. 213a

      Computer Lab 1, building 16.3, r. 213a

      Jülich Supercomputing Centre

      Forschungszentrum Jülich Jülich Supercomputing Centre 52425 Jülich Germany
    • 12:00 12:30
      Parallel NetCDF 30m Computer Lab 1, building 16.3, r. 213a

      Computer Lab 1, building 16.3, r. 213a

      Jülich Supercomputing Centre

      Forschungszentrum Jülich Jülich Supercomputing Centre 52425 Jülich Germany
    • 12:30 13:30
      Break 1h Casino

      Casino

    • 13:30 14:30
      Parallel NetCDF 1h Computer Lab 1, building 16.3, r. 213a

      Computer Lab 1, building 16.3, r. 213a

      Jülich Supercomputing Centre

      Forschungszentrum Jülich Jülich Supercomputing Centre 52425 Jülich Germany
    • 14:30 15:00
      Break 30m Computer Lab 1, building 16.3, r. 213a

      Computer Lab 1, building 16.3, r. 213a

      Jülich Supercomputing Centre

      Forschungszentrum Jülich Jülich Supercomputing Centre 52425 Jülich Germany
    • 15:00 16:30
      Parallel NetCDF 1h 30m Computer Lab 1, building 16.3, r. 213a

      Computer Lab 1, building 16.3, r. 213a

      Jülich Supercomputing Centre

      Forschungszentrum Jülich Jülich Supercomputing Centre 52425 Jülich Germany
    • 09:00 10:30
      HDF5 1h 30m Computer Lab 1, building 16.3, r. 213a

      Computer Lab 1, building 16.3, r. 213a

      Jülich Supercomputing Centre

      Forschungszentrum Jülich Jülich Supercomputing Centre 52425 Jülich Germany
    • 10:30 11:00
      Break 30m Computer Lab 1, building 16.3, r. 213a

      Computer Lab 1, building 16.3, r. 213a

      Jülich Supercomputing Centre

      Forschungszentrum Jülich Jülich Supercomputing Centre 52425 Jülich Germany
    • 11:00 12:30
      HDF5 1h 30m Computer Lab 1, building 16.3, r. 213a

      Computer Lab 1, building 16.3, r. 213a

      Jülich Supercomputing Centre

      Forschungszentrum Jülich Jülich Supercomputing Centre 52425 Jülich Germany
    • 12:30 13:30
      Break 1h Casino

      Casino

    • 13:30 14:30
      SIONlib 1h Computer Lab 1, building 16.3, r. 213a

      Computer Lab 1, building 16.3, r. 213a

      Jülich Supercomputing Centre

      Forschungszentrum Jülich Jülich Supercomputing Centre 52425 Jülich Germany
    • 14:30 15:00
      Break 30m Computer Lab 1, building 16.3, r. 213a

      Computer Lab 1, building 16.3, r. 213a

      Jülich Supercomputing Centre

      Forschungszentrum Jülich Jülich Supercomputing Centre 52425 Jülich Germany
    • 15:00 16:30
      SIONlib 1h 30m Computer Lab 1, building 16.3, r. 213a

      Computer Lab 1, building 16.3, r. 213a

      Jülich Supercomputing Centre

      Forschungszentrum Jülich Jülich Supercomputing Centre 52425 Jülich Germany
    • 09:00 10:30
      MPI I/O 1h 30m Computer Lab 1, building 16.3, r. 213a

      Computer Lab 1, building 16.3, r. 213a

      Jülich Supercomputing Centre

      Forschungszentrum Jülich Jülich Supercomputing Centre 52425 Jülich Germany
    • 10:30 11:00
      Break 30m Computer Lab 1, building 16.3, r. 213a

      Computer Lab 1, building 16.3, r. 213a

      Jülich Supercomputing Centre

      Forschungszentrum Jülich Jülich Supercomputing Centre 52425 Jülich Germany
    • 11:00 12:30
      MPI I/O 1h 30m Computer Lab 1, building 16.3, r. 213a

      Computer Lab 1, building 16.3, r. 213a

      Jülich Supercomputing Centre

      Forschungszentrum Jülich Jülich Supercomputing Centre 52425 Jülich Germany
    • 12:30 13:30
      Break 1h Casino

      Casino

    • 13:30 14:30
      Discussion/Special Topics 1h Computer Lab 1, building 16.3, r. 213a

      Computer Lab 1, building 16.3, r. 213a

      Jülich Supercomputing Centre

      Forschungszentrum Jülich Jülich Supercomputing Centre 52425 Jülich Germany
    • 14:30 15:00
      Break 30m Computer Lab 1, building 16.3, r. 213a

      Computer Lab 1, building 16.3, r. 213a

      Jülich Supercomputing Centre

      Forschungszentrum Jülich Jülich Supercomputing Centre 52425 Jülich Germany
    • 15:00 16:00
      Discussion/Special Topics 1h Computer Lab 1, building 16.3, r. 213a

      Computer Lab 1, building 16.3, r. 213a

      Jülich Supercomputing Centre

      Forschungszentrum Jülich Jülich Supercomputing Centre 52425 Jülich Germany