Hdf5

Pour faire fonctionner les tests, voir les fichiers sources :

version 1.10.6 parallèle (module lib/hdf5/intel18/1.10.6)

  • version compilée avec mpt et intel compiler version 18
  • Module associé : module add lib/hdf5/intel18/1.10.6
[thor ~]$ module add lib/hdf5/intel18/1.10.6
[thor ~]$ module li
Currently Loaded Modulefiles:
  1) intel-cc-18/18.2.199      3) mpt/2.12
  2) intel-fc-18/18.2.199      4) lib/hdf5/intel18/1.10.6
  • vérifier que la variable MPICC_CC soit égale à icc (ou faire export MPICC_CC=icc).
  • utiliser mpirun systématiquement (en test/interactif ou mpiexec_mpt si utilisation avec pbs) avant tous les binaires (du fait de l'utilisation de la lib mpt).
  • Enfin, l'usage des I/O MPI avec le système de fichier LUSTRE fait qu'il n'est pas possible d'accéder au fichier sur le noeud de login, il faut pour cela accéder à un noeud de calcul avec un job interactif (qsub -I ...) :
# FORTRAN EXAMPLE 
[homer@thor code_bench]$ module add lib/hdf5/intel18/1.10.6
[homer@thor code_bench]$ h5pfc hdf5_test.f90
ifort: warning #10145: no action performed for file '/usr/lib64/libcpuset.so.1'
ifort: warning #10145: no action performed for file '/usr/lib64/libbitmask.so.1'
[@thor code_bench]$ qsub -I -q express -l walltime=01:00:00 -l select=1:ncpus=20
qsub: waiting for job 3072497.thor to start
qsub: job 3072497.thor ready

Start Prologue v2.5.3 Wed Nov 18 09:21:32 CET 2020
End Prologue v2.5.3 Wed Nov 18 09:21:32 CET 2020

cd /home//pbs.3072497.thor.x8z
[homer@r2i1n4 ~]$ cd /home/gueguenm/pbs.3072497.thor.x8z
[homer@r2i1n4 pbs.3072497.thor.x8z]$ cd $PBS_O_WORKDIR
[homer@r2i1n4 code_bench]$ module add lib/hdf5/intel18/1.10.6
[homer@r2i1n4 code_bench]$ mpirun -n 1 ./a.out
 Using filename =
 sds.h5

[homer@r2i1n4 code_bench]$ mpirun -n 1  h5ls -d sds.h5
IntArray                 Dataset {8, 5}
    Data:
        (0,0) 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
        (3,4) 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34,
        (7,0) 35, 36, 37, 38, 39

# EXAMPLE WITH c code
[@thor code_bench]$ h5pcc -o ctest_mpt hdf5_test.c
icc: warning #10145: no action performed for file '/usr/lib64/libcpuset.so.1'
icc: warning #10145: no action performed for file '/usr/lib64/libbitmask.so.1'
[@thor code_bench]$ ldd ctest_mpt
    linux-vdso.so.1 =>  (0x00007fffedae6000)
    libz.so.1 => /lib64/libz.so.1 (0x00007fffed8b9000)
    libdl.so.2 => /lib64/libdl.so.2 (0x00007fffed6b4000)
    libm.so.6 => /lib64/libm.so.6 (0x00007fffed430000)
    libmpi.so => /sw/sgi/mpt/mpt-2.12/lib/libmpi.so (0x00007fffed0a9000)
    libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fffece8b000)
    libcpuset.so.1 => /usr/lib64/libcpuset.so.1 (0x00007fffecc7e000)
    libbitmask.so.1 => /usr/lib64/libbitmask.so.1 (0x00007fffeca7b000)
    libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fffec864000)
    libc.so.6 => /lib64/libc.so.6 (0x00007fffec4d0000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fffedae7000)
    librt.so.1 => /lib64/librt.so.1 (0x00007fffec2c8000)
[@thor code_bench]$ qsub -I  -l select=2:ncpus=20:mpiprocs=20:ompthreads=1 -l walltime=10:00:00
qsub: waiting for job 3090171.thor to start
qsub: job 3090171.thor ready

Start Prologue v2.5.3 Wed Jan 20 12:10:01 CET 2021
End Prologue v2.5.3 Wed Jan 20 12:10:02 CET 2021

cd /home//pbs.3090171.thor.x8z
[@r1i5n2 ~]$ cd /home/gueguenm/pbs.3090171.thor.x8z
[@r1i5n2 pbs.3090171.thor.x8z]$ cd $PBS_O_WORKDIR
[@r1i5n2 code_bench]$ module purge
[@r1i5n2 code_bench]$ module add lib/hdf5/intel18/1.10.6
[@r1i5n2 code_bench]$ mpiexec_mpt -n 2 ./ctest_mpt # will create SDS_row file
[@r1i5n2 code_bench]$ mpirun -n 1 h5ls SDS_row.h5
IntArray                 Dataset {8, 5}

version 1.10.6 parallèle (module lib/hdf5/gcc_imp18/1.10.6)

  • librairie hdf5 avec intelmpi18 + gcc (parallèle/fortran
  • Module associé : module add lib/hdf5/gcc_imp18/1.10.6
  • mpirun non nécessaire avant les binaires crées (h5ls...), mais le "blocage" est identique avec l'utilisation du binaire sur le noeud de login :
[@thor code_bench]$ ./a.out
Using filename = sds.h5
This requires fcntl(2) to be implemented. As of 8/25/2011 it is not. Generic MPICH Message: File locking failed in ADIOI_Set_lock(fd 3,cmd F_SETLKW/7,type F_RDLCK/0,whence 0) with return value FFFFFFFF and errno 26.
- If the file system is NFS, you need to use NFS version 3, ensure that the lockd daemon is running on all the machines, and mount the directory with the 'noac' option (no attribute caching).
- If the file system is LUSTRE, ensure that the directory is mounted with the 'flock' option.
ADIOI_Set_lock:: Function not implemented
ADIOI_Set_lock:offset 0, length 1
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0
[unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1
:
system msg for write_line failure : Bad file descriptor
HDF5: infinite loop closing library
L,T_top,P,P,FD,E,SL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL,FL
  • usage :
[@thor code_bench]$ module purge
[@thor code_bench]$ module add lib/hdf5/gcc_imp18/1.10.6
[@thor code_bench]$ which h5pfc
/sw/lib/hdf5/1.10.6/impi18/bin/h5pfc
[@thor code_bench]$ h5pfc hdf5_test.f90
[@thor code_bench]$ qsub -I -q express -l walltime=01:00:00 -l select=1:ncpus=20
qsub: waiting for job 3072502.thor to start
qsub: job 3072502.thor ready

Start Prologue v2.5.3 Wed Nov 18 09:29:46 CET 2020
End Prologue v2.5.3 Wed Nov 18 09:29:47 CET 2020

cd /home//pbs.3072502.thor.x8z
[@r2i1n4 ~]$ cd /home//pbs.3072502.thor.x8z
[@r2i1n4 pbs.3072502.thor.x8z]$ cd $PBS_O_WORKDIR
[@r2i1n4 code_bench]$ module purge
[@r2i1n4 code_bench]$ module add lib/hdf5/gcc_imp18/1.10.6
[@r2i1n4 code_bench]$ mpirun -n 2 ./a.out
Using filename = sds.h5
Using filename = sds.h5
[@r2i1n4 code_bench]$ which h5ls
/sw/lib/hdf5/1.10.6/impi18/bin/h5ls
[@r2i1n4 code_bench]$ h5ls -d sds.h5
IntArray Dataset {8, 5}
Data:
(0,0) 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31,
(6,2) 32, 33, 34, 35, 36, 37, 38, 39

hdf5_test.c Magnifier - example parallel HDF5 c (3.46 KB) Gueguen Mikael, 01/20/2021 12:15 PM

hdf5_test.f90 - example parallel HDF5 fortran (4.58 KB) Gueguen Mikael, 01/20/2021 12:15 PM