R195/clusters/user8b2b21cc8ff2master
Readme.md
SCITAS users benchmarks
This page describes the users benchmakrs than SCITAS runs on their machines and when testing new architectures. These benchmarks are also used for tenders.
You need Intel Compilers and MKL, as well as Intel MPI to run these benchmarks.
Settings, general
Settings are recorded in the file cores.txt. List there on how many cores you want to run.
Settings, scheduler
Settings for the scheduler can be found in ../cluster.job. Put in this file what is between the SLURM directives and the actual run command. An example is provided.
Quantum Espresso
Quantum Espresso (hereafter abbreviated by QE) stands for opEn Source Package for Research in Electronic Structure, Simulation, and Optimization. It is an integrated suite of computer codes for electronic-structure calculations and materials modeling at the nanoscale. It is based on Density-Functional Theory (DFT), using a Plane-Wave basis set and pseudo-potentials.
More details can be found at http://www.quantum-espresso.org
The code is written in Fortran 95.
Two kinds of simulation are performed in this benchmark :
- Car-Parrinello Molecular Dynamics (CP package)
- Born-Oppenheimer Molecular Dynamics (PWscf package)
How to compile
- Enter working directory and configure
cd espresso-5.3.0/ ./configure MPIF90=mpiifort F90=ifort --enable-parallel --with-scalapack=intel
It is important to be sure that the program finds the MKL libraries. Have a look at the make.sys file.
- You can compile only the parts of Quantum Espresso which are needed for this benchmark with
make cp pw
- After the compilation successfully exits, you can find the binaries in the espresso-5.3.0/bin directory
Execution
- Go to the QE directory
cd ../QE
- if you want to change the input of the program (problem size for example), you should modify PW/generate-pw-test and CP/generate-cp-test. For example, you can change the size of the problem by setting the ecutwfc to 70.0 for a run using 64 GB of RAM.
- have a look at the runme.sh (for interactive runs) or runme_slurm.sh (for runs with the SLURM scheduler). The only part you should want to edit are the environment variables at the top.
- depending if you are on a cluster of on a machine in interactive mode, run runme.sh or runme_slurm.sh.
GEAR
GEAR is a fully parallel code for cosmological N-body simulations. It is based on the freely available code GADGET-2 (version 2.0) that it supplements with the complex treatment of the baryon physics (Y. Revaz and P. Jablonka, Astronomy & Astrophysics, 2012). It computes gravitational forces with a hierarchical tree algorithm (optionally in combination with a particle-mesh scheme for long-range gravitational forces) and represents fluids by means of smoothed particle hydrodynamics (SPH).
It is written in C and parallelized with the MPI communication interface.
For this benchmark, higher CPU frequency provides higher performance. Our test cases produce a large amount of data. Performances also depend on the speed of the storage.
Please note that the GEAR benchmark depends on the FFTW and GSL libraries.
Compiling
We will use in this example the gsl-1.15 and fftw-2.1.5 libraries. In order to compile these pre-requisites, do the following:
- pick a prefix where to install the libraries
export PREFIX=/opt
- configure, compile, install
export CC=icc export CPP="icpc -E" export F77=ifort export I_MPI_CC=icc export MPICH_CC=icc export MPICC=mpiicc wget http://www.fftw.org/fftw-2.1.5.tar.gz tar -zxf fftw-2.1.5.tar.gz cd fftw-2.1.5 ./configure --prefix=$PREFIX/fftw-2.1.5 --enable-type-prefix --enable-mpi --enable-float CFLAGS="-O3 -xHost" FFLAGS="-O3 -xHost" make sudo make install cd .. wget http://mirror.switch.ch/ftp/mirror/gnu/gsl/gsl-1.15.tar.gz tar -zxf gsl-1.15.tar.gz cd gsl-1.15/ ./configure --prefix=$PREFIX/gsl-1.15 CFLAGS="-O3 -xHost" FFLAGS="-O3 -xHost" make sudo make install
Now, you can compile GEAR. Enter the GEAR directory. In the src directory, edit the Makefile in the following way:
- look for the string modify to suit your machine and verify the variables
- type make
- you should end up with a binary called Gadget2
Execution
- choose the input file. A small problem (32 GB of RAM) is available here: http://scitas-data.epfl.ch/benchmarks/inputs/snap-118.dat.bz2. A larger problem (64 GB of RAM) is available here: http://scitas-data.epfl.ch/benchmarks/inputs/snap-284.dat.bz2. Download the input file to the GEAR directory, decompress it and rename it (or link it to) snap.dat.
- have a look at the runme.sh (for interactive runs) or runme_slurm.sh (for runs with the SLURM scheduler). The only part you should want to edit are the environment variables at the top.
- make sure the GEAR directory is clean. No dIrr-* directory from previous runs must be present
- execute runme.sh or runme_slurm.sh
CPMD
CPMD (Car-Parrinello Molecular Dynamics) is an ab-initio molecular dynamics simulation software that provides a way to perform molecular dynamic simulations from first principles, using a plane wave/pseudopotential implementation of Density Functional Theory. Its development is coordinated by the CPMD consortium (http://www.cpmd.org/).
It is written in Fortran 90 and parallelized with MPI.
The CPMD kernel can perform:
- Car-Parrinello Molecular Dynamics
- Ab-initio Molecular Dynamics
- Response calculations
- Metadynamics
- QM/MM
This code is memory bound.
Compilation
- Enter the CPMD directory
- Type make
Execution
- have a look at the runme.sh (for interactive runs) or runme_slurm.sh (for runs with the SLURM scheduler). The only part you should want to edit are the environment variables at the top.
- execute runme.sh or runme_slurm.sh
MiniFE
MiniFE is a Finite Element mini-application which implements a couple of kernels representative of implicit finite-element applications. It assembles a sparse linear-system from the steady-state conduction equation on a brick-shaped problem domain of linear 8-node hex elements. It then solves the linear-system using a simple un-preconditioned conjugate-gradient algorithm.
MiniFE contains kernels responsible for :
- computation of element-operators (diffusion matrix, source vector)
- assembly (scattering element-operators into sparse matrix and vector)
- sparse matrix-vector product (during CG solve)
- vector operations (level-1 blas: axpy, dot, norm)
More details can be found at http://www.nersc.gov/users/computational-systems/cori/nersc-8-procurement/trinity-nersc-8-rfp/nersc-8-trinity-benchmarks/minife
The code is written in C++ and parallelized with MPI.
Compilation
Enter the miniFE-1.4 directory and compile with:
make -f makefile.intel.mpi
Execution
The problem sizes you want to run must be specified in the ../miniFE-sizes.txt file. Then, you can have a look at the runme.sh (for interactive runs) or runme_slurm.sh (for runs with the SLURM scheduler), and run with either runme.sh or runme_slurm.sh.
Post-processing
A post-processing script post.sh is available in each benchmark directory. A post-processing script post.sh in this directory takes care of calling all post-processing scripts in the benchmarks directories. This means, that after you have completed all the benchmarks, you can come back here and run
sh post.sh
This will generate all the data needed to plot graphs and will copy all the results files to the Results directory. If you need to take home only the final results, you can copy over only this directory. You can find there a onecluster.R file, which takes care of reading all the data and outputting to the user.Rdata file.
About this file
Convert to HTML with pandoc Readme.md -f markdown -t html -s -o Readme.html