Commit bd1b893f authored by Victor Yu's avatar Victor Yu
Browse files

Update installation instructions

* Theta
* Cori
* Midway3
* macOS 11
parent e20e0db3
......@@ -7,7 +7,7 @@ Installation
In order to install WEST you need to download the `QuantumEspresso 6.1 <https://gitlab.com/QEF/q-e/-/archive/qe-6.1.0/q-e-qe-6.1.0.tar>`_.
`QuantumEspresso <http://www.quantum-espresso.org/>`_ (QE) is an integrated suite of Open-Source computer codes for electronic-structure calculations and materials modeling at the nanoscale, based on density-functional theory (DFT), plane waves (PW), and pseudopotentials (PP).
Configure QuantumEspresso by running the ``configure`` script that comes with the QE distribution. WEST requires `MPI <https://en.wikipedia.org/?title=Message_Passing_Interface>`_ support (`Scalapack <http://www.netlib.org/scalapack/>`_ and `OpenMP <http://openmp.org/>`_ support is also recommended, but optional). If all the environment variables (compilers, libraries etc.) have been set according to the QE configure guide, this would simply be:
Configure QuantumEspresso by running the ``configure`` script that comes with the QE distribution. WEST requires `MPI <https://en.wikipedia.org/?title=Message_Passing_Interface>`_ support (`ScaLAPACK <http://www.netlib.org/scalapack/>`_ and `OpenMP <http://openmp.org/>`_ support is also recommended, but optional). If all the environment variables (compilers, libraries etc.) have been set according to the QE configure guide, this would simply be:
.. code-block:: bash
......@@ -17,7 +17,7 @@ Configure QuantumEspresso by running the ``configure`` script that comes with th
$ ./configure
.. note::
Note that since v4.0.0 WEST requires dynamic linking and python3.
Note that since v4.0.0 WEST requires dynamic linking and Python3.
It's now time to create the ``pw.x``, ``wstat.x``, ``wfreq.x``, and ``westpp.x`` executables by doing:
......@@ -49,6 +49,7 @@ Suggested configuration options
.. toctree::
:maxdepth: 1
installations/cori.rst
installations/theta.rst
installations/midway.rst
installations/macosx.rst
installations/midway3.rst
installations/macos.rst
.. _cori:
=================
Cori-NERSC (XC40)
=================
Cori is a Cray XC40 located at National Energy Research Scientific Computing Center (`NERSC <https://www.nersc.gov/>`_).
.. code-block:: bash
$ ssh <username>@cori.nersc.gov
Building WEST
~~~~~~~~~~~~~
WEST executables can be compiled using the following script (tested on May 12, 2021):
.. code-block:: bash
$ cat build_west.sh
#!/bin/bash
module unload cray-libsci
module load cray-python/3.8.2.1
export CRAYPE_LINK_TYPE=dynamic
export LD_LIBRARY_PATH=$MKLROOT/lib/intel64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/intel/compilers_and_libraries_2019.3.199/linux/compiler/lib/intel64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/python/3.8.2.1/lib:$LD_LIBRARY_PATH
export MPIF90=ftn
export F77=ftn
export CC=cc
export SCALAPACK_LIBS="$MKLROOT/lib/intel64/libmkl_scalapack_lp64.so -Wl,--start-group $MKLROOT/lib/intel64/libmkl_intel_lp64.so $MKLROOT/lib/intel64/libmkl_intel_thread.so $MKLROOT/lib/intel64/libmkl_core.so $MKLROOT/lib/intel64/libmkl_blacs_intelmpi_lp64.so -Wl,--end-group"
./configure --enable-openmp=yes --enable-parallel=yes --enable-shared=yes --with-scalapack=intel --with-hdf5=no
make -j 8 pw
cd West
make conf PYT=python3 PYT_LDFLAGS="`python3-config --ldflags --embed`"
make all
To use the script do:
.. code-block:: bash
$ bash build_west.sh
Running WEST Jobs
~~~~~~~~~~~~~~~~~
The following is an example executable script `run_west.sh` to run the `wstat.x` WEST executable on two nodes of Cori (Haswell partition) with 32 MPI ranks per node. The <project_name> must be replaced with an active project allocation.
.. code-block:: bash
$ cat run_west.sh
#!/bin/bash
#SBATCH --job-name=WEST
#SBATCH --time=00:20:00
#SBATCH --account=<project_name>
#SBATCH --constraint=haswell
#SBATCH --qos=debug
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=32
#SBATCH --cpus-per-task=2
module unload cray-libsci
module load cray-python/3.8.2.1
export LD_LIBRARY_PATH=$MKLROOT/lib/intel64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/intel/compilers_and_libraries_2019.3.199/linux/compiler/lib/intel64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/python/3.8.2.1/lib:$LD_LIBRARY_PATH
export OMP_NUM_THREADS=1
export OMP_PLACE=threads
export OMP_PROC_BIND=spread
export MKL_NUM_THREADS=$OMP_NUM_THREADS
NTASKS=$(($SLURM_NTASKS_PER_NODE * $SLURM_JOB_NUM_NODES))
srun -N $SLURM_JOB_NUM_NODES -n $SLURM_NTASKS_PER_NODE -c $SLURM_CPUS_PER_TASK ./wstat.x -i wstat.in &> wstat.out
Job submission is done with the following:
.. code-block:: bash
$ sbatch run_west.sh
.. seealso::
For more information, visit the `NERSC user guide <https://docs.nersc.gov/systems/cori/>`_.
.. _macosx:
.. _macos:
======
MacOSX
======
=====
macOS
=====
The following instructions have been tested on MacOSX 10.14.6.
The following instructions have been tested on macOS 11.3.
Requirements:
- Gcc/Gfortran 9
- gcc/gfortran (e.g. GCC 9)
- MPICH
- Blas/Lapack/Scalapack
- BLAS/LAPACK/ScaLAPACK
- FFTW3
- Python3
......@@ -40,11 +40,12 @@ WEST executables can be compiled using the following script:
export SCALAPACK_LIBS=${MY_LIB_PATH}/SCALAPACK/libscalapack.a
export FFT_LIBS="${MY_LIB_PATH}/FFTW3/lib/libfftw3.a ${MY_LIB_PATH}/FFTW3/lib/libfftw3_omp.a"
./configure --with-scalapack --enable-openmp
./configure --enable-openmp=yes --enable-parallel=yes --enable-shared=yes --with-scalapack --with-hdf5=no
make -j 4 pw
cd West
make conf PYT=python3 PYT_LDFLAGS="`python3-config --ldflags --embed`"
make all
......
.. _midway:
===============
Midway-UChicago
===============
Midway is the HPC cluster of the University of Chicago, maintained by UChicago's `RCC <https://rcc.uchicago.edu/>`_.
.. code-block:: bash
$ ssh -Y <username>@midway.rcc.uchicago.edu
Building WEST
~~~~~~~~~~~~~
WEST executables can be compiled using the following script:
.. code-block:: bash
$ cat build_west.sh
#!/bin/bash
module load intelmpi/5.1+intel-16.0 mkl/2017.up4 python/cpython-3.8.5
export F77=mpiifort
export CC=mpiicc
export MPIF90=mpiifort
export FC=mpiifort
export CFLAGS="-O3 -fno-alias -ansi-alias -g -mkl -Bdynamic"
export FFLAGS="-O3 -fno-alias -ansi-alias -g -mkl -Bdynamic"
export BLAS_LIBS_SWITCH="external"
export BLAS_LIBS=" -lmkl_intel_lp64 -lmkl_sequential -lmkl_core"
export LAPACK_LIBS_SWITCH="external"
export LAPACK_LIBS=" "
export SCALAPACK_LIBS=" -lmkl_scalapack_lp64 -Wl,--start-group -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lmkl_blacs_intelmpi_lp64 -Wl,--end-group"
./configure --enable-openmp=yes --enable-parallel=yes --with-scalapack=intel --with-hdf5=no
make -j 2 pw
cd West
make conf PYT=python3 PYT_LDFLAGS="`python3-config --ldflags --embed`"
make all
To use the script do:
.. code-block:: bash
$ bash build_west.sh
Running WEST Jobs
~~~~~~~~~~~~~~~~~
The following is an example executable script `run_west.sh` to run the `wstat.x` WEST executable on two nodes of the `broadwl` partition with 28 MPI ranks per node.
.. code-block:: bash
$ cat run_west.sh
#!/bin/bash
#SBATCH --time=00:30:00
#SBATCH --partition=broadwl
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=28
#SBATCH --cpus-per-task=1
module load intelmpi/5.1+intel-16.0 mkl/2017.up4 python/cpython-3.8.5
export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK
NTASKS=$(($SLURM_NTASKS_PER_NODE * $SLURM_JOB_NUM_NODES))
mpirun -np $NTASKS ./wstat.x -i wstat.in > wstat.out
Job submission is done with the following:
.. code-block:: bash
$ sbatch run_west.sh
.. seealso::
For more information, visit the RCC user guide (`https://rcc.uchicago.edu/docs/ <https://rcc.uchicago.edu/docs/>`_).
.. _midway3:
================
Midway3-UChicago
================
Midway3 is the HPC cluster of the University of Chicago, maintained by UChicago's `RCC <https://rcc.uchicago.edu/>`_.
.. code-block:: bash
$ ssh <username>@midway3.rcc.uchicago.edu
Building WEST
~~~~~~~~~~~~~
WEST executables can be compiled using the following script:
.. code-block:: bash
$ cat build_west.sh
#!/bin/bash
module load intel/19.1.1
module load intelmpi/2019.up7+intel-19.1.1
module load mkl/2020.up1
module load python/anaconda-2020.11
export MPIF90=mpiifort
export F90=ifort
export F77=ifort
export CC=icc
export SCALAPACK_LIBS="-lmkl_scalapack_lp64 -Wl,--start-group -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lmkl_blacs_intelmpi_lp64 -Wl,--end-group"
./configure --enable-parallel --with-scalapack --enable-openmp
make -j 8 pw
cd West
make conf PYT=python3 PYT_LDFLAGS="`python3-config --ldflags --embed`"
sed -i 's/-L.*config-3.8-x86_64-linux-gnu //' west_make.inc
make all
To use the script do:
.. code-block:: bash
$ bash build_west.sh
Running WEST Jobs
~~~~~~~~~~~~~~~~~
The following is an example executable script `run_west.sh` to run the `wstat.x` WEST executable on two nodes of Midway3 with 32 MPI ranks per node. The <project_name> and <account_name> must be replaced with an active project allocation.
.. code-block:: bash
$ cat run_west.sh
#!/bin/bash
#SBATCH --time=00:20:00
#SBATCH --partition=<partition_name>
#SBATCH --account=<account_name>
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=32
#SBATCH --cpus-per-task=1
module load intel/19.1.1
module load intelmpi/2019.up7+intel-19.1.1
module load mkl/2020.up1
module load python/anaconda-2020.11
export LD_LIBRARY_PATH=/software/python-anaconda-2020.11-el8-x86_64/lib:$LD_LIBRARY_PATH
export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK
NTASKS=$(($SLURM_NTASKS_PER_NODE * $SLURM_JOB_NUM_NODES))
mpirun -np $NTASKS ./wstat.x -i wstat.in > wstat.out
Job submission is done with the following:
.. code-block:: bash
$ sbatch run_west.sh
.. seealso::
For more information, visit the `RCC user guide <https://rcc.uchicago.edu/docs/>`_.
......@@ -8,12 +8,12 @@ Theta is a Cray XC40 located at Argonne National Laboratory, maintained by `ALCF
.. code-block:: bash
$ ssh -Y <username>@theta.alcf.anl.gov
$ ssh <username>@theta.alcf.anl.gov
Building WEST
~~~~~~~~~~~~~
WEST executables can be compiled using the following script:
WEST executables can be compiled using the following script (tested on May 12, 2021):
.. code-block:: bash
......@@ -21,18 +21,24 @@ WEST executables can be compiled using the following script:
#!/bin/bash
module unload cray-libsci
module load cray-python/3.6.5.3
module load cray-python/3.8.2.1
export CRAYPE_LINK_TYPE=dynamic
export LD_LIBRARY_PATH=$MKLROOT/lib/intel64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/intel/compilers_and_libraries_2018.0.128/linux/compiler/lib/intel64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/python/3.6.5.3/lib:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/intel/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/python/3.8.2.1/lib:$LD_LIBRARY_PATH
export MPIF90=ftn
export F77=ftn
export CC=cc
export SCALAPACK_LIBS="$MKLROOT/lib/intel64/libmkl_scalapack_lp64.so -Wl,--start-group $MKLROOT/lib/intel64/libmkl_intel_lp64.so $MKLROOT/lib/intel64/libmkl_intel_thread.so $MKLROOT/lib/intel64/libmkl_core.so $MKLROOT/lib/intel64/libmkl_blacs_intelmpi_lp64.so -Wl,--end-group"
./configure MPIF90=ftn CC=cc CXX=CC --enable-openmp=yes --enable-parallel=yes --enable-shared=yes --with-scalapack=intel SCALAPACK_LIBS="${MKLROOT}/lib/intel64/libmkl_scalapack_lp64.so -Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_intel_lp64.so ${MKLROOT}/lib/intel64/libmkl_intel_thread.so ${MKLROOT}/lib/intel64/libmkl_core.so ${MKLROOT}/lib/intel64/libmkl_blacs_intelmpi_lp64.so -Wl,--end-group" FFLAGS=" -xMIC-AVX512 -qopenmp -align array64byte -fp-model fast=2 -no-prec-div -assume byterecl" --with-hdf5=no CFLAGS=" -xMIC-AVX512" LDFLAGS=" -shared-intel -qopenmp"
./configure --enable-openmp=yes --enable-parallel=yes --enable-shared=yes --with-scalapack=intel --with-hdf5=no
make pw -j 16
make -j 8 pw
cd West
make conf PYT=python3 PYT_LDFLAGS="`python3-config --ldflags`"
make conf PYT=python3 PYT_LDFLAGS="`python3-config --ldflags --embed`"
make all
To use the script do:
......@@ -62,11 +68,11 @@ The following is an example executable script `run_west.sh` to run the `wstat.x`
HT=1
module unload cray-libsci
module load cray-python/3.6.5.3
export CRAYPE_LINK_TYPE=dynamic
module load cray-python/3.8.2.1
export LD_LIBRARY_PATH=$MKLROOT/lib/intel64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/intel/compilers_and_libraries_2018.0.128/linux/compiler/lib/intel64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/python/3.6.5.3/lib:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/intel/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/python/3.8.2.1/lib:$LD_LIBRARY_PATH
echo "Running Cobalt Job $COBALT_JOBID."
......@@ -77,7 +83,7 @@ Make the script executable:
.. code-block:: bash
$ chmod +x run_west.sh
$ chmod 755 run_west.sh
Job submission is done with the following:
......@@ -86,4 +92,4 @@ Job submission is done with the following:
$ qsub run_west.sh
.. seealso::
For more information, visit the ALCF user guide (`https://www.alcf.anl.gov/user-guides/xc40-system-overview <https://www.alcf.anl.gov/user-guides/xc40-system-overview/>`_).
For more information, visit the `ALCF user guide <https://www.alcf.anl.gov/user-guides/xc40-system-overview/>`_.
......@@ -382,7 +382,7 @@ wfreq_control
* - **Type**
- float
* - **Default**
- dynamically se to the cutoff energy of the density, read from the ground state
- dynamically set to the cutoff energy of the density, read from the ground state
* - **Description**
- Cutoff for the imaginary frequencies (in Ry).
......
%% Cell type:markdown id: tags:
This tutorial can be downloaded [link](http://greatfire.uchicago.edu/west-public/West/raw/master/doc/tutorials/west_100.ipynb).
%% Cell type:markdown id: tags:
# 1.0 Getting Started: GW calculation
%% Cell type:markdown id: tags:
In order to compute the GW electronic structure of the silane molecule you need to run `pw.x`, `wstat.x` and `wfreq.x` in sequence.
%% Cell type:markdown id: tags:
## Step 1: Ground State
%% Cell type:markdown id: tags:
The ground state electronic structure of silane molecule with [QuantumEspresso](https://www.quantum-espresso.org/) is obtained by running `pw.x`. The pseudopotential files for **Si** and **H** in UPF format can be downloaded from: [QE-PP](https://www.quantum-espresso.org/pseudopotentials) database, or from [SG15](http://www.quantum-simulation.org/potentials/sg15_oncv/upf/) database. Check out the `pw.x` [input description](https://www.quantum-espresso.org/Doc/INPUT_PW.html) in order to generate an input file for QuantumEspresso called `pw.in`.
%% Cell type:markdown id: tags:
Download these files in your current working directory:
%% Cell type:code id: tags:
``` python
%%bash
wget -N -q http://www.west-code.org/doc/training/silane/pw.in
wget -N -q http://www.quantum-simulation.org/potentials/sg15_oncv/upf/H_ONCV_PBE-1.0.upf
wget -N -q http://www.quantum-simulation.org/potentials/sg15_oncv/upf/Si_ONCV_PBE-1.1.upf
```
%% Cell type:markdown id: tags:
Let's inspect the ``pw.in`` file, input for ``pw.x``.
%% Cell type:code id: tags:
``` python
%%bash
cat pw.in
```
%%%% Output: stream
&control
calculation = 'scf'
restart_mode = 'from_scratch'
pseudo_dir = './'
outdir = './'
prefix = 'silane'
wf_collect = .TRUE.
/
&system
ibrav = 1
celldm(1) = 20
nat = 5
ntyp = 2
ecutwfc = 25.0
nbnd = 10
assume_isolated ='mp'
/
&electrons
diago_full_acc = .TRUE.
/
ATOMIC_SPECIES
Si 28.0855 Si_ONCV_PBE-1.1.upf
H 1.00794 H_ONCV_PBE-1.0.upf
ATOMIC_POSITIONS bohr
Si 10.000000 10.000000 10.000000
H 11.614581 11.614581 11.614581
H 8.385418 8.385418 11.614581
H 8.385418 11.614581 8.385418
H 11.614581 8.385418 8.385418
K_POINTS {gamma}
%% Cell type:markdown id: tags:
Run ``pw.in`` on 2 cores.
%% Cell type:code id: tags:
``` python
%%bash
mpirun -n 2 pw.x -i pw.in > pw.out
```
%% Cell type:markdown id: tags:
The output file ``pw.out`` contains information about the ground state calculation.
%% Cell type:markdown id: tags:
## Step 2: Screening
%% Cell type:markdown id: tags:
The static dielectric screening is computed using the projective dielectric eigendecomposition (PDEP) technique. Check out the ``wstat.x`` [input description](http://www.west-code.org/doc/West/latest/) and generate an input file for WEST called ``wstat.in``.
%% Cell type:markdown id: tags:
Download this file in your current working directory:
%% Cell type:code id: tags:
``` python
%%bash
wget -N -q http://www.west-code.org/doc/training/silane/wstat.in
```
%% Cell type:markdown id: tags:
Let's inspect the ``wstat.in`` file, input for ``wstat.x``.
%% Cell type:code id: tags:
``` python
%%bash
cat wstat.in
```
%%%% Output: stream
input_west:
qe_prefix: silane
west_prefix: silane
outdir: ./
wstat_control:
wstat_calculation: S
n_pdep_eigen: 50
%% Cell type:markdown id: tags:
Run ``wstat.in`` on 2 cores.
%% Cell type:code id: tags:
``` python
%%bash
mpirun -n 2 wstat.x -i wstat.in > wstat.out
```
%% Cell type:markdown id: tags:
The output file ``wstat.out`` contains information about the PDEP iterations, and the dielectric eigenvalues can be found in the file ``west_prefix.wstat.save/wstat.json``.
%% Cell type:markdown id: tags:
## Step 3: GW
%% Cell type:markdown id: tags:
The GW electronic structure is computed treating the frequency integration of the correlation part of the self energy with the Contour Deformation techinique and by computing the dielectric screening at multipole frequencies with Lanczos iterations. Check out the ``wfreq.x`` [input description](http://www.west-code.org/doc/West/latest/) and generate an input file for WEST called ``wfreq.in``.
%% Cell type:markdown id: tags:
Download this file in your current working directory:
%% Cell type:code id: tags:
``` python
%%bash
wget -N -q http://www.west-code.org/doc/training/silane/wfreq.in
```
%% Cell type:markdown id: tags:
Let's inspect the ``wfreq.in`` file, input for ``wfreq.x``.
%% Cell type:code id: tags:
``` python
%%bash
cat wfreq.in
```
%%%% Output: stream
input_west:
qe_prefix: silane
west_prefix: silane
outdir: ./
wstat_control:
wstat_calculation: S
n_pdep_eigen: 50
wfreq_control:
wfreq_calculation: XWGQ
n_pdep_eigen_to_use: 50
qp_bandrange: [1,5]
n_refreq: 300
ecut_refreq: 2.0
%% Cell type:markdown id: tags:
Run ``wfreq.in`` on 2 cores.
%% Cell type:code id: tags:
``` python
%%bash
mpirun -n 2 wfreq.x -i wfreq.in > wfreq.out
```
%% Cell type:markdown id: tags:
The output file ``wfreq.out`` contains information about the GW self-energy corrected electronic structure can be found in the file ``west_prefix.wstat.save/wfreq.json``.
The output file ``wfreq.out`` contains information about the calculation of the GW self-energy, and the corrected electronic structure can be found in the file ``west_prefix.wstat.save/wfreq.json``.
%% Cell type:code id: tags:
``` python
```
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment