Skip to content

GitLab

Commit bd1b893f authored by Victor Yu's avatar Victor Yu
Browse files

Update installation instructions 

* Theta
* Cori
* Midway3
* macOS 11
parent e20e0db3
Hide whitespace changes
Inline Side-by-side
Doc/installation.rst
View file @ bd1b893f
......@@ -7,21 +7,21 @@ Installation
In order to install WEST you need to download the `QuantumEspresso 6.1 <https://gitlab.com/QEF/q-e/-/archive/qe-6.1.0/q-e-qe-6.1.0.tar>`_.
`QuantumEspresso <http://www.quantum-espresso.org/>`_ (QE) is an integrated suite of Open-Source computer codes for electronic-structure calculations and materials modeling at the nanoscale, based on density-functional theory (DFT), plane waves (PW), and pseudopotentials (PP).
Configure QuantumEspresso by running the ``configure`` script that comes with the QE distribution. WEST requires `MPI <https://en.wikipedia.org/?title=Message_Passing_Interface>`_ support (`Scalapack <http://www.netlib.org/scalapack/>`_ and `OpenMP <http://openmp.org/>`_ support is also recommended, but optional). If all the environment variables (compilers, libraries etc.) have been set according to the QE configure guide, this would simply be:
Configure QuantumEspresso by running the ``configure`` script that comes with the QE distribution. WEST requires `MPI <https://en.wikipedia.org/?title=Message_Passing_Interface>`_ support (`ScaLAPACK <http://www.netlib.org/scalapack/>`_ and `OpenMP <http://openmp.org/>`_ support is also recommended, but optional). If all the environment variables (compilers, libraries etc.) have been set according to the QE configure guide, this would simply be:
.. code-block:: bash
.. code-block:: bash
$ git clone -b 'qe-6.1.0' --single-branch --depth 1 https://gitlab.com/QEF/q-e.git QEdir
$ cd QEdir
$ git clone -b 'v4.2.1' --single-branch --depth 1 http://greatfire.uchicago.edu/west-public/West.git West
$ ./configure
$ ./configure
.. note::
Note that since v4.0.0 WEST requires dynamic linking and python3.
.. note::
Note that since v4.0.0 WEST requires dynamic linking and Python3.
It's now time to create the ``pw.x``, ``wstat.x``, ``wfreq.x``, and ``westpp.x`` executables by doing:
.. code-block:: bash
.. code-block:: bash
$ cd QEdir
$ make pw
......@@ -31,7 +31,7 @@ It's now time to create the ``pw.x``, ``wstat.x``, ``wfreq.x``, and ``westpp.x``
You have succefully installed QuantumEspresso and WEST if you see the executables ``pw.x``, ``wstat.x``, ``wfreq.x``, and ``westpp.x`` created in the QEdir/bin directory.
.. code-block:: bash
.. code-block:: bash
$ ls QEdir/bin/
pw.x
......@@ -40,7 +40,7 @@ You have succefully installed QuantumEspresso and WEST if you see the executable
westpp.x
... (other content) ...
Congratulations, you are all set for running QuantumEspresso and WEST!
Congratulations, you are all set for running QuantumEspresso and WEST!
Suggested configuration options
......@@ -49,6 +49,7 @@ Suggested configuration options
.. toctree::
:maxdepth: 1
installations/cori.rst
installations/theta.rst
installations/midway.rst
installations/macosx.rst
installations/midway3.rst
installations/macos.rst
Doc/installations/cori.rst 0 → 100644
View file @ bd1b893f
.. _cori:
=================
Cori-NERSC (XC40)
=================
Cori is a Cray XC40 located at National Energy Research Scientific Computing Center (`NERSC <https://www.nersc.gov/>`_).
.. code-block:: bash
$ ssh <username>@cori.nersc.gov
Building WEST
~~~~~~~~~~~~~
WEST executables can be compiled using the following script (tested on May 12, 2021):
.. code-block:: bash
$ cat build_west.sh
#!/bin/bash
module unload cray-libsci
module load cray-python/3.8.2.1
export CRAYPE_LINK_TYPE=dynamic
export LD_LIBRARY_PATH=$MKLROOT/lib/intel64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/intel/compilers_and_libraries_2019.3.199/linux/compiler/lib/intel64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/python/3.8.2.1/lib:$LD_LIBRARY_PATH
export MPIF90=ftn
export F77=ftn
export CC=cc
export SCALAPACK_LIBS="$MKLROOT/lib/intel64/libmkl_scalapack_lp64.so -Wl,--start-group $MKLROOT/lib/intel64/libmkl_intel_lp64.so $MKLROOT/lib/intel64/libmkl_intel_thread.so $MKLROOT/lib/intel64/libmkl_core.so $MKLROOT/lib/intel64/libmkl_blacs_intelmpi_lp64.so -Wl,--end-group"
./configure --enable-openmp=yes --enable-parallel=yes --enable-shared=yes --with-scalapack=intel --with-hdf5=no
make -j 8 pw
cd West
make conf PYT=python3 PYT_LDFLAGS="`python3-config --ldflags --embed`"
make all
To use the script do:
.. code-block:: bash
$ bash build_west.sh
Running WEST Jobs
~~~~~~~~~~~~~~~~~
The following is an example executable script `run_west.sh` to run the `wstat.x` WEST executable on two nodes of Cori (Haswell partition) with 32 MPI ranks per node. The <project_name> must be replaced with an active project allocation.
.. code-block:: bash
$ cat run_west.sh
#!/bin/bash
#SBATCH --job-name=WEST
#SBATCH --time=00:20:00
#SBATCH --account=<project_name>
#SBATCH --constraint=haswell
#SBATCH --qos=debug
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=32
#SBATCH --cpus-per-task=2
module unload cray-libsci
module load cray-python/3.8.2.1
export LD_LIBRARY_PATH=$MKLROOT/lib/intel64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/intel/compilers_and_libraries_2019.3.199/linux/compiler/lib/intel64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/python/3.8.2.1/lib:$LD_LIBRARY_PATH
export OMP_NUM_THREADS=1
export OMP_PLACE=threads
export OMP_PROC_BIND=spread
export MKL_NUM_THREADS=$OMP_NUM_THREADS
NTASKS=$(($SLURM_NTASKS_PER_NODE * $SLURM_JOB_NUM_NODES))
srun -N $SLURM_JOB_NUM_NODES -n $SLURM_NTASKS_PER_NODE -c $SLURM_CPUS_PER_TASK ./wstat.x -i wstat.in &> wstat.out
Job submission is done with the following:
.. code-block:: bash
$ sbatch run_west.sh
.. seealso::
For more information, visit the `NERSC user guide <https://docs.nersc.gov/systems/cori/>`_.
Doc/installations/macosx.rst → Doc/installations/macos.rst
View file @ bd1b893f
.. _macosx:
.. _macos:
======
MacOSX
======
=====
macOS
=====
The following instructions have been tested on MacOSX 10.14.6.
The following instructions have been tested on macOS 11.3.
Requirements:
Requirements:
- Gcc/Gfortran 9
- MPICH
- Blas/Lapack/Scalapack
- gcc/gfortran (e.g. GCC 9)
- MPICH
- BLAS/LAPACK/ScaLAPACK
- FFTW3
- Python3
Building WEST
~~~~~~~~~~~~~
WEST executables can be compiled using the following script:
WEST executables can be compiled using the following script:
.. code-block:: bash
.. code-block:: bash
$ cat build_west.sh
#!/bin/bash
......@@ -40,17 +40,18 @@ WEST executables can be compiled using the following script:
export SCALAPACK_LIBS=${MY_LIB_PATH}/SCALAPACK/libscalapack.a
export FFT_LIBS="${MY_LIB_PATH}/FFTW3/lib/libfftw3.a ${MY_LIB_PATH}/FFTW3/lib/libfftw3_omp.a"
./configure --with-scalapack --enable-openmp
./configure --enable-openmp=yes --enable-parallel=yes --enable-shared=yes --with-scalapack --with-hdf5=no
make -j 4 pw
cd West
make conf PYT=python3 PYT_LDFLAGS="`python3-config --ldflags --embed`"
make all
To use the script do:
To use the script do:
.. code-block:: bash
.. code-block:: bash
$ bash build_west.sh
......@@ -60,7 +61,7 @@ Running WEST
We can run the `wstat.x` WEST executables on 2 cores using the following command:
.. code-block:: bash
.. code-block:: bash
$ export OMP_NUM_THREADS=1
$ mpirun -np 2 ./wstat.x -i wstat.in > wstat.out
Doc/installations/midway.rst deleted 100644 → 0
View file @ e20e0db3
.. _midway:
===============
Midway-UChicago
===============
Midway is the HPC cluster of the University of Chicago, maintained by UChicago's `RCC <https://rcc.uchicago.edu/>`_.
.. code-block:: bash
$ ssh -Y <username>@midway.rcc.uchicago.edu
Building WEST
~~~~~~~~~~~~~
WEST executables can be compiled using the following script:
.. code-block:: bash
$ cat build_west.sh
#!/bin/bash
module load intelmpi/5.1+intel-16.0 mkl/2017.up4 python/cpython-3.8.5
export F77=mpiifort
export CC=mpiicc
export MPIF90=mpiifort
export FC=mpiifort
export CFLAGS="-O3 -fno-alias -ansi-alias -g -mkl -Bdynamic"
export FFLAGS="-O3 -fno-alias -ansi-alias -g -mkl -Bdynamic"
export BLAS_LIBS_SWITCH="external"
export BLAS_LIBS=" -lmkl_intel_lp64 -lmkl_sequential -lmkl_core"
export LAPACK_LIBS_SWITCH="external"
export LAPACK_LIBS=" "
export SCALAPACK_LIBS=" -lmkl_scalapack_lp64 -Wl,--start-group -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lmkl_blacs_intelmpi_lp64 -Wl,--end-group"
./configure --enable-openmp=yes --enable-parallel=yes --with-scalapack=intel --with-hdf5=no
make -j 2 pw
cd West
make conf PYT=python3 PYT_LDFLAGS="`python3-config --ldflags --embed`"
make all
To use the script do:
.. code-block:: bash
$ bash build_west.sh
Running WEST Jobs
~~~~~~~~~~~~~~~~~
The following is an example executable script `run_west.sh` to run the `wstat.x` WEST executable on two nodes of the `broadwl` partition with 28 MPI ranks per node.
.. code-block:: bash
$ cat run_west.sh
#!/bin/bash
#SBATCH --time=00:30:00
#SBATCH --partition=broadwl
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=28
#SBATCH --cpus-per-task=1
module load intelmpi/5.1+intel-16.0 mkl/2017.up4 python/cpython-3.8.5
export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK
NTASKS=$(($SLURM_NTASKS_PER_NODE * $SLURM_JOB_NUM_NODES))
mpirun -np $NTASKS ./wstat.x -i wstat.in > wstat.out
Job submission is done with the following:
.. code-block:: bash
$ sbatch run_west.sh
.. seealso::
For more information, visit the RCC user guide (`https://rcc.uchicago.edu/docs/ <https://rcc.uchicago.edu/docs/>`_).
Doc/installations/midway3.rst 0 → 100644
View file @ bd1b893f
.. _midway3:
================
Midway3-UChicago
================
Midway3 is the HPC cluster of the University of Chicago, maintained by UChicago's `RCC <https://rcc.uchicago.edu/>`_.
.. code-block:: bash
$ ssh <username>@midway3.rcc.uchicago.edu
Building WEST
~~~~~~~~~~~~~
WEST executables can be compiled using the following script:
.. code-block:: bash
$ cat build_west.sh
#!/bin/bash
module load intel/19.1.1
module load intelmpi/2019.up7+intel-19.1.1
module load mkl/2020.up1
module load python/anaconda-2020.11
export MPIF90=mpiifort
export F90=ifort
export F77=ifort
export CC=icc
export SCALAPACK_LIBS="-lmkl_scalapack_lp64 -Wl,--start-group -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lmkl_blacs_intelmpi_lp64 -Wl,--end-group"
./configure --enable-parallel --with-scalapack --enable-openmp
make -j 8 pw
cd West
make conf PYT=python3 PYT_LDFLAGS="`python3-config --ldflags --embed`"
sed -i 's/-L.*config-3.8-x86_64-linux-gnu //' west_make.inc
make all
To use the script do:
.. code-block:: bash
$ bash build_west.sh
Running WEST Jobs
~~~~~~~~~~~~~~~~~
The following is an example executable script `run_west.sh` to run the `wstat.x` WEST executable on two nodes of Midway3 with 32 MPI ranks per node. The <project_name> and <account_name> must be replaced with an active project allocation.
.. code-block:: bash
$ cat run_west.sh
#!/bin/bash
#SBATCH --time=00:20:00
#SBATCH --partition=<partition_name>
#SBATCH --account=<account_name>
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=32
#SBATCH --cpus-per-task=1
module load intel/19.1.1
module load intelmpi/2019.up7+intel-19.1.1
module load mkl/2020.up1
module load python/anaconda-2020.11
export LD_LIBRARY_PATH=/software/python-anaconda-2020.11-el8-x86_64/lib:$LD_LIBRARY_PATH
export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK
NTASKS=$(($SLURM_NTASKS_PER_NODE * $SLURM_JOB_NUM_NODES))
mpirun -np $NTASKS ./wstat.x -i wstat.in > wstat.out
Job submission is done with the following:
.. code-block:: bash
$ sbatch run_west.sh
.. seealso::
For more information, visit the `RCC user guide <https://rcc.uchicago.edu/docs/>`_.
Doc/installations/theta.rst
View file @ bd1b893f
......@@ -4,40 +4,46 @@
Theta-ALCF (XC40)
=================
Theta is a Cray XC40 located at Argonne National Laboratory, maintained by `ALCF <https://www.alcf.anl.gov/>`_.
Theta is a Cray XC40 located at Argonne National Laboratory, maintained by `ALCF <https://www.alcf.anl.gov/>`_.
.. code-block:: bash
.. code-block:: bash
$ ssh -Y <username>@theta.alcf.anl.gov
$ ssh <username>@theta.alcf.anl.gov
Building WEST
~~~~~~~~~~~~~
WEST executables can be compiled using the following script:
WEST executables can be compiled using the following script (tested on May 12, 2021):
.. code-block:: bash
.. code-block:: bash
$ cat build_west.sh
#!/bin/bash
module unload cray-libsci
module load cray-python/3.6.5.3
module load cray-python/3.8.2.1
export CRAYPE_LINK_TYPE=dynamic
export LD_LIBRARY_PATH=$MKLROOT/lib/intel64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/intel/compilers_and_libraries_2018.0.128/linux/compiler/lib/intel64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/python/3.6.5.3/lib:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/intel/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/python/3.8.2.1/lib:$LD_LIBRARY_PATH
export MPIF90=ftn
export F77=ftn
export CC=cc
export SCALAPACK_LIBS="$MKLROOT/lib/intel64/libmkl_scalapack_lp64.so -Wl,--start-group $MKLROOT/lib/intel64/libmkl_intel_lp64.so $MKLROOT/lib/intel64/libmkl_intel_thread.so $MKLROOT/lib/intel64/libmkl_core.so $MKLROOT/lib/intel64/libmkl_blacs_intelmpi_lp64.so -Wl,--end-group"
./configure MPIF90=ftn CC=cc CXX=CC --enable-openmp=yes --enable-parallel=yes --enable-shared=yes --with-scalapack=intel SCALAPACK_LIBS="${MKLROOT}/lib/intel64/libmkl_scalapack_lp64.so -Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_intel_lp64.so ${MKLROOT}/lib/intel64/libmkl_intel_thread.so ${MKLROOT}/lib/intel64/libmkl_core.so ${MKLROOT}/lib/intel64/libmkl_blacs_intelmpi_lp64.so -Wl,--end-group" FFLAGS=" -xMIC-AVX512 -qopenmp -align array64byte -fp-model fast=2 -no-prec-div -assume byterecl" --with-hdf5=no CFLAGS=" -xMIC-AVX512" LDFLAGS=" -shared-intel -qopenmp"
./configure --enable-openmp=yes --enable-parallel=yes --enable-shared=yes --with-scalapack=intel --with-hdf5=no
make pw -j 16
make -j 8 pw
cd West
make conf PYT=python3 PYT_LDFLAGS="`python3-config --ldflags`"
make all
To use the script do:
make conf PYT=python3 PYT_LDFLAGS="`python3-config --ldflags --embed`"
make all
To use the script do:
.. code-block:: bash
.. code-block:: bash
$ bash build_west.sh
......@@ -46,7 +52,7 @@ Running WEST Jobs
The following is an example executable script `run_west.sh` to run the `wstat.x` WEST executable on two nodes of Theta with 64 MPI ranks per node. The <project_name> must be replaced with an active project allocation.
.. code-block:: bash
.. code-block:: bash
$ cat run_west.sh
#!/bin/bash
......@@ -62,28 +68,28 @@ The following is an example executable script `run_west.sh` to run the `wstat.x`
HT=1
module unload cray-libsci
module load cray-python/3.6.5.3
export CRAYPE_LINK_TYPE=dynamic
module load cray-python/3.8.2.1
export LD_LIBRARY_PATH=$MKLROOT/lib/intel64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/intel/compilers_and_libraries_2018.0.128/linux/compiler/lib/intel64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/python/3.6.5.3/lib:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/intel/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/python/3.8.2.1/lib:$LD_LIBRARY_PATH
echo "Running Cobalt Job $COBALT_JOBID."
export OMP_NUM_THREADS=$NTHREADS
aprun -n $MPIRANKS -N $MPIRANKS_PERNODE -cc depth -d $NTHREADS -j $HT ./wstat.x -i wstat.in &> wstat.out
Make the script executable:
Make the script executable:
.. code-block:: bash
.. code-block:: bash
$ chmod +x run_west.sh
$ chmod 755 run_west.sh
Job submission is done with the following:
Job submission is done with the following:
.. code-block:: bash
.. code-block:: bash
$ qsub run_west.sh
.. seealso::
For more information, visit the ALCF user guide (`https://www.alcf.anl.gov/user-guides/xc40-system-overview <https://www.alcf.anl.gov/user-guides/xc40-system-overview/>`_).
For more information, visit the `ALCF user guide <https://www.alcf.anl.gov/user-guides/xc40-system-overview/>`_.
Doc/manual.rst
View file @ bd1b893f
......@@ -3,19 +3,19 @@
Manual
======
The complete **WEST** reference for input parameters.
The complete **WEST** reference for input parameters.
.. contents:: :local:
:depth: 1
.. note::
Not all input parameters listed below are mandatory. Check :ref:`quickreference` or :ref:`tutorial` pages to see examples of input files.
.. note::
Not all input parameters listed below are mandatory. Check :ref:`quickreference` or :ref:`tutorial` pages to see examples of input files.
.. seealso::
**WESTpy** is a Python package, designed to assist users of the WEST code in pre- and post-process massively parallel calculations. Click `here <http://www.west-code.org/doc/westpy/latest/>`_ to know more.
.. seealso::
The input file is given according to the YAML Notation (`https://yaml.org/ <https://yaml.org//>`_).
The input file is given according to the YAML Notation (`https://yaml.org/ <https://yaml.org//>`_).
|
......@@ -48,7 +48,7 @@ input_west
* - **Default**
- "west"
* - **Description**
- Prefix prepended to the WEST save and restart folders.
- Prefix prepended to the WEST save and restart folders.
.. data:: outdir
......@@ -62,8 +62,8 @@ input_west
* - **Default**
- "./"
* - **Description**
- Directory for: input, temporary, and output files.
- Directory for: input, temporary, and output files.
|
......@@ -86,8 +86,8 @@ wstat_control
- "S" : Start from scratch.
- "R" : Restart from an interrupted run. You should restart with the same number of cores, and images.
- "E" : Calculation of the response is external, i.e. outsourced to a server.
- "E" : Calculation of the response is external, i.e. outsourced to a server.
.. data:: n_pdep_eigen
......@@ -284,7 +284,7 @@ wfreq_control
- "XWO" : Compute the optical properties.
- "XWGQP" : Compute the QP corrections, and plot spectral functions.
- "XWGQOP" : Compute all.
.. data:: n_pdep_eigen_to_use
......@@ -382,7 +382,7 @@ wfreq_control
* - **Type**
- float
* - **Default**
- dynamically se to the cutoff energy of the density, read from the ground state
- dynamically set to the cutoff energy of the density, read from the ground state
* - **Description**
- Cutoff for the imaginary frequencies (in Ry).
......@@ -568,7 +568,7 @@ westpp_control
- "C"
* - **Description**
- Available options for the output fortmat are:
- "c" : Cube.
- "x" : Planar average yz.
- "y" : Planar average xz.
......@@ -679,5 +679,5 @@ server_control
- The document is serialized into a JSON string and passed to the server (see `West/Pytools/west_clientserver.py`).
|
Doc/tutorials/west_100.ipynb
View file @ bd1b893f
......@@ -143,11 +143,11 @@
mpirun -n 2 wfreq.x -i wfreq.in > wfreq.out
```
%% Cell type:markdown id: tags:
The output file ``wfreq.out`` contains information about the GW self-energy corrected electronic structure can be found in the file ``west_prefix.wstat.save/wfreq.json``.
The output file ``wfreq.out`` contains information about the calculation of the GW self-energy, and the corrected electronic structure can be found in the file ``west_prefix.wstat.save/wfreq.json``.
%% Cell type:code id: tags:
``` python
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment