midway.rst 2.14 KB
Newer Older
Marco Govoni's avatar
Marco Govoni committed
1
.. _midway:
Marco Govoni's avatar
Marco Govoni committed
2
3
4
5
6
7
8
9
10

===============
Midway-UChicago
===============

Midway is the HPC cluster of the University of Chicago, maintained by UChicago's `RCC <https://rcc.uchicago.edu/>`_. 

.. code-block:: bash 

Marco Govoni's avatar
Marco Govoni committed
11
   $ ssh -Y <username>@midway.rcc.uchicago.edu
Marco Govoni's avatar
Marco Govoni committed
12
13
14
15
16
17
18
19

Building WEST
~~~~~~~~~~~~~

WEST executables can be compiled using the following script: 

.. code-block:: bash 

Marco Govoni's avatar
Marco Govoni committed
20
   $ cat build_west.sh
Marco Govoni's avatar
Marco Govoni committed
21
22
   #!/bin/bash
   
Marco Govoni's avatar
Marco Govoni committed
23
24
   module load intelmpi/5.1+intel-16.0 mkl/2017.up4 python/cpython-3.8.5 
  
Marco Govoni's avatar
Marco Govoni committed
25
26
27
28
   export F77=mpiifort
   export CC=mpiicc
   export MPIF90=mpiifort
   export FC=mpiifort
29
30
   export CFLAGS="-O3 -fno-alias -ansi-alias -g -mkl -Bdynamic"
   export FFLAGS="-O3 -fno-alias -ansi-alias -g -mkl -Bdynamic"
Marco Govoni's avatar
Marco Govoni committed
31
32
33
34
35
36
   export BLAS_LIBS_SWITCH="external"
   export BLAS_LIBS=" -lmkl_intel_lp64  -lmkl_sequential -lmkl_core"
   export LAPACK_LIBS_SWITCH="external"
   export LAPACK_LIBS=" "
   export SCALAPACK_LIBS=" -lmkl_scalapack_lp64 -Wl,--start-group  -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lmkl_blacs_intelmpi_lp64 -Wl,--end-group"
   
Marco Govoni's avatar
Marco Govoni committed
37
38
   ./configure --enable-openmp=yes --enable-parallel=yes --with-scalapack=intel --with-hdf5=no
   make -j 2 pw
Marco Govoni's avatar
Marco Govoni committed
39
40
   
   cd West
Marco Govoni's avatar
Marco Govoni committed
41
   make conf PYT=python3 PYT_LDFLAGS="`python3-config --ldflags --embed`"
Marco Govoni's avatar
Marco Govoni committed
42
   make all
Marco Govoni's avatar
Marco Govoni committed
43

Marco Govoni's avatar
Marco Govoni committed
44
45
46
47
48
49
To use the script do: 

.. code-block:: bash 

   $ bash build_west.sh

Marco Govoni's avatar
Marco Govoni committed
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65

Running WEST Jobs
~~~~~~~~~~~~~~~~~

The following is an example executable script `run_west.sh` to run the `wstat.x` WEST executable on two nodes of the `broadwl` partition with 28 MPI ranks per node.

.. code-block:: bash 

   $ cat run_west.sh
   #!/bin/bash
   #SBATCH --time=00:30:00
   #SBATCH --partition=broadwl
   #SBATCH --nodes=2
   #SBATCH --ntasks-per-node=28
   #SBATCH --cpus-per-task=1

Marco Govoni's avatar
Marco Govoni committed
66
   module load intelmpi/5.1+intel-16.0 mkl/2017.up4 python/cpython-3.8.5
Marco Govoni's avatar
Marco Govoni committed
67

Marco Govoni's avatar
Marco Govoni committed
68
   export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK
Marco Govoni's avatar
Marco Govoni committed
69
70
71
72
73
74
75
76
77
78
79
80
   NTASKS=$(($SLURM_NTASKS_PER_NODE * $SLURM_JOB_NUM_NODES))

   mpirun -np $NTASKS ./wstat.x -i wstat.in > wstat.out

Job submission is done with the following: 

.. code-block:: bash 

   $ sbatch run_west.sh

.. seealso::
   For more information, visit the RCC user guide (`https://rcc.uchicago.edu/docs/ <https://rcc.uchicago.edu/docs/>`_).