midway.rst 2.09 KB
Newer Older
Marco Govoni's avatar
Marco Govoni committed
1
.. _midway:
Marco Govoni's avatar
Marco Govoni committed
2
3
4
5
6
7
8
9
10

===============
Midway-UChicago
===============

Midway is the HPC cluster of the University of Chicago, maintained by UChicago's `RCC <https://rcc.uchicago.edu/>`_. 

.. code-block:: bash 

Marco Govoni's avatar
Marco Govoni committed
11
   $ ssh -Y <username>@midway.rcc.uchicago.edu
Marco Govoni's avatar
Marco Govoni committed
12
13
14
15
16
17
18
19

Building WEST
~~~~~~~~~~~~~

WEST executables can be compiled using the following script: 

.. code-block:: bash 

Marco Govoni's avatar
Marco Govoni committed
20
   $ cat build_west.sh
Marco Govoni's avatar
Marco Govoni committed
21
22
23
24
25
26
27
28
   #!/bin/bash
   
   module load intelmpi/5.1+intel-16.0 mkl/2017.up4 Anaconda3/5.1.0 
   
   export F77=mpiifort
   export CC=mpiicc
   export MPIF90=mpiifort
   export FC=mpiifort
29
30
   export CFLAGS="-O3 -fno-alias -ansi-alias -g -mkl -Bdynamic"
   export FFLAGS="-O3 -fno-alias -ansi-alias -g -mkl -Bdynamic"
Marco Govoni's avatar
Marco Govoni committed
31
32
33
34
35
36
   export BLAS_LIBS_SWITCH="external"
   export BLAS_LIBS=" -lmkl_intel_lp64  -lmkl_sequential -lmkl_core"
   export LAPACK_LIBS_SWITCH="external"
   export LAPACK_LIBS=" "
   export SCALAPACK_LIBS=" -lmkl_scalapack_lp64 -Wl,--start-group  -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lmkl_blacs_intelmpi_lp64 -Wl,--end-group"
   
Marco Govoni's avatar
Marco Govoni committed
37
   ./configure --enable-parallel --with-scalapack --enable-openmp LD_LIBS="`python3-config --ldflags`"
Marco Govoni's avatar
Marco Govoni committed
38
   make -j 6 pw
Marco Govoni's avatar
Marco Govoni committed
39
40
   
   cd West
Marco Govoni's avatar
Marco Govoni committed
41
42
   make conf PYT=python3
   make all
Marco Govoni's avatar
Marco Govoni committed
43

Marco Govoni's avatar
Marco Govoni committed
44
45
46
47
48
49
To use the script do: 

.. code-block:: bash 

   $ bash build_west.sh

Marco Govoni's avatar
Marco Govoni committed
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67

Running WEST Jobs
~~~~~~~~~~~~~~~~~

The following is an example executable script `run_west.sh` to run the `wstat.x` WEST executable on two nodes of the `broadwl` partition with 28 MPI ranks per node.

.. code-block:: bash 

   $ cat run_west.sh
   #!/bin/bash
   #SBATCH --time=00:30:00
   #SBATCH --partition=broadwl
   #SBATCH --nodes=2
   #SBATCH --ntasks-per-node=28
   #SBATCH --cpus-per-task=1

   module load intelmpi/5.1+intel-16.0 mkl/2017.up4 Anaconda3/5.1.0

Marco Govoni's avatar
Marco Govoni committed
68
   export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK
Marco Govoni's avatar
Marco Govoni committed
69
70
71
72
73
74
75
76
77
78
79
80
   NTASKS=$(($SLURM_NTASKS_PER_NODE * $SLURM_JOB_NUM_NODES))

   mpirun -np $NTASKS ./wstat.x -i wstat.in > wstat.out

Job submission is done with the following: 

.. code-block:: bash 

   $ sbatch run_west.sh

.. seealso::
   For more information, visit the RCC user guide (`https://rcc.uchicago.edu/docs/ <https://rcc.uchicago.edu/docs/>`_).