Modules
Modules provide a way for a dynamic
modification of a user's environment via modulefiles. They are very useful
in managing large numbers of applictions, especially when there are multiple
versions and dependencies present.
Each modulefile contains the information needed to set up your environment for the application.
In most cases it will alter or define shell variables such as
PATH, LD_LIBRARY_PATH, etc.
module and ml
ml is essentially an alias for module although ml means module list and ml <package> means module load <package>.
Finding available software
You are likely to have an idea of the software you want to use, the "ml spider" command will allow you to see available modules that fit the named package
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24 | k1234567@erc-hpc-login2:~$ ml spider python
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
python:
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Versions:
python/3.11.6-gcc-11.4.0
python/3.11.6-gcc-12.3.0
python/3.11.6-gcc-13.2.0
Other possible modules matches:
py-meson-python py-mysql-connector-python py-python-dateutil
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
To find other possible module matches execute:
$ module -r spider '.*python.*'
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
For detailed information about a specific "python" package (including how to load the modules) use the module's full name.
Note that names that have a trailing (E) are extensions provided by other modules.
For example:
$ module spider python/3.11.6-gcc-13.2.0
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
Loading modules
To set-up your environment for a specific software you need to load the
relevant modulefile. The command should include the name of the application
and the desired version
| module load nwchem/7.2.0-gcc-13.2.0-openmpi-4.1.6-python-3.11.6
|
If an application has dependencies such as libraries, compilers, etc,
those will be automatically loaded for you.
Tip
We recommend that you always include relevant module load statements in
your job scripts.
You can check which modules are currently enabled within your environment with
This should result in an output like:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16 | k1234567@erc-hpc-login1:~$ module list
Currently Loaded Modules:
1) libpciaccess/0.17-gcc-13.2.0 14) openssl/3.1.3-gcc-13.2.0 27) netlib-scalapack/2.2.0-gcc-13.2.0-openmpi-4.1.6-python-3.11.6
2) libiconv/1.17-gcc-13.2.0 15) krb5/1.20.1-gcc-13.2.0 28) libmd/1.0.4-gcc-13.2.0
3) xz/5.4.1-gcc-13.2.0 16) libedit/3.1-20210216-gcc-13.2.0 29) libbsd/0.11.7-gcc-13.2.0
4) zlib-ng/2.1.4-gcc-13.2.0 17) libxcrypt/4.4.35-gcc-13.2.0 30) expat/2.5.0-gcc-13.2.0
5) libxml2/2.10.3-gcc-13.2.0 18) openssh/9.5p1-gcc-13.2.0 31) readline/8.2-gcc-13.2.0
6) ncurses/6.4-gcc-13.2.0 19) libevent/2.1.12-gcc-13.2.0 32) gdbm/1.23-gcc-13.2.0
7) hwloc/2.9.1-gcc-13.2.0 20) pmix/5.0.1-gcc-13.2.0 33) libffi/3.4.4-gcc-13.2.0
8) numactl/2.0.14-gcc-13.2.0 21) libnl/3.3.0-gcc-13.2.0 34) sqlite/3.43.2-gcc-13.2.0
9) bzip2/1.0.8-gcc-13.2.0 22) rdma-core/41.0-gcc-13.2.0-python-3.11.6 35) util-linux-uuid/2.38.1-gcc-13.2.0
10) pigz/2.7-gcc-13.2.0 23) ucx/1.14.1-gcc-13.2.0-python-3.11.6 36) python/3.11.6-gcc-13.2.0
11) zstd/1.5.5-gcc-13.2.0 24) openmpi/4.1.6-gcc-13.2.0-python-3.11.6 37) nwchem/7.2.0-gcc-13.2.0-openmpi-4.1.6-python-3.11.6
12) tar/1.34-gcc-13.2.0 25) fftw/3.3.10-gcc-13.2.0-openmpi-4.1.6-python-3.11.6
13) gettext/0.22.3-gcc-13.2.0-libxml2-2.10.3 26) openblas/0.3.24-gcc-13.2.0
|
If you are used to using Rosalind you may notice fewer modules are loaded.
This is due to the new packaging system which makes greater use of RPATH for making dependencies available to a module.
Unloading modules
You can remove any modules loaded within the current session using module unload or module rm
command
| module unload nwchem/7.2.0-gcc-13.2.0-openmpi-4.1.6-python-3.11.6
|
Tip
You do not need to explicitly unload modules when your job, or session terminates.
The changes made by the module files are temporary and will only last for the
duration of the job/session.
If you want to remove all of the currently loaded modules, use
Warning
Purging might remove utility modules loaded as part of your shell
set-up/login process.
Finding out more about specific module
You may be able to find out more about a particular module by running module what-is command:
| k1234567@erc-hpc-login1:~$ module whatis python/3.11.6-gcc-13.2.0
python/3.11.6-gcc-13.2.0 : The Python programming language.
|
Listing available software
To list all software available on the cluster use
This should produce something similar to
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29 | k1234567@erc-hpc-login1:~$ module avail
abseil-cpp/20230125.3-gcc-11.4.0 nextflow/22.10.1-gcc-13.2.0 (D)
adios2/2.9.2-gcc-13.2.0-flex-2.6.4-python3+-chk-version nghttp2/1.57.0-gcc-11.4.0
adios2/2.9.2-gcc-13.2.0-openmpi-4.1.6-python3+-chk-version (D) nghttp2/1.57.0-gcc-13.2.0 (D)
admixture/1.3.0-gcc-13.2.0 ninja/1.11.1-gcc-11.4.0-python3+-chk-version
alsa-lib/1.2.3.2-gcc-13.2.0 ninja/1.11.1-gcc-13.2.0-python3+-chk-version (D)
anaconda3/2021.05-gcc-13.2.0 nlohmann-json/3.11.2-gcc-13.2.0
... lines omitted for brevity ...
berkeley-db/18.1.40-gcc-13.2.0 (D) p7zip/16.02-gcc-13.2.0
binutils/2.41-gcc-11.4.0 pango/1.50.13-gcc-11.4.0-python3+-chk-version
binutils/2.41-gcc-13.2.0 (D) pango/1.50.13-gcc-13.2.0-python3+-chk-version (D)
bismark/0.23.0-gcc-13.2.0-python3+-chk-version parallel/20220522-gcc-13.2.0
bison/3.8.2-gcc-11.4.0 patchelf/0.17.2-gcc-13.2.0
bison/3.8.2-gcc-13.2.0 (D) pcre/8.44-gcc-13.2.0
... lines omitted for brevity ...
ncurses/6.4-gcc-13.2.0 (D) zlib-ng/2.1.4-gcc-12.3.0
netcdf-c/4.9.2-gcc-13.2.0-openmpi-4.1.6-python3+-chk-version zlib-ng/2.1.4-gcc-13.2.0 (D)
netlib-scalapack/2.1.0-gcc-13.2.0-openmpi-4.1.6-python3+-chk-version zlib/1.2.8-gcc-13.2.0
netlib-scalapack/2.2.0-gcc-13.2.0-multithreading-openmp-python3+-chk-version zlib/1.3-gcc-13.2.0 (D)
netlib-scalapack/2.2.0-gcc-13.2.0-openmpi-4.1.6-python3+-chk-version (D) zstd/1.5.5-gcc-11.4.0
nettle/3.9.1-gcc-13.2.0 zstd/1.5.5-gcc-12.3.0
nextflow/21.04.3-gcc-13.2.0 zstd/1.5.5-gcc-13.2.0 (D)
Where:
D: Default Module
Use "module spider" to find all possible modules and extensions.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".
|
Use spacebar key to paginate through the list.
The (D) tag next to the module names indicates the version
of the software that will be loaded if no explicit version has been
provided during the module load, e.g. module load nextflow would load
nextflow/22.10.1-gcc-13.2.0.
Tip
To avoid surprises when the defaults change, we recommend using explicit
version numbers when loading the modules.
In some cases the same version of the software might be offered with different
build options, for example python/3.11.6-gcc-13.2.0 will refer to a
version built using gcc 13.2.0 with python 3.11.6 support.
Choose the version that is the most appropriate for your needs.
Personal modules
In some circumstances, especially if you build your own software,
you might benefit from your own environment that integrates with
the modules ecosystem.
Details of how to run personal modules will be here soon...