Modules

Modules provide a way for a dynamic modification of a user's environment via modulefiles. They are very useful in managing large numbers of applictions, especially when there are multiple versions and dependencies present.

Each modulefile contains the information needed to set up your environment for the application. In most cases it will alter or define shell variables such as PATH, LD_LIBRARY_PATH, etc.

module and ml

ml is essentially an alias for module although ml means module list and ml <package> means module load <package>.

Finding available software

You are likely to have an idea of the software you want to use, the "ml spider" command will allow you to see available modules that fit the named package

k1234567@erc-hpc-login2:~$ ml spider python

---------------------------------------------------------------------------------------------------------------------------------------------------
  python:
---------------------------------------------------------------------------------------------------------------------------------------------------
     Versions:
        python/2.7.18-gcc-9.4.0
        python/3.8.12-gcc-9.4.0
     Other possible modules matches:
        py-python-dateutil

---------------------------------------------------------------------------------------------------------------------------------------------------
  To find other possible module matches execute:

      $ module -r spider '.*python.*'

---------------------------------------------------------------------------------------------------------------------------------------------------
  For detailed information about a specific "python" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

     $ module spider python/3.8.12-gcc-9.4.0
---------------------------------------------------------------------------------------------------------------------------------------------------

Loading modules

To set-up your environment for a specific software you need to load the relevant modulefile. The command should include the name of the application and the desired version

module load nwchem/7.0.2-gcc-9.4.0-openmpi-4.1.1

If an application has dependencies such as libraries, compilers, etc, those will be automatically loaded for you.

Tip

We recommend that you always include relevant module load statements in your job scripts.

You can check which modules are currently enabled within your environment with

module list

This should result in an output like:

k1234567@erc-hpc-login1:~$ module list
Currently Loaded Modules:
  1) nwchem/7.0.2-gcc-9.4.0-openmpi-4.1.1

If you are used to using Rosalind you may notice fewer modules are loaded. This is due to the new packaging system which makes greater use of RPATH for making dependencies available to a module.

Unloading modules

You can remove any modules loaded within the current session using module unload or module rm command

module unload nwchem/7.0.2-gcc-9.4.0-openmpi-4.1.1

Tip

You do not need to explicitly unload modules when your job, or session terminates. The changes made by the module files are temporary and will only last for the duration of the job/session.

If you want to remove all of the currently loaded modules, use

module purge

Warning

Purging might remove utility modules loaded as part of your shell set-up/login process.

Finding out more about specific module

You may be able to find out more about a particular module by running module what-is command:

k1234567@erc-hpc-login1:~$ module whatis Python/2.7.18-GCCcore-9.4.0
python/2.7.18-gcc-9.4.0                         : The Python programming language.

Listing available software

To list all software available on the cluster use

module avail

This should produce something similar to

k1234567@erc-hpc-login1:~$ module avail
---------------------------------------- /software/spackages_prod/modules/linux-ubuntu20.04-zen2 ----------------------------------------
   alsa-lib/1.2.3.2-gcc-9.4.0                      libxcb/1.14-gcc-9.4.0                                 psi4/1.3.2-gcc-9.4.0
   amdblis/3.0-gcc-9.4.0                           libxcomposite/0.4.4-gcc-9.4.0                         psimd/2020-05-17-gcc-9.4.0
   amdfftw/3.0-gcc-9.4.0-openmpi-4.1.1             libxcursor/1.1.14-gcc-9.4.0                           pthreadpool/2021-04-13-gcc-9.4.0
   amdlibflame/3.0-gcc-9.4.0                       libxdamage/1.1.4-gcc-9.4.0                            py-bottleneck/1.3.2-gcc-9.4.0
   anaconda3/2021.05-gcc-9.4.0                     libxdmcp/1.1.2-gcc-9.4.0                              py-certifi/2021.10.8-gcc-9.4.0
... lines omitted for brevity ...
   bracken/1.0.0-gcc-9.4.0                         lz4/1.9.3-gcc-9.4.0                                   py-markdown/3.1.1-gcc-9.4.0
   bwa/0.7.17-gcc-9.4.0                            m4/1.4.19-gcc-9.4.0                                   py-markupsafe/1.1.1-gcc-9.4.0
   bzip2/1.0.8-gcc-9.4.0                           magma/2.6.1-gcc-9.4.0-cuda35                          py-markupsafe/2.0.1-gcc-9.4.0           (D)
   cairo/1.16.0-gcc-9.4.0                          magma/2.6.1-gcc-9.4.0                          (D)    py-matplotlib/2.2.5-gcc-9.4.0
   cereal/1.3.0-gcc-9.4.0                          manta/1.6.0-gcc-9.4.0                                 py-matplotlib/3.4.3-gcc-9.4.0           (D)
   cgal/5.0.3-gcc-9.4.0                            meme/5.3.0-gcc-9.4.0-openmpi-4.1.1                    py-multiqc/1.7-gcc-9.4.0
... lines omitted for brevity ...
   libx11/1.7.0-gcc-9.4.0                          postgresql/14.0-gcc-9.4.0                             zstd/1.5.0-gcc-9.4.0
   libxau/1.0.8-gcc-9.4.0                          protobuf/3.18.0-gcc-9.4.0
   libxc/5.1.5-gcc-9.4.0                           prsice/2.3.5-gcc-9.4.0

  Where:
   D:  Default Module

Use spacebar key to paginate through the list.

The (D) tag next to the module names indicates the version of the software that will be loaded if no explicit version has been provided during the module load, e.g. module load apps/matlab.

Tip

To avoid surprises when the defaults change, we recommend using explicit version numbers when loading the modules.

In some cases the same version of the software might be offered with different build options, for example Python/2.7.18-GCCcore-9.4.0 will refer to a version built using gcc 9.4.0 with python 2.7.18 support. Choose the version that is the most appropriate for your needs.

Personal modules

In some circumstances, especially if you build your own software, you might benefit from your own environment that integrates with the modules ecosystem. Details of how to run personal modules will be here soon...