Singularity¶
Singularity is an open source container platform designed to be simple, fast, and secure. It allows unprivileged users to run untrusted containers in a trusted way. It has been designed with reproducability and HPC workloads in mind.
It is available as an installed package on CREATE.
Take note
Singularity is only accessible from the compute nodes, to use Singularity and avoid the memory limitations of the login nodes, please request an interactive session.
1 2 |
|
Usage¶
You can also download containers using singularity pull
command from
the Singularity Container Library
1 |
|
or from the Docker Hub
1 |
|
Alternatively you can build your own containers. Please see the Singularity documentation for more information.
Warning
Using third party containers is a great way to get started, but you need to make sure that the container is behaving as expected before using it in your work.
Running containers¶
Containers can be used like any other application via an interactive shell or a batch job. You can either run the container
1 |
|
execute a command
1 |
|
or start a shell
1 2 |
|
Alternative Cache location¶
File caching during the initial image build process is by default written to your home directory: /users/k1234566/.singularity/cache
.
Setting the following Singularity environment variable can be used to redirect caching to a larger scratch location where there is more free space available to you:
1 2 |
|
It would be useful to make this alternative location more perminent by setting the above environment variable in your /users/k1234567/.bashrc
file or at least include it in the scope of your batch scripts or interactive sessions.
This is to ensure your cached image file is re-used so that you can avoid unnessarily consuming memory from your allocated resources to download images that are again not present in your default cache location.
Using the Remote Builder¶
Greater flexibility with permissions on the HPC is available with Singularity as the containerisation tool does not have to assume your user has root access to the system. A method to make use of this beneficial assumption is with the remote builder option.
Once you have created an account with Sylabs, you will then be able to generate a secret access token in your remote builder portal. With this access token, you will then be able to authenticate your remote login from CREATE HPC:
1 |
|
This will then prompt for the generated access token, updating the contents of: /users/k1234567/.singularity/remote.yaml
, to configure both your personal HPC Singularity setup and new Singularity remote builder account.
And once complete, with the use of your own definition files, you should now be able to build your own containers whilst on CREATE with a command similar to the following:
1 |
|