TRE HPC¶
TRE HPC combines the security of a Trusted Research Environment with the familiar job scheduling process of SLURM and the Spack package manager. This guide explains how to load software, submit jobs, use virtual environments, and troubleshoot common issues.
Note
This is a newly developed service created by e-Research and is not the default TRE-configuration. If you believe you would benefit from a TRE HPC implementation, please email support@er.kcl.ac.uk where a member of the team will be able to guide you.
One critical difference between CREATE HPC and the TRE HPC is the use of a 'true scratch' working directory /job_scratch
and a /hpc_jobs
directory for job output. This can be evoked as part of your 'SBATCH' script using #SBATCH --workdir=/job_scratch
. The /job_scratch
directory is faster than your /home
storage.
1. Loading Software Modules¶
TRE HPC uses the same module environment system as CREATE HPC. This allows you to load modules and packages needed for your jobs without manual builds or lengthy approval processes.
Example SBATCH script:
1 2 3 4 5 6 7 8 9 10 11 12 |
|
Warning
Do not use the -l
flag in your SBATCH script; it is not required for job submission.
You can check module versions interactively or within your job script:
1 2 |
|
Note
If using a GPU, ensure you load the appropriate CUDA modules.
2. Submitting Jobs¶
Unlike CREATE HPC, srun
and sbatch
are disabled on TRE HPC. Instead, use the submit_job
wrapper:
1 |
|
Resource requests and SBATCH flags are the same as on CREATE HPC. See the SLURM documentation for details.
Example:
1 |
|
Monitor your jobs with:
1 |
|
3. Example: Matrix Multiplication Benchmark¶
Create a Python script to compare CPU and GPU performance:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
|
Corresponding SBATCH script:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
|
4. Using Virtual Environments¶
- Python venv module works as expected.
- Conda environments currently do not work due to package issues.
pip
can install packages from local and main module repositories, but not from outside PyPi.
Creating and activating a virtual environment using venv:
1 2 3 |
|
Installing PyTorch with CUDA support:
1 |
|
Note
If you need additional software, email support@er.kcl.ac.uk.
Proxy errors:
If you see errors like ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 403 Forbidden'))
, contact the e-Research team with details.
5. Example: PyTorch Benchmark with Virtualenv¶
Python script (pytorch_test.py
):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
|
SBATCH script:
1 2 3 4 5 6 7 8 9 10 |
|
Tip:
Runsubmit_job
from the directory containing your scripts, or provide the correct path.
6. Troubleshooting¶
- Ensure modules are loaded in your SBATCH script.
- Check paths and permissions.
- For proxy or package errors, contact support.
7. Support¶
For help with software installation or access issues, email: support@er.kcl.ac.uk
Last updated: 06/2025