Getting started with CREATE Cloud¶
Project Registration¶
Before getting started with a CREATE Cloud project you need to decide whether this will be a free tier (limited quota, best effort support) or paid tier project (paid per annum quotas, full support). For more details see CREATE Cloud pricing.
Once the tier is decided email support@er.kcl.ac.uk with the tier choice and project registration information.
Info
It is fine to start with a free tier project for experimentation at the pre-award stage and then have paid tier quota applied to the same project if your funding bid is succesful. In these cases it helps if you can provide registration details that reflect the long term aims of the project and let us know of any changes to these detail at the point of grant award (e.g. end date).
House Rules¶
- With great power comes great responsibility. CREATE Cloud provides administrative capabilties (e.g. system root, Internet exposure) on your projects. Use these powers with great care and understanding.
- CREATE Cloud allows software to be exposed to the Internet for external access (via e-Research load balancers). Exposed applications must be security scanned prior to exposure and will be security scanned on a regular basis. Any software with published security vulnerabilties may be taken offline while they are addressed.
- You must adhere to CREATE Terms of Use.
- You must adhere to the King's Code of Connection.
If you or your research group lack the systems administration capability to comply with these rules we can almost certainly help so please get in touch to discuss your project requirements.
Web interface¶
The web interface is accessed at https://cloud.er.kcl.ac.uk using standard King's credentials for single sign on over the e-Research VPN.
This documentation covers the basic settings required to get VMs running. For more advanced configurations or detail please see our "going further" doc or the offical OpenStack Documentation.
Setting up your first VM¶
Navigate to your project¶
On logging in you will see an overview of the current resources available in your default project and other summary details.
If you have more than one project, use the drop-down combo box next to the King's logo at the top of the page to select the project you want to work on.
Create a key pair¶
Configuring an (SSH) Key Pair in OpenStack allows the public key of the pair to be automatically installed on to instances (virtual machines) as part of the build process.
Keys created in OpenStack cannot have passwords attached to them so we strongly recommend following the guidance on creating keys in SSH Clients and Keys and importing the public key to OpenStack during the configuration process.
Select 'Key Pairs' from the main menu in the 'Compute' section and click 'Import Public Key'. Enter a meaningful name e.g. 'KCL laptop key'.
Launch an instance¶
To create a virtual machine (VM) select 'Instances' from the main menu in the 'Compute' section and press the button 'Launch Instance'.
Details¶
Every instance must have a unique name, if you want to create multiple identical VMs the count can be increased and this will append a number in series, starting at 1, to the instance name.
Source¶
When creating your first VM you will want to choose a source, available sources are listed at the bottom of the dialogue box when 'Source' is selected.
We currently provide a standard Ubuntu 20.04 image if your project requires another flavour of Linux or Windows please get in touch to discuss your requirements.
Our flavours all have a disk size of 10GB. The general expectation is that research data is stored on RDS. If your application requires a larger disk size adjust the 'Volume Size' accordingly. Upon selection of an Available Source the minimum volume size of 10GB will be automatically populated.
When creating VMs for the first time Select Boot Source will be left as the default choice 'Image'. It is also possible to create VMs from Volumes and Volume Snapshots (n.b. we disable creation from Image Snapshots in our deployment).
Attention
We intentionally prevent the upload of user created VM images by default. This allows us to focus our support and security operations on our standard Ubuntu image. If your research requires another flavour of Linux/Unix please email support@er.kcl.ac.uk to discuss your requirement further.
Flavour¶
Attention
We are in the process of deprecating the flavours prefixed "ae7" selecting these will result in failure to launch your instance. Please select the flavours which are named "cpuNramN".
The 'Flavour' refers to the different hardware configurations available and you should select the one most appropriate for your needs. A flavour defines the amount of vCPU and memory an instance will have. The choice of flavours may be restricted by the particular type of image chosen or the resources (e.g. GPUs) you have available in the project.
Networks¶
The most basic approach to networking is to connect your instances to the 'external_4003' network directly. This will assign them an IP address in the 10.211.112.0/20 subnet which is then reachable over the e-Research VPN. For details on creating networks internal to your project see our "going further" doc.
The 'external_4003' network is automatically selected in the Launch Instance wizard, click Next.
Network Ports¶
Click Next.
Security Groups¶
Attention
Do not select the er_ssh_form_bastions security group during this launch wizard, doing so will cause the instance launch to fail. It can be selected after launch (see below). We hope this will be addressed in an impending fix to the OpenStack Nova service.
Key Pair¶
It is possible to create a key pair in the Launch Instance dialogue window although if you have added / created a key pair already it will be available for selection.
Be sure to select a key pair as this will be required for SSH access to the VM.
Launch instance¶
You can now click the 'Launch instance' button. You will see your instance building and also the IP address in 10.211.112/20 that has been assigned.
Select the SSH Security Group¶
Attention
Security Groups control what traffic is allowed to access your VMs. Caution and understanding are required when making any changes to the defaults e-Research provide.
e-Research provide a number of automatically generated Security Groups to your projects when they are created. These are prefixed 'er_'.
Using the drop down in the instance's 'Actions' column choose 'Edit Secuirty Groups'.
Select the 'er_ssh_from_bastions' Security Group to allow SSH traffic over the e-Research VPN.
Connect via SSH¶
Once the status shows as 'Active' and the er_ssh_from_bastions Security Group has been
assigned you should be able to connect via SSH using the account 'ubuntu' and the SSH key pair
you configured while connected to the e-Research VPN or via our SSH bastion
servers with the following SSH incantation:
1 | |
Our Ubuntu 20.04 image¶
Our Ubuntu image is close to vanilla with a few notable customisations:
- We install an administrator account and SSH key so that we can access the instance to respond to any critical incidents
- We install a Wazuh security scanning agent so that we can monitor vulnerability exposure across CREATE
- We enable automatic patching (including required) reboots on a weekly basis
- We configure a password strength policy for local accounts (should they be needed, n.b. we recommend SSH keys instead wherever possible)
We expect these configurations to be left untouched in the majority of use cases while acknowledging that users may need to adjust automatic reboots to better suit their use case. Any changes to the above must not prevent compliance with our house rules.
Connect to RDS or HPC storage¶
Attention
It is generally expected that you use RDS to store all your research project data as it is backed up. Connecting with HPC scratch storage may be useful to integrate HPC processing with your OpenStack applications but all datasets and results should ultimately be moved to RDS for permanent storage.
First you will need to contact us to request an SMB role account associated with your project storage. Once requested you will receive the account name and password via an encrypted email.
Save these credentials in to the file /etc/smb_creds in the format below and restrict the permissions with sudo chmod 600 /etc/smb_creds.
1 2 3 | |
RDS mount details¶
Create the mount point with mkdir -p /rds/<RDS share name>.
Add the following line at the bottom of /etc/fstab:
1 | |
HPC scratch mount details¶
Create the mount point with mkdir -p /hpc/<HPC directory name>.
Add the following line at the bottom of /etc/fstab:
1 | |
n.b. /users or /scratch locations e.g.:
1 | |