TensorFlow is an end-to-end open source platform for machine learning.
There are a range of available installation options for TensorFlow on CREATE.
To avoid memory limitations on the login nodes, request an interactive session to complete your installation process. Please see the documentation on how to request more resources when using CREATE HPC.
To save space in your home directory, the following example assumes you have created a non-standard conda package cache location, however, this is not a requirement and the standard method will work just as well.
Using a Virtual Environment¶
If you require the latest stable version of TensorFlow, pip is recommended as TensorFlow is only officially released to PyPI. Make sure to first setup your virtual environment:
1 2 3 4
Or you can use conda, although it may not have the latest version, it is still a great option for repeatable analysis and is much easier to use for dependency management:
1 2 3
TensorFlow is also available as a module on CREATE:
module load py-tensorflow
1 2 3
Testing GPU Access with Tensorflow¶
If you installed TensorFlow with one of the virtual environment examples above, for working GPU access, you will need to load the following modules also:
1 2 3
Using TensorFlow in a Jupyter notebook¶
For a complete guide on how to launch Jupyter on CREATE HPC, then please refer to our guide document here. The following example makes use of the virtual environment created above and installs jupyterlab directly there:
However, when using CREATE modules and self-installed software, please make note of what python version is being used to avoid potential incompatibility issues.
Create a batch script for TensorFlow and Jupyter¶
Due to the resource overhead of both Jupyter and TensorFlow, please make sure you request a sufficient amount of compute resources via sbatch to avoid potential kernel instability issues when using your Jupyter notebook.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
Once you have submitted your batch script via sbatch, you can use the instructions printed in the slurm output to launch Jupyter notebook in your browser and test your TensorFlow:
1 2 3 4 5 6