Building Software
A High-Performance Computing (HPC) environment like the Sol and Phoenix supercomputer varies greatly from a desktop environment like a personal workstation. When you build software for supercomputers, you must consider architecture, interconnects, binary compatibility, and permissions, to name a few. These considerations mean you will spend more time and effort building software.
Research Computing staff will make a best-effort attempt to install software upon request. Ultimately, it is the user’s responsibility to find a viable solution for their workflow, whether by compiling the code manually, using others' pre-compiled binaries or containers, or identifying alternative software.
Some software is central to a HPC environment, such as compiler suites, Python, R, and their library dependencies; Research Computing admins will always install and make Environment Modules to allow quick and easy access to these highly-requested software programs.
In other cases, such as 1) highly domain-specific software, 2) largely-untested code, and 3) development versions of software, it may be imprudent to create public Environment modules. Research Computing admins can still assist by building this software in more personalized locations where you can execute it without the use of the “module load” system.
If you build software for use by only one user, install it to your HOME directory; software like this typically only needs to be invoked by the filepath to operate, whether interactively or in a job script.
If you want software usable by a group/lab, install it to the group share directory, e.g., /data/grp_XXXX where XXXX is your group (visible with the command groups). Software installed in this location can be used by all users sharing that group, with both read and write permissions available. Source code that needs to be modified with custom changes will typically fall in this category.
If you request software that has wide appeal to many groups and users, this software will typically be built in the public fileshare, e.g., /packages/apps, and will be accompanied by a public environment modulefile which you can load with module load <package/version>.
Requesting software to be built
Submit a support ticket to request software installations.
Be sure to include all pertinent information:
- URL to software website/source code
- specific version requirements
- plugin/addon requirements
- location of downloaded source code (if behind paywall or login, or the file path downloaded to)
The supercomputer runs 100% Linux using Rocky Linux 8.x, a Redhat variant (RHEL). The supercomputer does not support Windows software.
We will try to build software that explicitly requires Debian, Ubuntu, etc., if it can compile. Beyond that, seek a container solution, as other Linux distributions rely on software managers and libraries likely incompatible with RedHat-compatible Rocky Linux.
Compiling software from source
Luckily, many of the compilation considerations mentioned above can be handled automatically, with provided build scripts and makefiles from the software author. You must understand Linux filesystem permissions to build software, as most software defaults to installing files in locations not permissible by regular, non-admin users.
Filesystem Permissions & Shared Storage
A valid--albeit abbreviated--installation of python might go like this:
wget https://www.python.org/ftp/python/3.9.7/Python-3.9.7.tgz
tar -xf Python-3.9.7.tgz
cd Python-3.9.7
./configure
make
make install
This example shows how to install software, as described on a tutorial website. Note, the installation will likely stop at the make install step, as it defaults to installing software to /usr/local, a system directory.
sudo access
sudo access is reserved explicitly for Research Computing admins. Thus, you cannot install software to system locations, e.g., /usr/local/python3.9.7.
While on workstations, sudo is a familiar way to allow installations to write files to this location, this is not a valid route on supercomputer clusters; instead, redirect the installation path to a directory for which you have permissions, which can include your HOME, scratch, or group project storage.
In many cases, tutorials may advise installing packages through a package manager, e.g., dnf or apt; this, too, is not a practical method in Supercomputer environments (it is again only practical for workstations).
With sudo access, you can install software to the default location /usr/local/… on a single node. However, on subsequent attempts connecting to the supercomputer, you are likely to be assigned to a different node, which means your installed files would be inaccessible to your new job and your jobscript would fail.
Thus, rely on /home/[asurite]/ or /scratch/[asurite] due to their universal availability on every node or use software modules.
Changing build directories
You can adjust the installation in a very simple way to install to your HOME directory:
...
$ ./configure --prefix=/home/[asurite]/.local/opt/python-3.9.7
$ make
$ make install
Container solutions
Apptainer (aka Singularity) is available on the RC supercomputers. Many times, software is distributed in an easy-to-download package (.sif file) which you can upload to your HOME directory and use immediately.
In some cases where a pre-configured Singularity is not available, but one can be built with a recipe, you must build the container on your local workstation (rather than the supercomputer), due to sudo/root requirements.
Please check this dedicated page for more information on Apptainer usage.
Docker is not supported in the supercomputer environment, as it was architected to use root (in its primary deployment use-case), and those permissions are not accessible to supercomputer end-users. Apptainer is the most suitable supercomputer-compatible approach and typically can work with docker images without issue.