Skip to content

Containers

Containers enable consistent, reproducible environments by bundling code, libraries, and dependencies. They also provide secure environments for executing software, separated from data in the host system. JADE supports containers through Apptainer (a fork of Singularity). The advantage of Apptainer over the alternative Docker engine, is Apptainer does not require you have root/sudo/admin permissions to run.

Quick start

We have prepared a small set of useful containers for you to use, stored in folder /package/containers on the cluster:

/package/containers/
├── ubuntu            # Basic Ubuntu container, mimics JADE
├── R               
│   └── 4.5.2         # Ubuntu + R 4.5.2
├── rstudio-server  
│   ├── rserver-2025  # RStudio Server 2025 + R 4.5.1
│   └── rserver       # latest RStudio Server + R
├── miniforge         # Ubuntu + Miniforge
├── alphafold         # AlphaFold 2.3.2 + CUDA 11 + Python 3.8
├── codex             # OpenAI Codex in a sandbox
├── almalinux         # Basic AlmaLinux container, for a RHEL environment not Ubuntu
└── builder           # Ubuntu + compilers & libraries

Inside each folder are two important files. The *.sif file is the container image. The *.def is its definition file, a text file that specifies how container was created, and which you can customise to make your own containers.

Each of our containers has a small help message, which you can view with:

$ apptainer run-help /package/containers/ubuntu/ubuntu-22.sif
Ubuntu 22.04 with basic set of tools

To get a Bash shell inside a container:

$ apptainer exec /package/containers/ubuntu/ubuntu-22.sif bash
Apptainer> # now inside running container
Apptainer> echo $SINGULARITY_NAME
ubuntu-22.sif

Note: apptainer shell *.sif also gives you a Bash shell but without loading your ~/.bashrc - this can affect Mamba/Conda.

Press Ctrl-D or type exit to exit.

Apptainer can also run Docker images (from online repositories):

$ apptainer exec docker://ubuntu:22.04 bash
apptainer> cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.3 LTS"

Apptainer will automatically mount your home folder inside the container and set as starting location:

Apptainer> pwd
/home/a/aowenson

Special cases

  • To use our RStudio Server container, please follow our specific guide.

  • Our Codex container should be run with our launch script to setup sandbox: /package/containers/codex/launch.sh --help

Binding folders

You may have noticed neither /project nor /package was available inside the container. This is a consequence of default security behaviour - separating software inside from data outside. To make these other folders available inside, you must bind them. For example, to mount the sysadmin project add argument -B /project/sysadmin:

$ apptainer exec -B /project/sysadmin /package/containers/ubuntu/ubuntu-22.sif bash
Apptainer> ls /project
sysadmin

Mount multiple folders by chaining multiple -B arguments:

$ apptainer exec -B /project/sysadmin -B /package/python-cbrg /package/containers/ubuntu/ubuntu-22.sif bash
apptainer> ls /project
sysadmin
Apptainer> ls /package
python-cbrg

Downloads cache

Apptainer default caches downloaded containers in your home folder - ~/.apptainer/ - but with our 10 GB quota this will quickly create problems. Change the cache location with Bash variable APPTAINER_CACHEDIR, e.g. put in your .bashrc:

export APPTAINER_CACHEDIR=/project/example/myname/mycontainers

Nvidia GPUs

If running on a Nvidia GPU node, you should specify the argument --nv to bind GPU(s) to your container, e.g.:

apptainer exec --nv -B /project/example docker://tensorflow/tensorflow:2.15.0-gpu-jupyter bash

See Apptainer GPU documentation for more information.

Running prebuilt software

Some containers come with additional software prebuilt in, such as Jupyter Notebook with TensorFlow:

apptainer exec $bind_options docker://tensorflow/tensorflow:2.15.0-jupyter \
  jupyter notebook --no-browser --allow-root --ip 0.0.0.0 --port `id -u` \
  --notebook-dir=/path/to/my/notebook

This launches a web server inside your container, so a SSH tunnel is needed to connect.

There are various image repositories from Docker, Quay, etc. You may be instructed by the application developer an image is available somewhere. Even though they may not mention Apptainer works for your case, you can refer to the Apptainer page for starting the images from these providers.

Creating a container

You are encouraged to make your own container images; you can precompile everything you need into one single package. This significantly improves the reproducibility of your work.

As a comprehensive example, look at the following definition file MyNotebook.def. It starts from the ubuntu:22.04 image, builds a custom environment with the apt install and then pip install. It also defines the default command to run, to be a Jupyter Lab.

bootstrap: docker
FROM: ubuntu:22.04

%post
export TZ=Etc/GMT
export DEBIAN_FRONTEND=noninteractive
apt update
apt install -y python3 python3-pip python-is-python3 r-base-core g++ gcc
pip install --uploaded-prior-to P3D jupyterlab numpy scipy matplotlib ipywidgets bash_kernel
python -m bash_kernel.install
python -m pip cache purge
apt clean

%environment
export TZ=Etc/GMT
export DEBIAN_FRONTEND=noninteractive

%runscript
jupyter lab --no-browser --allow-root --ip 0.0.0.0 --port `id -u` --notebook-dir=/path/to/my/notebook

Then you can build the container image file MyNotebook.sif with:

apptainer build /project/example/myname/MyNotebook.sif MyNotebook.def

Finally, test your container:

apptainer run /project/example/myname/MyNotebook.sif

To use one of our prepared container images as a foundation rather than a Docker image downloaded from an online repository, or to use one of your own container images, adjust the top bootstrap instruction to e.g.:

bootstrap: local
FROM: /package/containers/ubuntu/ubuntu-22.sif

Conda

You can use our Miniforge container to create your own container that auto-loads a Conda environment. This example downloads samtools from bioconda and installs into the system folder of the new container - this avoids modifying your home folder. Then the %environment section auto-loads samtools.

Bootstrap: localimage
From: /package/containers/miniforge/miniforge.sif

%post
  export DEBIAN_FRONTEND=noninteractive
  apt update -y ; apt-get clean
  rm -rf /var/lib/apt/lists/*

  mamba init --system  # create /etc/profile.d/conda.sh
  . /etc/profile.d/conda.sh  # init Conda
  mamba create -n my-samtools -c bioconda -c conda-forge samtools -y  # setup your Conda environment

%environment
  export SINGULARITY_SHELL=/bin/bash
  . /etc/profile.d/conda.sh
  conda activate my-samtools  # auto-load your Conda environment

To run:

$ apptainer shell samtools.sif
(my-samtools) samtools

Program: samtools (Tools for alignments in the SAM format)
Version: 1.20 (using htslib 1.20)

Sharing a container

Apptainer containers are in SIF format, plain compressed files that can copy to other systems but with a condition: running requires the other system have same CPU architecture and similar operating system as the build system. JADE CPUs are AMD EPYC Milan running Linux operating system, so building on an Apple M2 Mac will not work. Intel processors are largely compatible, but be careful not to apply non-AMD optimisations such as AVX512.

Common problems

Here are some common problems when you move your work to an Apptainer container. Containerisation is not a simple task, talk to us via help@imm.ox.ac.uk for assistance.

File not found

Double-check your bind points. Then check for symbolic links that point outside of the bind points. Then check if you specified relative paths of input/output files - instead use full paths.

Read-only filesystem

With few exceptions* containers are read-only, so carefully review your program. Maybe it is trying to install more software.

* writable folders include: your home, any bound locations, and /tmp/

Quota exceeded

Check if your program is writing its output to your home directory. Review your quota usage with getquota -l.

GLIBC not found

This is a known bug of our current apptainer installation. Let us know if you encounter it so we can sort out for you.

/var/run/* not writable

Add argument --writable-tmpfs, e.g. apptainer run --writable-tmpfs ...