Configure docker for NVIDIA acceleration

Introduction

The Docker container platform is a great way to deploy and install software while maintaining a consistent user and developer experience. If you would like to learn how to set up Docker on your Ubuntu system, follow this guide: Install Docker on Ubuntu. However some docker images may be more suited to graphics acceleration, which requires some configuration to utilise. This guide will show you how to set up the NVIDIA Container Toolkit, so that your Docker containers can take advantage of the host NVIDIA hardware, perfect for applications such as TensorFlow to turbocharge your AI/ML models.

Requirements

Before you continue, please ensure you have the following:

1. A computer with an NVIDIA graphics card/ chip
2. Ubuntu 22.04 LTS installed
3. NVIDIA graphics drivers installed (NVIDIA drivers are pre-installed on Entroware NVIDIA systems and do not need to be reinstalled)
4. Docker installed (Install Docker on Ubuntu)

Install the NVIDIA Container Toolkit package

Ensure the curl command is installed with

1user@machine:~$ sudo apt install curl

Add the NVIDIA Container Toolkit GPG keys and add the repository

1user@machine:~$ curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
2  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
3    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
4    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

1user@machine:~$ sudo sed -i -e '/experimental/ s/^#//g' /etc/apt/sources.list.d/nvidia-container-toolkit.list

Update the apt package list

1user@machine:~$ sudo apt-get update

Install the NVIDIA Container Toolkit package

1user@machine:~$ sudo apt-get install -y nvidia-container-toolkit

Configure the Container Toolkit and Docker

1user@machine:~$ sudo nvidia-ctk runtime configure --runtime=docker

1user@machine:~$ sudo systemctl restart docker

1user@machine:~$ nvidia-ctk runtime configure --runtime=docker --config=$HOME/.config/docker/daemon.json

1user@machine:~$ sudo nvidia-ctk config --set nvidia-container-cli.no-cgroups --in-place

Test Docker GPU detection

nvidia-smi

The following test will use an Ubuntu Docker container to run the nvidia-smi command, which will determine whether an NVIDIA GPU is being detected within the container.

1user@machine:~$ docker run --rm --runtime=nvidia --gpus all \
2    --device=/dev/nvidia-uvm \
3    --device=/dev/nvidia-uvm-tools \
4    --device=/dev/nvidia-modeset \
5    --device=/dev/nvidiactl \
6    --device=/dev/nvidia0 \
7    ubuntu nvidia-smi

If a graphics card has been successfully detected, the output should display a table, showing the model of the GPU detected, plus a table showing the processes that are currently being accelerated by the GPU (if applicable).

TensorFlow

The next test will use tensorflow:latest-gpu Docker container to test for GPU detection

1user@machine:~$ docker run --rm --runtime=nvidia --gpus all \
2    --device=/dev/nvidia-uvm \
3    --device=/dev/nvidia-uvm-tools \
4    --device=/dev/nvidia-modeset \
5    --device=/dev/nvidiactl \
6    --device=/dev/nvidia0 \
7    tensorflow/tensorflow:latest-gpu \
8   python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"

Successful GPU detection output should look similar to

1[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]

Unsuccessful GPU detection output will look like

1CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected

Conclusion

You should now be able to run Docker containers with GPU acceleration to maximise the resources available to your applications. Whether you need to accelerate video transcoding, AI/ML models or other GPU based workloads, you can now reap the benefits as both a user and a developer.