Nvidia GTX 1080 on Ubuntu 16.04 for Deep Learning

Sunday, June 12, 2016

I got a Nvidia GTX 1080 last week and want to make it run Caffe on Ubuntu 16.04. After some trial-and-errors, I findally made it work. The speed is very fast and the price of card is reasonable($699) and the power consumption is low(180Watts maximum).

Testing setup

Hardware:

  • Dell Precision Workstation T7500
  • Nvidia GeForce 5000
  • Nvidia GTX 1080

Software:

  • Ubuntu 16.04 x86_64

  • CUDA Toolkit 8 Now Available. Install it as follows:
    • Download CUDA 8.0 and install as follows:
    wget http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_8.0.61-1_amd64.deb
wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/libcudnn6_6.0.21-1%2Bcuda8.0_amd64.deb
wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/libcudnn6-dev_6.0.21-1%2Bcuda8.0_amd64.deb
sudo dpkg -i cuda-repo-ubuntu1604_8.0.61-1_amd64.deb
sudo dpkg -i libcudnn6_6.0.21-1+cuda8.0_amd64.deb
sudo dpkg -i libcudnn6-dev_6.0.21-1+cuda8.0_amd64.deb
sudo apt-get update
sudo apt-get install cuda=8.0.61-1
sudo apt-get install libcudnn6-dev

    • Reboot the system to load the NVIDIA drivers.

    • Skip the following steps(deprecated kept for historical reasons) to the step of “set up the development environment”.

  • Install CUDA 8.0RC: (These steps are obsolete, keep them here for record, if you want to upgrade to CUDA 8.0, follow the above steps)
    • Install Nvidia driver 367.18 or later from ppa:graphics-drivers/ppa as follows:
    sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt-get update
sudo apt-get install nvidia-367
sudo apt-get install mesa-common-dev
sudo apt-get install freeglut3-dev

    • Reboot the system to load the NVIDIA drivers.

    • Download CUDA 8.0RC. In order to download the installers, you need to login or register(free) first. Choose Linux -> x86_64 -> Ubuntu -> 16.04 -> runfile(local). Download both Base Installer and Patch 1. Run command:

      sudo sh cuda_8.0.27_linux.run

      Follow the command-line prompts. When asked the following question:

      Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 361.77?

      (y)es/(n)o/(q)uit: n

      make sure to type n as above to decline on installing the old drivers. Otherwise the latest driver installed in the last step will be overwritten.

      If the installation failed and you see the following error message:

      Toolkit: InstallationFailed. Using unsupported Compiler.

      please change the installing command to:

      sudo sh cuda_8.0.27_linux.run --override

    • Similarly install Patch 1 as follows:

    sudo sh cuda_8.0.27.1_linux.run

  • Install cuDNN. Download cuDNN 5.1 from NVIDIA CUDA Deep Neural Network library (cuDNN). Choose

    Download cuDNN v5.1 (August 10, 2016), for CUDA 8.0 RC

    cuDNN v5.1 Library for Linux

    to download. Uncompress and copy the cuDNN files into the CUDA directory. Assuming the CUDA toolkit is installed in /usr/local/cuda, run the following commands:

    tar xvzf cudnn-8.0-linux-x64-v5.1.tgz
sudo cp cuda/include/cudnn.h /usr/local/cuda/include
sudo cp cuda/lib64/libcudnn* /usr/local/cuda/lib64
sudo chmod a+r /usr/local/cuda/include/cudnn.h /usr/local/cuda/lib64/libcudnn*

  • Set up the development environment by modifying the PATH and LD_LIBRARY_PATH variables, also add them to the end of .bashrc file:

    export PATH=/usr/local/cuda-8.0/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

Test run CUDA sample nbody

First let’s see more details about the card:

~$ nvidia-smi
Mon Jun 13 11:43:53 2016
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 367.27                 Driver Version: 367.27                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Quadro 5000         Off  | 0000:03:00.0      On |                  Off |
| 30%   64C   P12    N/A /  N/A |    376MiB /  2493MiB |      3%      Default |
+-------------------------------+----------------------+----------------------+
|   1  GeForce GTX 1080    Off  | 0000:04:00.0     Off |                  N/A |
| 27%   38C    P8     6W / 180W |      1MiB /  8113MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Build and run the nbody sample:

$ cd NVIDIA_CUDA-8.0_Samples/5_Simulations/nbody
$ make
$ ./nbody -benchmark -numbodies=256000 -device=0
> Windowed mode
> Simulation data stored in video memory
> Single precision floating point simulation
> 1 Devices used for simulation
gpuDeviceInit() CUDA Device [0]: "Graphics Device
> Compute 6.1 CUDA device: [Graphics Device]
number of bodies = 256000
256000 bodies, total time for 10 iterations: 2384.865 ms
= 274.800 billion interactions per second
= 5495.992 single-precision GFLOP/s at 20 flops per interaction

Install Caffe

There are two ways to install Caffe: