Showing posts with label cuda. Show all posts
Showing posts with label cuda. Show all posts

Thursday, January 18, 2024

Build pytorch cpp_extention with cuda arch error

 Error:

"cpp_extension.py", line 1561, in _get_cuda_arch_flags
    arch_list[-1] += '+PTX'

IndexError: list index out of range

Solution:

Tried to investigate a bit this issue since I've faced the same problem in one of my Docker container.

If you're currently running your code through a setup.py , you should first add TORCH_CUDA_ARCH_LIST="YOUR_GPUs_CC+PTX" to run:

python TORCH_CUDA_ARCH_LIST="YOUR_GPUs_CC+PTX" setup.py install

(or an ARG TORCH_CUDA_ARCH_LIST="YOUR_GPUs_CC+PTX" in your Dockerfile for instance )

Additional infos. can be found here: https://pytorch.org/docs/stable/cpp_extension.html


CUDA_VERSION=$(/usr/local/cuda/bin/nvcc --version | sed -n 's/^.*release \([0-9]\+\.[0-9]\+\).*$/\1/p')
if [[ ${CUDA_VERSION} == 9.0* ]]; then
    export TORCH_CUDA_ARCH_LIST="3.5;5.0;6.0;7.0+PTX"
elif [[ ${CUDA_VERSION} == 9.2* ]]; then
    export TORCH_CUDA_ARCH_LIST="3.5;5.0;6.0;6.1;7.0+PTX"
elif [[ ${CUDA_VERSION} == 10.* ]]; then
    export TORCH_CUDA_ARCH_LIST="3.5;5.0;6.0;6.1;7.0;7.5+PTX"
elif [[ ${CUDA_VERSION} == 11.0* ]]; then
    export TORCH_CUDA_ARCH_LIST="3.5;5.0;6.0;6.1;7.0;7.5;8.0+PTX"
elif [[ ${CUDA_VERSION} == 11.* ]]; then
    export TORCH_CUDA_ARCH_LIST="3.5;5.0;6.0;6.1;7.0;7.5;8.0;8.6+PTX"
else
    echo "unsupported cuda version."
    exit 1
fi

If the gpu driver is loaded correctly, execute the following statement in the python console

>>> torch.cuda.get_device_capability(0)
(6, 1)

that means TORCH_CUDA_ARCH_LIST="6.1"However, in most cases, cuda is unavailable because you have specified gpu incorrectly.

Wednesday, May 20, 2020

Install cuda 10.2

1. Uninstall existing nvidia driver

nvidia-installer --uninstall

or

The correct way to uninstall just cuda and keep your nvidia drivers would be:
sudo apt purge "libcublas*" "cuda-*" cuda
Possible, because they were installed as requirements, you can also purge,
sudo apt purge "nsight-*" nvidia-modprobe
After that you can, if you want, also remove the nvidia drivers with:
sudo apt purge "*nvidia*"
Of course, if you installed cuda using nvidia's .run file then this won't work; in that case you probably have some uninstall script.

2. Close X server
# To stop:
sudo init 3
# To resume:
sudo init 5
3. Download & install
wget http://developer.download.nvidia.com/compute/cuda/10.2/Prod/local_installers/cuda_10.2.89_440.33.01_linux.runsudo sh cuda_10.2.89_440.33.01_linux.run

Monday, July 10, 2017

Install cudnn

Default Tensorflow supports cuda 8.0 and cudnn5.1, but I installed cudnn 6.0
I came across the same issue
In [1]: import tensorflow
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
/usr/local/lib/python3.5/site-packages/tensorflow/python/pywrap_tensorflow.py in <module>()
     40     sys.setdlopenflags(_default_dlopen_flags | ctypes.RTLD_GLOBAL)
---> 41   from tensorflow.python.pywrap_tensorflow_internal import *
     42   from tensorflow.python.pywrap_tensorflow_internal import __version__

/usr/local/lib/python3.5/site-packages/tensorflow/python/pywrap_tensorflow_internal.py in <module>()
     27             return _mod
---> 28     _pywrap_tensorflow_internal = swig_import_helper()
     29     del swig_import_helper

/usr/local/lib/python3.5/site-packages/tensorflow/python/pywrap_tensorflow_internal.py in swig_import_helper()
     23             try:
---> 24                 _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
     25             finally:

/usr/local/lib/python3.5/imp.py in load_module(name, file, filename, details)
    241         else:
--> 242             return load_dynamic(name, filename, file)
    243     elif type_ == PKG_DIRECTORY:

/usr/local/lib/python3.5/imp.py in load_dynamic(name, path, file)
    341             name=name, loader=loader, origin=path)
--> 342         return _load(spec)
    343

ImportError: libcudnn.so.5: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

ImportError                               Traceback (most recent call last)
<ipython-input-1-a649b509054f> in <module>()
----> 1 import tensorflow

/usr/local/lib/python3.5/site-packages/tensorflow/__init__.py in <module>()
     22
     23 # pylint: disable=wildcard-import
---> 24 from tensorflow.python import *
     25 # pylint: enable=wildcard-import
     26

/usr/local/lib/python3.5/site-packages/tensorflow/python/__init__.py in <module>()
     49 import numpy as np
     50
---> 51 from tensorflow.python import pywrap_tensorflow
     52
     53 # Protocol buffers

/usr/local/lib/python3.5/site-packages/tensorflow/python/pywrap_tensorflow.py in <module>()
     50 for some common reasons and solutions.  Include the entire stack trace
     51 above this error message when asking for help.""" % traceback.format_exc()
---> 52   raise ImportError(msg)
     53
     54 # pylint: enable=wildcard-import,g-import-not-at-top,unused-import,line-too-long

ImportError: Traceback (most recent call last):
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/pywrap_tensorflow.py", line 41, in <module>
    from tensorflow.python.pywrap_tensorflow_internal import *
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
    _pywrap_tensorflow_internal = swig_import_helper()
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
  File "/usr/local/lib/python3.5/imp.py", line 242, in load_module
    return load_dynamic(name, filename, file)
  File "/usr/local/lib/python3.5/imp.py", line 342, in load_dynamic
    return _load(spec)
ImportError: libcudnn.so.5: cannot open shared object file: No such file or directory


Failed to load the native TensorFlow runtime.

See https://www.tensorflow.org/install/install_sources#common_installation_problems

for some common reasons and solutions.  Include the entire stack trace
above this error message when asking for help.
I have installed cudnn 6.0 while it needs libcudnn.so.5, apparently it couldn't find libcudnn.so.5. It seems that your tensorflow needs cudnn 5.x, so install cudnn 5.x CUDNN INSTALLATION
Make sure you have already installed cuda 8.0 and exported the PATH and LD_LIBRARY_PATH
To install cudnn 5.x, try the following commands

Extract tgz files

$ tar -zxvf cudnn-8.0-linux-x64-v5.1.tgz

Check the files

$ cd cuda/lib64/
$ ls -l
total 150908
lrwxrwxrwx 1 doom doom       13 Nov  7  2016 libcudnn.so -> libcudnn.so.5
lrwxrwxrwx 1 doom doom       18 Nov  7  2016 libcudnn.so.5 -> libcudnn.so.5.1.10
-rwxr-xr-x 1 doom doom 84163560 Nov  7  2016 libcudnn.so.5.1.10
-rw-r--r-- 1 doom doom 70364814 Nov  7  2016 libcudnn_static.a
Here you will see 2 symbolic link files, and just copy libcudnn.so.5.1.10 and libcudnn_static.a to /usr/local/cuda/lib64

Make symbolic link files

$ cd /usr/local/cuda/lib64/
$ sudo ln -s libcudnn.so.5.1.10 libcudnn.so.5
$ sudo ln -s libcudnn.so.5 libcudnn.so
$ ls -l libcudnn*
lrwxrwxrwx 1 root root       13 May 24 09:24 libcudnn.so -> libcudnn.so.5
lrwxrwxrwx 1 root root       18 May 24 09:24 libcudnn.so.5 -> libcudnn.so.5.1.10
-rwxr-xr-x 1 root root 84163560 May 24 09:23 libcudnn.so.5.1.10
-rw-r--r-- 1 root root 70364814 May 24 09:23 libcudnn_static.a

Copy cudnn.h in include directory to /usr/local/cuda/include

$ sudo cp cudnn.h /usr/local/cuda/include/

Check

$ cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2

Install cuda 8.0 and cudnn in ubuntu 16.04

Original article link https://askubuntu.com/questions/799184/how-can-i-install-cuda-on-ubuntu-16-04

Install CUDA for Ubuntu

There is an Linux installation guide. However, it is basically only those steps:
  1. Download CUDA: I used the 16.04 version and "runfile (local)". That is 1.5 GB + 93MB.
  2. Check the md5 sum: md5sum cuda_7.5.18_linux.run. Only continue if it is correct.
  3. Remove any other installation (sudo apt-get purge nvidia-cuda* - if you want to install the drivers too, then sudo apt-get purge nvidia-*.)
    1. If you want to install the display drivers(*), logout from your GUI. Go to a terminal session (ctrl+alt+F2)
    2. Stop lightdm: $ sudo service lightdm stop or $ sudo init 3
    3. Create a file at /etc/modprobe.d/blacklist-nouveau.conf with the following contents: blacklist nouveau options nouveau modeset=0
    4. Then do: sudo update-initramfs -u
  4. sudo sh cuda_7.5.18_linux.run --override. Make sure that you say y for the symbolic link.
    1. Start lightdm again: sudo service lightdm start
  5. Follow the command-line prompts
  6. Need to reboot computer.
Notes: Yes, there is the possibility to install it via apt-get install cuda. I strongly suggest not to use it, as it changes the paths and makes the installation of other tools more difficult.
You might also be interested in How can I install CuDNN on Ubuntu 16.04?.
*: Don't install the display drivers with this script. They are old. Download the latest ones from http://www.nvidia.com/Download/index.aspx

Verify CUDA installation

The following command shows the current CUDA version (last line):
# Should add cuda path first, don't use $ sudo apt-get install nvidia-tool, the default version is 7.5, will be conflict later.
add two lines in ~/.bashrc
export PATH="/usr/local/cuda-8.0/bin:$PATH"
export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64:$LD_LIBRARY_PATH
$ source ~/.bashrc
May logout to make sure the path is added. can check by using $ echo $PATH

$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2016 NVIDIA Corporation
Built on Sun_Sep__4_22:14:01_CDT_2016
Cuda compilation tools, release 8.0, V8.0.44
The following command shows your driver version and how much GPU memory you have:
$ nvidia-smi
Fri Jan 20 12:19:04 2017       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 367.57                 Driver Version: 367.57                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce 940MX       Off  | 0000:02:00.0     Off |                  N/A |
| N/A   75C    P0    N/A /  N/A |   1981MiB /  2002MiB |     98%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0      1156    G   /usr/lib/xorg/Xorg                             246MiB |
|    0      3198    G   ...m,SecurityWarningIconUpdate<SecurityWarni   222MiB |
|    0      6645    C   python                                        1510MiB |
+-----------------------------------------------------------------------------+

Help! The new driver does not work!

Don't panic. Even if you can't see anything on your computer, the following steps should get you back to the state before:
  1. Press shift while startup
  2. Go into a root shell
  3. Make it writable by mount -o remount,rw / (- is ? and / is - in the american layout)
  4. sh cuda_7.5.18_linux.run --uninstall
  5. sudo apt-get install nvidia-361 nvidia-common nvidia-prime nvidia-settings

Graphic drivers

Installing the graphic drivers is a bit tricky. This has to be done without graphics support.
  1. Logout from your current X session.
  2. Ctrl+Alt+F4 (you can switch back with Ctrl+Alt+F7)
  3. You should remove all other drivers before.
    1. Search them via dpkg -l | grep -i nvidia
    2. Remove them via sudo apt-get remove --purge nvidia-WHATEVER
  4. Stop lightdm via sudo service lightdm stop
  5. You might need to reboot your pc / blacklist the nouveau driver (German tutorial)

The installation of CuDNN is just copying some files. Hence to check if CuDNN is installed (and which version you have), you only need to check those files.

Install CuDNN

Step 1: Register an nvidia developer account and download cudnn here (about 80 MB). You might need nvcc --version to get your cuda version.
Step 2: Check where your cuda installation is. For most people, it will be /usr/local/cuda/. You can check it with which nvcc.
Step 3: Copy the files:
$ cd folder/extracted/contents
$ sudo cp include/cudnn.h /usr/local/cuda/include
$ sudo cp lib64/libcudnn* /usr/local/cuda/lib64
$ sudo chmod a+r /usr/local/cuda/lib64/libcudnn*

Check version

You might have to adjust the path. See step 2 of the installation.

$ cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2