Useful link

Monday, July 10, 2017

Install cuda 8.0 and cudnn in ubuntu 16.04

Original article link https://askubuntu.com/questions/799184/how-can-i-install-cuda-on-ubuntu-16-04

Install CUDA for Ubuntu

There is an Linux installation guide. However, it is basically only those steps:

Download CUDA: I used the 16.04 version and "runfile (local)". That is 1.5 GB + 93MB.
Check the md5 sum: md5sum cuda_7.5.18_linux.run. Only continue if it is correct.
Remove any other installation (sudo apt-get purge nvidia-cuda* - if you want to install the drivers too, then sudo apt-get purge nvidia-*.)
1. If you want to install the display drivers^(*), logout from your GUI. Go to a terminal session (ctrl+alt+F2)
2. Stop lightdm: $ sudo service lightdm stop or $ sudo init 3
3. Create a file at /etc/modprobe.d/blacklist-nouveau.conf with the following contents: blacklist nouveau options nouveau modeset=0
4. Then do: sudo update-initramfs -u
sudo sh cuda_7.5.18_linux.run --override. Make sure that you say y for the symbolic link.
1. Start lightdm again: sudo service lightdm start
Follow the command-line prompts
Need to reboot computer.

Notes: Yes, there is the possibility to install it via apt-get install cuda. I strongly suggest not to use it, as it changes the paths and makes the installation of other tools more difficult.

You might also be interested in How can I install CuDNN on Ubuntu 16.04?.

*: Don't install the display drivers with this script. They are old. Download the latest ones from http://www.nvidia.com/Download/index.aspx

Verify CUDA installation

The following command shows the current CUDA version (last line):

# Should add cuda path first, don't use $ sudo apt-get install nvidia-tool, the default version is 7.5, will be conflict later.

add two lines in ~/.bashrc

export PATH="/usr/local/cuda-8.0/bin:$PATH"

export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64:$LD_LIBRARY_PATH

$ source ~/.bashrc

May logout to make sure the path is added. can check by using $ echo $PATH

$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2016 NVIDIA Corporation
Built on Sun_Sep__4_22:14:01_CDT_2016
Cuda compilation tools, release 8.0, V8.0.44

The following command shows your driver version and how much GPU memory you have:

$ nvidia-smi
Fri Jan 20 12:19:04 2017       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 367.57                 Driver Version: 367.57                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce 940MX       Off  | 0000:02:00.0     Off |                  N/A |
| N/A   75C    P0    N/A /  N/A |   1981MiB /  2002MiB |     98%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0      1156    G   /usr/lib/xorg/Xorg                             246MiB |
|    0      3198    G   ...m,SecurityWarningIconUpdate<SecurityWarni   222MiB |
|    0      6645    C   python                                        1510MiB |
+-----------------------------------------------------------------------------+

Help! The new driver does not work!

Don't panic. Even if you can't see anything on your computer, the following steps should get you back to the state before:

Press shift while startup
Go into a root shell
Make it writable by mount -o remount,rw / (- is ? and / is - in the american layout)
sh cuda_7.5.18_linux.run --uninstall
sudo apt-get install nvidia-361 nvidia-common nvidia-prime nvidia-settings

Graphic drivers

Installing the graphic drivers is a bit tricky. This has to be done without graphics support.

Logout from your current X session.
Ctrl+Alt+F4 (you can switch back with Ctrl+Alt+F7)
You should remove all other drivers before.
1. Search them via dpkg -l | grep -i nvidia
2. Remove them via sudo apt-get remove --purge nvidia-WHATEVER
Stop lightdm via sudo service lightdm stop
You might need to reboot your pc / blacklist the nouveau driver (German tutorial)

The installation of CuDNN is just copying some files. Hence to check if CuDNN is installed (and which version you have), you only need to check those files.

Install CuDNN

Step 1: Register an nvidia developer account and download cudnn here (about 80 MB). You might need nvcc --version to get your cuda version.

Step 2: Check where your cuda installation is. For most people, it will be /usr/local/cuda/. You can check it with which nvcc.

Step 3: Copy the files:

$ cd folder/extracted/contents
$ sudo cp include/cudnn.h /usr/local/cuda/include
$ sudo cp lib64/libcudnn* /usr/local/cuda/lib64
$ sudo chmod a+r /usr/local/cuda/lib64/libcudnn*

Check version

You might have to adjust the path. See step 2 of the installation.

$ cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2

[Solved] Problems in installing caffe in ubuntu 16.04

A very complete guidline
https://github.com/BVLC/caffe/wiki/Ubuntu-16.04-or-15.10-Installation-Guide

It is easier to use cmake than to use the Makefile.config.

1. Problems caused by Anaconda.
Since Anaconda use gcc 4.7 and the Ubuntu 16 system use gcc 5.4, there occur a lot problems.

Solution:
Don't use Anaconda.
$ vim ~/.bashrc
comment line #export PATH="/home/ks/anaconda2/bin:$PATH"
$ sudo apt install python-pip
$ cd <caffe-master>/python
$ for req in $(cat requirements.txt); do pip install $req; done
$ cd ..
$ mkdir build1
$ cd build1
$ cmake ..
$ make -j<nproc>

Description copied from http://www.cnblogs.com/kingstrong/p/6015157.html

CXX/LD -o .build_release/tools/compute_image_mean.bin
.build_release/tools/test_net.o：在函数‘boost::system::system_error::what() const’中：
test_net.cpp:(.text._ZNK5boost6system12system_error4whatEv[_ZNK5boost6system12system_error4whatEv]+0x74)：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_replace(unsigned long, unsigned long, char const*, unsigned long)’未定义的引用
test_net.cpp:(.text._ZNK5boost6system12system_error4whatEv[_ZNK5boost6system12system_error4whatEv]+0xa1)：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_append(char const*, unsigned long)’未定义的引用
test_net.cpp:(.text._ZNK5boost6system12system_error4whatEv[_ZNK5boost6system12system_error4whatEv]+0xe3)：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_append(char const*, unsigned long)’未定义的引用
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libgflags.so：对��.build_release/tools/train_net.o：在函数‘boost::system::system_error::what() const’中：
train_net.cpp:(.text._ZNK5boost6system12system_error4whatEv[_ZNK5boost6system12system_error4whatEv]+0x74)：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_replace(unsigned long, unsigned long, char const*, unsigned long)’未定义的引用
train_net.cpp:(.text._ZNK5boost6system12system_error4whatEv[_ZNK5boost6system12system_error4whatEv]+0xa1)：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_append(char const*, unsigned long)’未定义的引用
train_net.cpp:(.text._ZNK5boost6system12system_error4whatEv[_ZNK5boost6system12system_error4whatEv]+0xe3)：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_append(char const*, unsigned long)’未定义的引用
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libgflags.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::compare(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long)@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_replace_aux(unsigned long, unsigned long, unsigned long, char)@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_append(char const*, unsigned long)@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_replace(unsigned long, unsigned long, char const*, unsigned long)@GLIBCXX_3.4.21’未定义的引用
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libgflags.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct(unsigned long, char)@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘vtable for std::__cxx11::basic_stringbuf<char, std::char_traits<char>, std::allocator<char> >@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::assign(char const*)@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_assign(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libgflags.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::compare(std::__cxx11::basic_string<char, std::<char_traits<char>char>, std:> :allocator<char> > const&) const@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long)@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_replace_aux(unsigned long, unsigned long, unsigned long, char)@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_append(char const*, unsigned long)@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_replace(unsigned long, unsigned long, char const*, unsigned long)@GLIBCXX_3.4.21’未定义的引用
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libgflags.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct(unsigned long, char)@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/�libglog.so：对‘vtable for std::__cxx11::basic_stringbuf<char, std::char_traits<char>, std::allocator<char> >@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::assign(char const*)@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_assign(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)@GLIBCXX_3.4.21’未定义的引用
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libgflags.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_erase(unsigned long, unsigned long)@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘vtable for std::__cxx11::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >@GLIBCXX_3.4.21’未定义的引用
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libgflags.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::reserve(unsigned long)@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：collect2: error: ld returned 1 exit status
对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::compare(char const*) const@GLIBCXX_3.4.21’未定义的引用
collect2: error: ld returned 1 exit status
/../x86_64-linux-gnu/libgflags.so：�对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_replace(unsigned long, unsigned long, char const*, unsigned long)’未定义的引用
finetune_net.cpp:(.text._ZNK5boost6system12system_error4whatEv[_ZNK5boost6system12system_error4whatEv]+0xa1)：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_append(char const*, unsigned long)’未定义的引用
finetune_net.cpp:(.text._ZNK5boost6system12system_error4whatEv[_ZNK5boost6system12system_error4whatEv]+0xe3)：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_append(char const*, unsigned long)’未定义的引用
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libgflags.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::compare(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long)@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_replace_aux(unsigned long, unsigned long, unsigned long, char)@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_append(char const*, unsigned long)@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_replace(unsigned long, unsigned long, char const*, unsigned long)@GLIBCXX_3.4.21’未定义的引用
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libgflags.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct(unsigned long, char)@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘vtable for std::__cxx11::basic_stringbuf<char, std::char_traits<char>, std::allocator<char> >@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::assign(char const*)@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_assign(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)@GLIBCXX_3.4.21’未定义的引用
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libgflags.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_erase(unsigned long, unsigned long)@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘vtable for std::__cxx11::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >@GLIBCXX_3.4.21’未定义的引用
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libgflags.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::reserve(unsigned long)@GLIBCXX_3.4.21’未定义的引用
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libgflags.so：对‘std::__throw_out_of_range_fmt(char const*, ...)@GLIBCXX_3.4.20’未定义的引用
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libgflags.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::rfind(char, unsigned long) const@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘std::__cxx11::basic_stringbuf<char, std::char_traits<char>, std::allocator<char> >::_M_sync(char*, unsigned long, unsigned long)@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string()@GLIBCXX_3.4.21’未定义的引用
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libgflags.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::append(char const*)@GLIBCXX_3.4.21’未定义的引用
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libgflags.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::find(char, unsigned long) const@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘std::__cxx11::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >::~basic_ostringstream()@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘VTT for std::__cxx11::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >@GLIBCXX_3.4.21’未定义的引用
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libgflags.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::find(char const*, unsigned long, unsigned long) const@GLIBCXX_3.4.21’未定义的引用
/usr/local/lib/libglog.so：对‘std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::compare(char const*) const@GLIBCXX_3.4.21’未定义的引用
collect2: error: ld returned 1 exit status
。。。很长一串问题

对于这个问题找了N久都没有找到答案，在caffe的issue中看到有人有同样的问题，但并没有得到答案，后来在另一个问题中看到了一个方法https://github.com/BVLC/caffe/issues/2348，

cd caffe
cp Makefile.config.example Makefile.config
and change setting in Makefile.config
mkdir build
cd build
cmake ..
./configure (??not sure??)
make all

执行一遍后发现有戏啊，可是又有一个新的关于opencv_imgcodecs的问题，同样在刚才的链接中的方法

add "opencv_imgcodecs" in Makefile.(LIBRARIES += glog gflags protobuf leveldb snappy \
lmdb boost_system hdf5_hl hdf5 m \
opencv_core opencv_highgui opencv_imgproc opencv_imgcodecs)
就是在makefile的“LIBRARIES +=”中添加一个“opencv_imgcodecs”,make clean，再make all这样刚才的问题就没有了

这样就可以了？？？结果又出了一个新的问题，/usr/lib/libopencv_highgui.so.2.4: undefined reference to TIFFRGBAImageOK@LIBTIFF_4.0' 1>

网上说是opencv编译的问题，于是就重新编译opencv，这次编译cmake加上一个参数：

-D BUILD_TIFF=ON
于是又屁颠屁颠的开始编译了，这一次已经有了报错的心里准备了，果然新的问题来了，//usr/lib/x86_64-linux-gnu/libsoxr.so.0:对‘GOMP_parallel@GOMP_4.0’未定义的，
对于这个问题我以为百度一下就能解决，可事实证明我还是too young too simple，这个问题找了很久都没找到答案，但是有个发现，就是稍微沾边的解释里都有anaconda，突然醒悟，我现在遭的罪不都是安装了anaconda后造成的么，搞不好是anaconda与系统兼容造成的，于是我决定anaconda
零时屏蔽了再来编译opencv试试，屏蔽的具体方法是在

sudo gedit ~/.bashrc
在下面这两句话前面加上“#”注释掉
#export PATH="/home/ks/anaconda2/bin:$PATH"
#export LD_LIBRARY_PATH="/home/ks/anaconda2/lib:$LD_LIBRARY_PATH"

然后再次来编译opencv，奇迹出现啦，经过漫长的等待，opencv终于编译完成啦，现在回头想想，caffe之所以编译报那么多错肯定也是由anaconda造成的,但是屏蔽anaconda后用以前的方法直接编译caffe还是会报很多错，不过用上面提到的cmake的方式进行编译就顺利的完成了。

Ubuntu Install conda with python2.7

Sample of install Anaconda with python 3
https://www.digitalocean.com/community/tutorials/how-to-install-the-anaconda-python-distribution-on-ubuntu-16-04

1. Download from the official website https://www.continuum.io/downloads
or

$ curl -O https://repo.continuum.io/archive/Anaconda2-2.4.0-Linux-x86_64.sh

2. compare meta data

$ sha256sum Anaconda2-2.4.0-Linux-x86_64.sh

3. Install

$ sh Anaconda2-2.4.0-Linux-x86_64.sh

4. source

$ source ~/.bashrc

[Solved] Ubuntu log in GUI problem

Solve Ubuntu log in GUI problem

Problem definition:

I have ubuntu-desktop 16.04 to access it via GUI. Now I get the GUI login screen with my username (display name) but when i enter my password the screen just flicks and comes back to login screen ( doesnt give me any error). the same user works fine with same credentials in CLI.

strange thing is when i click on guest, it logs me in straight away and works fine.

Solution from stackflow:

down vote

This looks like the .Xauthority file is corrupted. Try this in a tty:

sudo mv .Xauthority .Xauthority_backup

Then restart lightdm and try to log in again:

sudo service lightdm restart

If that was not the cause, restore the file to its previous state:

sudo mv .Xauthority_backup .Xauthority