The following is detailed, tested sequence of steps to setup universal deep learning environment on a minimal install of Centos 7. While Ubuntu is a bit more documented platform – Centos 7 has it’s own advantages and I wanted to target this specific version. My main reason is that Splunk is more suitable to run on Centos / RedHat and I want to make sure to have instructions that support abilities to apply deep learning in this environment.
That took me about a week to figure out and polish this sequence. This setup is somewhat caters to Jeremy Howard’s Deep Learning course, however it enables to setup frustration-free, universal environment suitable for other variations of practical deep learning.
You start with totally blank, minimal install of Centos 7 on a deciated server and end up with:
- Nvidia CUDA* stuff fully installed.
- Anaconda, iPython, Keras, Theano and Tensorflow + tons of other data science, math and machine learning software installed.
- Both Python 2.x and Python 3.x setup and configured to run simultaneously (on a different ports: Python 2.x version on port 8882, Python 3.x version on port 8883) in a non-conflicting environment.
With my love to Nvidia hardware, Nvidia CUDA installation documentation is full of vague statements lacking clear steps to accomplish the task, and it took me many hours to assemble bits and pieces from all over the web to finally make Nvidia CUDA drivers and software work and operational.
I plan to add more comments and explanations to this section – but here are the step by step instructions to setup deep learning platform on fresh Centos 7 from zero to hero:
// Secure SSH side of the server and make it less anal // (note: disabling firewall was suitable for me because I had another firewall in front of it, but make sure you understand consequences) // # yum -y install nano policycoreutils-python # nano /etc/ssh/sshd_config // uncomment and change port to 7729 # semanage port -a -t ssh_port_t -p tcp 7729 // allow SSH port to be anything other than 22 # service sshd restart # systemctl stop firewalld // disable firewall. ISP/Router does firewalling. # systemctl disable firewalld // At this point you can login from any computer via putty/SSH to your server, port 7729. # adduser johnsmith # => your main user account # adduser p2 # account where Python 2.x stuff will be installed # adduser p3 # account where Python 3.x stuff will be installed # passwd johnsmith # passwd p2 # passwd p3 # usermod -aG wheel johnsmith // make user sudoer. # usermod -aG wheel p2 # usermod -aG wheel p3 $ yum -y update $ yum -y install screen mc net-tools lsof wget bzip2 # su - johnsmith $ echo "alias ll='ls -la' alias s='screen -d -r johnny' alias j='jupyter notebook'" >> ~/.bashrc $ source ~/.bashrc // Installing CUDA* according to: https://gist.github.com/wangruohui/df039f0dc434d6486f5d4d098aa52d07: $ sudo yum -y remove nvidia* $ sudo nano /etc/modprobe.d/blacklist-nouveau.conf // Add this to above file and save: blacklist nouveau options nouveau modeset=0 $ sudo mv /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).img.bak $ sudo dracut -v /boot/initramfs-$(uname -r).img $(uname -r) $ sudo reboot // These are not needed in bare bone server case: // $ sudo systemctl stop lightdm // $ sudo systemctl stop gdm // $ sudo systemctl stop kdm // Install NVIDIA driver. // $ sudo yum groups mark convert $ sudo yum -y group install "Development Tools" $ sudo yum -y --disablerepo="*" --enablerepo="epel" install dkms $ sudo yum -y install epel-release dkms libstdc++.i686 kernel-devel-$(uname -r) kernel-headers-$(uname -r) // Go to: http://www.nvidia.com/Download/index.aspx // Select drivers for your graphics card. Mine is: Nvidia EVGA 980 Ti, Linux, 64 bit. // Move it ( NVIDIA-Linux-x86_64-375.26.run ) via mountain Duck (great software!) to Linux // $ chmod +x NVIDIA-Linux-x86_64-375.26.run $ sudo ./NVIDIA-Linux-x86_64-375.26.run --dkms $ nvidia-smi // command will report all your CUDA-capable devices in the system. // Install CUDA from runfile installer. // Goto: https://developer.nvidia.com/cuda-downloads // Make proper selections on a web site, get link and download *runfile local*: // $ wget https://developer.nvidia.com/compute/cuda/8.0/prod/local_installers/cuda_8.0.44_linux-run $ mv cuda_8.0.44_linux-run cuda_8.0.44_linux.run $ chmod +x cuda_8.0.44_linux.run $ mkdir $HOME/1/2 // Just my weird choice of temporary directory names $ ./cuda_8.0.44_linux.run --extract=$HOME/1/2 $ cd 2 $ sudo ./cuda-linux64-rel-8.0.44-21122537.run // This is CUDA installer $ sudo ./cuda-samples-linux-8.0.44-21122537.run // This is samples installer // Follow prompts $ sudo bash -c "echo /usr/local/cuda/lib64/ > /etc/ld.so.conf.d/cuda.conf" $ sudo ldconfig $ echo "export PATH=\"/usr/local/cuda-8.0/bin:\$PATH\"" >> ~/.bashrc $ echo "export LD_LIBRARY_PATH=\"/usr/local/cuda-8.0/lib64:\$LD_LIBRARY_PATH\"" >> ~/.bashrc $ source ~/.bashrc $ cd .. // Install CuDNN. // Start here: https://developer.nvidia.com/cudnn // Create account and download stuff: // Goto: https://developer.nvidia.com/rdp/cudnn-download // Click on latest ( "Download cuDNN v5.1 (August 10, 2016), for CUDA 8.0" ) // Click on "cuDNN v5.1 Library for Linux" // It will save file as: cudnn-8.0-linux-x64-v5.1.solitairetheme8 // This is actually tar-compatible file // Transfer it to Linux (I user Mountain Duck software to transfer files from Windows 10 to Linux via SSH and via versa) $ sudo cp cudnn-8.0-linux-x64-v5.1.solitairetheme8 /usr/local/ $ su - # cd /usr/local # tar xzvf cudnn-8.0-linux-x64-v5.1.solitairetheme8 # rm -f cudnn-8.0-linux-x64-v5.1.solitairetheme8 # ldconfig # exit // Install Anaconda/iPython for Python 2.x . Reference: https://github.com/fastai/courses/blob/master/setup/install-gpu.sh $ su - p2 $ echo "alias ll='ls -la' alias s='screen -d -r johnny' alias j='jupyter notebook'" >> ~/.bashrc $ echo "export PATH=\"/usr/local/cuda-8.0/bin:\$PATH\"" >> ~/.bashrc $ echo "export LD_LIBRARY_PATH=\"/usr/local/cuda-8.0/lib64:\$LD_LIBRARY_PATH\"" >> ~/.bashrc $ source ~/.bashrc // 1. Goto: https://www.continuum.io/downloads // 2. Copy appropriate installation link (Linux, Python 2.x, 64 bit) // $ wget https://repo.continuum.io/archive/Anaconda2-4.2.0-Linux-x86_64.sh $ bash Anaconda2-4.2.0-Linux-x86_64.sh -b $ echo "export PATH=\"$HOME/anaconda2/bin:\$PATH\"" >> ~/.bashrc $ source ~/.bashrc $ conda install -y bcolz $ conda install -y -c anaconda toolz=0.8.2 $ conda upgrade -y --all $ mkdir ~/KEYS $ openssl req -x509 -nodes -days 3650 -newkey rsa:1024 -keyout ~/KEYS/mykey.key -out ~/KEYS/mycert.pem $ jupyter notebook --generate-config $ jupass=`python -c "from notebook.auth import passwd; print(passwd())"` $ echo "c.NotebookApp.password = u'"$jupass"'" >> $HOME/.jupyter/jupyter_notebook_config.py echo "c.NotebookApp.ip = '*' c.NotebookApp.open_browser = False c.NotebookApp.keyfile = u'"$HOME"/KEYS/mykey.key' c.NotebookApp.certfile = u'"$HOME"/KEYS/mycert.pem' c.NotebookApp.port = 8882 c.NotebookApp.ip = '*' c.NotebookApp.open_browser = False" >> $HOME/.jupyter/jupyter_notebook_config.py // Now you can do “jupyter notebook”, to listen on specified port: // $ screen -S johnny // $ jupyter notebook // Ctrl+A,D // Install Deep Learning course notebooks. $ cd $ git clone https://github.com/fastai/courses.git // It will create ‘~/courses/deeplearning1/nbs’ and other dirs. // Install Theano. $ pip install theano $ echo "[global] floatX = float32 device = gpu [nvcc] fastmath = True [lib] cnmem = 0.95 [cuda] root = /usr/local/cuda" > ~/.theanorc // Install Keras. $ cd $ pip install keras $ mkdir ~/.keras $ echo '{ "image_dim_ordering": "th", "epsilon": 1e-07, "floatx": "float32", "backend": "theano" }' > ~/.keras/keras.json // Install Tensorflow. $ cd $ pip install tensorflow-gpu // Alternative installation is listed here: https://www.tensorflow.org/get_started/os_setup#pip_installation // Replace with the latest versions: // $ export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.10.0-cp35-cp35m-linux_x86_64.whl // $ pip install --ignore-installed --upgrade $TF_BINARY_URL // Install Anaconda/iPython for Python 3.x . $ su - p3 $ echo "alias ll='ls -la' alias s='screen -d -r johnny' alias j='jupyter notebook'" >> ~/.bashrc $ echo "export PATH=\"/usr/local/cuda-8.0/bin:\$PATH\"" >> ~/.bashrc $ echo "export LD_LIBRARY_PATH=\"/usr/local/cuda-8.0/lib64:\$LD_LIBRARY_PATH\"" >> ~/.bashrc $ source ~/.bashrc // 1. Goto: https://www.continuum.io/downloads // 2. Copy appropriate installation link (Linux, Python 3.x, 64 bit) // $ wget https://repo.continuum.io/archive/Anaconda3-4.2.0-Linux-x86_64.sh $ bash Anaconda3-4.2.0-Linux-x86_64.sh -b $ echo "export PATH=\"$HOME/anaconda3/bin:\$PATH\"" >> ~/.bashrc $ source ~/.bashrc $ conda install -y bcolz $ conda install -y -c anaconda toolz=0.8.2 $ conda upgrade -y --all $ mkdir ~/KEYS $ openssl req -x509 -nodes -days 3650 -newkey rsa:1024 -keyout ~/KEYS/mykey.key -out ~/KEYS/mycert.pem $ jupyter notebook --generate-config $ jupass=`python -c "from notebook.auth import passwd; print(passwd())"` $ echo "c.NotebookApp.password = u'"$jupass"'" >> $HOME/.jupyter/jupyter_notebook_config.py echo "c.NotebookApp.ip = '*' c.NotebookApp.open_browser = False c.NotebookApp.keyfile = u'"$HOME"/KEYS/mykey.key' c.NotebookApp.certfile = u'"$HOME"/KEYS/mycert.pem' c.NotebookApp.port = 8883 c.NotebookApp.ip = '*' c.NotebookApp.open_browser = False" >> $HOME/.jupyter/jupyter_notebook_config.py // Now you can can jupyter notebook, to listen on specified port: // // $ screen -S johnny // $ jupyter notebook // Ctrl+A,D // Install Theano. $ pip install theano $ echo "[global] floatX = float32 device = gpu [nvcc] fastmath = True [lib] cnmem = 0.95 [cuda] root = /usr/local/cuda" > ~/.theanorc // Install Keras. $ cd $ pip install keras $ mkdir ~/.keras $ echo '{ "image_dim_ordering": "th", "epsilon": 1e-07, "floatx": "float32", "backend": "theano" }' > ~/.keras/keras.json // Install Tensorflow. $ cd $ pip install tensorflow-gpuConnect with me on LinkedIn
Gleb Esman is currently working as Senior Product Manager for Security/Anti-fraud solutions at Splunk leading efforts to build next generation security products covering advanced fraud cases across multiple industry verticals.
Contact Gleb Esman.