Setting up Deep Learning environment on Centos 7: Nvidia CUDA, Anaconda, iPython, Keras, Theano and Tensorflow

By |

The following is detailed, tested sequence of steps to setup universal deep learning environment on a minimal install of Centos 7. While Ubuntu is a bit more documented platform – Centos 7 has it’s own advantages and I wanted to target this specific version. My main reason is that Splunk is more suitable to run on Centos / RedHat and I want to make sure to have instructions that support abilities to apply deep learning in this environment.

That took me about a week to figure out and polish this sequence. This setup is somewhat caters to Jeremy Howard’s Deep Learning course, however it enables to setup frustration-free, universal environment suitable for other variations of practical deep learning.

You start with totally blank, minimal install of Centos 7 on a deciated server and end up with:

Nvidia CUDA* stuff fully installed.
Anaconda, iPython, Keras, Theano and Tensorflow + tons of other data science, math and machine learning software installed.
Both Python 2.x and Python 3.x setup and configured to run simultaneously (on a different ports: Python 2.x version on port 8882, Python 3.x version on port 8883) in a non-conflicting environment.

With my love to Nvidia hardware, Nvidia CUDA installation documentation is full of vague statements lacking clear steps to accomplish the task, and it took me many hours to assemble bits and pieces from all over the web to finally make Nvidia CUDA drivers and software work and operational.

I plan to add more comments and explanations to this section – but here are the step by step instructions to setup deep learning platform on fresh Centos 7 from zero to hero: […]

Comments Off on Setting up Deep Learning environment on Centos 7: Nvidia CUDA, Anaconda, iPython, Keras, Theano and Tensorflow
  • Tracking Identity of DDOS-for-Bitcoins criminal service operator with Splunk, Maltego and Domaintools

Connecting the Dots: Tracking Identity of DDOS-for-Bitcoins criminal service operator with Maltego, Splunk and Domaintools

By |

This post will demonstrate the ways to investigate and track real identity of an anonymous website operator promoting and selling DDOS attacking services for Bitcoins.

I built a system consisting of Maltego visual link analysis tool powered by DomainTools and third party bitcoin intelligence database as data providers. Maltego was connected to bitcoin intelligence data to combine it with domaintools data for powerful, interactive visual link anlysis. […]

  • Shift Card

Shift Card: Bitcoin Visa Debit Card that changes the game

By |

Bitcoin subject was on my inter […]

  • gleb_esman_splunk-500x

Joining Splunk as Senior Product Manager

By |

This summer been full of great news […]

  • detecting_web_malware_2

Real time detection and automated root cause analysis of web malware, exploits and backdoors with Splunk. Part 2, Detection and alerting.

By |

Continued from Part 1…
Adding alert on file system modification events
Let’s setup alert that will send email to administrator when some executable script will be modified on Web server under user’s file system space. We will run scheduled search every 5 minutes to scan last 5 minutes worth of modifications. […]

  • detecting_web_malware

Real time detection and automated root cause analysis of web malware, exploits and backdoors with Splunk. Part 1, Architecture.

By |

In this article I’ll demonstrate step by step how to setup Splunk analytics to detect successful known and unknown malware attacks on web hosting systems in real time.

In addition the same solution will include instructions to deploy fully automated investigative analytics to discover the origins of attackers (IP addresses) as well as any modifications within the file system.

This information is essential to discover and immediately eliminate all possible backdoors and exploits that attacker tried to plant.

Real time alerts will be delivered via email to system administrator as soon as attack occurs. The same information will be available via Splunk web interface for further analysis. […]

  • Detecting Human Emotions In Data

Predicting Unknown Threats: Detecting Human Emotions Through Machine Data Analytics

By |

Wouldn’t it be nice if your SIEM would send a “possible insider threat!” alert when it detects that employee is in fearful, paranoid or even panicky emotional state while trying to access secure, confidential corporate documents repository?

Or receive real time “possible account takeover!” alert when it detects that currently logged in user is in deep anxiety or experiencing severely agitated emotional conditions while trying to initiate money transfer to an outside bank account?

This approach is used very successfully to detect potential threats by the world’s most secure airlines.
Trained security officers are able to see if passenger feels nervous or agitated or otherwise exhibits emotionally unusual behaviors and then follow up with further checks and investigation. On one occasion by interrogating the nervous passenger the actual bomb was found inside his luggage while the passenger mistakenly thought he had been hired to smuggle diamonds.
The Step Up from User Behavior Analytics
With some creativity, knowledge of human psychology and analytics approach we can apply the same methods to today’s machine data generated by users, clients and employees of financial institutions, banks, governments facilities and corporations to prevent known and unknown attacks from outside as well as from inside of enterprise.

A while ago analyzing an account takeover cyber attack I’ve isolated a complete data set belonging to the attacker who’ve accessed another user account with legitimate credentials.

Attacker’s session activity was almost identical to legitimate user’s activity across all metrics:
Pages accessed, session duration, session hit length, browser user agent used, geographical region of originated IP address, order in which pages were accessed, approximately the same time of the date as legitimate user would use, etc…

[…]

  • session_velocity

User Behavior Analysis with Splunk: Detecting Threats and Fraudulent Activity in the Ocean of Behaviors: Part 2 – Detecting Abnormal User Session Velocity and Density

By |

One of my enterprise clients observed that certain class of attacks having a number of distinctive characteristics: attacker who possessed correct user account credentials won’t try to engage into malicious behavior right away.

Initial activity would involve certain degree of reconnaissance and gathering of future victim’s specific data, such as account balances, trade histories and other. So normal “red flags” and risk scoring metrics won’t generate any alerts.

However in many cases such pre-fraudulent activity was still carrying an unusual behavior marks: either session velocity (average number of seconds per hit) or session density (number of hits within the session) or both exceeded normal, baseline session patterns typical for the average client application user’s behavior.

Abnormally high session velocity is also a typical pattern of an automated script-driven session that both fraudsters and competition were using to siphon data from the client’s servers.

One of the possible solutions to detect these activities would be to calculate average session velocity and density and then apply these values to trigger alerts when session metrics exceeded thresholds.

The issue here is that due to the client’s business specific these averages vary greatly depending on the time of the day, time of the week and also the month of the year.

So stuffing some fixed “guessed” threshold values won’t work and will either generate lots of false positives or miss many suspicious sessions. […]