Gleb – Mensk Technologies Inc

Setting up Deep Learning environment on Centos 7: Nvidia CUDA, Anaconda, iPython, Keras, Theano and Tensorflow

The following is detailed, tested sequence of steps to setup universal deep learning environment on a minimal install of Centos 7. While Ubuntu is a bit more documented platform – Centos 7 has it’s own advantages and I wanted to target this specific version. My main reason is that Splunk is more suitable to run on Centos / RedHat and I want to make sure to have instructions that support abilities to apply deep learning in this environment.

That took me about a week to figure out and polish this sequence. This setup is somewhat caters to Jeremy Howard’s Deep Learning course, however it enables to setup frustration-free, universal environment suitable for other variations of practical deep learning.

You start with totally blank, minimal install of Centos 7 on a deciated server and end up with:

Nvidia CUDA* stuff fully installed.
Anaconda, iPython, Keras, Theano and Tensorflow + tons of other data science, math and machine learning software installed.
Both Python 2.x and Python 3.x setup and configured to run simultaneously (on a different ports: Python 2.x version on port 8882, Python 3.x version on port 8883) in a non-conflicting environment.

With my love to Nvidia hardware, Nvidia CUDA installation documentation is full of vague statements lacking clear steps to accomplish the task, and it took me many hours to assemble bits and pieces from all over the web to finally make Nvidia CUDA drivers and software work and operational.

I plan to add more comments and explanations to this section – but here are the step by step instructions to setup deep learning platform on fresh Centos 7 from zero to hero: (more…)

Connecting the Dots: Tracking Identity of DDOS-for-Bitcoins criminal service operator with Maltego, Splunk and Domaintools

This post will demonstrate the ways to investigate and track real identity of an anonymous website operator promoting and selling DDOS attacking services for Bitcoins.

I built a system consisting of Maltego visual link analysis tool powered by DomainTools and third party bitcoin intelligence database as data providers. Maltego was connected to bitcoin intelligence data to combine it with domaintools data for powerful, interactive visual link anlysis. (more…)

Shift Card: Bitcoin Visa Debit Card that changes the game

Shift Card

Bitcoin subject was on my interest radar for quite a while.

Being involved in a number of projects related to modern e-commerce always saw disconnect between bitcoin as a

[crypto] currency and the real world of products, stores and services where it’s hard to impossible to use bitcoin for day to day transactions. In other words if you have few bitcoins available for joy – it’s hard to find an easy way to spend it as you see fit.

Sure, there were few geeky stores here and there that had weird “Bitcoin Accepted Here” signs, but nothing much beyond that.

This is about to change. Shift Payments just released Bitcoin Visa Debit Card that is (as of December 2015) usable in 24 states in USA anywhere where Visa is accepted. Currently supported states are:

Alabama, Arizona, California, Delaware, District of Columbia, Georgia, Idaho, Iowa, Kansas, Maine, Mississippi, Nebraska, Nevada, New Jersey, North Carolina, North Dakota, Oklahoma, Oregon, Pennsylvania, Puerto Rico, South Dakota, Texas, Vermont, Washington, West Virginia.

That suddenly makes life much easier for bitcoin owners – now you can buy stuff against your bitcoin balance. Which bitcoin balance? Shift Payments integrates with Dwolla and Coinbase, which means you can make dollar-based purchases that will auto-convert bitcoins at your Coinbase account to dollar with backup funds (in case you don’t have enough bitcoins) to be pulled out of Dwolla to fit the bill. You do not need to have Dwolla, just having Coinbase account is enough.

Getting Shift Card

I have account at Coinbase. When I got notification about bitcoin-based debit card availability – I ordered one right away to fully test it.

Shift Card is not the world’s first offering of bitcoin based debit card. There are few players here and there including some shady ones, although for me personally this was the easiest one to get because of my Coinbase account and California location.

When I ordered Shift card (which cost $10 flat, one time fee) the transaction at my Coinbase looked like that:

In about 2 weeks I got an envelope with the actual card, some basic information and activation procedures.

Bitcoin Visa Debit Card: Shift Card

First things I wanted to test is how many places actually accept this card, how many merchants would cause hassles or glitches regarding this card and what’s actual exchange rate that is being applied on purchases.

As a bonus test, I wanted to see what’s the deal with withdrawing the cold hard USD cash out of ATM machine against my bitcoin balance using this card. And of course – what are the fees and exchange rates.

For exchange rate matching I used coinbase own bitcoin charts.

Using bitcoin debit card to pay for parking

As soon as I got card and activated it – very simple call to the 1-800 number provided – I took my daughter and we went to San Francisco Union Square for a stroll.

After initial struggle of finding street parking on a busy Saturday morning I decided to reward local street parking pay machine with my new Shift card payment.

This went flawless – and payment went without a glitch. For a few seconds I event felt like a thief – I was almost sure that my card ought to be declined or reported as invalid or something like that.

But more than that – within a few seconds I’ve received an email confirmation from that my payment went through just fine.

Here’s how parking charge got reflected at Coinbase account:

Every time I used Shift Card the Coinbase bitcoin exchange rate snapshot was made. According to this data the exchange rate of transaction was $377.9378 – which is quite fairly in the middle of bid/ask spread. So far so good!

Exchange rate when paid for parking

Buying Nespresso coffee with bitcoin Shift debit Card

Next test was done at local Nespresso store. This time Shift card passed through payment processing terminal. Again – no problems or issues whatsoever:

06-buying-nespresso-coffee-with-bitcoins

Paying with bitcoin visa debit card at Nespresso coffee store

Payment notification was delivered to me immediately by email.

Exchange rate at Nespresso purchase

The total transaction was $35.50. Coinbase record shown dollar amount charge to be $35.49 which took 0.093933 bitcoins out of my account.

The exchange rate of that transaction was 377.82. Which is again – almost exactly in the middle of that moment’s bid/ask spread (377.86). Which is fair enough.

Paying for gas with bitcoins: using bitcoin visa debit card: Shift card

Next test was done at self service gas pump in San Mateo:

After typing necessary ZIP code – the pump was enabled and I was able to fill-in the gas tank without issues.

There is one thing to mention: some merchants like self service gas pumps, rental car companies, hotels and some others authorize your credit card for an initial amount larger than the purchase amount.

Then once everything is settled – the difference is released back to your card. This present some extra actions that are happening on the background.

Gas pump transactions: Authorization and refund

Even though I purchased gas for the amount of $32.77, my Shift card was charged $75.99 at an exchange rate of 391.79. The average exchange rate at the moment was: (396.16 + 387.98)/2 = 392.07. Which is again – in the middle of bid/ask spread considering it is being quite wide. A day later I was refunded $43.22 (to bring total spent back to actual $32.77). The refund was done at exchange rate of 392.99 – which was different from the time of original transaction rate. This is likely the rate that happened to be at the time of refund. Which could be hit or miss, depending on where bitcoin is going – but on the average we may consider this as a fair deal again.

No extra fees was charged on any of these transactions, email notification was instant and exchange rate was applied at the time of transaction to be in between of bid/ask spread.

Next purchase test I tried in Costco – but my transaction failed because I haven’t figured the PIN code. This was totally my fault.

Buying Apple iPhone 6s for bitcoins directly at Apple store: using Shift card

I planned to upgrade my phone to Apple iPhone 6s and decided to use my Shift card for that. To purchase iPhone 6s with 64GB memory outright I needed to have close to $800 available on my Coinbase balance, so I transferred 2 BTC to my Coinbase account.

Then I headed to Apple store in Hillsdale shopping center in San Mateo.

08-buying-apple-iphone-6s-with-bitcoin-shift-debit-card

Buying Apple iPhone 6s with Shift card – bitcoin Visa debit card

During the actual transaction I had to enter my debit card PIN code and this time transaction went through just flawlessly.

Bitcoin exchange rate during the purchase of iPhone in Apple store

The exchange rate applied to bitcoin based Apple iPhone purchase was 818.38/2.078431 = 393.26

The average exchange rate at that moment was (397.65+389.76)/2 = 393.7. Which is very close again.

After iPhone purchase I headed down to amazon.com to purchase protective case and some other stuff, testing bitcoin Shift card there at online e-tailer. No problem there again – card went through to place an order and amazon actually placed a charge on it a day or so later when stuff was about ready to be shipped.

Using Bitcoin Visa Debit Card – Shift card to buy stuff at amazon.com

Withdrawing cash from ATM using Bitcoin Debit Card: Shift Visa Debit Card

The last test I did is for direct ATM cash withdrawals. For that I headed to Bank of America ATM in San Mateo on 3rd Street.

09-withdrawing-cash-from-atm-using-bitcoin-debit-card

Withdrawing cash from ATM using Shift Card – Bitcoin Visa Debit Card. Step 1

09-withdrawing-cash-from-atm-using-bitcoin-debit-card--atm-fees

Withdrawing cash from ATM using Shift Card – Bitcoin Visa Debit Card. Step 2

This of course involved paying rather hefty fee to ATM operator – $3 in this case. I accepted that for sake of $100 withdrawal test.

09-withdrawing-cash-from-atm-using-bitcoin-debit-card-USD100

Withdrawing cash from ATM using Shift Card – Bitcoin Visa Debit Card. The cash is out! Step 3

This worked just fine as well!

ATM Cash withdrawal transaction record

Exchange rate at the moment of cash withdrawal

The average bitcoin exchange rate at the moment of cash withdrawal was $411.2. The exchange rate I was charged by Coinbase for this transaction was $411.31 – which was again in the middle of bid/ask.

Cash withdrawals carrying higher fees: ATM fee ($3 for my case but may vary from ATM to ATM) + Coinbase fee or $2.49.

Conclusion

The main advantages of Shift card :

Being an enabler of bitcoin-based payments in everyday life for millions of consumers and merchants nationwide.
This is true win for Shiftpayments and Coinbase.
Shift card offers transparent and hassle free bitcoin-backed payments in US dollars at any merchant who accepts Visa or debit cards.
Easy to get: I just moved to USA from Canada (in August 2015) and even with traditional “lack of credit history” there were no problems for me to open Coinbase account and then to apply and get Shift card.
Simplicity of operation: It can be used *anywhere* where Visa credit card is accepted as well as where debit cards are accepted. That includes traditional retailers, Costco, Apple store, online stores like Amazon and similar as well as smaller merchants and automated paying machines.
Fairness: there are no hidden tricks – exchange rate applied to your purchases is fair. No extra fees for purchases.
Low fees: no fees for purchases.
Fee for ATM cash withdrawals but nothing really surprising here.
Easy way to convert bitcoins to small amount of cash if don’t mind ~5% fees and want to avoid shady dealers.
Secure: any activity on the card and payment notifications are delivered instantly by email.
All regular security and support procedures are in place.

Notes:

Shift card does carry restrictions to daily cash withdrawals and daily spending limits.
While somewhat low – these are still perfectly acceptable to make normal and even high priced daily purchases as I’ve tested.
No anonymouty – you have to submit personal information to open Coinbase account.
To have your limits raised you need to be an account holder in good standing with verified bank account.
This may pose issues for some, but should be no problem to an average American geek or entrepreneur or just smart citizen who wants to be part of the rising wave of cryptocurrency adoption in daily life!

Connect with me on LinkedIn
Gleb Esman is currently working as Senior Product Manager for Security/Anti-fraud solutions at Splunk leading efforts to build next generation security products covering advanced fraud cases across multiple industry verticals.
Contact Gleb Esman.

Joining Splunk as Senior Product Manager

This summer been full of great news in my personal career development.

I’ve been accepted as a speaker on Fraud/Security subjects at Splunk 2015 yearly users conference to talk about using Splunk in financial/banking industry to detect, alert and investigate advanced accounts takeover cyberattacks.

My Splunk 2015 conference session talk slides are here.

The session attracted plenty of interest from multiple financial organizations and banks and based on their detailed use cases and feedback Splunk is on a target to become powerful security solution to detect complicated fraud events.

In addition to that right before the conference I been made and accepted an offer from Splunk directly to join their team in a Senior Product Manager role within security organization overseeing development of anti-fraud products.

I really excited to be able to utilize all my past experience in security research and practical skills in building anti-fraud products.

This of course involved me moving from cold Canada to warm San Francisco Bay area.

With my new office to be located in San Francisco this is quite a change – yet I am seriously excited to be involved into making world’s greatest data analytics software even better!

Connect with me on LinkedIn
Gleb Esman is currently working as Senior Product Manager for Security/Anti-fraud solutions at Splunk leading efforts to build next generation security products covering advanced fraud cases across multiple industry verticals.
Contact Gleb Esman.

Real time detection and automated root cause analysis of web malware, exploits and backdoors with Splunk. Part 2, Detection and alerting.

Continued from Part 1…

Adding alert on file system modification events

Let’s setup alert that will send email to administrator when some executable script will be modified on Web server under user’s file system space. We will run scheduled search every 5 minutes to scan last 5 minutes worth of modifications. (more…)

Real time detection and automated root cause analysis of web malware, exploits and backdoors with Splunk. Part 1, Architecture.

In this article I’ll demonstrate step by step how to setup Splunk analytics to detect successful known and unknown malware attacks on web hosting systems in real time.

In addition the same solution will include instructions to deploy fully automated investigative analytics to discover the origins of attackers (IP addresses) as well as any modifications within the file system.

This information is essential to discover and immediately eliminate all possible backdoors and exploits that attacker tried to plant.

Real time alerts will be delivered via email to system administrator as soon as attack occurs. The same information will be available via Splunk web interface for further analysis. (more…)

Predicting Unknown Threats: Detecting Human Emotions Through Machine Data Analytics

Wouldn’t it be nice if your SIEM would send a “possible insider threat!” alert when it detects that employee is in fearful, paranoid or even panicky emotional state while trying to access secure, confidential corporate documents repository?

Or receive real time “possible account takeover!” alert when it detects that currently logged in user is in deep anxiety or experiencing severely agitated emotional conditions while trying to initiate money transfer to an outside bank account?

This approach is used very successfully to detect potential threats by the world’s most secure airlines.
Trained security officers are able to see if passenger feels nervous or agitated or otherwise exhibits emotionally unusual behaviors and then follow up with further checks and investigation. On one occasion by interrogating the nervous passenger the actual bomb was found inside his luggage while the passenger mistakenly thought he had been hired to smuggle diamonds.

The Step Up from User Behavior Analytics

With some creativity, knowledge of human psychology and analytics approach we can apply the same methods to today’s machine data generated by users, clients and employees of financial institutions, banks, governments facilities and corporations to prevent known and unknown attacks from outside as well as from inside of enterprise.

A while ago analyzing an account takeover cyber attack I’ve isolated a complete data set belonging to the attacker who’ve accessed another user account with legitimate credentials.

Attacker’s session activity was almost identical to legitimate user’s activity across all metrics:
Pages accessed, session duration, session hit length, browser user agent used, geographical region of originated IP address, order in which pages were accessed, approximately the same time of the date as legitimate user would use, etc…

(more…)

User Behavior Analysis with Splunk: Detecting Threats and Fraudulent Activity in the Ocean of Behaviors: Part 2 – Detecting Abnormal User Session Velocity and Density

One of my enterprise clients observed that certain class of attacks having a number of distinctive characteristics: attacker who possessed correct user account credentials won’t try to engage into malicious behavior right away.

Initial activity would involve certain degree of reconnaissance and gathering of future victim’s specific data, such as account balances, trade histories and other. So normal “red flags” and risk scoring metrics won’t generate any alerts.

However in many cases such pre-fraudulent activity was still carrying an unusual behavior marks: either session velocity (average number of seconds per hit) or session density (number of hits within the session) or both exceeded normal, baseline session patterns typical for the average client application user’s behavior.

Abnormally high session velocity is also a typical pattern of an automated script-driven session that both fraudsters and competition were using to siphon data from the client’s servers.

One of the possible solutions to detect these activities would be to calculate average session velocity and density and then apply these values to trigger alerts when session metrics exceeded thresholds.

The issue here is that due to the client’s business specific these averages vary greatly depending on the time of the day, time of the week and also the month of the year.

So stuffing some fixed “guessed” threshold values won’t work and will either generate lots of false positives or miss many suspicious sessions. (more…)

User Behavior Analysis with Splunk: Detecting Threats and Fraudulent Activity in the Ocean of Behaviors: Part 1 – Setting Alerts on User Session Risk Factors

Back in my days at IBM T.J. Watson Research Center where we were working on techniques to detect known and unknown malware, the fast growing challenge was the rising threat of malware’s abilities to become polymorphic.

Malicious snippets of code encrypted themselves and made it very difficult to apply conventional signature based detection techniques.

We’ve developed a tiny virtual machine in C language that was able to load malware code in real time and analyze it’s behavior without need to figure out how to decrypt it. Certain score metrics were assigned to keypoints and function calls and logic was put in place to trigger an alert if “risk” score exceeded certain heuristic threshold.

That technique allowed us to deliver top quality enterprise security solution (purchased by Symantec later on) that was capable of detecting previously unknown threats. That was more than 15 years ago.

While working with financial clients and technology companies today I can see that old behavior pattern analysis stays as strong as ever helping enterprises to discover new types of suspicious behaviors and investigate malicious activities with previously unknown patterns from previously unknown sources.

Industry leaders seems to agree that some of the recent high profile breaches could of been thwarted with properly configured behavior analysis SIEM system in place. (more…)

Detecting Bank Accounts Takeover Fraud Cyberattacks with Splunk. Part3: The Advanced Negative Look Behind Query

…Continued from Part 2.

In the final part of this writeup I’ll show you the actual query that does it all and explain how it works.

To remind – this is the challenge – what we want to accomplish:

Detect and alert when C-class IP subnet tries to access at least 5 different accounts within an hour and at least 75% of total accounts touched has never been accessed from this subnet *and* from this USER_AGENT within the last 45 days.

And, as you may remember from Part 1, here’s the basic logic that we need to implement to make it happen:

Scan last hour of access log data and find the list of subnets that tried to access multiple accounts within that hour.
For each of these accounts – take username, IP, USER_AGENT and scan the previous 45 days of traffic history to find usernames that has never been touched by this IP/USER_AGENT combo.
Alert if number of found accounts is above threshold.

I’ve spent quite a bit of effort to come up with a single query that does all of the above and in a pretty efficient manner.

The biggest part of challenge is that the query needs to find events (#1 above) but then it needs to run very custom search for each event against summary index that we’ve created (#2 above). And added icing on this cake is that the query needs to return results only if there are *no matches* found for the second part of search.

This quickly gets mind-boggling and it is a rather interesting puzzle to solve with SPL.

The way I solved it – is with a combination of macros + advanced subsearch. But instead of returning traditional results – the subsearch will return new, custom crafted Splunk search query to be executed by the outer search.

I named this approach Advanced Negative Look Behind (ANLB) query.

ANLB query is the query that has these capabilities: (more…)

Admin

About Gleb

Getting Shift Card

Using bitcoin debit card to pay for parking

Buying Nespresso coffee with bitcoin Shift debit Card

Paying for gas with bitcoins: using bitcoin visa debit card: Shift card

Buying Apple iPhone 6s for bitcoins directly at Apple store: using Shift card

Withdrawing cash from ATM using Bitcoin Debit Card: Shift Visa Debit Card

Conclusion

Adding alert on file system modification events

The Step Up from User Behavior Analytics