As part of being professional consultant building client-specific fraud detection solutions I often witness product pitches by different vendors in a security / fraud detection space.
The recent wave of successful high profile cyberattacks and disastrous data leaks added new level of activity into search for the perfect fraud detection and early alerting solution.
With attackers changing their activity vectors, attack patterns and techniques on a daily basis this makes many legacy fraud detection tools to lose their efficiency and get outdated very quickly.
In the never ending quest to protect enterprise against fraud losses the ideas of Automated Anomaly Detection are picking up steam.
The way it generally works – anomaly detection system would establish baseline for certain predefined dimensions and system would then monitor (often in real time) for deviations from established baseline. Once sufficiently abnormal condition is detected – the alert is issued. Such system could operate pretty automatically and learn from historical and present data constantly updating it’s baselines as well as trigger thresholds.
I’ve observed a number of tools in this space and have to say they are pretty efficient to catch interesting anomalous events that indicate some sort of problem.
Having said that – the whole space of fraud detection is somewhat different beast.
Successful fraudsters are more likely being sophisticated humans rather than automated scripts. They are trying to stay under the radar to avoid detection as much as possible.
The new solutions are being pitched pretty aggressively selling automated anomaly detection concepts as a “new and better approach” against fraud, promising fully automated, never-need-to-be-updated, catch-it-all products.
Before you’ll buy into disappointment, here are few bullet points to be aware of when evaluating new security products and vendors covering the fraud space:
- Beware of “blackbox”-type, “closed” solutions with vendor-proprietary “ip” inside and a few knobs outside to configure it. When these type of solutions need to be customized for new fraud types, integrated or adjusted for new deployment architectures – this usually involves pricey mandatory “professional services” from the vendor. This cycle has rinse and repeat tendency and the whole thing could get very costly.
Gravitate toward full stack security frameworks allowing for deep customizations of all layers, and ability of building vendor and business specific solutions on top of them. - Beware of pure anomaly detection based solutions for fraud detection. These usually come with a pitch of requiring very little configuration and ability to automatically adjust themselves to incoming data in real time and detect fraud “forever”. Sort of set-and-forget scenario that sounds like a music to ears of CISO’s.
These type of tools are an ultimate utopia for vendors themselves to strive for because it would mean that they won’t have to care about business domain specifics and just work on detecting anomalies covering 100% of all industries. Anomalies – yes, fraud – not so fast.
In reality successful fraud detection tasks are very vertical-specific, require deep knowledge of business domain, and demand complex real time evaluations of multiple activity points, correlation with external unstructured threat feeds and complex references to past profile activities for each incoming user session in real time. If solution is based only on[automated] anomaly detection – beware – they likely won’t be able to handle these tasks efficiently.
Having said that – automated anomaly detection solutions are great for enterprise monitoring and generic application support tasks. They could be very efficient for early alerting when “something” start going awry and quite often their efficiency might even exceed other tools. They might be beneficial for automated intrusion detection tasks as well.
But not for fraud detection challenges – these are totally different!- Beware of “our machine learning” is better then “theirs” (or “everyone elses”) claims.
Definition of “machine learning” is different from vendor to vendor and more complex doesn’t mean “better”.
Demand facts, independent reviews and verifiable references.
If vendor in question hides behind “it’s proprietary” statement – it’s a red flag.- […Continues:] Beware of “anomaly detection is the better way to detect fraud” – type of pitches.
Not every anomaly is fraud (in fact most are not) and not every fraud is anomaly (lately more and more are not as well).
Beware of pre-baked, simplified, unrealistic demos at this stage. These might be inapplicable to your enterprise domain specifics.- Do not: pay much attention to “our solution generates very low level of false positives” statements. These are very domain and case-specific and need to be re-tested for your enterprise.
In fraud space false positives are fact of life and are lesser evil than missed fraud.
Demand demos over datasets and models that are very similar to the ones generated within your business.
The impressive demo of catching gift card fraud doesn’t mean anything for insider derivatives trading fraud.- Do: Welcome evaluations of vendor tool over your own, historical datasets containing fraud events. Do not: disclose fraud specifics (IP addresses, usernames, account numbers) but let vendor system prove itself.
Observe an amount of adjustments and customizations the system would require before it can be “turned on”. Likely similar process will need to be repeated few times.- Beware: the low entry price pitch “our solution is cheaper”. This is especially true for closed “black box” products. They *are* often cheaper because of the hidden costs.
Price wise it works like buying an inkjet printer – low initial entry point + expensive, vendor-specific, recurring ink purchases for a long time to come. “ink” == “professional services” + extra fee addons, plugins, extensions, etc…
Steer toward clearly licensed or flat fee licensed solutions. These are easier to budget with predictable cost structures.
Make sure to clarify the level of customizations and adjustments that can be made using your own resources before vendor’s consulting needs to be involved.- Do: ask for demos specific to your business domain. Don’t put much value on generic, simplified, catch-all demos.
It’s ok to be the first in industry to test new vendor or new solution but be aware that past performance of a fraud detection solution elsewhere definitely does not guarantee results within your business vertical.
Being the first is industry vertical to test the solution will likely give you the cost negotiation advantage as well.- Consider: powerful machine learning and anomaly detection tools as an extensions to your existing tools and frameworks.
The good example of this would be Splunk as a base analytics framework and custom Splunk apps or addons designed specifically for anomaly detection, such as Prelert anomaly detective.
If your enterprise already using Splunk for enterprise monitoring tasks – this would be the perfect platform to build tailored fraud detection solution without need to invest into yet another security vendor.
That’s exactly how Morgan Stanley leveraged an existing investment in Splunk, saved resources and came up with customized and efficient fraud and account takeover detection system based on Splunk Enterprise.
Contact me for more information on this use case to see how your enterprise might benefit from this approach for security and fraud detection needs.Connect with me on LinkedIn
Gleb Esman is currently working as Senior Product Manager for Security/Anti-fraud solutions at Splunk leading efforts to build next generation security products covering advanced fraud cases across multiple industry verticals.
Contact Gleb Esman. - Beware of “our machine learning” is better then “theirs” (or “everyone elses”) claims.
