past July, Capital One was hit by one of the largest financial hacks in history,
impacting the personal data of over 100 million people. Unlike other attacks, however,
this one was notable because it originated from a cloud vendor. The attack targeted data
stored on AWS servers, with access coming through a misconfigured firewall.
is only as strong as the weakest link in your organization’s line of defense, and
the many threats facing the financial sector are escalating both in terms of quantity
and complexity. So it’s projected that by 2021, the amount of damages due
to malware will likely hit $6 trillion. It’s not surprising, then, that in
a new report from the World Economic Forum, cyberattacks are again ranked the number one risk to doing business… and by a very large margin.
is immense pressure on cybersecurity and fraud detection experts to identify
and neutralize threats in near real-time. Yet today’s detection tools – including
graph search databases – face daunting challenges, particularly when parsing
and analyzing the massive datasets upon which so many financial institutions now
response, banks and other large organizations are exploring new graph search technologies
to identify malware patterns. Cutting-edge in-memory computing and graph search
tools can identify cyber risks in near real-time, condensing what typically
takes weeks down to just minutes.
– if you’re a security expert who works with especially large datasets – there
are now three steps you can take to reduce the time it takes to find and
neutralize malware. These include:
Migrate from graph databases
to graph search tools.
There is an overwhelming amount of data out there; too much for conventional tools to scan in a practical time frame. Companies must regularly scan their network log data to identify lateral movement. At the same time, banks easily generate terabytes of network log data per day… which means threats cannot be identified in a reasonable time frame. Conventional tools will simply never catch up to the amount of data being generated and the number of incoming threats hitting the network This is why the average dwell-time for malware on a bank’s network is an astounding 71 days.
dramatically reduce this figure, it’s essential to look beyond conventional
graph databases. While graph databases are ideally suited for smaller datasets
because they scale both vertically and horizontally without introducing data consistency
or integrity issues, they simply don’t scale well once you have terabytes of
data to analyze. At this scale, graph database performance declines for two
- Scaling horizontally results in practically every memory fetch (edge traversal) requiring a message to be sent across the network to another node.
- Storing data on disk, and working with only a small fraction of that data in-memory, results in data thrashing between RAM and disk to traverse edges.
graph search tools are specifically built for very large datasets. For example,
the Department of Defense helped develop the Trovares
graph search tool, which adopts supercomputing techniques such as extreme
multithreading and fine-grain locks to achieve substantial increases in both
scale and speed. A team of data scientists applied analytics and supercomputing
expertise to deliver a significantly different graph search tool, which returns
queries hundreds of times faster than conventional graph tools, while
supporting large in-memory graphs for fast queries. It also enables the direct
ingest of data into the system to avoid database performance issues.
Don’t search just a slice of
your data. Search all of it.
common solution to scanning large datasets is to slice-and-dice, or analyze
just a piece of the overall dataset at a time, to try and find malware
patterns. But this is no longer the best option.
search tools can ingest more data while answering complex searches. The
performance boost of graph search – particularly when combined with symmetric
multiprocessor systems (SMP) systems – lets it leapfrog conventional search
tools when searching much larger datasets. Thus it can quickly find intrusions
that have continued to reside in the data. Critically, increased speed and
scale allows organizations to scan the full dataset, in a matter of seconds or
minutes, to neutralize malware.
Consider migrating to an SMP
you should consider which compute platform will meet your need for extreme
performance. Server clusters are popular, but they are not ideal for graph
search, and indeed the typical computation over a graph data structure is among
the worst for clusters.
on the other hand, are excellent for graph search. Implementations from the
team commercializing the DoD technology were built on SMP systems such as HPE’s
SMP system today can range in size from 1000+ threads of execution, and 3 to 48
terabytes of memory, providing the balance of processing capacity and storage to
meet the demands of scaling graph search performance. These platforms enable
high performance ingest of data and are built on industry standard x86
processor technology and PCIe-based IO. They support the full range of software
needed to complete a workflow around graph search.
data for these systems shows near linear scalability when querying 3 terabytes
of data with 20 billion graph edges and 212 billion edge properties. The
combination of the graph search tool and an SMP system demonstrates orders of
magnitude improvements in speed, reducing query time from days to minutes.
should expect graph search tools on SMP systems to outperform conventional
tools on datasets of all sizes… but they become particularly compelling when your
dataset reaches a billion or more records.
Why this matters
malware challenge continues to evolve and grow, and organizations must adopt
new technologies to remain one step ahead. Today’s leading enterprises can find
the performance needed to overcome the speed and scale challenges of
identifying and neutralizing malware by adding graph search and SMP systems to
their cybersecurity roadmaps. With these tools, it’s now possible for
organizations to quickly scan their entire datasets to address data breaches as
they happen in near real-time.
James Rottsolk is co-founder, president and CEO of Trovares