Endpoint security software vendor Carbon Black has been found to be exfiltrating data from several Fortune 1000 companies due to the architecture of its Cb Response software, the information security services and managed services provider DirectDefense claims.
A blog post written by its president, Jim Broome, described the Cb Response software as “the world’s largest pay-for-play data exfiltration botnet”.
He claimed that experts in his company had been able to rec cover the following kinds of information pertaining to Fortune 1000 companies:
Cloud keys (AWS, Azure, Google Compute) – which can provide access to all cloud resources;
App store keys (Google Play Store, Apple App Store) – which can be used to upload rogue applications that will be updated in place;
Internal usernames, passwords, and network intelligence;
Communications infrastructure (Slack, HipChat, SharePoint, Box, Dropbox, etc.);
Single sign-on/two factor keys;
Customer data; and
Proprietary internal applications (custom algorithms, trade secrets).
Broome said that the leaked data existed primarily around a number of executable formats. “We haven’t seen evidence of this in documents or PDFs yet. However, if handled incorrectly, even executables can easily contain serious data leakage of information that can be hazardous to a company’s security posture.”
Carbon Black, formerly Bit9, provides a whitelisting solution in the Cb Response software. The software connects to endpoints and checks files, using a cloud look-up service to do the work due to the sheer number of files that need to be checked.
Broome said there was a problem in this methodology because “without a large sample set to determine what was good or bad already available to their users, Carbon Black deferred the decision to a cloud-based multiscanner service”.
“Ultimately Carbon Black would have a bunch of anti-virus solutions decide which files were bad, and remove the offending file from the set of things customers could whitelist,” Broome said.
“This worked well for their customers, but it brought a new wrinkle into the equation. What about the good files that haven’t been seen on the cloud-based multiscanner? The answer was obvious. The files must be uploaded, have all the A-V engines scan them, and then use those scores. So, Carbon Black began uploading files from their customers to their cloud, and from their cloud to the multiscanner solution.”
Cloud-based multiscanners are operated as a for-profit business and allow access to anyone who paid the requisite fee.
“Access to these tools includes access to the files submitted to the multiscanner corpus (it’s hard to analyse malware that you don’t have),” Broome wrote.
“This means that files uploaded by Cb Response customers first go to Carbon Black (or their local Carbon Black server instance), but then are immediately forwarded to a cloud-based multiscanner, where they are dutifully spread to anyone who wants them and is willing to pay.”
He said the amount of data available on the multiscanner could be gauged from Carbon Black’s statement that by the end of 2015 it had sold more than seven million software licences and had about 2000 customers worldwide.
Explaining the operation that was causing so many files to be available on the multiscanner, Broome said when a new file appeared on a protected endpoint, a cryptographic hash was calculated.
“This hash is then used to look up the file in Carbon Black’s cloud. If Carbon Black has a score for this file, it gives the existing score, but if no entry exists, it requests an upload of the file. Since Carbon Black doesn’t know if this previously unseen file is good or bad, it then sends the file to a secondary cloud-based multiscanner for scoring. This means that all new files are uploaded to Carbon Black at least once.”
He said his staff had stumbled across the vulnerability while analysing potential malware and utilising the analyst interface of a large cloud-based multiscanner to do so.
A useful feature of this multiscanner was that it allowed searching for similar malware to get some context. “..,in doing so, we stumbled across a couple of files that were very different. These seemed to be internal applications from a very large (and completely unrelated to our original customer) telecommunications equipment vendor.
“After determining they were unrelated, we became curious about how such files could have gotten up onto the multiscanner corpus to begin with.”
Broome said all the other files were discovered to have been uploaded by a similar uploader which was obscured behind an API key.
“By doing some research, we determined that this is the primary key for uploading files by Carbon Black for Cb Response. By searching for similar uploads from this key, we found hundreds of thousands of files comprising terabytes of data. We started downloading some of these and digging a little deeper,” he said.
“We downloaded about 100 files (we found JAR files and script files to be the easiest to analyse by script), and ran these files through some simple pattern matching. When we got hits, we’d try to extrapolate where they came from. We were not trying to be exhaustive in analysis, and only repeated this operation a few times to see if it still held true.”
Broome said his staff had found files from a large streaming media company, a social media company, a financial services company.
In response, Carbon Black’s chief technology officer Michael Viscuso said DirectDefense was wrong to say there was an architectural flaw in Cb Response that leaked customer data.
“In fact, this is an optional feature (turned off by default) to allow customers to share information with external sources for additional ability to detect threats,” he said.
To this, DirectDefense responded by stating that while the setting was off by default, “the recommendations or messaging from Carbon Black’s professional services team during the course of installing the product is to turn this feature on to help accelerate the analysis of the file scans”.