The topic of ASV scanning came up as usual at the 2014 PCI Community Meeting. The questions all seemed to revolve around how to obtain a passing scan. What the Council representatives suggested is that multiple scans can be put together to create a passing scan. Unfortunately, what the Council keeps suggesting as the solution is impossible to implement and here is why.
In a typical environment, an ASV customer logs onto their account with the ASV and schedules their ASV scans of their PCI in-scope assets. The customer may also add or subtract the number of IP addresses that are scanned as the scope of their external environment may change. Depending on a number of factors, there may be one scan or multiple scans. The vulnerability scans are executed on the schedule and the results are returned to the customer.
If there are false positive results or results the customer does not agree, they can apply back to the ASV to have those results removed. If there are actual vulnerabilities, the customer can contact the ASV with how they have mitigated the vulnerabilities and the ASV can either accept those mitigates and give the customer a passing scan or allow the results to stand.
So where are the problems?
Whether or not the Council acted on facts that cheating was occurring or anecdotal evidence is unknown. But because of the potential for cheating by customers, the Council mandated a number of years ago that ASVs lock down their scanning solutions so that customers cannot modify anything regarding testing other than the IP addresses involved. The ASV Program Guide v2.0 on page 11, states:
“However, only an authorized ASV employee is permitted to configure any settings (for example, modify or disable any vulnerability checks, assign severity levels, alter scan parameters, etc), or modify the output of the scan. Additionally, the ASV scan solution must not provide the ability for anyone other than an authorized ASV employee to alter or edit any reports, or reinterpret any results.”
So right off the bat, the Council’s recommendation of “putting together multiple reports” is not as easily accomplished based on their earlier directives. That is because it will require the ASV’s customer to get the ASV to agree to put together multiple reports so that they can achieve a passing scan. That implies that the ASV’s solution will even accommodate that request, but then the ASV needs to be agreeable to even do that task. Based on the Council’s concerns regarding manipulation of scanning results and the threat of the Council putting ASVs in remediation, I do not believe the ASVs will be agreeable to combining reports as that would clearly be manipulating results to achieve a passing scan.
But it gets worse. As a lot of people have experienced, they can scan one day and get a passing scan and then scan a day or even hours later and get a failing scan. The reason this happens is that the vulnerability scanning vendors are adding vulnerabilities to their signature sets as soon as they can, sometimes even before vendors have a patch. As a result, it is very easy to encounter different results from scan to scan including failing due to a vulnerability that does not yet have a solution or the vendor only just provided a patch.
But if that is not enough, it gets even worse. Statistically, the odds of getting a passing scan are nearly impossible and gets even worse if you are only doing quarterly scanning. A review of the National Vulnerability Database (NVD) shows that 94% of vulnerabilities from 2002 to 2014 have a common vulnerability scoring system (CVSS) score of 4.0 or greater. That means that it is almost impossible to obtain a passing vulnerability scan, particularly if you are only scanning quarterly, when vulnerabilities are announced almost daily and vendors such as Microsoft are coming out monthly with patches. Those of you scanning monthly can attest that even on a 30 day schedule, a passing scan is nearly impossible to get.
For an organization that has only one Web site, this situation is likely not a problem. But when organizations have multiple Web sites which a lot of organizations large and small have, you are really struggling in some cases to get passing scans.
But let us add insult to injury. A lot of organizations have their eCommerce environments running on multiple platforms such as Oracle eCommerce or IBM Websphere. In those examples, this situation becomes a nightmare.
Platforms such as those from Oracle and IBM may run on Windows or Linux, but Oracle and IBM do not allow the customer to patch those underlying OSes as they choose. These vendors ship quarterly, semi-annually or on some other schedule, a full update that patches not only their eCommerce frameworks, but also the underlying OS. The vendors test the full compatibility of their updates to ensure that the update will not break their frameworks. In today’s 24x7x365 world, these vendors can run into serious issues if eCommerce sites begin to not function due to an update. However, that also means there is the possibility that critical patches may be left out of an update due to compatibility and stability reasons. As a result, it is not surprising that in some updates, vulnerabilities may still be present both those that are new and those that have been around for a while.
But if Oracle and IBM are not patching on 30 day schedules, that means there is a high likelihood that the scans will not be passing. This means that the customer must go to their ASV with compensating controls (CCW) to mitigate these vulnerabilities to obtain passing scans.
The bottom line is that the deck is stacked against an organization obtaining a passing scan. While the Council and the card brands do not recognize this, the rest of the world sure has come to that determination.
In Part 2, I will discuss the whole ASV approach and how I believe the drive to be the cheapest has turned the ASV process into a mess.
PCI Guru 
Our most recent scan failure was for a flaw which allegedly had “no known fix at this time”! That’s not very helpful, especially when you’re a small company with no clout with an ASV (SecurityMetrics in our case). In the end I found a workaround (fix?) via an IT community (http://community.spiceworks.com/topic/577169-securitymetrics-failed-scan-the-remote-mail-server-is-affected-by-an-informat?page=1&source=homepage-feed#entry-3853012).
We have been finding that CVSS scores are changing in the National Vulnerability Database (NVD) which is causing some vulnerabilities to now exceed 4.0. As a result, environments that were getting passing scans for years are now failing. And with Poodle just coming out, I am guessing that we will see a lot of scan failures.
During research I often find that CVSS scores in the NVD are exaggerated, quite often “due to insufficient information”, but also because it doesn’t properly take into account the exploitability (e.g. a non-default and not too common configuration that is rated with low access complexity, when it should be medium or even high). If a vulnerability, or the severity of a vulnerability, is disputed and the CVSS base score is >= 4.0 the ASV is required to investigate it. It’s not necessarily clear in the ASV Program Guide, but under “Managing False Positives and Other Disputes” one of the possible disputes is “Vulnerabilities that have a disputed CVSS Base score”, and under “Exceptions to Scoring Vulnerabilities with the NVD” one exception is “The ASV disagrees with the CVSS score noted in the NVD.” — i.e. the ASV /can/ modify the base score from that in the NVD.
One ASV scanning bugbear I have, is the vulnerabilities for which we get a FAIL because the ASV can’t tell whether a particular patch has been applied or not. This is crazy. We go to great lengths to implement information hiding tactics, as this is a good security practice.
I share your pain as a lot of people have similar stories. However, ASVs are supposed to allow standing CCWs and vulnerability explanations so that the process is not as painful as some make it.
While hiding banners and other information is not a bad practice, it’s not necessarily a good practice as long as you have verified vulnerability management and patch management processes in place.
While vulnerability assessments do include behavioural tests, a large number of tests are based on fingerprinting and version checks. Often the reported version is all they have to go on. On the other hand, many attackers will opportunitistically try actual exploits anyway without even caring what version the software says it is. Even a targeted attacker will likely find other ways to get the information they need to penetrate your environment, barely slowing down to do so.
By showing the real product banners, the vulnerability scanner can more accurately determine the software present and avoid reporting false positives. By having a continuous vulnerability and patch management process, you reduce the small advantage an attacker can gain from learning the product versions, because of course you have already applied the latest security updates.
Unfortunately, this will by no means make false positives due to applied patches go away. Many GNU/Linux distributions will backport fixes without updating the visible version information, or the precise version information isn’t displayed anyway.