Mr. Robert Gezelter’s blog post entitled ‘Securitization: A Risk To Compliance Integrity’ discusses his organization’s encounter with an approved scanning vendor (ASV) and their vulnerability scanning. Mr. Gezelter discusses in his posting that the ASV conducted testing through a firewall and intrusion detection system against a system that was powered down. While I would question the ASV’s qualifications, Mr. Gezelter brings up a valid point regarding his experience and concerns with the vulnerability scanning process.
One of the problems lies with the fact that the ASV certification process certifies an entire firm, not an individual. That means that a firm can use its best personnel to get their ASV certification. Once certified, they then turn it over to low skilled personnel (i.e, low cost) to conduct the customers’ scans. Or worse yet, the ASV implements an automated solution that customers set up for scanning. All of this increases their margin on their work, not their accuracy or customer service. A reputable ASV will use highly qualified personnel to configure and conduct the scanning and interpret the results. These personnel will have a good understanding of the PCI compliance process and what the scanning is to accomplish. This is why there is such a variance in scanning costs and results. This would all be addressed by making the ASV certification by individual and not by firm and then requiring the scans be conducted by a certified individual.
Another part of the problem comes from the sales cycle by the ASV. The sales cycle for those cost conscious customers usually results in a customer keying in the IP addresses of what they think is their PCI in-scope systems into a Web site and then setting up a scanning schedule. Whether or not the scan will be properly conducted is anyone’s guess. A reputable ASV will have informed personnel walk a client through the scanning process and ask appropriate questions to determine the amount of effort required to get the correct results. Again, you get what you pay for. However, this would be addressed by requiring scans to be conducted by a certified individual using tools, not just a tool. Until tools cannot generate false positives and false negatives, they will always require an experienced human to interpret the results.
Finally, using an automated tool is only part of the compliance process. The results produced by the tool need to be interpreted, false positives determined and documented and then the real vulnerabilities dealt with. Tools produce false positives and false negatives and these must be resolved by someone with experience so that the correct results are addressed. Most organizations using these automated solutions are not qualified to interpret the results and therefore are likely only complying with the scanning and not with the remediation. Again, this can be addressed by requiring a certified individual to conduct the scanning and determine what remediation is required.
Only time will tell if the PCI SSC will address this situation.
PCI Guru 
I just stopped by your blog and thought I would say hello. I like your site design. Looking forward to reading more down the road.