Changes Coming To A QSA Near You

As I stated in an earlier post, I had my recertification training back around the first part of April in Chicago.  One of the outcomes of this training was identification of a number of work program areas that are being significantly changed or expanded under v1.2 of the PCI DSS.  The purpose of this post is to give all my readers a ‘heads up’ on these changes as they have the potential for significantly increasing the amount of time it takes to get your Report On Compliance completed and, hence, likely increase the cost.

There is a reason for these changes to the procedures.  The PCI SSC is just trying to improve consistency between QSAs as well as obtaining more accurate and reliable results from the information provided by the QSA in the Report On Compliance.  This is probably one of the biggest complaints regarding QSAs and this is just one of the ways that the PCI SSC is attempting to address the QSA consistency issue.

First, we were told point blank that we would have to document and justify our sampling strategy.  We had heard for years stories of QSAs conducting ridiculously small samples such as two retail locations out of a 1,500-store retail chain.  Or sampling one Windows, one Linux and one Unix server each out of 2,000 total servers.  That is around 1/10th of one percent and, no matter how good your procedures are and the fact that you maintain consistent configurations, a QSA cannot justify these as good samples.  In addition, some QSAs were also sampling the same locations or servers year after year, which had always been not allowed, but without the PCI SSC QA program, there was no way to know that this sort of procedure was being followed.

One option to keep QSA sampling to a minimum is to get the client’s internal audit function to include PCI coverage in their retail outlet audit program.  Most large retailers conduct periodic on-site internal audits of their retail locations.  These audits are typically conducted on a three year rotating schedule, so every retail location will always get reviewed at lest every three years.  Potentially more often if issues are identified during a review.  We have added a number of basic PCI procedures to our large merchant audit programs to cover those PCI requirements that are relevant out in their retail locations.  We then review the results and conduct our own testing based on a much smaller sample size.  If issues are uncovered in our sampling, we then increase our sample size.

Another area that is being expanded is in the area of infrastructure configuration reviews.  Short of demanding that automated analysis tools be used, we were told that the analysis of firewall, router, switch and other infrastructure device configurations need to be much, much more robust and detailed.  Essentially, QSAs are being held responsible for ensuring that the analysis of the network is complete enough to give reasonable assurance that all of the relevant network security requirements can be answered from the detailed analysis of the devices’ configuration files.  We were also told that using tools such as Cisco’s Output Interpreter is not going to be acceptable.  The PCI SSC and card brands want a very detailed analysis of the network conducted to ensure that the PCI in-scope network is truly secure and controlled.  Based on my understanding of the results they are looking for, there is no way a manual assessment will meet their requirements unless the network is very simplistic.  As a result, you can expect QSAs to start using a variety of network configuration analysis tools such as Secure Passage’s FireMon or Athena Security’s FirePac and Verify.  Depending on the number of devices involved, this sort of analysis can be expensive.  If your network is consistently configured to specific standards, your QSA should be able to pick a sample of devices to obtain an end-to-end analysis of your network and, therefore, be able to get detailed answers.  If your network is not consistent, then this analysis is going to end up being expensive.

We were told of a number of QSAs that are under probation for issues with their Reports On Compliance uncovered during their QA review.  We were also given the grading scale that the PCI SSC uses to assess Reports On Compliance.  This grading scale is essentially the Report On Compliance with a rating and weighting scale for each requirement.  Any QSA company not scoring at least 85% on their reports is placed on probation for at least three months.  While under probation, the QSA company must submit every Report On Compliance to the PCI SSC for review before it can be issued.  Any QSA company scoring less than 75% can be removed as a QSA company by the PCI SSC.  I can tell you that my firm is going through its QA review and we are all on pins and needles.

Finally, there is a change coming in all QSA’s proposals and engagement/project legal agreements.  For this year’s QA reviews, all Reports On Compliance must be redacted to ensure client confidentiality is maintained.  This can be a very painful process for a QSA company that issues a lot of Reports On Compliance.  The change is to require a statement in all engagement/project legal agreements that allow the PCI SSC to have access to Reports On Compliance for the purposes of the QA review process.  This will effectively create a non-disclosure agreement between the PCI SSC and the QSA company’s clients.

The times they are a changing.