In the November 2016 Assessor Newsletter from the PCI SSC, there is a clarification on what ‘Not Tested’ actually means and implies. I am sure this will really get some service providers whipped up as it will create some issues with work they perform on behalf of their customers.
The following is taken directly from that newsletter.
“Recently, AQM has received some questions about the impact of using “Not Tested” as a response within a completed ROC. This article is intended to address a few points briefly, with published documentation to follow.
- Due to an oversight, the option for “Not Tested” was not included in the summary findings table within the summary overview when that table was introduced with the ROC Reporting Template for use with PCI DSS v3.2. We will publish an errata for the ROC Reporting Template shortly.
- Some have asked whether one can have a compliant AOC in instances where “Not Tested” was used. While PCI SSC is not able to comment on matters of compliance, we would direct you to read the verbiage at Part 3 PCI DSS Validation of the Attestation of Compliance below:
How to achieve “all questions answered affirmatively” is the question. PCI SSC does not consider “Not Tested” to be an affirmative statement. The difference between “Not Tested” and “Not Applicable” is that no testing at all is performed for “Not Tested” whereas for “Not Applicable” some testing is performed to confirm a given control is truly not applicable. As such, between “Not Tested” and “Not Applicable,” only “Not Applicable” can be considered an affirmative response.
The intent in introducing “Not Tested” was to achieve a better level of transparency as to the level of compliance and this clarification supports that intent. If you have questions or suggestions, please reach out to the QSA Program Manager.”
It is that second to the last paragraph that will likely send most people off of the deep end. Their comment that the “PCI SSC does not consider “Not Tested” to be an affirmative statement” really got me going. What exactly then was the point of using ‘Not Tested’ if you did not consider it an affirmative statement? Which by the way, when using affirmative as an adjective, means “asserting the truth, validity, or fact of something.” Last I checked, ‘Not Tested’ would be considered a truth or fact.
There are a number of options for the Council to take here.
- Change the wording in the ‘Compliant’ box in Part 3 to reflect that an entity is compliant with all of the requirements tested.
- Give us a box in Part 3 that says ‘Compliant with Exceptions’ or something of that ilk which would allow those entities not testing certain requirements to still be judged compliant with what was tested.
- Tell QSAs that an AOC cannot be filled out for assessments that mark any requirements as ‘Not Tested’ because an AOC is not relevant.
I remember at a number of past Community Meetings various Council representatives repeatedly and emphatically told those of us from the Accounting community that PCI assessments were not SAS 70 (now SSAE 16) engagements when we would invoke SAS 70 like rules for sampling, testing and the like. Well, I hate to say it, but the Council is sure turning them into one with all of these pronouncements.
UPDATE: On the Council’s Webinar on Thursday, December 15, it was announced that the Council will be making changes to the AOC and will issue new guidance on this topic sometime in the first Quarter of 2017. So stay tuned for an update.
UPDATE: From the January 2017 Assessor Newsletter.
“Update on use of “Not Tested” with PCI DSS
In the November 2016 Assessor Newsletter, PCI SSC addressed the use of “Not Tested” in PCI DSS and the resulting impact on the Attestation on Compliance (AoC). The article included no change in intent, but rather acknowledgment of existing confusion. PCI SSC recognizes the challenges that the clarification on “NotTested” highlighted and is working internally on providing further documentation to support assessors in addressing such cases.
PCI SSC cannot comment on matters of compliance, as those decisions – as always – are business decisions that should be made by the assessor with the payment card brands and/or acquirers. The article ‘Clarification – “Not Tested” use in PCI DSS’ was never intended to change this, and assessors should continue to work with these entities as they have since the “Not Tested” designation was introduced.
Please note that FAQ 1331 “Can SAQ eligibility criteria be used for determining applicability of PCI DSS requirements for onsite assessments?” has been updated this month to better support the intended use of the “Not Tested” and “Not Applicable” findings.”
The bottom line is that “Not Tested” cannot be a response to any requirement and then expect to have a compliant ROC/SAQ and resulting AOC.
PCI Guru 