We are starting to hear rumblings within some organizations regarding conducting their own PCI Report On Compliance (ROC) assessment. While as a QSA I have a vested interest in not seeing this become a common practice, I can understand why organizations would want to at least examine this option. Particularly since the latest study from The Ponemon Institute says that the cost of a PCI ROC is between $225,000 and $500,000.
Remember, a ROC is only required when an organization is conducting a minimum of six million Visa, MasterCard or Discover transactions or two and a half million American Express transactions or one million JCB transactions. We are not talking about your Mom & Pop store around the corner or even your local chain. We are talking about organizations such as Wal*Mart, Exxon/Mobil and Amazon.com. The obvious reason organizations want to conduct their own PCI ROC is the belief that in-sourcing the assessment will save them money. However, as I will discuss later, there will likely be few if any cost savings.
The main benefit of conducting your own ROC assessment is that you use your own personnel. While the PCI DSS is silent on this subject, MasterCard and Visa Southeast Asia require the use of internal audit personnel. The use of internal audit personnel would seem to be the obvious choice because of their independence, their familiarity with assessing business processes against a standard and organizing and retaining the necessary documentation. Most major merchants already have an internal audit function, so there is no need to necessarily increase headcount. On the face of things, conducting a ROC internally sounds fairly straight forward.
As usual before you can save any money, there are some issues to conducting your own PCI ROC assessment.
- Most internal audit personnel are financial auditors not IT auditors and, as such, they have limited technology skills. And even with organizations that have an IT audit capability, the level of technical skills required by the PCI ROC is more than the skills available. The PCI ROC requires a significant amount of technological background in order to be conducted. Skills such as analyzing firewall, router and switch configurations, understanding access control mechanisms and conducting vulnerability scans and penetration tests are required for an assessment to be properly performed. As a result, your internal audit group will likely lack the necessary technical skills to conduct the assessment. This means investing in either training your existing audit staff or contracting for the skills necessary, neither of which is cheap.
- Until recently, there was a limited amount of PCI compliance training available for internal audit personnel. With MasterCard’s requirement of training internal audit personnel by July 2011 if an organization desires to conduct their own assessment, the PCI SSC has developed training for non-QSAs and that training will be available in 2010. However, such training is also not inexpensive. In addition, training is required annually to stay on top of the PCI DSS, so it is an ongoing cost.
- Since the ROC assessment process is an annual occurrence, the trained internal audit personnel will likely not retain all of the knowledge necessary to conduct the program from year to year because they do not conduct ROCs all year long like a QSA. This will mean ramp up time as well as the potential that the assessment will miss possible compliance issues or issues may be misinterpreted.
I have personally seen the results of a number of internally conducted PCI ROCs. None of them properly interpreted the PCI DSS requirements. In all cases the internal assessment judged the organization as being in full PCI compliance. However a review of the work showed that none of these organizations were PCI compliant. In some cases, the evidence collected in no way meets the documentation requirements of the PCI SSC let alone documented compliance with the PCI ROC requirement. And the worst problem, these PCI ROCs were all signed off by the organization’s Chief Financial Officer representing that the PCI ROC accurately represented their PCI compliance.
In the end, I am not convinced that organizations that desire to conduct their own PCI ROC will achieve the cost savings they believe they should achieve. And while on the surface it may appear to be cheaper to do the assessment internally, the skills and high level of training required to obtain a proper PCI ROC are likely beyond the investment required to get the job done. Finally, because your internal auditors are not conducting PCI ROCs every day there is a higher risk that your assessment may miss potential threats to your cardholder data environment.
I am not saying that you should not consider conducting your own PCI ROC. I am just pointing out that it is not a simple internal IT audit. It is also not as inexpensive as you might think. So please think about this before you go down the internal assessment road and get your organization in trouble.
UPDATE: The PCI SSC now offers the Internal Security Assessor (ISA) certification program for internal personnel. It is supposedly equivalent to their QSA certification program. Like the QSA program, ISAs are required to re-certifiy annually, so it is not a one shot deal. It is also not cheap at around $2,500 per person not including expenses. For MasterCard Level 2 merchants that want to do their own assessment after June 30, 2011, you will be required to either hire a QSA or have internal assessors attend and pass the ISA certification.