“Usually when PCI-compliant companies are breached, the real culprit is the assessor, the person who confirmed the company had met the PCI Requirements.” Jeff Multz, Dell SecureWorks
This is a very interesting approach for an employee at a qualified security assessor company (QSAC) to use to drum up business, toss all QSAs, including his own organization’s QSAs, under the bus. I know that is not what he meant to do, but that is certainly what he did with this statement in his posting a few days ago.
I think most QSAs know where Mr. Multz is coming from. He is more than likely venting over losses to QSACs that we all know are more interested in revenue generation than security. They further that goal by incenting their QSAs to do as many PCI assessments as possible in the shortest amount of time as well as identify opportunities for selling the QSAC’s security appliances to solve compliance problems. And to just pile on, they further their revenue generation by being the low cost provider through a focus on volume of work over quality. As Kurt Vonnegut said in Cat’s Cradle, “In this world, you get what you pay for.”
Getting back though to Mr. Multz and his statement that QSAs are responsible for all breaches, let us see how that plays out with a few breaches.
During the Target breach, it was the QSA that was socially engineered and gave away the keys to the kingdom and missed all of the alerts generated by the FireEye software. At Neiman Marcus, it was the QSA that missed the alerts for 60+ days that the malware was reinstalling nightly. It was the QSA that swapped out the points of interaction (POI) at Barnes & Noble for malware infested POI.
Sorry Mr. Multz, but it was employees and/or contractors at all of these organizations, not the QSA that had a part in these breaches and all breaches for that matter. I really do not see how you can hold a QSA responsible for the inaction and errors of employees/contractors. Organizations are not going to pay to have QSAs on site, 24×7, to babysit all of their employees to maintain compliance with PCI or any other compliance program. Not only that, no security framework is ever going to stop breaches, all they do is hopefully minimizing the impact when a breach occurs.
However, Mr. Multz was not done.
“The PCI Requirements were created so that organizations would focus on securing their networks, but many assessors only focus on meeting the requirements rather than security.”
From this statement it is painfully obvious that Mr. Multz does not understand what an assessment is about and how the assessment process works. The job of a QSA is to execute the tests as defined in the PCI DSS Reporting Template and report the results of that testing – nothing more, nothing less. Organizations are judged by a QSA as compliant with the PCI DSS whether they are just squeaking by or if they have a full on security program next to none. Organizations do not get “extra credit” or “atta boys” if they have gone beyond the requirements.
While the original intent of the standards was to focus on securing cardholder data, that got morphed by the wonderfully misdirected marketing job that was done by certain card brands before the PCI standards came together. For those of us around the security industry more than a decade ago, we advised Visa and MasterCard to stop pushing their cardholder information security program (CISP) and site data protection (SDP) standards as “The Way” that was going to stop breaches. We explained that, properly implemented, CISP and SDP should minimize the number of PANs obtained, but it would not completely stop breaches. It was only recently that the card brands started to realize this fact and stop pushing the PCI standards as a panacea of security. If you have noticed with the rollout of EMV, Visa, MasterCard and the PCI SSC have stated that EMV is not a “silver bullet” solution and in other statements stated there are no “silver bullet” solutions. That is a long way from a decade ago when their security standards were sold as the “be all to end all” for stopping breaches. Unfortunately for QSAs everywhere, that message is out there and we have to deal with it every day.
All of this is not to say that QSAs cannot and do not make recommendations to organizations regarding their security programs and how and where it needs to improve. I constantly make suggestions during my PCI assessments on how my client needs to improve their security posture. However, it is ultimately up to the organization to put such changes in place, not the QSA’s responsibility. If an organization chooses inaction, I will bring it up again and again. But as the old proverb states, “you can lead a horse to water, but you cannot make them drink”.
Where the PCI DSS assessment process truly fails is the point in time approach (with the exception of vulnerability scanning and a few other select requirements). To address that shortcoming, the Council has introduced the concept of business as usual (BAU) and it is my guess that we will see that concept placed into the standard in the next version. It will be then that QSAs will have to test PCI compliance over a 12 month period similar to testing procedures financial auditors perform for annual financial audits.
As a result, the inclusion of BAU as part of the PCI DSS will likely be the straw that breaks the camel’s back for a lot of organizations. This is because BAU will require organizations to track their compliance with the PCI DSS 24x7x365 as they should have been doing all along. But from experience, I can tell you that there is no organization I have ever encountered that was compliant with any standard all of the time because people make mistakes. As such, BAU is designed to shed light on those mistakes and require organizations to identify them and remediate them. For organizations just squeaking by, this will probably make PCI compliance truly impossible to achieve. If you are one of those organizations complaining about compliance with the current PCI DSS, just wait until BAU gets added. Organizations that are truly interested in security are already implementing BAU because they see the operational value in integrating security controls with their other business controls. BAU will show the true colors of those organizations that want security versus those that are checking a box.
And that gets me to Mr. Multz’s actual reason for his post, what makes a good QSA? Good QSAs understand that the world is not perfect nor is security. Good QSAs know that compliance with the PCI DSS does not and will not eliminate breaches. Good QSAs know that the goal of PCI compliance is to minimize security control errors, provide an ability to recognize security control errors as soon as possible and then remediate those security control errors such that the security controls are only non-compliant for the shortest possible amount of time.
But just because a company has such errors does not automatically mean that they are not PCI compliant. A good QSA only judges an organization non-compliant when the QSA has evidence that problems are consistently recurring and are not being corrected in a timely manner or corrected at all.
I appreciate Mr. Multz’s frustration but as a QSA I do not appreciate him tossing me under the bus with the QSAs that are doing a disservice to PCI compliance. Like any industry, there are good service providers and there are bad service providers. Those of us in this industry all know who the bad ones are and we hope they will get weeded out. But from my own long experience in consulting, that does not always happen.
So in my very humble opinion, Mr. Multz needs to suck it up and deal with it, but stop tossing QSAs under the bus in the process. QSAs are only the messengers.
PCI Guru 