How Many Auditors Does It Take …

The title of this post sounds like the start of one of those bad jokes involving the changing of light bulbs.  But this is a serious issue for all organizations because, in today’s regulatory environment, it can be a free for all of audit after audit after assessment after assessment.  A never ending cascade of interruptions to their business operations as they go through audits and assessments, all in the name of ensuring controls are designed and functioning properly.

But another reason I have written this post is because of all of the comments that I have received that seem to paint my position as a reason why QSAs are not needed to conduct PCI DSS assessments.  I wanted to clarify for everyone my rationale for my position.

Besides those reasons, the larger reason this issue needs to be brought up and discussed is that the PCI SSC is pushing for organizations to adopt business as usual (BAU).  For those of you that did not read the preamble of the PCI DSS v3, BAU is the integration of relevant portions of the PCI DSS into an organization’s everyday activities.  A rather noble goal and only a recommendation at this time, one has to believe that BAU will at some point become part of the PCI DSS in a future version.

Any organization that takes the time to implement BAU is going to want to assess their implementation of BAU.  They will do this through internal/external audit activities, automated real-time monitoring via dashboards and other internal assessment processes.  Why bother with BAU if you are not going to use it to spot control issues before they become major problems?  That is, after all, the whole point of BAU.

Which brings me back to this year’s Community Meeting and the question I asked about reliance on other auditor’s/assessor’s work.  The reason for the question is to minimize, as best we can, the disruptive effects of the myriad of audits/assessments that some organizations are required to submit.  The answer provided by the Council was an emphatic “NO!” followed by some backtracking after the audience apparently showed its displeasure to the Council members on stage to their take it or leave it answer.

The reason for the audiences’ displeasure though is genuine.  A lot of organizations question the number of times user management controls such as identification of generic UIDs, last password change date, last logon date and the like need to be performed before such activities are deemed adequate?  How many times do facilities people need to be interrupted to prove that video monitoring is performed and the video is retained?  How many times do facilities have to be visited and reviewed for physical access controls?  There are numerous areas in all control assessment programs where those programs cover the same ground in varying levels of detail and focus.  It is these areas of commonality where the most pain is felt and we hear the lament, “Why do I have to keep covering this ground over and over with every new auditor that comes through?”

It is not like the PCI DSS cornered the market on control assessments.  Organizations have to comply with ISO, HIPAA, GLBA, FISMA, NIST and a whole host of other security and privacy control audits or assessments.  All of these audits/assessments share certain common controls for user management, physical security, facilities management, etc.  What differentiates the programs is the focus of what they are trying to protect.

One easy approach to address this situation is to combine audit/assessment meetings with personnel in physical security, facilities management, user management and the like.  Each auditor/assessor can ask their specific questions and gather evidence and conduct testing as they need.  Unfortunately, due to timing of reporting requirements, having common meetings might not always be possible.

But another approach would be to use internal auditors performing testing monthly, quarterly, etc. and then the QSA reviewing those results during their annual PCI assessment process.  There might be some independent testing required by the QSA for areas such as device configurations, change control and application development changes, but the sample sizes of any testing could be greatly reduced because of the testing done throughout the year due to the implementation of BAU.

If we as QSAs work with other auditors/assessors and agree to common criteria in our respective work programs that satisfy our common controls then we will not have to interrupt an organization to ask the same questions and alienate people as we do today.

Success of compliance programs is the result of making them as unintrusive and automatic as possible.  BAU is a great idea, but it will only succeed if the Council understands how BAU will be implemented in the real world and then adjusts their compliance programs and assessment approach to take BAU into account.  The quickest way to kill BAU is to make it painful and cumbersome which the Council is doing very effectively at the moment.


3 Responses to “How Many Auditors Does It Take …”

  1. September 17, 2014 at 11:09 AM

    If it weren’t for the fact that my intrusive activities didn’t continually uncover really, really significant deficiencies in the processes that have always been implied in the PCI DSS I would tend to agree with you. Full Disclosure – I work for a company that provides a continuous monitoring solution that I think most QSAs/Auditors would love to use to simply sit in one spot and review dashboards/reports that demonstrate the majority of technical controls in the PCI DSS (which would cover most other regulatory requirements as well) for not only the cardholder environment but the entire enterprise.

    Forget all the agonizing hours and money spent trying to rationalize a segmented environment that ends up being a half-dozen “enclaves” scattered throughout the enterprise, and then trying to convince me that all those connected systems I see aren’t really in scope – just secure everything and then monitor everything on a near real-time (e.g. continuous) basis. What bliss. (I’m not blatantly trying to sell our tools here but I actually think they would greatly help the cause).

    But again, I stop short of fully endorsing this approach because I never conducted an assessment where I didn’t uncover something new and previously unaccounted for – even after six years onsite! I’m not talking about nitpicking policies or procedures, I’m talking about discovering undocumented cardholder data flows, repositories, and worse (and sadly far too often) retention of sensitive authentication data. I was pretty good at conducting these assessments, and very often my customers would say, “that was the toughest assessment we’ve ever been through, but boy I’m sure glad you did it that way” – but I would always caveat my work with, “I don’t guarantee that I’ve found everything – keep looking, and keep up the due diligence.”

    In theory, the QSA is a seasoned security professional, and IMO a cut above the run of the mill auditor. I would rather see all those auditors out there rely on the work that has been performed by the QSA rather than vice versa. I trust my work a whole lot more than some unknown, last-week-was-a-CPA , has more certifications than you can count on one hand, really great test taker that doesn’t even understand the rationale behind three quarters of the requirements in the PCI DSS in the first place. Of course, many of these auditors also moonlight as QSAs…ugh.

    Death to the auditor (metaphorically of course)…long live the QSA!

    • September 17, 2014 at 11:43 AM

      Please take this in the context of the assessee having implemented business as usual (BAU). And BAU does NOT require more appliances and widgets nor does it replace the need for QSAs. What BAU focuses on is getting organizations to walk the walk, not just talk the talk.

      I did assessments of financial institutions for years and they are possibly the most regulated and assessed entities on the face of the Earth and implemented the BAU concept decades ago. However, I always found issues that needed to be addressed in audits. What I didn’t find was issues all over the place and that is what will happen as organizations implement BAU.

      It is not that issues go away under BAU, they just become fewer which is a good thing. As things stand today with most organizations, that would be a VERY good thing.

      That said, there are areas under the BAU model where you are going to end up covering ground that internal audit has already covered. If you don’t trust the internal audit function, fine then do your procedures and don’t expect to be invited back. A lot of organizations have exemplary internal audit functions and those are the organizations that are tired of QSAs from reconducting audits of areas that they already rake over with a fine toothed comb. What I am suggesting is that the Council needs to take a page from the financial audit industry and stop the audit/assessment insanity of redoing everyones’ work. I know plenty of internal auditors that take apart their organizations and do much more testing than the PCI DSS requires, find many more findings than you ever would and it’s arrogant for anyone to think that they are going to better them. But arrogance at times is the Council’s forte.

      Yes, one more set of eyes MIGHT uncover something, but in my 30+ years of experience that is a very, very rare occurrence.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Welcome to the PCI Guru blog. The PCI Guru reserves the right to censor comments as they see fit. Sales people beware! This is not a place to push your goods and services.

September 2014

%d bloggers like this: