18
Sep
10

The 2010 PCI Community Meeting

It is that time of the year.  Time for another get together with the PCI SSC, the card brands, participating organizations and QSAs.  This year’s meetings are in Orlando and Barcelona.  Unfortunately, I am not going to be in attendance due to scheduling conflicts.  Since I will not be able to attend, I thought I would provide a topic for discussion.  I want to get the PCI SSC to repeal their inane Report On Compliance (ROC) report writing standard.  This standard has become onerous and, in the end, has become “make do” work.

To understand this situation, you need a bit of history.  Until last year, the only proof that the PCI SSC and the acquiring banks had to prove a QSA had done their job properly was to read the ROC.  The ROC was required to contain references to all of the documentation, interviews and procedures they had observed to ensure that an organization was complying with the PCI DSS.  As a result, this caused the PCI SSC to develop an extensive grading and scoring spreadsheet that is used to determine if a ROC covers everything it is required to cover.  Each test may have any of the following components.

  • Observation;
  • Interview;
  • Documentation;
  • Process/action/state; and
  • Network monitoring.

Each of these components may be assessed one to four scoring points depending on the number of occurrences that may be contained in the given test.    A ROC must score better than 75% in possible points to avoid remediation.  But the PCI SSC expects that a ROC should score no lower than 95% of possible points.

The PCI SSC has instructed QSAs that each test in the ROC must be able to stand on its own.  This means that one test is not allowed to reference another test.  As a result, QSAs must replicate of a lot of information throughout the report.  This obviously introduces the potential for errors and omissions in the report as well as making the report unnecessarily long.

To ensure the report writing process is truly questionable, the PCI SSC recommends that QSACs develop pre-written templates so that all of the components get covered for each test.  While a template speeds the report writing process, I would still estimate that the report writing process consumes at least one-third to one-half of a PCI assessment’s budget.  Not only does it take time to write, but it takes a lot of time to proof and to review.

As I stated earlier, last year, the PCI SSC began requiring language in all QSA contracts that grants the PCI SSC the right to examine any QSA’s work papers.  AS a result, one would think that this report writing standard would no longer be needed, but it is still in place.

Because a lot of our clients use hosting services, we get to see a lot of ROCs that have been prepared by other QSAs.  You can really tell those QSAs that have not been through the PCI SSC QA process by the fact that their ROCs are very short and lack detail.  But for those QSAs that have been through the QA process, based on my review of their ROCs, the grading scale seems to have caused QSAs to worry more about how the ROC is written and not necessarily on the actual assessment of the security practices of their client.  A lot of the writing is more about meeting the scoring template and not about the controls.  In some cases, the writing starts you to wonder if the control is really in place.

ROCs can become inordinately long because of the replication of the same information over and over.  During our QA remediation, we were told that the average ROC ran around 180 to 200 pages however I have yet to see one produced by us that is under 250 pages and we seem to average around 350 to 400 pages.  I have heard from some reviewers at acquiring banks that the only worthwhile information in these tomes is anything that is not in place and any compensating controls.  If that is all that is getting read, then what is the point of all of this other information that is being ignored?  The point is that it remains the way that the PCI SSC ensures that QSAs are doing their job.  And as I stated earlier if the writing makes you question if the control is in place, then what is the quality of all of this writing?

Now that the PCI SSC has the right to review a QSA’s work papers, there is no reason to require all of this pointless verbiage in the ROC.  QSAs should be able to have one column for each requirement in the report labeled ‘Status’ and the entry in the column is either ‘In Place’ or ‘Not In Place’.  If something is not in place, then the column next to it, labeled ‘Comments’, should document what is being done to bring a requirement into compliance and when that will occur.

If the PCI SSC is not comfortable with this approach, then maybe they have the wrong organizations as QSACs and they need to get rid of those that cannot conduct the work to professional standards.  This approach works for financial auditors, there is no reason it cannot work here.

Have a good time in Orlando or Barcelona.

Advertisements

1 Response to “The 2010 PCI Community Meeting”


  1. 1 Brian
    October 19, 2010 at 5:06 PM

    Consider this scenario. If a QSA assesses an entity and deems them compliant (i.e., “In Place” for all requirements) and the entity is breached, who gets blamed? On more than one occasion in the past year, the QSA has been blamed by the entity for not finding the vulnerability that led to the compromise. The QA scoring matrix is risk management for the QSA. If an entity was found to be compromised after an assessment, the QSA can defend their assessment by showing what was reviewed, what their observation was, and how they came to their conclusion. Without this level of documentation the QSA can fake it, and sometimes do so under pressure from their client.

    “Independently defensible” is the phrase I like to use, wherein the documentation should allow a third-party to review the test results and come to the same conclusion of the author. Some QSAs are not auditors, by training or profession, and it sounds like you are not either. While, there are some faults in the QA scoring matrix, I disagree with your premise that it should be eliminated entirely.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Announcements

If you are posting a comment, be patient, as the comments will not be published until they are approved.

If your organization has a PCI opportunity, is in need of assistance with a PCI issue or if you would like the PCI Guru to speak at your meeting, you can contact the PCI Guru at pciguru AT gmail DOT com.

I do allow vendors to post potential solutions in response to issues that I bring up in posts. However, the PCI Guru does not endorse any specific products, so "Caveat Emptor" - let the buyer beware. Also, if I feel that the response is too "sales-ee", I reserve the right to edit or not even authorize the response.

Calendar

September 2010
M T W T F S S
« Aug   Oct »
 12345
6789101112
13141516171819
20212223242526
27282930  

Enter your email address to subscribe to the PCI Guru blog and receive notifications of new posts by email.

Join 1,883 other followers


%d bloggers like this: