How The PCI Standards Will Really Die

Welcome to the new year.  I hope the holidays have been treating you well and the coming year is good as well.

There have been a number of articles written about why and how the PCI compliance process will die.  It is not that I look forward to the PCI standards dying as they have brought a needed visibility to information security and privacy as well as the fact that PCI keeps me gainfully employed.  However if things stay on their current trajectory, the PCI standards will eventually die, but not for the reasons being quoted in today’s articles.  The real killers of the PCI compliance process will be the card brands and the PCI Security Standards Council.  Yes, the very folks that brought us the PCI standards will bring the ultimate demise of their precious set of standards.

The first death knell I see is that it is very easy to issue edicts from on high when you do not have to implement them.  Over the years, clarifications have been issued, quality assurance reviews performed, forensic examinations conducted and a host of other activities have resulted in “enhancements” to how the PCI standards are assessed and enforced.  Do not get me wrong, a lot of what has been done was needed and appreciated.

However, by the same token, some of what has come down has been a nightmare to implement.  Any QSAC not using some sort of automated system to conduct their PCI assessments will find it impossible to meet the current and any future documentation and tracking standards now required by the PCI SSC’s QA process.  Under the current standards, QSACs need to document who they interviewed and what the persons were interviewed about as well as tying documentation and observations to the tests performed.  Without some sort of automated process, these requirements are just too intensive to perform manually.

Documentation received and reviewed needs to have its file name, date of issue and a description of its purpose in the PCI assessment process documented.  The basic PCI DSS has a minimum of around 200 discrete documents that are required for the PCI assessment process.  The average we see for most of our engagements is over 600 documents which also include not only policies, standards and procedures, but configuration files, interview notes and observations such as screen shots, log files and file dumps.  You really have to question any QSAC that tells you they manually manage the process.  They either have an amazing and magically efficient project management process, they have very, very inexpensive staff (i.e., overseas labor) or they are short cutting the processes and producing a work product that does not comply with the PCI SSC QA program and have yet to be assessed by the PCI SSC (the most likely scenario).

Even using simple SharePoint or Lotus Notes solutions are not cheap when you consider the cost of the server(s) and the storage of all of documentation collected, which can be around 5 to 10GB per project, as well as all of the requisite system maintenance.  Servers and storage may be cheap, but it all adds up, the more clients you assess.  And speaking of the storage of documentation, the PCI SSC requires that documentation related to PCI assessments be stored for at least three years.  For those of us with electronic work paper management systems, this is not a problem.  However, given the amount of paper generated by these projects, those QSACs using the traditional paper filing methods will find a lot of shelf space taken up by their PCI engagements if they are truly following the procedures required by the PCI SSC.

All of this drives up the cost of a proper PCI assessment, more than I think the card brands and the PCI SSC are willing to admit.  It is not that I think the card brands and PCI SSC do not care about this situation, but more related to they do not have an understanding of the operational ramifications of their edicts.  The card brands and PCI SSC tread a very fine line here and to this point they have been heavy handed in the issuing of their edicts.  Going forward, the PCI SSC needs to ask the QSACs, Participating Organizations and ASVs to assess the cost and time impacts of these edicts so that they can be weighed against their benefits versus what is done now which is more of a procedural and proofing review.  If this is not done, there will soon come a point where merchants and service providers will push back hard and refuse to go through the process due to the cost and the amount of time involved to be assessed.

The next death knell is the inane process that is called the PCI Report On Compliance (ROC).  When the PCI SSC did not have access to the QSACs’ work papers, the current ROC writing process made some sense as there was no other way for the PCI SSC or the processors and acquiring banks to know if the QSACs had really done the work they were saying they had done.  However, all of that changed a number of years ago when the PCI SSC required QSACs to add a disclaimer to their contracts stating that the PCI SSC had the right to review all work products.  Yet even with this change, we continue to have to write an insanely detailed ROC, typically numbering in a minimum of 300+ pages for even the most basic of ROCs.

Unfortunately, there are QSACs out there that apparently have not been through the PCI SSC QA process and that dreaded of all states – Remediation.  As a result, they have much lower costs because they are not documenting their assessment work as completely as they need to and are not sampling, observing or interviewing like QSACs that have been through the QA process.  In addition, based on some work products we have seen, they also do not care about the quality of the resulting ROC as it looks entirely like a ‘find and replace’ of a template and makes no sense when you read it.  In talking to other large QSACs that have been through the QA process multiple times, the PCI SSC has indicated that they are monitoring the large QSACs more than the little QSACs because there is more risk with the large QSACs.  While true to an extent, we have encountered a number of smaller QSACs that perform assessments for large clients due to their much lower cost structure and their willingness to ‘overlook’ compliance issues.  If the PCI SSC does not go after these QSACs soon, there will likely be a number of breaches that occur due to the QSACs’ lack of diligence in performing their assessments.

I know of a number of QSACs that would like to see Bob Russo and the representatives of the various card brands to actually work as staff on a few PCI assessment engagements so that they can better appreciate the inordinate amount of work involved in generating a ROC.  I think they would be shocked at the amount of work effort they have driven into a process that is already too complicated and prone for error.

As it stands today, the ROC writing, review and proofing process is probably 50% to 60% of a good QSAC’s project costs.  To address this, the PCI SSC QA group tells QSACs to develop one or more templates for writing the ROC which, from what we have seen from some other QSACs, means a lot of mass ‘find and replace’ to speed the ROC writing process.  For the last few years, a number of QSACs have brought the ROC writing process up at the Community Meetings.  However the card brands continue to shoot down any sort of changes to the process.  As a result, the cost of producing a ROC is driven by the size and complexity of the merchants’ or service providers’ cardholder data environment (CDE).  These costs will only continue to rise as long as the PCI SSC does not allow QSACs to mark items as ‘In Place’ with only a check box and rely on the QSAC’s work papers versus the verbosity required now.  If this sort of process can work for financial auditors, it can work here as well.

A third death knell is the PCI SSC and card brands continuing to quote that the majority of breaches are the result of organizations not complying with the PCI DSS.  In discussions with a number of the PCI forensic examination companies, I am hearing that the card brands cannot believe the fact that more and more organizations were PCI compliant at the time of their breach.  The PCI SSC and card brands have apparently convinced themselves that the PCI standards are “perfect” and they cannot imagine that an organization could be breached unless that organization was not complying with the PCI standards.  There is no security standard that I am aware that totally prevent breaches.  So while the PCI standards are good baseline security standards, the card brands and PCI SSC seem to have forgotten that security is not perfect and that any security standard only minimizes the damage done when a breach occurs if the standard is truly followed.

And as organizations have gotten the PCI “religion,” the effort required to compromise them from the outside via traditional attacks has increased significantly.  As a result, successful attackers have changed strategy and work on social engineering their way past the bulk of an organization’s security measures.  The PCI DSS only has a little bit on social engineering in requirement 12.6 regarding security awareness training.  And even those organizations with the most robust of security awareness programs will tell you that, even after extensive security awareness training, human beings are still fallible and that some people still do very questionable things that continue to put organizations at risk, sometimes significant risk.  Even when you have the most diligent of employees, they still make mistakes in judgment from time to time.

Until the human element can be totally removed, there will always be a certain amount of risk that will never go away.  Again, the PCI SSC and card brands seem to not want to acknowledge the failings of the human element and appear to believe that technology is the savior based on the focus of the PCI standards.  However time and again, every security professional has seen very sophisticated security technologies circumvented by human error or just plain apathy towards security (i.e., “it always happens to someone else, not my organization” or “we’re too small to be a target”).

Until the PCI SSC and the card brands drop the “holier than thou” attitude toward the PCI standards and stop the public pillory of organizations that have been breached, there will continue to be editorial commentary regarding the pointlessness of the standards and ever more serious push back to complying with the standards.

These are the reasons why the PCI SSC and the card brands will be the ones that will kill the PCI standards.  At the moment, they are so far removed from the process; they do not understand how complicated and expensive the process has become which is why merchants and service providers are complaining about the ever increasing costs and effort related to the PCI assessment process.

The PCI SSC and card brands also seem to have forgotten that QSACs have to make money doing these assessments and, when you pile on clarifications and edicts that do nothing to streamline and simplify the process; you are only driving the costs of the process higher.  And higher costs only make merchants and service providers, who are on thin margins to being with, even more incentivized to use the much lower cost QSACs, driving the diligent QSACs out of the market, thus increasing the likelihood of breaches.

Again, it is not that I want the PCI standards to go away as I think they have brought a real benefit.  However, if these issues are not addressed, the PCI standards will end up going away.  I fear that, with them gone, there will be no carrot to ensure the security of cardholder information and we will end up back where we were before the PCI standards existed.

Another Year, Another QSA Re-Certification

It is that time of the year when I have to go through the PCI SSC’s Qualified Security Assessor (QSA) re-certification process.

To add to the re-certification process this year, I have been sick for the last two months with a cold that turned into a nasty case of bronchitis along with laryngitis that then caused a severe case of sinusitis.  I just could not catch a break this Spring.  The good news is that I am finally on the mend and should be back to normal in another couple of weeks.

However, even illness does not get you out of the QSA re-certification process.  So, I put it off as long as I could and took the examination this morning.

As I expected, there was not a lot of new material in this year’s QSA update.  The biggest focus of this year’s training seemed to be:

• The interrelationship of the various PCI standards;
• Roles and responsibilities of QSAs, ASVs, merchants, service providers, acquirers, PCI SSC and the card brands;
• Scoping of the cardholder data environment and cardholder data discovery; and
• The integration of the PA-DSS with the PCI DSS.

Other than that, it was for the most part a reinforcement of the changes in the PCI DSS v2.0 to make sure that QSAs really understand the standard.

There is an interesting section on what not to write in the In Place column.  The unfortunate aspect about this section of the training was that the examples that were presented were straight out of ROCs that the PCI SSC QA program had reviewed.  Some of those responses were very difficult to read they were so bad.

There is also a discussion on network segmentation.  Unfortunately, the examples were very simple.  I wish our clients had such simplified networks.  However, because this discussion is in this year’s presentation materials indicates there are apparently still a lot of QSAs that do not understand the concept of network segmentation and what constitutes good segmentation from poor segmentation.

As I am finishing this post, I have been told I passed the QSA re-certification examination.  So I am a QSA for another year.

PCI DSS Compliance Certificates

In this month’s PCI SSC QSA Newsletter, the FAQ of the Month is about so called ‘PCI DSS Compliance Certificates’.  I started to hear about these a couple of years ago, but it got really big last year when I started running into processors and acquiring banks demanding them.  I had a particularly troubling conversation with a processor who demanded that we produce one for one of our clients.  When offered the PCI DSS Attestation Of Compliance (AOC), this processor acted as though we were trying to put something over on them.  When I asked him where I was supposed to get such a certificate when it does not exist on the PCI SSC Web site, he accused me of not being a QSA because all QSAs know what the certificate looks like and where to get it.

As a result, a lot of QSAs must have submitted a question regarding these certificates like I did.  Here is the PCI SSC’s response.

“In addition to the official PCI SSC reporting forms and templates, some QSA or ASV companies provide certificates, letters or other documentation as confirmation that an organization is PCI DSS compliant. The PCI SSC does not prevent QSAs or ASVs from producing this type of documentation, as it is considered an additional service which the assessor company may elect to provide and is therefore outside of the purview of the Council.  However, in accordance with the ethical requirements for QSA and ASV companies, any such certificates, letters and other documentation must be accurate and not be in any way misleading.  Additionally, these certificates, letters and other documentation should be clearly identified as supplemental materials provided by the QSA or ASV; they should not be presented as documents endorsed by the PCI SSC, nor should they be considered replacements for the official PCI SSC templates and forms which have been approved by the payment brands.

The PCI SSC website contains reporting templates and forms which have been approved by all payment brands, including ROC templates, Attestations of Compliance, Self-Assessment Questionnaires, and Attestations of Scan Compliance for ASV scans. Compliance validation and reporting requirements are determined by the individual payment card brands and, irrespective of whether an organization is performing a self-assessment or has an onsite review completed by a QSA company, acceptance of a validation method outside of those listed on the Council website is ultimately up to the entity accepting the validation (that is, the acquiring bank or payment card brand). In many cases, certificates, letters or other documentation issued by QSA or ASV companies outside of the official PCI SSC templates may not be accepted by acquiring banks or payment card brands. ASVs and QSAs should encourage their clients to check with their acquirer or the payment brands directly to determine their compliance reporting requirements, including whether the submission of such certificates is acceptable.”

So all of you processors and acquiring banks that seem to think the only acceptable proof of PCI compliance is some mystical PCI DSS Compliance Certificate, stop demanding them.  They do not exist and never have existed.  The document you need for proof of PCI compliance is the Attestation Of Compliance (AOC), period.  All self-assessment questionnaires (SAQ) contain the AOC and there is a separate AOC form for those submitting a Report On Compliance (ROC).

And all of you QSAs and ASVs out there differentiating yourselves because you produce these nice, but essentially worthless, certificates, stop misinforming merchants, processors and acquiring banks by implying that QSAs and ASVs not producing such a certificate are somehow doing something wrong or worse, dishonest.

Now that the PCI SSC has clarified this situation, hopefully, this marketing ploy will stop.

PCI SSC Updates The ASV Program and Issues New Information Supplement

March 2011 has been a busy month thus far for the PCI SSC.  On Thursday, March 10, they announced a new ASV training program and on Friday, March 18, they released an Information Supplement on protecting telephone-based cardholder data.

The ASV training program has blindsided the ASV community as it was a total surprise.  Yes, there has been talk over the years at the Community Meetings and in other venues regarding ASV qualifications and training, but nothing ever seemed to come from those discussions.

According to the press release, the ASV training program, “… is a direct response to the feedback we’ve received from the merchant community …”  One can only assume that the complaints that have been voiced about ASVs from merchants and service providers as well as the comments in the media have finally caught the attention of the PCI SSC and they are going to address the problem with training.

The consistent complaint I have heard from clients over the years was in regards to the fact that no two ASVs ever scoped their network the same for vulnerability scanning.  The next most common complaint was always about the different results of the vulnerability scanning which was most often voiced after a company changed ASVs.  While there are always some bad apples in the bunch, I do believe that most ASVs know what they are doing.

As a result of all of this, I am sure this new training requirement is to address the complaints and make the program better.  This training is in addition to the network security test ASV companies already had to pass.  For those of you that did not know, ASVs have to conduct network security scanning against a test network with predefined vulnerabilities operated and configured by the PCI SSC.  ASVs are expected to produce a sample ASV report and document all of the predefined vulnerabilities.

The ASV training is comprised of an eight hour online course and an examination.  A minimum of two employees are required to take the course and pass the examination.  Once this has been accomplished, the organization is designated as having Qualified ASV employees.  As with a lot of PCI requirements, there are some important dates involved.  ASVs that are renewing their status prior to June 1, 2011 need to get two personnel trained and passed the examination by June 15, 2011.  ASVs that will certify after June 1, 2011 need to have two staff trained and passed the examination by their renewal date.

Given the additional cost of this new training plus the requirement to have a minimum of two people trained, it will be interesting to see if any of the existing 130 ASVs drop out because of this new requirement.

Telephony Cardholder Data

The other issuance of interest this month was in regards to telephone conversation recordings that contain cardholder data.  While this is a longer dissertation on FAQ 5362, there is really nothing new presented in this information supplement.  The bottom line still is that if you have call recordings that contain cardholder data you are required to either; (a) do not record it in the first place, (b) if recorded, redact it if possible, or (c) make sure that it cannot be searched, is encrypted and access is restricted.  The best thing about this information supplement is the flowchart that was created for this situation.

The “Magic” Vulnerability – Revised

What started this post is that I have recently received a number of calls and messages from clients and colleagues.  The conversations have all gone basically the same.  They were calling me and telling me that their ASV had failed their vulnerability scan because the OS detected was unsupported and they are wondering whether or not I have encountered this before.

My first question usually was along the lines of; “So, what vulnerabilities did they detect?”

“None,” was the confused answer at the other end of the line.

“What?  They must have detected at least one high, severe or critical vulnerability?  That is the only way you can fail,” I would ask, now also confused.

“Nope.  Nothing.  Just the fact that the OS is unsupported,” I was told.

Do not get me wrong.  I am not advocating the use of unsupported operating systems, particularly unsupported versions of Windows.  The risk of course is that one or more vulnerabilities show up that the vendor will not fix because the OS is no longer supported.  So there is good reason to avoid this situation.  However, there are also situations when you just get no other choice either due to your own organization’s issues and politics or software vendor issues.

This situation got me thinking and doing some research since I did not remember ever seeing or being told that an unsupported OS was an automatic vulnerability scan failure.  I no longer do external vulnerability scanning, so my recollections of training and working on the ASV side of our business is a bit fuzzy and very rusty.  However, I had never failed a client for an unsupported OS.  So when this issue came up, my only action was to determine what had changed.

The first thing I did was review the latest version of the PCI ASV Scanning Procedures, v1.1.  I searched for terms such as ‘old’, ‘unsupported’, ‘out of date’, ‘OS’ and ‘operating system’.  No matches.  So there is nothing in the ASV scanning procedures that fail an organization for running an unsupported OS.  Even the PCI DSS does not call out unsupported software, so procedurally; I am thinking there is nothing explicit regarding unsupported OSes causing a failed vulnerability scan.

So when I made the original posting, I got a comment from one of my readers pointing me to the ASV Program Guide.  Low and behold on the top of page 16 is the following:

“The ASV scan solution must be able to verify that the operating system is patched for these known exploits. The ASV scanning solution must also be able to determine the version of the operating system and whether it is an older version no longer supported by the vendor, in which case it must be marked as an automatic failure by the ASV. “

So there is no “magic” vulnerability I was missing as the PCI SSC does specify that a scan automatically fails if the OS is unsupported.

But that is not the entire story.  The key to this whole process is that the vulnerability scanner used must be able to verify the operating system.  While all vulnerability scanners attempt to identify the operating system, the reliability of this identification process is suspect at best.  I am not aware of any vendor of security testing tools that makes a claim that they will identify an operating system 100% of the time.  This is because of the fact that there are many, many things that can influence the OS signature that the tools cannot control and therefore can greatly affect the ability of the tool to identify the OS, particularly when talking about external scanning.  And if an organization follows the OS security hardening guidelines, a lot of unsupported OSes will not be properly or reliably identified by vulnerability scanners.  As a result, I find it hard to believe that the PCI SSC intended to have ASVs only rely on the results of a vulnerability scanner, but that seems to be the case.

So with this clarification, I contacted our ASV personnel and they have told me that they too have been failing vulnerability scans if they run across unsupported operating systems.  I ask if the OS signature is inconclusive, then there is not a failure?  Yes, if the scan comes back and does not identify the OS, then they have nothing to go on to fail the scan and the scan passes.  Given the difficulties vulnerability scanners can have identifying the target operating systems such as when scanning through network firewalls, Web application firewalls, load balancers and the like, I now ask if they feel that these identifications are reliable enough to fail a scan.  I am told this is why they confirm the information with the client before issuing the report so that the report is accurate.  So if a client is not honest, they could influence the results of their scan?  I am reluctantly told that is probably true.

Then there is the issue that not all operating systems are created equal.  Operating systems such as MVS, VMS and MCP are nowhere as risky, if they are even risky to begin with, as Windows and Linux.  A lot of ASVs would argue that they never come across these operating systems running Web services.  However, all of them have the capability of running Web services and I personally know of a number of organizations that run their Web services from such environments.  Organizations are running these older versions of operating systems mostly because of the financial considerations of migrating to something else.  However, I can guarantee that none of the dozens of vulnerability scanners that I have used in the last 10 years would accurately identify any of these operating systems, let alone tell you the version unless some service message header information was retrieved by these tools.  And even then, most tools do not parse the header to determine the OS so it would take human intervention to make that determination.

Regardless of the failure, most ASVs do have a review or appeal process that allows organizations to dispute findings and to submit compensating controls to address any failures.  So for organizations that cannot get rid of unsupported OSes, they can use a compensating control.  Like compensating controls for the PCI DSS, the organization is responsible for writing the compensating control and the ASV needs to assess the compensating control to ensure that it will address the issues identified by the vulnerability scan.

So, if you can fail an organization over an unsupported OS, why is it that you do not automatically fail on unsupported application software?  I went through the Program Guide and there are all sorts of other criteria for applications but nothing regarding the fact of what to do if they too are unsupported.  Applications such as IBM Websphere and Oracle Commerce can become unsupported just as easily as their OS brethren.  And in my experience, use of unsupported application software is even more prevalent than unsupported OSes under the idea that if it is not broken and does not have vulnerabilities, why upgrade?  When I asked our ASV group if they fail organizations on unsupported applications I got silence and then the response that they will fail an application if the vulnerability scanner provides a high, severe or critical vulnerability.  To tell you the truth, while vulnerability scanners regularly return text header information for a lot of applications, I would be hard pressed without doing a lot of research to find out if the version being reported was unsupported.  However, scanners could provide this feedback if they were programmed to provide it.

Then there are all of the conspiracy theories out there that say the PCI SSC and technology companies are working together to drive software sales by forcing organizations to upgrade and there would appear to be a lot of anecdotal evidence that would seem to support this argument.  In reality it is not that the software companies are working together with regulators such as the PCI SSC so much as software companies operate this way in order to focus development and support resources on fewer, more current versions.  As a result, it is just happenstance that regulations cause organizations to have to update their software.

The bottom line in all of this is that you have options to avoid a failing vulnerability scan because of an unsupported OS.  The best method, and the one I most recommend, is do not use unsupported operating systems in the first place.  However, as a former CIO, I do understand the real world and the issues IT departments face.  As a result, I recommend all of the following which may or may not require you to develop a compensating control.

• Implement not only a network firewall, but also a Web application firewall (WAF) and make sure that the rules are extremely restrictive for servers running unsupported operating systems.
• Configure your firewalls to block the broadcasting of any OS signature information.  Masking the OS signature will provide the benefit of not advertising to the world that the OS running whatever application is unsupported.  This is not a perfect solution as, 9 times out of 10, the application itself will likely advertise the fact that the underlying OS is unsupported.  It is very important to note that this is only a stop gap measure and you should still be actively in the process of migrating to a supported OS.
• Implement real-time monitoring of firewalls, servers and applications.  Define very specific alerting criteria to ensure that any suspicious activity is immediately reported and operations personnel immediately follow up on any alerts to determine whether they are a false positive.
• Implement a host-based intrusion detection/prevention solution on any servers that run the unsupported OS.  If using a HIPS solution, you may also want to consider using its preventative capabilities for certain critical incidents.
• Implement real-time log analysis for firewall, servers and applications.  Define very specific alerting criteria to ensure that any suspicious activity is immediately reported and operations personnel immediately follow up on any alerts to determine whether they are a false positive.
• Actively use your incident response procedures to address any incidents that are identified with any unsupported OS.

Intent Of Requirements – 11.2

As I discussed in my earlier post on the intent to requirement 6.1, requirement 11.2 is another requirement where there is a hard and fast metric of four “clean” quarterly vulnerability scans.  Nice idea, but tough to execute.

To remind everyone, the intent of vulnerability scanning is to; (a) ensure that if a patch has been applied, it is in fact in place and the vulnerability has been addressed and, (b) identify any new vulnerabilities.  11.2 is in response to at least two issues.  The first is that organizations that do not vulnerability scan have no way to assess the larger risk their technology infrastructure poses to the potential loss of data.  The second is to ensure that updates and patches are in fact truly correcting vulnerabilities.  I have personally seen situations where patches have removed other patches thus leaving a system at risk of compromise.  Granted these situations are rare, but without vulnerability scanning, the removal of these patches would have never been known.

Requirement 11.2 requires that vulnerability scanning is performed at least quarterly.  However, given the 30 day patching rule, the fact that most vendors have a monthly release of patches and the fact that scanning must be performed after all “significant” changes, an organization really needs to conduct monthly vulnerability scanning at a minimum just to stay compliant.  As a result, if you have an organization doing anything less than monthly scanning, you really need to question if they understand the vulnerability scanning requirement.  However, from a ROC reporting perspective, you really only need to report the results of four quarterly scans even though 12 or more scans are available.

Then there is the problem of the definition of a “clean” scan.  According to the PCI SSC, a “clean” scan is one that contains no severe, high or critical vulnerabilities.  You hear complaints about this time and again from organizations, “How do you ever get a ‘clean’ scan when new vulnerabilities are introduced almost every day?”  This is a very fair question when most security professionals know that the odds for obtaining a “clean” scan are against them because 94% of all vulnerabilities are rated severe, high or critical.  Let alone the fact that tests for vulnerabilities can show up in scanning engines long before the vendor has a patch just exacerbates this situation.  Is it any wonder security professionals are frustrated with this requirement?

This is an area that the PCI SSC has really failed to address and bring some reality to the requirement.  For a lot of large organizations, vulnerability scans that have no severe, high or critical vulnerabilities documented cannot physically be achieved without some rare alignment of the planets, stars and moons.  They could be scanning 24×7 and still could not have a complete, truly “clean” scan of their environment.  As a result, a “clean” scan is usually all in the eye of the beholder.

But that is not the entire problem.  A lot of organizations run packages such as Oracle Commerce, IBM Websphere or MICROS Simphony.  Oracle, IBM and MICROS typically have issues with some OS or system software vendor patches such as those for Windows, Linux or Apache.  They usually issue advisories to their customers indicating that these patches cannot be installed.  In these cases, severe, high or critical vulnerabilities may not get patched for months as these vendors develop an update to their solution as well as packaging the other vendors’ patches.  As a result, organizations with these solutions can have severe, critical or high vulnerabilities in their scans for quite a while.  In these cases, a QSA needs to determine if the organization has followed the vendors’ recommendations or used any other recommendations for mitigating the risks of having these vulnerabilities.  QSAs also need to see proof that once Oracle, IBM or MICROS issue their fixes for these situations, that these organizations implement them in a timely manner.

As a result of these constraints, I have developed my own definition of a “clean” scan.  Again, I am not advocating that vulnerability scanning does not have to result in “clean” results.  I am advocating that QSAs bring reality to the process.  In my world, a “clean” scan is a scan that results in severe, critical or high vulnerabilities that were not documented in any of the previous scans.  If there are repeated severe, critical or high vulnerabilities, those vulnerabilities should only be the result of vendor patching issues and the organization must have documentation that proves that management understands that they exist, what measures the organization has taken to mitigate their existence and when the vendor anticipates to have the vulnerabilities addressed.

Here is my rationale for this definition.  There are just too many factors that go against getting what the PCI SSC defines as a “clean” vulnerability scan.  Real world experience says that getting a “clean” scan is near to impossible, particularly in large environments.  I can count on one hand the number of organizations that I have worked with that have truly four quarters of “clean” scans and none of them are very large.  Therefore, as long as I can prove that severe, high or critical vulnerabilities are being addressed in a reliable and as timely a manner as possible, then the scans are clean.

All of you anal-retentive types I am sure are going to flame me as not following the standard.  However, what is a standard worth if it cannot be applied equitably to all?

Interesting Announcements From The PCI SSC

For those of you that are not QSAs, the PCI SSC over the last year has tried to keep QSAs in the loop by issuing a monthly Assessor Update newsletter via email.  These usually are not noteworthy, but the November 2010 issue contains a number of items that need to be shared just in case you miss your edition or you are not a QSA.

PCI DSS Timeline Clarification

The Council apparently got the message that they did not communicate the sunset date for the PCI DSS v1.2.1 and the start date for PCI DSS v2.0 very well.  As a result, they issued a clarification in the November 2010 newsletter.  To quote the Council:

“Entities needing to comply with the PCI DSS are strongly encouraged to begin using the new standard immediately. However, version 1.2.1 will remain effective until December 31st, 2011 to allow everyone time to adopt any changes they may need to in order to maintain their PCI DSS compliance. This means that organizations assessing and reporting compliance during 2011 may validate to either version 1.2.1 or 2.0. However, the Council urges all organizations to complete their transition to the new standard as quickly as possible, especially where any new controls may enhance the protection of cardholder data.”

Since QSAs will not have the scoring template until sometime in January 2011, it makes planning and executing any assessments difficult until the scoring template is issued.  As a result, the earliest I can see any v2.0 assessments getting started is March 2011.

PCI DSS and PA-DSS v2.0 Scoring Templates

And speaking of those scoring templates, the scoring templates for v2.0 of the PCI DSS and PA-DSS should be published sometime in January 2011.  It would be nice to have these a bit earlier, but better late than never.

Expiration Of PABP v1.4 Extended 90 Days

The PABP v1.4 standard that was expected to expire tomorrow, December 2, 2010, has been extended to March 2, 2011.  To quote the Council:

“This updated deadline recognizes the challenges many merchants and Payment Application end users have in implementing system changes over the busy holiday period, and allows the Payment Application vendor community to consider submitting new versions of their products for assessment against the new PA-DSS 2.0 standard.

The Council is committed to reviewing all submissions for the updated versions of expiring PABP v1.4 applications, and this new March 2nd 2011 deadline will allow the review process to be completed before previous versions of these applications expire.  This extension will also provide more time for PA-QSAs to complete reviews of those Payment Applications that are currently in process.  Finally, this extension will allow Payment Application vendors, should they choose to hold off on assessment of expiring Payment Applications and instead submit (after January 1st, 2011) their Payment Applications for assessment against the new PA-DSS v2.0 standard.”

ASV Sampling And Scanning Do Not Mix

While sampling of devices is allowed under the PCI DSS, it is not allowed for ASV scans.  To quote the Council:

“Within a given quarter, all Internet accessible systems must pass an ASV scan. It is not necessary that they all be scanned at the same time, but they all must be scanned quarterly.”

Apparently, some ASVs were only scanning a sampling of PCI in-scope devices each quarter.  I am sure this will lead to consolidation of a lot of organization’s external network presence.

2011 PCI SSC Training Schedule

The training schedule for next year should be posted to the PCI SSC’s Web site by mid-December.

Telecom Private Circuit FAQ Issued

See the end of my post on MPLS for the text of the FAQ.

Why The PCI Standards Exist

After another spate of articles and speeches about the PCI DSS and why it is worthless, I thought it might be a good idea to explain why the PCI standards came to exist.

In 1999, Visa USA began to work on what became the Customer Information Security Program or CISP.  The first official version of the CISP was issued in the summer of 2003 with Visa asking select merchants to comply with the CISP as soon as they could.  The original impetus for the CISP was a response to increasing chargebacks that were the result of the fraudulent use of credit card accounts.  An analysis of these chargebacks had started to paint a picture of merchant employees that were increasingly using their access to point of sale (POS) and accounting systems to obtain credit card numbers and then using those numbers to commit fraud.

As development on the CISP progressed, Visa USA also started to see increasing instances where an e-Commerce site had been compromised and the credit card information stored on the site had been taken by an attacker and then was fraudulently used.  The reason these compromises had resulted in cardholder data being exposed was that application developers had used the same software design models for e-Commerce as those that were used by traditional POS.  This resulted in cardholder data being stored in databases that faced the Internet.

A year or so after the start of Visa USA’s efforts, MasterCard International began development of the Site Data Protection (SDP) standard.  Unlike the CISP, the SDP focused specifically on the security of e-Commerce sites.  MasterCard had monitored the rising fraud rate related to the compromising of e-Commerce sites.  Like Visa USA’s analysis, MasterCard’s analysis of the problem pointed to the fact that most e-Commerce sites were storing cardholder data in databases that faced the Internet and were not very well protected from compromise.  As a result, MasterCard approached the problem with the SDP which specified a basic level of information and network security for e-Commerce Web sites.

As work progressed on the CISP and further statistics on security issues were gathered, Visa USA began to recognize that the on-line payment applications themselves were the biggest problem related to the compromising of cardholder data.  As a result, Visa USA developed the Payment Application Best Practices (PABP) standard.  The PABP was published in late 2004 with Visa USA encouraging software vendors to use it as a benchmark for securing their software.

It has been suggested that the PCI standards were only developed to minimize the losses to the card brands and banks and do nothing for merchants.  However, the PCI standards were meant to protect everyone in the transaction process.  When a breach occurs, the first thing people remember is the name of the card brand(s) involved, the second name is likely the merchant or service provider and the third name is the franchisee if that is the case.  The card brands, service providers and franchisers discovered that their reputations were highly at risk, even though it was typically the franchisee merchant that actually created the problem.  Regardless of who caused the breach, the card brands further discovered that what people really remembered from breaches were the card brands’ names and everything else was forgotten.  As a result, the card brands became determined to protect their brand names and reputations.

There was another recent suggestion that the PCI DSS was not needed because market forces would resolve the security issues inherent in the conducting of credit card transactions.  The first problem with that idea is that most merchants and service providers are unconvinced that they are responsible for securing cardholder data, even after they might have suffered a breach.  They believe that it is the card brands’ problem to secure cardholder data because the card brands are the ones that generate the cards.  Unfortunately, the security of cardholder data is mostly outside of the control of the card brands, therefore the merchants and service providers need to be responsible for securing cardholder data as well.  The second problem with that idea is that for every security “expert”, there are a corresponding number of security baselines.  No one agrees on security, because everyone’s view is from their own perspective and the threats that they see or perceive.  As a result, some organizations have very strong and strict security (e.g., banks for example), while others have only marginal security.  The problem with this approach is that security is only as good as the weakest link in the chain.  So those organizations that have weak security practices become targets against the entire process chain.  As a result, in our interconnected world, that puts those organizations with strong security at risk if they are partners to those organizations that have weak security.  As a result, the card brands recognized that a single standard baseline was needed just to provide a basis for a consistent foundation on which to build additional security.

So, that is how we got where we are today.  Hopefully with this perspective you can now understand why the standards exist and their use in providing basic, essential security for cardholder data.

Where Do I Find PCI Information?

This is a common question that comes up.  Where do you find all of the information on the PCI standards?

The first place to go looking is at the PCI Security Standards Council (SSC) Web site.  The PCI SSC Web site has a number of resources that you need to check out.  These include:

• The PCI Data Security Standard (DSS), the Self-Assessment Questionnaires (SAQ), Payment Application Data Security Standard (PA-DSS) and PIN Transaction Security (PTS)
• Frequently Asked Questions (FAQ)
• Information Supplements
• Locate a Qualified Security Assessor (QSA), Qualified Security Assessor Company (QSAC), Payment Application QSAC (PA-QSAC) or an Approved Scanning Vendor (ASV)
• PCI training

The FAQ is where the PCI SSC posts all of the questions and answers about the PCI standards.  Questions can be asked by anyone accessing the Web site.  The answers come from representatives of the card brands with the PCI SSC staff collecting and publishing the card brands’ agreed to responses.  Answers to questions typically take four to six weeks to obtain an answer.  However, it is not unusual for answers to take quite a while.  For example, in the case of securing pre-authorization data, it has been almost four years and we are still waiting for a response which the PCI SSC has promised they will deliver in the coming year.

Information Supplements are white papers written by various authors (usually PCI SSC staff or the card brands) and approved by the PCI SSC and the card brands that discuss a PCI standard requirement in detail to further explain a requirement and explain how a merchant or service provider can meet the requirement.  Informational Supplements that have been published thus far include:

• Skimming Prevention: Best Practices for Merchants
• Wireless Guidelines
• Requirement 6.6 (Application Code Reviews and Application Firewalls)
• Requirement 11.3 (Penetration Testing)

The PCI SSC has indicated that more Information Supplements are going to be issued in the future instead of updates to the standards.

Locating a QSAC, PA-QSAC and ASV is provided by a PDF list for each type.  Individual QSAs can be looked up by their name so that you can confirm they are in fact QSAs.  The PCI SSC recently sent out a clarification in one of their newsletters to QSAs discussing the fact that once a QSA leaves their QSAC; they need to join another QSAC who applies to have them transferred by the PCI SSC to retain their QSA certification.  If they do not join another QSAC, they are no longer allowed to use the QSA designation.  So, it is important to confirm that the people and firms you are talking with are in fact QSAs and QSACs.

If a QSAC is listed as in remediation does not mean that the QSAC and their QSAs cannot continue to perform PCI assessments, this just means that the QSAC was found deficient in meeting the documentation and reporting standards of the PCI SSC.  Even though the PCI SSC issued a well written release on the meaning of remediation, a number of unscrupulous QSAs are telling prospects that because a QSAC is in remediation, it cannot perform PCI assessments.  This is patently false and any QSA that makes such a statement should be reported to the PCI SSC for telling such a falsehood.

Of course the most obvious thing provided by the PCI SSC’s Web site is the standards themselves.  Unfortunately, without the benefit of the PCI SSC’s training program, interpreting the various PCI standards can be difficult, if not impossible.  However, there are a number of independent resources for people to use to get interpretations of the PCI standards.  One that I have actively participated in is the Society of Payment Security Professionals (SPSP) Forum.  There is also the PCI Knowledge Base that has a large contingent of QSAs and other experts that can provide guidance regarding the PCI standards.  There are also forums provided through Yahoo and LinkedIn as well as other similar services, but I am not as well versed in the accuracy of the information provided through those venues so I cannot recommend them.

So the next time you are looking for information regarding PCI, here are some resources you can use.

“Passing” Vulnerability Scans

What is a “passing” scan?  The PCI DSS requirement 11.2.b defines a passing scan as a scan that does not report any urgent, critical, or high vulnerabilities, i.e., any vulnerability with a CVSS base score of 4.0 or greater.  So what is the big deal and why is a passing scan so difficult to obtain?

The National Vulnerability Database (NVD) maintained by the National Institute of Standards and Technology (NIST) has 40,029 vulnerabilities cataloged from 1999 through the end of 2009.  Of those vulnerabilities, almost 94% (37,523) have a CVSS base score of 4.0 or greater.  As a result, statistics say that there are going to be times when the vulnerability scan comes back with an urgent, critical or high vulnerability.  While requirement 11.2.b allows a minimum of quarterly scanning, requirement 6.1.b requires that all urgent, critical or high patches must be applied within a month.  As a result, once a vulnerability is identified by your scan, you essentially have 30 days to apply a patch and you must rescan to ensure that the patch has been applied.

Under a quarterly scanning program, when a non-passing scan occurs, you must now schedule a scan at least 30 days later to prove that the urgent, critical or high vulnerability was patched.  Given statistics say that 94% of all vulnerabilities have a CVSS base score of 4.0 or greater, it is highly likely that you will have to scan at least eight times during the year.  The four quarterly scans plus four more remediation scans.  However, given those previous statistics, it is also highly likely that those four remediation scans will reveal new vulnerabilities meaning that you will have to scan at least four more times.  That means at least 12 scans, possibly more.  This is why a lot of organizations just do monthly scans.

But, this is not the entire patching story.  Most of the time, vendors have a patch within days or a week or two of identification.  However, there are instances where vendors have taken months or even years to deliver a patch.  As a result, in certain instances, patches may simply not be available from a given vendor.  In some rare and bizarre scenarios, we have seen certain patches from vendors remove patches thus reintroducing old vulnerabilities.  When systems were reviewed, the system patch records still indicated that all patches had been applied however the vulnerability had reappeared and had to be patched again.

Addressing vulnerabilities can get even more delayed when we talk about packaged software.  For organizations running packaged solutions, they typically do not have the option to patch their software within the 30 day window required in 6.1.b.  This is because packaged software vendors need to test operating system patches and other system patches with their software prior to telling their customers that the patch is compatible with the packaged solution.  In some cases, the software vendor issues their own service packs on a quarterly, semi-annual or other periodic basis that contain compatible system patches as well as any updates to their own software.

This is where the experience of the QSA comes into play.  An experienced QSA understands the realities and that scans showing new vulnerabilities are a fact of life.  As a result, I recommend the following guidelines to determine if an organization is meeting their PCI compliance obligations regarding patching and scanning.

• Determine that vulnerability scanning, penetration testing and patch management processes are documented.  Obtain and review all policies, standards and procedures related to vulnerability scanning, penetration testing and patch management.
• Determine that there is proof that supports that the patch management process works as documented.  Proof that a patch management process is working includes any reports from tools such as Microsoft WSUS, Big Fix, Lumension, GFI LANguard, Shavlik HfnetchkPro and the like as well as reviews of system patching records from the systems themselves and the vulnerability scanning and penetration testing reports.
• Determine that the vulnerability scanning and penetration testing processes are functioning by reviewing all available reports from those processes for the PCI compliance reporting period.  Confirm that any new vulnerabilities identified are either addressed in the 30 day window or are documented as to why they were not addressed.  Determine that rescanning and retesting are performed after any patching has been completed.  Remember, only your external quarterly scans need to be done by an ASV.  Any other scanning done can be done by qualified internal resources, so you do not have to incur additional costs of an ASV for scans outside of the quarterly scans.
• Review change control records and determine if any significant changes have been made to either PCI in-scope applications or the in-scope networks.  If significant changes have occurred, match the completion dates of those changes to vulnerability scans and penetration tests to ensure that scanning and testing was performed after those significant changes were implemented.
• If a vulnerability is not patched, obtain documentation explaining why the vulnerability is not patched, the risk presented by the vulnerability, what has been implemented to mitigate any additional risks presented by the vulnerability and, if possible, when is the vulnerability expected to be addressed.  Determine that management has approved that the vulnerability has not been patched and they accept any additional risk presented.

If an organization can provide all of this documentation and proof, in my opinion they are meeting their PCI compliance obligations regardless of what their vulnerability scans and penetration tests document.