Compliance, Compliance Testing and Security

I was recently on a Webinar presented by a major security vendor and one of their points was that executive management is finally starting to realize that compliance does not equal security.  If you read this blog regularly, you know I really do not like the phrase “compliance does not equal security” and I view it as a convenient dodge by those who use it as a way to weasel out of their responsibilities.

But during this Webinar I had an epiphany regarding this topic.  It is the confusion between security, compliance testing and reporting and the act of compliance by your technology, employees and business partners with your organization’s security policies, standards and procedures that is the problem.

I know I am just asking for flame mail with this post, but I am so tired of people looking to blame everyone but themselves about their inadequacies surrounding information security.  As I have done before, to paraphrase Tom Hank’s character in ‘A League of Their Own’, “There’s a reason security is hard.  If it wasn’t hard, everyone would do it.”

Security is not always easy, particularly when upper management does not have buy in.  But even when upper management supports security efforts, I have seen security personnel not take advantage of that fact and get the job done.  Security does not have to be hard, but it does take more than just slamming some firewalls and intrusion prevention gear down, tossing a SIEM into the mix and thinking you are done.  Security is a never ending journey because someone is always coming up with new ways to attack you.

Anyway, to start off, let us take a look at some definitions first so we are all on the same page.

Compliance is defined as:

“Conformity in fulfilling official requirements.”

“Official requirements?”  Could that possible mean your organization’s security policies, standards and procedures?  You bet.  In this instance, we are talking about those that correspond to the PCI DSS, but this also applies to ISO 27K, FISMA, HIPAA, GLBA or any multitude of frameworks and regulatory requirements.

Conformity is defined as:

“Compliance with standards, rules, or laws.”

Based on these definitions, security is all predicated on complying with what are deemed an adequate set of security policies, standards and procedures.  Conversely, if you are not complying with an adequate set of security policies, standards and procedures, then your organization cannot be as secure as it could be.  As a result, compliance has to equal security as long as the security policies, standards and procedures are considered adequate.  Therefore security professionals that quote the mantra, “compliance does not equal security” either have a problem with the compliance side of the equation (most likely) or with the standards/frameworks (the dodge).

Over the years there have been a lot of discussions about the PCI DSS, ISO 27K, FISMA and other security frameworks and whether or not they are adequate.  The important thing to remember is that all of these standards or frameworks are merely ante into the information security game.  They are the bare minimum or a baseline to get to a basic level of security.  Should you being doing more?  Definitely, but what those efforts beyond the standard/framework are depends on what you are trying to secure, your network and application architectures and a multitude of other factors related to your computing environment and how it is used.  Those are factors that cannot be taken into account by any standard/framework because they would start to become impossible for others to follow and implement.  The bottom line here is that if you want someone to tell you exactly what to do to secure your networks and applications, go hire a consultant you trust and they will tell you everything you want to know.

The rub in all of this is that, based on the breach reports from Verizon Business Services, Trustwave, et.al. as well as compliance testing reports I have reviewed, none of you out there are 100% compliant to begin with, let alone even close.  Every organization I am aware has problems complying with the basics, let alone with any advanced security requirements in the published standards/frameworks.  So if you cannot comply with what you already have, explain to me how a different framework is going to change that fact unless it is less stringent than the framework you are already trying to use?  And if that other framework is less stringent, while that may solve the compliance issue (which I seriously doubt), exactly how is a less stringent framework going to make you secure?  The answer is that it will not make you secure.

What security professionals struggle with is that compliance is a never ending, 24x7x365 effort.  Drop your guard for an instant and it can be game over.  But provided your security policies, standards and procedures are appropriate and detailed (the reason why you want to use an appropriate standard/framework), your organization is not as secure as it can be unless your personnel and devices comply 100% of the time with every defined security policy, standard and procedure.  If you want confirmation of these facts, again, just look at the breach analysis reports year after year.  The reason there are breaches is because of non-compliance with one, but usually more, of an organization’s security policies, standards and/or procedures.

This brings me to the rumblings of late regarding a rethinking of defense in depth.  Defense in depth is predicated on using layers of security devices and controls to minimize the risk that a security incident occurs not to completely prevent an incident although you might get lucky.  For example, firewalls are the sledge hammer of security tools.  However, because we need to have ports open for outsiders to access applications, we follow our firewalls with intrusion detection/prevention devices to ensure that no one abuses the protocols used by the ports.  We follow that up with monitoring of log data from the firewalls, IDS/IPS, routers, switches and servers to identify any “sneaky” attacks using the protocols we allow.  The layers are there to cover the various holes we need to have in order to make our networks and applications function.  The tighter and smaller we can make those holes, the more secure we will be, but there will still be some amount of risk.  So we bring in more layers to cover those risks until it is more expensive to address the risk than to accept the risk.  That remaining risk is the residual risk that we therefore manage and control through detection and correction.

The other thing defense in depth relies on is the control triad.  The idea being that, because you cannot entirely prevent every security incident, you need a way to detect the incident so that you can take action to stop or minimize the impact of the incident.  You follow that up with periodic assessments of your control environment to identify and correct any deficiencies or improve your program based on new information regarding security.  The follow up assessments can be activities such as a root cause analysis (RCA) of an incident, an internal audit of user accounts and user rights or brining in a network security team to assess your security architecture and controls.  All of these activities will result in findings and recommendations to make your security systems and controls better.

And that brings us full circle to the PCI assessment.  It is merely a tool used by the acquiring banks, card brands, processors and others to obtain reasonable assurance that your organization is doing what it can to minimize the possibility of a breach of cardholder data.  It is not meant to be, nor could it ever be, an absolute complete assessment of an organization’s security posture and therefore provide absolute assurance that a breach will not occur (even though the PCI SSC and card brands tend to imply that fact).  Compliance assessments are only a snapshot of personnel and device compliance at the time the reports were written.  This is no different than going to the doctor for your annual physical which results in a snapshot of your health at that point in time.  It is not that those compliance reports are worthless; they just need to be referenced and used properly based on the fact that they are a snapshot.  Just as your doctor will tell you to lose weight or stop smoking, compliance reports provide recommendations on where you can make improvements or adjustments in your policies, standards and procedures based on what compliance evidence was found, or not found, during the assessment.

So, what are the lessons to be learned?

• Security is not and never will be perfect; there will always be residual risk that must be managed and controlled.
• Compliance does equal security, at least as best as your preferred standard or framework defines it plus whatever enhancements you have made.
• Compliance assessments and reports point out where your organization was not compliant and needs to do better, not to prove your organization is secure.

Use the tools at your disposal correctly, stay current on threats and monitor your security posture and you will likely live a long, prosperous and secure life.

Keep hiding behind “compliance does not equal security” and you will forever be living off of your “luck” until it runs out (usually sooner rather than later).

How The PCI Standards Will Really Die

Welcome to the new year.  I hope the holidays have been treating you well and the coming year is good as well.

There have been a number of articles written about why and how the PCI compliance process will die.  It is not that I look forward to the PCI standards dying as they have brought a needed visibility to information security and privacy as well as the fact that PCI keeps me gainfully employed.  However if things stay on their current trajectory, the PCI standards will eventually die, but not for the reasons being quoted in today’s articles.  The real killers of the PCI compliance process will be the card brands and the PCI Security Standards Council.  Yes, the very folks that brought us the PCI standards will bring the ultimate demise of their precious set of standards.

The first death knell I see is that it is very easy to issue edicts from on high when you do not have to implement them.  Over the years, clarifications have been issued, quality assurance reviews performed, forensic examinations conducted and a host of other activities have resulted in “enhancements” to how the PCI standards are assessed and enforced.  Do not get me wrong, a lot of what has been done was needed and appreciated.

However, by the same token, some of what has come down has been a nightmare to implement.  Any QSAC not using some sort of automated system to conduct their PCI assessments will find it impossible to meet the current and any future documentation and tracking standards now required by the PCI SSC’s QA process.  Under the current standards, QSACs need to document who they interviewed and what the persons were interviewed about as well as tying documentation and observations to the tests performed.  Without some sort of automated process, these requirements are just too intensive to perform manually.

Documentation received and reviewed needs to have its file name, date of issue and a description of its purpose in the PCI assessment process documented.  The basic PCI DSS has a minimum of around 200 discrete documents that are required for the PCI assessment process.  The average we see for most of our engagements is over 600 documents which also include not only policies, standards and procedures, but configuration files, interview notes and observations such as screen shots, log files and file dumps.  You really have to question any QSAC that tells you they manually manage the process.  They either have an amazing and magically efficient project management process, they have very, very inexpensive staff (i.e., overseas labor) or they are short cutting the processes and producing a work product that does not comply with the PCI SSC QA program and have yet to be assessed by the PCI SSC (the most likely scenario).

Even using simple SharePoint or Lotus Notes solutions are not cheap when you consider the cost of the server(s) and the storage of all of documentation collected, which can be around 5 to 10GB per project, as well as all of the requisite system maintenance.  Servers and storage may be cheap, but it all adds up, the more clients you assess.  And speaking of the storage of documentation, the PCI SSC requires that documentation related to PCI assessments be stored for at least three years.  For those of us with electronic work paper management systems, this is not a problem.  However, given the amount of paper generated by these projects, those QSACs using the traditional paper filing methods will find a lot of shelf space taken up by their PCI engagements if they are truly following the procedures required by the PCI SSC.

All of this drives up the cost of a proper PCI assessment, more than I think the card brands and the PCI SSC are willing to admit.  It is not that I think the card brands and PCI SSC do not care about this situation, but more related to they do not have an understanding of the operational ramifications of their edicts.  The card brands and PCI SSC tread a very fine line here and to this point they have been heavy handed in the issuing of their edicts.  Going forward, the PCI SSC needs to ask the QSACs, Participating Organizations and ASVs to assess the cost and time impacts of these edicts so that they can be weighed against their benefits versus what is done now which is more of a procedural and proofing review.  If this is not done, there will soon come a point where merchants and service providers will push back hard and refuse to go through the process due to the cost and the amount of time involved to be assessed.

The next death knell is the inane process that is called the PCI Report On Compliance (ROC).  When the PCI SSC did not have access to the QSACs’ work papers, the current ROC writing process made some sense as there was no other way for the PCI SSC or the processors and acquiring banks to know if the QSACs had really done the work they were saying they had done.  However, all of that changed a number of years ago when the PCI SSC required QSACs to add a disclaimer to their contracts stating that the PCI SSC had the right to review all work products.  Yet even with this change, we continue to have to write an insanely detailed ROC, typically numbering in a minimum of 300+ pages for even the most basic of ROCs.

Unfortunately, there are QSACs out there that apparently have not been through the PCI SSC QA process and that dreaded of all states – Remediation.  As a result, they have much lower costs because they are not documenting their assessment work as completely as they need to and are not sampling, observing or interviewing like QSACs that have been through the QA process.  In addition, based on some work products we have seen, they also do not care about the quality of the resulting ROC as it looks entirely like a ‘find and replace’ of a template and makes no sense when you read it.  In talking to other large QSACs that have been through the QA process multiple times, the PCI SSC has indicated that they are monitoring the large QSACs more than the little QSACs because there is more risk with the large QSACs.  While true to an extent, we have encountered a number of smaller QSACs that perform assessments for large clients due to their much lower cost structure and their willingness to ‘overlook’ compliance issues.  If the PCI SSC does not go after these QSACs soon, there will likely be a number of breaches that occur due to the QSACs’ lack of diligence in performing their assessments.

I know of a number of QSACs that would like to see Bob Russo and the representatives of the various card brands to actually work as staff on a few PCI assessment engagements so that they can better appreciate the inordinate amount of work involved in generating a ROC.  I think they would be shocked at the amount of work effort they have driven into a process that is already too complicated and prone for error.

As it stands today, the ROC writing, review and proofing process is probably 50% to 60% of a good QSAC’s project costs.  To address this, the PCI SSC QA group tells QSACs to develop one or more templates for writing the ROC which, from what we have seen from some other QSACs, means a lot of mass ‘find and replace’ to speed the ROC writing process.  For the last few years, a number of QSACs have brought the ROC writing process up at the Community Meetings.  However the card brands continue to shoot down any sort of changes to the process.  As a result, the cost of producing a ROC is driven by the size and complexity of the merchants’ or service providers’ cardholder data environment (CDE).  These costs will only continue to rise as long as the PCI SSC does not allow QSACs to mark items as ‘In Place’ with only a check box and rely on the QSAC’s work papers versus the verbosity required now.  If this sort of process can work for financial auditors, it can work here as well.

A third death knell is the PCI SSC and card brands continuing to quote that the majority of breaches are the result of organizations not complying with the PCI DSS.  In discussions with a number of the PCI forensic examination companies, I am hearing that the card brands cannot believe the fact that more and more organizations were PCI compliant at the time of their breach.  The PCI SSC and card brands have apparently convinced themselves that the PCI standards are “perfect” and they cannot imagine that an organization could be breached unless that organization was not complying with the PCI standards.  There is no security standard that I am aware that totally prevent breaches.  So while the PCI standards are good baseline security standards, the card brands and PCI SSC seem to have forgotten that security is not perfect and that any security standard only minimizes the damage done when a breach occurs if the standard is truly followed.

And as organizations have gotten the PCI “religion,” the effort required to compromise them from the outside via traditional attacks has increased significantly.  As a result, successful attackers have changed strategy and work on social engineering their way past the bulk of an organization’s security measures.  The PCI DSS only has a little bit on social engineering in requirement 12.6 regarding security awareness training.  And even those organizations with the most robust of security awareness programs will tell you that, even after extensive security awareness training, human beings are still fallible and that some people still do very questionable things that continue to put organizations at risk, sometimes significant risk.  Even when you have the most diligent of employees, they still make mistakes in judgment from time to time.

Until the human element can be totally removed, there will always be a certain amount of risk that will never go away.  Again, the PCI SSC and card brands seem to not want to acknowledge the failings of the human element and appear to believe that technology is the savior based on the focus of the PCI standards.  However time and again, every security professional has seen very sophisticated security technologies circumvented by human error or just plain apathy towards security (i.e., “it always happens to someone else, not my organization” or “we’re too small to be a target”).

Until the PCI SSC and the card brands drop the “holier than thou” attitude toward the PCI standards and stop the public pillory of organizations that have been breached, there will continue to be editorial commentary regarding the pointlessness of the standards and ever more serious push back to complying with the standards.

These are the reasons why the PCI SSC and the card brands will be the ones that will kill the PCI standards.  At the moment, they are so far removed from the process; they do not understand how complicated and expensive the process has become which is why merchants and service providers are complaining about the ever increasing costs and effort related to the PCI assessment process.

The PCI SSC and card brands also seem to have forgotten that QSACs have to make money doing these assessments and, when you pile on clarifications and edicts that do nothing to streamline and simplify the process; you are only driving the costs of the process higher.  And higher costs only make merchants and service providers, who are on thin margins to being with, even more incentivized to use the much lower cost QSACs, driving the diligent QSACs out of the market, thus increasing the likelihood of breaches.

Again, it is not that I want the PCI standards to go away as I think they have brought a real benefit.  However, if these issues are not addressed, the PCI standards will end up going away.  I fear that, with them gone, there will be no carrot to ensure the security of cardholder information and we will end up back where we were before the PCI standards existed.

A Reason Why The PCI Standards Get No Respect

Call it the “Rodney Dangerfield Effect.”  Conflicts of interest seem to pervade the PCI compliance process and it is something the PCI SSC and the card brands need to clear up before their precious standards get even more bloodied in the media.

I have run across another processor that dictates the use of a particular QSAC.  Now do not get me wrong, I am a capitalist and interested in making money just like the other guy.  But I have to say that I am not a shark like some of my competitors.  I know this post will sound like someone bemoaning sour grapes but, in my opinion, this situation just makes the whole PCI compliance process look like a worthless sham.

What prompts this post is a call with one of our clients that we have performed PCI assessments for years, even before the PCI SSC existed.  They are implementing a new point of sale (POS) terminal that requires them to use a different credit card transaction processor because their existing processor is not yet certified to process transactions from this new terminal.  Fair enough.

The new terminal is a test installation to see if the service should be expanded to all of our client’s locations.  Since the terminal will only generate a couple of thousand transactions in the coming year, the new processor has identified our client as a Level 4 merchant and is treating them accordingly.  In reviewing the processor’s contract, our client found that the contract dictates that they use a specific QSAC to “assist” them in filling out their PCI Self-Assessment Questionnaire (SAQ) A.  Knowing the SAQ A, our client cannot figure out what a QSAC would do to assist, but it is in the processor’s contract.

Our client’s first question was, “Since when does a processor have the right to force us to use a particular QSA?”  We explained that we have been told that while the PCI SSC and the card brands allow processors to have rules that go above and beyond the PCI SSC’s and card brand’s requirements.  While I understand that the processor is likely trying to ensure that their Level 4 merchants are not just checking the ‘Yes’ box on their SAQs, forcing the use of a particular QSAC seems a bit questionable.  Particularly when we have been told that some QSACs are giving processors payments for all of the customers they refer.

I have written about this issue before with processors charging fees to merchants for the filing of their SAQs.  There is also the scam of forcing merchants to use a specific PCI Approved Scanning Vendor (ASV) to scan the merchant’s networks even when the merchant does not have an ecommerce presence or outsources their ecommerce to a third party that already provides their ecommerce customers ASV reports.  This is just one more questionable requirement that processors demand that makes merchants and the media think PCI is a scam.

Their next question was, “Since you already do our ROC, can’t we just submit that to our new processor?”  You can do that, but you need the new processor’s approval as they do not have to accept our work.  What is the likelihood that the new processor will accept our client’s ROC?  No idea and I am anxious to hear what our client tells us in that regard.

The problem here is that the processor in question and the QSAC have numerous connections that give a distinct impression of conflicts of interest.  First, the QSAC in question runs the processor’s Level 4 merchant compliance program.  That program dictates that the QSAC perform some sort of assessment process in order for any of the processor’s Level 4 merchants to create and submit their SAQ.

The justification the processor gave our client was that the PCI SSC requires this action.  Last I checked, the PCI SSC and the card brands did not require a QSA to fill out an SAQ.  MasterCard has a deadline of June 30, 2012 for Level 2 merchants to have either an ISA fill out their SAQ or have a QSA conduct a PCI ROC.  Until October 2010, Visa Canada required that a QSA sign off on all SAQs.  But those are the only SAQ rules involving QSAs that I am aware.

Next, the QSAC and the processor have swapped various personnel over the years.  As a result, there is an appearance that the two are essentially one in the same given that the QSAC runs the processor’s compliance program and the processor dictates that their merchants use the QSAC for PCI assessments.  I know that people move between organizations in the same industry all the time, but the number of people that have gone between these two would seem to be higher than expected.

I guess since I am an employee of a public accounting firm in the United States, I have greater sensitivity to conflicts of interest than most.  The American Institute of Certified Public Accountants (AICPA) has very specific rules in regards to conflicts of interest.  We have an entire department dedicated to ensuring that we avoid conflicts of interest.  As a result, we regularly look at the services provided to our clients and ensure that we are not in conflict or even give an appearance of a conflict of interest.

Now I am not suggesting that the PCI SSC and card brands go to the levels that the AICPA requires.  But let us face it, it is the Wild West out there and some of the QSACs do not care what conflicts they may have and how it might hurt the PCI compliance processes.  The PCI SSC only requires its assessors document the services they provide to the organizations they assess in their assessment reports.  While that offers a certain amount of transparency, when you read some of these ROCs, it becomes painfully obvious that some QSACs are assessing their own security services.  In some cases, the organization being assessed has outsourced almost everything related to their PCI compliance to the QSAC doing their assessment.  What do you think the likelihood is that those services will be assessed as not compliant if there are compliance issues?  One would assume it will be very unlikely.

But it can get even worse.  A certain QSAC operates one of the card brand’s merchant PCI compliance program.  Merchants submit their Attestations of Compliance (AOC) and Reports On Compliance (ROC) to this card brand through the QSAC which manages the process.  Does this QSAC inform their clients that accept this card brand’s cards of this fact?  Not that I have ever been told by any prospects.  Does the QSAC list the management of the card brand’s merchant program on their ROCs?  Not that I have seen and I have seen a number of their ROCs for merchants that accept the card brand’s cards and I have never seen the program listed.  Does the QSAC submit their ROCs to the program that they manage?  They must as the ROCs I have seen are from merchants that accept the card brand’s cards.  Is this a conflict of interest?  One would think, but this is how things operate today.

The bottom line is that in this age of openness and transparency, it is these sorts of relationships and actions that give a very bad impression to the outside world.  The PCI SSC and the card brands need to enhance their rules for QSACs that define conflicts of interest.  Until this is done, the PCI standards will continue to be ridiculed and viewed as pointless.

PCI SSC Updates The ASV Program and Issues New Information Supplement

March 2011 has been a busy month thus far for the PCI SSC.  On Thursday, March 10, they announced a new ASV training program and on Friday, March 18, they released an Information Supplement on protecting telephone-based cardholder data.

The ASV training program has blindsided the ASV community as it was a total surprise.  Yes, there has been talk over the years at the Community Meetings and in other venues regarding ASV qualifications and training, but nothing ever seemed to come from those discussions.

According to the press release, the ASV training program, “… is a direct response to the feedback we’ve received from the merchant community …”  One can only assume that the complaints that have been voiced about ASVs from merchants and service providers as well as the comments in the media have finally caught the attention of the PCI SSC and they are going to address the problem with training.

The consistent complaint I have heard from clients over the years was in regards to the fact that no two ASVs ever scoped their network the same for vulnerability scanning.  The next most common complaint was always about the different results of the vulnerability scanning which was most often voiced after a company changed ASVs.  While there are always some bad apples in the bunch, I do believe that most ASVs know what they are doing.

As a result of all of this, I am sure this new training requirement is to address the complaints and make the program better.  This training is in addition to the network security test ASV companies already had to pass.  For those of you that did not know, ASVs have to conduct network security scanning against a test network with predefined vulnerabilities operated and configured by the PCI SSC.  ASVs are expected to produce a sample ASV report and document all of the predefined vulnerabilities.

The ASV training is comprised of an eight hour online course and an examination.  A minimum of two employees are required to take the course and pass the examination.  Once this has been accomplished, the organization is designated as having Qualified ASV employees.  As with a lot of PCI requirements, there are some important dates involved.  ASVs that are renewing their status prior to June 1, 2011 need to get two personnel trained and passed the examination by June 15, 2011.  ASVs that will certify after June 1, 2011 need to have two staff trained and passed the examination by their renewal date.

Given the additional cost of this new training plus the requirement to have a minimum of two people trained, it will be interesting to see if any of the existing 130 ASVs drop out because of this new requirement.

Telephony Cardholder Data

The other issuance of interest this month was in regards to telephone conversation recordings that contain cardholder data.  While this is a longer dissertation on FAQ 5362, there is really nothing new presented in this information supplement.  The bottom line still is that if you have call recordings that contain cardholder data you are required to either; (a) do not record it in the first place, (b) if recorded, redact it if possible, or (c) make sure that it cannot be searched, is encrypted and access is restricted.  The best thing about this information supplement is the flowchart that was created for this situation.

If Not The PCI Standards, Then What?

I have just read a couple of articles as well as attended a couple of meetings where the topic du jour was the PCI standards.  They were a bash fest of the highest order.  Frustrated, I asked the participants at my last meeting, “If not the PCI standards, then what standard do you want to follow to ensure the security of cardholder data?”  Roaring silence.

This is the frustration that I and others have with people who complain about the PCI standards or any standards.  People complain and complain, yet they offer no solutions to address their complaints.  One thing I have always stressed with and required of people who work with me, if you are going to complain about something, you better have an idea for a solution.  Constructive criticism is fine, but if you do not have any ideas on how to make things better, then all you are doing is whining.  Children whine, adults have solutions.

But then you have the complainers who do offer a solution but that solution is to allow the marketplace to address the problem.  Hello!  How long was it going to take before merchants and service providers got a clue about securing cardholder data?  If it was such a priority, why did it take the card brands to come out with the standards?  For merchants and service providers, cardholder data security was not a priority, it was some other merchants’ or service providers’ problem.

The other problem with the marketplace approach is that each organization learns from its own incidents and possibly from incidents suffered by their business partners, not from the incidents experienced by all.  Under the marketplace approach, security protection only improves as each individual institution suffers a particular incident.  As a result, organizations reinvent the wheel with the majority of incidents.

Standards allow organizations to learn from the collective experience of all organizations, not just their own organization.  For example, if your organization does not have wireless networking but decides to implement wireless, a standard provides a guideline as to how to implement wireless securely.  Without a standard, you are on your own to do the best you can.  On your own you will likely get some things right, but you will also get some things wrong.  It is those mistakes due to lack of experience that come back to bite organizations.  With a standard to follow, the chance of getting bitten after the fact is often greatly minimized.

However, standards are not a guarantee.  Going back to wireless, just look at how things went wrong with WEP.  WEP is a standard and was well documented on how to implement it; supposedly securely.  WEP was also known to have the potential for security problems, but those problems were not widely publicized until organization began to have security incidents.  So a stop gap standard was provided called WPA which turned out to have its own security issues.  Ultimately, WPA was replaced by WPA2 which is the secure, permanent solution.

This is why early adopters of technology can end up getting burned.  When an organization decides to hop onboard the latest and greatest technology, there is a high risk that the security learning curve is not very far advanced.  As a result, the organization will be at a higher risk of suffering a security incident than an organization using a more tried and true approach.  As a new technology matures, typically its security posture matures and with a more mature security posture, the lower the likelihood that a security incident will occur.  However, the time it takes for that security maturity to occur can take quite a while and it is where things take quite a while where organizations are at the highest risk.

Unfortunately, in some instances, a new technology gets quickly usurped by an even newer technology and the original new technology never matures.  The bad news is that the early adopters get stuck with a solution that will never have its security shortcomings addressed, leaving the early adopters to either convert to the newer technology or find another alternative.  Many a career has been ended over such technology leap frogging events.

The PCI standards were not developed in a vacuum.  They are a consolidation of a lot of other security standards and guidance gained through root cause analysis of security incidents gathered over the years with the express purpose of protecting cardholder data.  If you follow another security standard such as ISO 27K or FISMA, a lot of what is in those standards is also in the PCI standards.  But there are also a lot of requirements in the PCI standards that are not in other standards as well.

The bottom line is if you do not like the PCI standards, then get involved in the process to make things better and stop whining.

The “Magic” Vulnerability – Revised

What started this post is that I have recently received a number of calls and messages from clients and colleagues.  The conversations have all gone basically the same.  They were calling me and telling me that their ASV had failed their vulnerability scan because the OS detected was unsupported and they are wondering whether or not I have encountered this before.

My first question usually was along the lines of; “So, what vulnerabilities did they detect?”

“None,” was the confused answer at the other end of the line.

“What?  They must have detected at least one high, severe or critical vulnerability?  That is the only way you can fail,” I would ask, now also confused.

“Nope.  Nothing.  Just the fact that the OS is unsupported,” I was told.

Do not get me wrong.  I am not advocating the use of unsupported operating systems, particularly unsupported versions of Windows.  The risk of course is that one or more vulnerabilities show up that the vendor will not fix because the OS is no longer supported.  So there is good reason to avoid this situation.  However, there are also situations when you just get no other choice either due to your own organization’s issues and politics or software vendor issues.

This situation got me thinking and doing some research since I did not remember ever seeing or being told that an unsupported OS was an automatic vulnerability scan failure.  I no longer do external vulnerability scanning, so my recollections of training and working on the ASV side of our business is a bit fuzzy and very rusty.  However, I had never failed a client for an unsupported OS.  So when this issue came up, my only action was to determine what had changed.

The first thing I did was review the latest version of the PCI ASV Scanning Procedures, v1.1.  I searched for terms such as ‘old’, ‘unsupported’, ‘out of date’, ‘OS’ and ‘operating system’.  No matches.  So there is nothing in the ASV scanning procedures that fail an organization for running an unsupported OS.  Even the PCI DSS does not call out unsupported software, so procedurally; I am thinking there is nothing explicit regarding unsupported OSes causing a failed vulnerability scan.

So when I made the original posting, I got a comment from one of my readers pointing me to the ASV Program Guide.  Low and behold on the top of page 16 is the following:

“The ASV scan solution must be able to verify that the operating system is patched for these known exploits. The ASV scanning solution must also be able to determine the version of the operating system and whether it is an older version no longer supported by the vendor, in which case it must be marked as an automatic failure by the ASV. “

So there is no “magic” vulnerability I was missing as the PCI SSC does specify that a scan automatically fails if the OS is unsupported.

But that is not the entire story.  The key to this whole process is that the vulnerability scanner used must be able to verify the operating system.  While all vulnerability scanners attempt to identify the operating system, the reliability of this identification process is suspect at best.  I am not aware of any vendor of security testing tools that makes a claim that they will identify an operating system 100% of the time.  This is because of the fact that there are many, many things that can influence the OS signature that the tools cannot control and therefore can greatly affect the ability of the tool to identify the OS, particularly when talking about external scanning.  And if an organization follows the OS security hardening guidelines, a lot of unsupported OSes will not be properly or reliably identified by vulnerability scanners.  As a result, I find it hard to believe that the PCI SSC intended to have ASVs only rely on the results of a vulnerability scanner, but that seems to be the case.

So with this clarification, I contacted our ASV personnel and they have told me that they too have been failing vulnerability scans if they run across unsupported operating systems.  I ask if the OS signature is inconclusive, then there is not a failure?  Yes, if the scan comes back and does not identify the OS, then they have nothing to go on to fail the scan and the scan passes.  Given the difficulties vulnerability scanners can have identifying the target operating systems such as when scanning through network firewalls, Web application firewalls, load balancers and the like, I now ask if they feel that these identifications are reliable enough to fail a scan.  I am told this is why they confirm the information with the client before issuing the report so that the report is accurate.  So if a client is not honest, they could influence the results of their scan?  I am reluctantly told that is probably true.

Then there is the issue that not all operating systems are created equal.  Operating systems such as MVS, VMS and MCP are nowhere as risky, if they are even risky to begin with, as Windows and Linux.  A lot of ASVs would argue that they never come across these operating systems running Web services.  However, all of them have the capability of running Web services and I personally know of a number of organizations that run their Web services from such environments.  Organizations are running these older versions of operating systems mostly because of the financial considerations of migrating to something else.  However, I can guarantee that none of the dozens of vulnerability scanners that I have used in the last 10 years would accurately identify any of these operating systems, let alone tell you the version unless some service message header information was retrieved by these tools.  And even then, most tools do not parse the header to determine the OS so it would take human intervention to make that determination.

Regardless of the failure, most ASVs do have a review or appeal process that allows organizations to dispute findings and to submit compensating controls to address any failures.  So for organizations that cannot get rid of unsupported OSes, they can use a compensating control.  Like compensating controls for the PCI DSS, the organization is responsible for writing the compensating control and the ASV needs to assess the compensating control to ensure that it will address the issues identified by the vulnerability scan.

So, if you can fail an organization over an unsupported OS, why is it that you do not automatically fail on unsupported application software?  I went through the Program Guide and there are all sorts of other criteria for applications but nothing regarding the fact of what to do if they too are unsupported.  Applications such as IBM Websphere and Oracle Commerce can become unsupported just as easily as their OS brethren.  And in my experience, use of unsupported application software is even more prevalent than unsupported OSes under the idea that if it is not broken and does not have vulnerabilities, why upgrade?  When I asked our ASV group if they fail organizations on unsupported applications I got silence and then the response that they will fail an application if the vulnerability scanner provides a high, severe or critical vulnerability.  To tell you the truth, while vulnerability scanners regularly return text header information for a lot of applications, I would be hard pressed without doing a lot of research to find out if the version being reported was unsupported.  However, scanners could provide this feedback if they were programmed to provide it.

Then there are all of the conspiracy theories out there that say the PCI SSC and technology companies are working together to drive software sales by forcing organizations to upgrade and there would appear to be a lot of anecdotal evidence that would seem to support this argument.  In reality it is not that the software companies are working together with regulators such as the PCI SSC so much as software companies operate this way in order to focus development and support resources on fewer, more current versions.  As a result, it is just happenstance that regulations cause organizations to have to update their software.

The bottom line in all of this is that you have options to avoid a failing vulnerability scan because of an unsupported OS.  The best method, and the one I most recommend, is do not use unsupported operating systems in the first place.  However, as a former CIO, I do understand the real world and the issues IT departments face.  As a result, I recommend all of the following which may or may not require you to develop a compensating control.

• Implement not only a network firewall, but also a Web application firewall (WAF) and make sure that the rules are extremely restrictive for servers running unsupported operating systems.
• Configure your firewalls to block the broadcasting of any OS signature information.  Masking the OS signature will provide the benefit of not advertising to the world that the OS running whatever application is unsupported.  This is not a perfect solution as, 9 times out of 10, the application itself will likely advertise the fact that the underlying OS is unsupported.  It is very important to note that this is only a stop gap measure and you should still be actively in the process of migrating to a supported OS.
• Implement real-time monitoring of firewalls, servers and applications.  Define very specific alerting criteria to ensure that any suspicious activity is immediately reported and operations personnel immediately follow up on any alerts to determine whether they are a false positive.
• Implement a host-based intrusion detection/prevention solution on any servers that run the unsupported OS.  If using a HIPS solution, you may also want to consider using its preventative capabilities for certain critical incidents.
• Implement real-time log analysis for firewall, servers and applications.  Define very specific alerting criteria to ensure that any suspicious activity is immediately reported and operations personnel immediately follow up on any alerts to determine whether they are a false positive.
• Actively use your incident response procedures to address any incidents that are identified with any unsupported OS.