Removing The Drama Of A PCI DSS Assessment

I had to prepare a presentation for a client a while back giving them some tips on how to prepare and get through a PCI assessment as easy as possible.  I thought it might be good to share those thoughts.

Trust But Verify

This famous quote from US President Ronald Reagan is the mantra of a PCI assessment.

The PCI DSS is based on the “trust” that organizations are complying with the PCI DSS.  However self-assessment processes and QSAs are used to “verify” that the organization is, in fact, complying with the PCI DSS.  As a result, the organization being assessed not only has to produce documentation to that effect, but the QSA must also observe that the PCI DSS requirements are being followed.

The net is that, just because you say something is fact, your QSA must substantiate your statements so that they, too, will treat them as fact.  If you remember nothing else but this simple truth, you will understand why a QSA must do what they do.

Scope

If PCI assessments go wrong for any reason, this is probably the primary reason.  It fascinates me that people often profess ignorance of the PCI DSS, yet somehow become experts on the subject when it comes to scoping.

Remember point number one, trust but verify.  Under that premise, the PCI SSC makes a QSA’s primary responsibility to confirm the scope of the PCI assessment as they verify the facts.  As a result, in order to confirm that scope, the QSA must look at everything and then, through investigation and evaluation, determine that the areas you deem out of scope are, in fact, truly out of scope.

Let your QSA ask their questions and conduct their observations without arguing with them about scope.  They are only doing this because they are required to confirm the facts and your fighting with them about scope is only going to making them wonder what you are trying to hide.  The bottom line is that arguing with your QSA about scope only makes your assessment all the more painful and time consuming.

If you truly want to avoid arguing over scoping, get a copy of the Open Source PCI Scoping Toolkit.  Go through your environment and determine the categories of all of your systems and networks.  This is a good annual exercise because you need to prove your scope every year.

Applicability

According to the PCI SSC, there are five PCI DSS requirements that can never, ever be marked as ‘Not Applicable’: 1.2.3, 3.2.1, 3.2.2, 3.2.3 and 11.1.  I have discussed these all before but they deserve another quick discussion here.

Clients will argue ad nauseam that wireless is not implemented or is out of scope and therefore refuse to discuss wireless.  For requirement 1.2.3, a QSA is required to document the procedures they followed to rule wireless in or out of scope.  That of course means the QSA must investigate any wireless networks and evaluate if the controls are rigorous enough to keep wireless out of scope.  For requirement 11.1, the QSA must investigate and evaluate if the organization’s controls surrounding the detection of rogue wireless are appropriate regardless of whether or not the organization has implemented wireless networking.

3.2.1, 3.2.2 and 3.2.3 are all related to the securing of cardholder data when it is stored.  Even if an organization is not storing cardholder data on their systems, a QSA must document the procedures they used to confirm that cardholder data is not stored on the organization’s systems.  This usually involves a review of flat files and database schemas and the running of utilities and queries against those systems and databases looking for cardholder data.

The bottom line is do not argue about something being ‘Not Applicable’ and then hinder the QSA’s investigation to prove it is ‘Not Applicable’.  Do not get me wrong, you need to keep your QSA on point, but remember that QSAs are required to evaluate the situation and then document the process used to determine that a particular requirement is ‘Not Applicable’.  All you do by complicating that investigation is add more time to your assessment and, potentially, cause a requirement to be marked as ‘Not In Place’ instead of ‘Not Applicable’.

Yes, I Did Kind Of Ask That Earlier

Like security, the PCI DSS also works from a ‘defense in depth’ approach.  A lot of the questions QSAs ask are very similar just asked from a different perspective.  The people that develop assessment and audit programs will tell you that this is the most effective way to uncover the level of compliance with a given program.  The reason is that organizations who have not integrated a compliance program into their day-to-day operations will typically provide inconsistent or confusing answers to the similar questions.  Not that this is a perfect technique mind you, but it does work the majority of the time.

Please be patient with your QSA.  They did not write these procedures, but they are required to execute them.

Answer The Question

Most people suck when being questioned, particularly in a legal proceeding, including yours truly.  Lawyers always instruct anyone that will be called to testify in a legal proceeding to take their time, focus on the question being asked and only answer the question being asked.  Never, ever, ever provide any information outside of the question, i.e., do not elaborate.  The trouble is that lawyers know that silence is a vacuum and it is human nature to fill that vacuum with extraneous information.  Hence why they typically have long pauses between questions.

QSAs and auditors tend to operate under the same principle as a lawyer.  People get into trouble when they start talking about things that are outside of the question, out of scope or not relevant to the assessment.  Such responses will at first confuse the QSA for a moment as they try to reconcile your remarks.  But then, the QSA may question whether they truly understand the environment and, possibly, the scope of the assessment.  It is then that they may start quizzing you and your staff as they go back and reconfirm their understanding of the environment.  All of this takes time, time away from the assessment process as you cover old ground while the QSA re-verifies the facts.

The lesson to be learned here is that there is nothing wrong with saying, “I do not know.”  Or “I will have to look into that question and get back to you.”  The worst thing you can do is try and “tap dance” around the question or never really answer the question.  If you do not have the answer, then find out who does have the answer and point the QSA to that person.

Prepare

And finally, the best thing you can do to avoid all of these issues is to walk through the PCI assessment process and requirements with those of your staff that will be interviewed/observed and make sure they understand the questions to be asked and how they should be answered.

If you really want to know what the QSA will ask, why they will ask and the evidence they will require, get a copy of the PCI DSS ROC Reporting Instructions from the PCI SSC Document Library.  The Reporting Instructions document is the “Bible” for QSAs as it documents how they will be assessed in a PCI SSC Quality Assurance review.  Reviewing and understanding this document will go a long way to minimizing the “What do you need that for?” questions that all QSAs encounter.

For each requirement’s tests, the Reporting Instructions will tell you:

• What observations, if any, need to be performed and documented.
• What documents, if any, need to be collected and reviewed and what information needs to be identified in those documents.
• What people, if any, need to be interviewed and about what topic(s).
• What processes, actions taken or states of equipment, if any, need to be observed and documented.
• Whether or not sampling can be used.

Using the Reporting Instructions, you can also gather a lot of the observations ahead of time.  Your QSA will still have to conduct some observations such as that default passwords are not used, that timeouts occur, that change management operates and the like.  But by gathering screen shots and documenting what you used as testing conditions will go a long way to making your assessment go much more smoothly and quickly.

Hopefully this discussion will help you get through your next PCI assessment without all of the associated drama that can come from such an exercise.

Advertisement

Advice To Merchants

Barring the card brands developing a truly secure card processing process, the PCI DSS and related standards are likely to be with us for quite a while.  That said, what is the future of complying with the PCI DSS?

For merchants, if you are not seeking out point-of-sale (POS) solutions that do not store cardholder information, you should be as soon as you possibly can.  That includes finding card processors that do not require you to store cardholder information and can provide you access to cardholder information when you need it for resolving disputes and chargebacks.  According to Robert McMullen, CEO of TrustWave, the majority of breaches TrustWave investigated occurred with POS systems.  So the rational approach to resolving this problem is to get rid of the cardholder data stored on these systems.

The problem with this is that most merchants, large or small, think that they need to store this information for some reason.  If you are a merchant in the United Kingdom, France, Italy and other select European countries, then you do need to have the PAN unencrypted, however it only is required on an original printed receipt, it is not required to be stored anywhere else.  So, all merchants need to put POS solutions in place that do not store cardholder data.  You do not need it and it puts you at risk if you do store it.

The next thing merchants need to do is to find a card processor that does not require the merchant to store cardholder data.  This can be a processor that uses tokenization or whatever, but the bottom line is that the processor does not return cardholder data to the merchant’s systems.  These processors typically provide secure Web-based systems that allow the merchant to view all of their transactions processed and, if necessary, provide a method to decrypt the PAN for dispute research and chargebacks.  Merchants need to restrict access to the processor’s applications to only those people that absolutely need access to perform their job.  These people should be reviewed at least quarterly to ensure that they continue to require access.

For those of you that just cannot get rid of cardholder data, there is the option of hashing.  Hashing allows applications such as fraud discovery, member tracking, rewards programs and similar functions to continue, they just do not have access to the actual PAN.  A hashed PAN results in the same hashed value, so research and analysis of PANs can still occur.  It is just that if you need to see the real PAN, you will have to go to the processor’s system to obtain the real PAN.

The travel industry, in particular hoteliers, are really behind the eight ball on PCI because of their need to keep the PAN for sometimes years because of the way reservations work.  However, this is where tokenization can earn its keep.  If a hotel takes a reservation and gets back a token when the credit card is authenticated, then the hotel can use the token however many times in the future for check in and check out.  Again, there is no reason for the hotel to need to retain the actual PAN.

The bottom line to all of this is that there are ways to minimize your organization’s PCI compliance efforts just by getting rid of the data in the first place.  So, stop putting forth efforts to comply and get with the movement to get rid of the cardholder data in the first place.  I have had a few clients go down this road and PCI compliance is now a piece of cake.  Their networks are still in scope for transmission, their applications in some cases do process cardholder data, but there is not storage which makes them much, much less of a target.

Call Center FAQ Significantly Changes

On January 22, 2010 the PCI SSC issued an update to their clarification regarding their FAQ on call centers storing CVV/CVC/CID in their call recordings.  The bottom line is that call centers are no longer allowed to store such cardholder information in recordings that are maintained digitally, regardless of whether they are encrypted.

The new FAQ (#5362) states that:

“It is a violation of PCI DSS requirement 3.2 to store any sensitive authentication data, including card validation codes and values, after authorization even if encrypted.

It is therefore prohibited to use any form of digital audio recording (using formats such as wav, mp3 etc) for storing CAV2, CVC2, CVV2 or CID codes after authorization; as card data can easily be extracted using freely available software.

On an exception basis, storage of CAV2, CVC2, CVV2 or CID codes in an analog format after authorization is allowed; as these recordings cannot be data mined easily. However the physical and logical protections defined in PCI DSS must still be applied to these analog call recording formats.

Audio recording solutions that prevent the storage or facilitate the deletion of CAV2, CVC2, CVV2 or CID codes and other card data are commercially available from a number of vendors. All other recordings containing cardholder data captured by call centers must be protected in accordance with the PCI DSS, including PCI DSS requirement 3.4.”

I am assuming that this change in policy is the result of the PCI SSC determining that, regardless of what we have been told in the latest reports from Verizon Business Services and other sources, there have been breaches of cardholder data that were the result of compromising digital audio recordings.

A big thank you to Emma Jenkins at VeriTape for pointing out this change.

UPDATE:  It appears that the FAQ has been changed again.  It now reads “It is therefore prohibited to use any form of digital audio recording (using formats such as wav, mp3 etc) for storing CAV2, CVC2, CVV2 or CID codes after authorization if that data can be queried; recognizing that multiple tools exist that potentially could query a variety of digital recordings.” adding back the “if that data can be queried” language.  There are plenty of commercial and open source tools that will search audio recordings, so of course this data can be queried.  Not sure of the motive here in this change.

How Email Ends Up In-Scope And What To Do About It

Let us clarify this issue.  I am not talking about the occasional email that contains cardholder data.  Try as your organization might, a small percentage of customers are going to email their cardholder data to you no matter how often and how strictly you remind them not to do such things.  In the immortal words of the comedian Ron White, “You can’t fix stupid.”  The way to address these customer indiscretions is to immediately print them out and delete them from the email system so that you minimize the chance that they end up on the email system’s backups.  Once done with the print out, shred or redact it.

These occasional customer “brain farts” does not, in my humble opinion, place an organization’s email system in-scope.  If your QSA says that such occasional messages do place it in-scope, I would seriously push back on this issue.  The risk involved along with the randomness of occurrence does not warrant the time and resources to totally eradicate such problems.  However, your management needs to understand the risk this presents and agree to accept that risk.

With that said, there is nothing worse than telling a client that their email system is in-scope for PCI compliance.  Most of the time, they look like a deer caught in a car’s headlights.  They just stare at you like you just slapped them across their face or dumped a bucket of ice cold water on them.  So, just how does this happen?

Interestingly, the primary cause of an email system being in-scope is due to the fact that it is not segmented away from the ecommerce environment that processes, stores or transmits cardholder data.  Some of this is not due to improper implementation.  In a number of instances, this situation occurs because the ecommerce environment requires integrated access to the email system and putting a firewall between the two is not an option.

The second reason email ends up in-scope is that in almost every email system implementation, the email system has been extended to handle other forms of communication.  As a result, the email system becomes the communications hub of an organization.  The most common extension is the integration of a facsimile system.  These integrated facsimile solutions were a God send when they were introduced as they improved efficiency and accuracy in handling facsimile communications.  They allow facsimiles to be received and automatically delivered directly to either an individual’s or a team’s Inbox based on pre-defined rules.  However as any IT person will tell you, these facsimile solutions have ended up being a compliance nightmare when security and privacy requirements like PCI were introduced.

Another application that gets integrated with email is instant messaging (IM).  IM inside an organization is typically just as secure as internal email.  However, users typically want the ability to not only IM their fellow employees, they also want to IM customers and business partners.  It is the use of IM outside of the organization that creates the problem because the security measures available inside are not typically available through Yahoo, AOL and Microsoft.  Not that IM applications cannot be secured; it is just that they usually are not secured outside of an organization.

With all of these potential risks with email, what should an organization do to ensure PCI compliance?

• Do whatever you can to get your email system out of scope.  The last thing you need is to have the email system in-scope.
• Physically or logically segregate your organization’s email system from your cardholder data environment.  If it cannot be segregated, then implement separate email solutions for your cardholder data environment and the rest of your organization.
• If you are using your email system for communicating cardholder data, stop that practice immediately and begin remediating your current email database and the backups of that database.
• If you need to use facsimile for transmission of cardholder data, have those facsimiles delivered to secure devices, not through your integrated facsimile/email system.  Most printer and multi-function device manufacturers produce devices that can provide a secure facsimile delivery capability including encrypted storage of facsimile transmissions.  If you really need the automation capabilities, then implement a separate instance of email and facsimile solutions for this purpose.
• Make sure that everyone in your organization understands the risks that email, instant messaging, facsimile and other forms of communication present to the cardholder data environment.  Train all personnel to never transmit cardholder data via any electronic communications.
• If you are printing out electronic communications that contain cardholder data, make sure that you also have proper destruction or redaction procedures documented and implemented.  Periodically test those destruction and redaction procedures to ensure that they are operating as expected.
• As best you can, restrict IM capability to only internal use.  If IM connectivity is required outside of your organization, implement it securely over an encrypted VPN link or other secure communications channels.

Decommissioning Applications

Here is a question that comes up from time to time.  Particularly because a lot of my clients are remediating their PCI compliance issues by replacing older applications with PCI compliant new ones.

What do I need to do in regards to PCI DSS compliance if I’m replacing an application?

There is no guidance in the PCI DSS regarding the decommissioning of applications that are in-scope.  So what should an organization do when they are getting rid of an in-scope application?

The first problem is the application’s cardholder data.  Cardholder data usually ends up everywhere, particularly with systems that are not PCI compliant.  Cardholder data is not only on hard disks and disk arrays; it is also on backup tapes and other backup media.  In the case of point-of-sale (POS) systems, cardholder data can end up on every POS as well as the POS servers.

The bottom line is that you need to track down all of this cardholder data and make sure that you properly dispose of it.  The key problem is making sure you have located all of the cardholder data.  You should use this opportunity to scan all of the systems to be decommissioned with a tool to locate cardholder data.  While this is not necessarily a perfect technique, it will identify all of those systems that likely have cardholder data and those that do not.  Those that do have cardholder data will be remediated first.  Those that do not have cardholder data will be remediated last.

Since these non-compliant applications typically did not securely store cardholder data, you need to make sure that the data that remains is properly disposed.  That means performing Department of Defense (DoD) grade erasing of data from hard disks and tapes.  If the hard drives are old and are not going to be reused, then I would recommend contracting with a reputable DoD certified firm to have them degaussed with your tapes.  Industrial strength degaussing will usually damage the electronics of the hard drive, so if you intend to reuse the hard drive, do not have it degaussed.  If you are going to reuse the hard disks, then they should be erased with a DoD grade disk wiping utility.  There are plenty of these available on the Internet.

The next issue is proving that the application is decommissioned.  Make sure to document all of the steps you took to ensure that the cardholder data has been removed from all systems.  Have management sign off on this documentation so that they are aware of what was done and how it was done.  This documentation will be useful for your filing of a Report On Compliance or Self-Assessment Questionnaire as well as should anything happen in the future that comes back to try an haunt you.

Hopefully this will assist those of you that are going through such a process to become PCI compliant.

PCI SSC Issues Clarification On Encrypted Data Being In-Scope

On October 27, 2009 the PCI SSC and the card brands issued a clarification on whether or not encrypted cardholder data is still in-scope (Article Number 1086).  For such a simple clarification, there was a lot of verbiage, so I thought it might be good to take a closer look at the FAQ in question.

The FAQ starts out stating the obvious facts regarding encryption that we all know.

“Encryption solutions are only as good as the industry-approved algorithms and key management practices used, including security controls surrounding the encryption/decryption keys (“Keys”). If Keys are left unprotected and accessible, anyone can decrypt the data. The DSS has specific encryption key management controls (DSS 3.5 and 3.6), however, other DSS controls such as firewalls, user access controls, vulnerability management, scanning, logging and application security provide additional layers of security to prevent malicious users from gaining privileged access to networks or cardholder data environments that may grant them access to Keys. It is for this reason that encrypted cardholder data is in scope for PCI DSS.”

The FAQ then goes on to give an exception.

“However, encrypted data may be deemed out of scope if, and only if, it has been validated that the entity that possesses encrypted cardholder data does not have the means to decrypt it. Any technological implementation or vendor solution should be validated to ensure both physical and logical controls are in place in accordance with industry best practices, prohibiting the entity, or malicious users that may gain access to the entity’s environment, from obtaining access to Keys.”

It appears that the PCI SSC and the card brands have recognized that if you do not have the ability to decrypt, then the data cannot be treated as in-scope.

However, if an encryption algorithm is used, there must be a way to decrypt.  As a result, the next sentence addresses this fact.

“Furthermore, service providers or vendors that provide encryption solutions to merchants who have administrative access and controls to Keys along with the management of termination points for encryption to process transactions, are required to demonstrate physical and logical controls to protect cryptographic keys in accordance with industry best practices (such as NIST referenced in PCI DSS requirement 3.6), along with full compliance with PCI DSS. Merchants should ensure their solution providers who provide key management services and/or act as the point of encryption/decryption are in compliance with PCI DSS.”

While we now have an exception, it appears to be only valid if the merchant do not have access to the encryption keys.  Notice, this exception does not include service providers that may also not have access to encryption keys, only to merchants.  In addition, the exception specifically calls out that vendors and service providers that have control of the encryption keys must prove that the keys are protected in accordance with industry best practices such as the NIST standards called out in requirement 3.6.  Notice, they MUST prove.  Therefore this nonsense of “trust us, we do this” is no longer acceptable.  They must provide proof that they are managing encryption keys under industry best practices.

Finally, merchants do not necessarily get off scot-free if their encryption keys are managed by a third party and the merchant does not have access to the keys.  The final statement in the FAQ covers this point.

“Merchants should be aware that encryption solutions most likely do not remove them completely from PCI DSS. Examples of where DSS would still be applicable include usage policies, agreements with service providers that deploy payment solutions, physical protection of payment assets and any legacy data and processes (such as billing, loyalty, marketing databases) within the merchant’s environment that may still store, process or transmit clear text cardholder data, as that would remain in scope for PCI DSS.”

So, in my opinion, here is the bottom line for this FAQ.  You can rule encrypted cardholder data out as long as the merchant does not have access to the encryption keys and the third party in control of the encryption keys can prove that it is properly managing the keys.  However, all other relevant PCI DSS requirements still apply.

UPDATE:  It has been pointed out to me that as long as there is an independent internal party available to manage the key, that that would also satisfy the exception.  While I agree with that assessment, with most small and mid-size merchants and service providers, I would argue that the only way they can comply with this exception is to use an external, third party to manage the keys.  Therefore, I stand by my interpretation that a third party needs to manage the keys.

UPDATE: The PCI SSC changed their FAQ system and the referenced FAQ number has changed to 1086 which I have changed in the post.

In Scope – Example I

I thought I would write about a recent meeting I had with a client regarding whether or not their proposed point-of-sale (POS) solution was going to be in or out of scope.  Obviously, the hope was that it would be out of scope.

I would like to commend this organization for taking the high road to PCI compliance.  I wish there were more organizations like them.  After slugging through a very ugly PCI compliance process last year with a multitude of compensating controls to get them compliant, senior management set a goal of removing all cardholder data (CHD) from their point-of-sale (POS) systems over the next year and a half.  To accomplish this, they are going to implement a tokenization solution on their centralized transaction switch that they are developing in-house.  As part of this effort, the company is purchasing a new POS solution that is PABP compliant with the vendor stating that the next version will be PA-DSS compliant.  The discussion at my meeting was to determine if the POS solution would be in or out of scope.

Prior to the meeting, the client had provided me with some documentation describing their plans for the POS implementation.  I have taken some editorial license to protect the identity of the client, but here is the verbiage that I was provided regarding the configuration of the POS solution.

“The POS PC is connected to a separate credit card terminal by a USB connection.  There is a software client running that communicates with their centralized transaction hub that then submits transactions for authentication as well as handling settlement.  The only interaction between the PC and the terminal is to pass the amount of the transaction from the PC to the terminal and for the terminal to tell the PC whether or not the transaction was approved or declined.”

On the face of things, the solution appears to keep the POS solution out of scope.  However, this is a prime example of why documentation must be complete, clear and accurate.  Interpreting the documentation and lacking a detailed network diagram, I assumed that the credit card terminal and the PC had separate network connections.

As Paul Harvey was so famous for saying, “And now for the rest of the story.”  It turned out that the documentation had a slight flaw.  The person who wrote the majority of the documents had inconsistently used the terms ‘terminal’ and ‘POS’ through out.  A common mistake since most people that develop documentation are usually very close to the implementation process and make the assumption that everyone that reads the document has the same background with the topic in question.  As a result, it was not clear whether the terminal was the PC or the terminal was the credit card terminal.  What the meeting uncovered was that the client software that communicated with the centralized transaction server was in fact running on the PC, not the credit card terminal.  This meant that all communications to and from the credit card terminal were routed through the PC to the centralized transaction server.  That means that the POS solution is in scope.

The moral of the story is that a good QSA never assumes they know everything about a solution and why they seem to ask many questions about something you would think they would know.  The reason is that a good QSA knows that it is your organization’s implementation of the product, not another organization’s implementation and there will be differences.  And those differences could mean the difference between in scope and out of scope.