Sins Of The Past

This was a question we got from our last PCI Dream Team session on the Cloud.

“Issue – found CVV in historical call recordings that need to be purge/delete. We are not able to purge the entire call record and still need to keep it for record retention. What tools should be evaluated to help address this issue?”

A lot of organizations are discovering that how they did things in the past did not meet PCI, GDPR or other legal or regulatory requirements when data in their possession needs to be protected.  Gone are those freewheeling days of collecting data without worrying about how to secure and protect it.  Customers and the public at large are realizing the value of their information and the need to protect it.  All of which starts organizations thinking about how to reduce the risk they have because they have all of this data and they are being held responsible for having it.  The patchwork of state laws in the US hold a lot of organizations at risk, some higher than others.

There are also the sins that come to light down the road.  It is not unusual to have a PCI in scope application crawl out of the woodwork years down the road at large organizations.  It should have been identified way back when the organization was starting out in PCI, but somehow was missed and just now turned up.  Unfortunately, these discoveries tend to turn up at the 11th hour of the organization’s current PCI assessment and there is no way to include the application without causing a delay in issuing the ROC.


So, let us talk about the last case first.  The application that we uncover very late in the PCI assessment.

What should happen and in the example cited did happen, was a conversation with the acquiring bank.  The situation was explained as well as the risk involved (it was storing encrypted PAN) and the bank was asked do we delay filing the ROC and assess this application (likely a delay of longer than 90 days) or do we keep moving ahead as planned and pick up the newly disclosed application in the next assessment?

The bank decided that they did not want to delay the ROC filing since it was just out of our QA process, had been sent to the client for their review and was due in around 30 days.

The client looked further into the application and determined that it could be easily remediated with tokenization from their gateway.  As a result, when time came for the next year’s assessment, the application had been remediated with tokenization.  We took a look at it and confirmed it no longer contained encrypted PAN and explained to the bank that it would no longer be in scope.

However, things do not always end that well.  I have also had occasions where no remediation was possible for a variety of reasons and had to go in the following year and assess the new discovered application in all its PCI compliance (and in some cases non-compliance) glory.


Getting back to our original sin, so to speak.

First and foremost, you may not be able to remediate your files due to legal or regulatory constraints.  So, before you go charging ahead on your remediation efforts, make sure you discuss it with your legal and compliance folks to ensure you are not creating an even bigger problem.  Assuming you are allowed to remediate data, you can proceed with reading the rest of this section.

Structured data is typically easy to remediate.  You find out what XML tags to look for, fields or what database columns are involved, you develop a program to remediate the data to first six and/or last four for the PAN or erasing data for any information you were not supposed to keep and execute.  Easy.  Well, easy until you take into account backups which can complicate remediation if you cannot just erase the backups.

Unstructured data as with call recordings and notes/comments fields can be a nightmare to remediate.  The reason of course is that the data has no structure and does not necessarily occur in the same place.  Unlike XML or a database where data is at least tagged or in a column, unstructured data exists wherever it exists and programs to remediate the sensitive data need to find it and then eradicate it.  That introduces the problem of false positive results.  I wrote all about the “fun” of trying to find cardholder data (CHD) five years ago and it has not necessarily gotten any better.  The bottom line with unstructured data is that it may not be possible to completely remediate the problem.

However, the best you may be able to do is to remediate the data when it is encountered.  Going back to call recordings, if the quality assurance review process or any process that has someone review recordings encounters CHD they redact the information from the file so that it is no longer in that file.  Not perfect, but slowly you reduce the amount you are storing.  You still have to encrypt the files for protection, but you are making an effort to reduce risk by reducing the amount of viable data.

Isolate It

This most commonly occurs with call recordings, but I have encountered the occasional “legacy” application that it applied to as well.

In the either case, the old system is being decommissioned and a new solution (usually outsourced) is being implemented.  The question comes up, “what do we do with the old system?”  The reason is that for customer service, legal and/or regulatory reasons it cannot just be wiped and destroyed.  It needs to be retained for some period of time before that can happen.

The answer is to keep the system powered up, but off any other network.  If people need access, they need to go to a PC or workstation that is connected to a private, air gapped, isolated network that consists of the old system and the PCs or workstations to be used to access the old system.  No internet or other network access is provided, only a network that contains those few isolated systems.  This will allow the system and workstations to age yet remain protected because of the air gap.  Remember, the PCs and workstations will also age along because it is highly likely that new software may not allow connectivity to the old system.  This is why everything will need to be air gapped.

I usually get asked for the reason to keep the old solution powered up.  That comes from a study done long ago by IBM.  What the IBM study found was that systems that get powered off after years of operation have a tendency to fail after being powered off for any extended length of time (i.e., long enough to cool down).  As a result, if you intend to keep the system around and available, you best keep it powered up albeit isolated as discussed earlier.

One of the larger issues with isolation will be monitoring of the air gapped network to ensure it remains air gapped and how you respond if that air gapped is breached.  There are a number of ways to address this issue, so pick the solution that best fits your environment.

Isolation is not a perfect solution.  It will likely require a number of compensating control worksheets (CCW) to address the fact that you have a number of “antique” systems around.  So be prepared for that work effort as it will likely not be small.

6 Responses to “Sins Of The Past”

  1. 1 naqwi
    December 18, 2020 at 10:58 AM

    Dear PCI Guru,
    We are level one merchant and recently deployed P2PE (PEDs) in over branches that will activate in January 2021.
    Cardholder data will directly transmit to the P2PE service provider through a separate network, and no cardholder will store or transmit in our environment.
    I need your expertise to resolve one problem. Currently, we are storing cardholder (PAN)in our server for six months for fraud investigation or bank enquiry. That system has all relevant (Encryption, AV, FIM, Access, Retention controls) control in place to protect cardholder, after P2PE implementation six months worth of data left in that system which hosted our vendor (not the same as P2PE solution provider) environment, data will delete automatically after six months.
    We are looking for some solution to obfuscate or tokenised the data, but it cost lots.
    Do you advise any solution that removes that system from our audit scope?
    Do you think we need to bring that system in our audit scope?

    • December 18, 2020 at 12:43 PM

      If it stores encrypted CHD, it is in-scope and needs to be assessed. That said, once you decommission that server and securely destroy the data (i.e., secure wipe or physically destroy the drives) then there will be no more scope there.

      As you are finding out, it is just cheaper to keep the server, minimize access and just let nature run its course. In seven months you will be able to address the problem for good.

      In regards to P2PE, make sure that you are not getting PANs back from your provider. Biggest problem with P2PE is that merchants find out after the fact that PAN is still being returned.

  2. 3 CR
    March 12, 2019 at 5:03 AM

    (Disclosure: I work for a company selling products and services in this space.)

    Could not agree more. The simplest solution (“air gapping”) is the easiest. We often counsel clients to take their historic call recordings, move them to a harddrive on a single PC, then disconnect that PC entirely from the network and securely erase the old location. (Sure, back them up by all means, but stick those offline backups in a safe somewhere.) Whilst technically still retaining CVV (and the compensating controls paperwork which comes from that), our clients have always found acquirers to be sympathetic to the practical approach for this problem.

    You may also be surprised how infrequently people need to walk to the air-gapped PC and find a recording. There is a perception (and sometimes a legal requirement) that the recordings need to be retained for years, but the practical reality is that are almost never accessed.

    There is no point spending money on speech detection, cleansing, or secure third-party encrypted online storage systems which (as PCI Guru points out) don’t practically work even 5 years after his original article, and in many cases negate the legal requirement that call recordings are not altered post recording.

    So: stop storing PAN/CVV from now onwards, and offline the old stuff.

    • March 12, 2019 at 6:12 AM

      The only problem with copying the recording files is that the majority of solutions I have encountered are appliances and you need the appliance to access the recordings that could be on storage contained in the appliance to on a NAS/SAN.

      But you are correct about access needs. In my experience, once you get passed 180 days, access usually only occurs if some sort of legal action is initiated.

  3. 5 John
    March 8, 2019 at 4:55 AM

    Fortunately with cardholder data like CVV, it all goes stale pretty soon — does any bank still issue cards with an expiration date of > 4 years? At that point you ought to be able to justify a finding that it’s no longer cardholder data, it’s just a string of characters that isn’t good for anything.

    • March 9, 2019 at 7:30 AM

      You would like to think you are safe, but you are not. While the CVV does change when a new card is issued, the primary account number (PAN) does not always change. And since a lot of merchants doing card not present (CNP) transactions do not ask for CVV, it really doesn’t matter if the CVV is breached or not as you can commit fraud without it.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Welcome to the PCI Guru blog. The PCI Guru reserves the right to censor comments as they see fit. Sales people beware! This is not a place to push your goods and services.

March 2019

%d bloggers like this: