Why vSkimmer Should Not Matter

It was announced this week by McAfee that a new threat to merchants has been discovered called vSkimmer.  This is a very insidious threat as most merchants will likely not know they have been infected until it is too late.

The net of vSkimmer is that it is malware of the highest order built for the explicit purpose of collecting track 2 data from Windows point of sale (POS) systems.  Worse yet, whoever wrote this little gem of software intends to enhance it in 2013 to include the ability to skim EMV cards’ “track” data as well.

vSkimmer can be deployed like Stuxnet through a USB thumb drive, as malware in an email message or on a Web site or any number of ways.  When installed, vSkimmer determines the operating system and version, hostname, active username and various other operational characteristics of the POS system.  It then inventories running tasks and memory to determine where track 2 data is stored and begins recording that data.

vSkimmer works whether the POS system is connected to the Internet or not.  When the POS is connected to the Internet, it transmits the data obtained to a control server using HTTP.  When the POS is not connected to the Internet, the information is stored until someone connects a USB device labeled ‘KARTOXA007’ and copies all the information it obtained onto the USB device.

As usual, the Internet is abuzz regarding how this will be addressed by the PCI DSS.  Sorry to disappoint, but it is already addressed.  Here are some key requirements in the PCI DSS that should mitigate vSkimmer.

• Requirement 1.2.1.a requires that only that network traffic that is necessary is allowed through the firewall.  Merchants should be only allowing connectivity from the POS or card terminal to their processor and nowhere else.  Any traffic attempting to go anywhere else should be flagged and IT alerted to investigate.
• Requirement 5.1 requires that you have anti-virus and anti-malware software installed on POS devices.  Given this is a Windows specific threat and Windows is highly susceptible to being infected, you should have done this already.  While anti-virus solutions are not perfect in always identifying such malware, since McAfee and other anti-virus solution vendors are the ones that found vSkimmer, I would imagine that they all have or will very soon have signatures for vSkimmer.
• Requirement 6.1 requires that systems are patched current.  The problem with patching POS systems is that a lot of vendors issue POS updates for the OS and their application on a quarterly or even annual basis and do not recommend that merchants patch their POS systems directly from Microsoft because of compatibility issues.
• Requirement 10.2 which requires the logging of events.  In the case of a USB device being plugged into the POS system, at a minimum you should see that the portable device enumerator service going active when a USB device is plugged in and if the device is new, you should see system event log entries regarding the loading of device drivers to support the USB device.  None of these actions should be seen in your log data, so if you monitor for these events, you will know that USB devices are potentially being plugged into your POS systems.
• Requirement 10.5.5 requires the use of file integrity monitoring which would catch the installation of vSkimmer as a foreign piece of software even though it masks itself as ‘svchost.exe’.  This would provide a backup control for requirement 5.1 if vSkimmer changes its approach as to its file name and is not caught by the anti-virus solution.

In addition to the PCI requirements, you can do the following to increase your security in regards to vSkimmer.

• Do not allow USB devices to be connected to your POS systems.  Most card terminals are RS232 devices, but USB is becoming more common.  The Windows Group Policy function can be used to disable USB ports on Windows systems.  There are also third party solutions that will disable USB ports.  A lot of these third party solutions can offer additional granularity in what types of USB devices can be connected.  This can be very advantageous when you are using USB card terminals which still need to connect, but other USB devices should not be allowed.  One of my more imaginative clients hot glues the ports shut on their POS systems.
• Train your staff on the vSkimmer threat.  Explain how it works and what they can do to minimize this threat such as not allowing anyone to manipulate the POS systems other than employees responsible for the care and maintenance of the systems.
• Lock your POS systems in a sealed cabinet or cage and only allow the manager on duty to have the key.  This may also involve additional security on POS servers if those are also used by your POS solution.
• Periodically review video of your POS stations to determine if cashiers or other personnel appear to be manipulating the POS system.

If you adopt all of these measures, you will significantly reduce the threat presented by vSkimmer and will likely never encounter it.

Bring Your Own Device And PCI Compliance

Bring your own device or BYOD is all the latest rage.  I believe that the reason for that exuberance is the consumerization of technology.  It is that exuberance through BYOD that has made everyone an “IT expert.”  Just ask any user of a smartphone or tablet and they will baffle you with vendor lingo regarding BYOD.  However, regardless of what these people think, there is a big difference from consumer use and enterprise use, the first of which is security.  In this post I am going to look at the minimum PCI DSS requirements BYOD will have to comply in order for your organization to maintain PCI compliance.

From a PCI perspective, requirement 12.3 is very relevant in the BYOD discussion.

“Develop usage policies for critical technologies (for example, remote-access technologies, wireless technologies, removable electronic media, laptops, tablets, personal data/digital assistants (PDAs), e-mail usage and Internet usage) and define proper use of these technologies. “

Under 12.3, there are eight sub-requirements that require:

• Explicit approval by authorized parties.  Just because you have a smartphone or tablet, does not imply you get an automatic pass to connect, you must be approved for that privilege.  In most organizations, that means you need to provide a business need to have such a privilege granted.  While access to email through a smartphone or tablet is one thing, access to cardholder data should be another and be granted very judiciously, if at all.
• Authentication for use of the technology.  If you think you are using a PIN, think again.  You still have to log onto any PCI in-scope system using a password that meets the PCI DSS requirements.  By the way, two-factor authentication is required to access cardholder data remotely, so BYOD does not get you a pass there either.
• A list of all such devices and personnel with access.  For organizations issuing BYOD, this is not a problem.  For organizations allowing any Apple iOS, Android or Windows device to connect is problematic.  While it can be done, you will need to document who was granted access, for what reason/purpose, and what they are using to obtain that access (i.e., make, model, etc.) even if your organization is not providing the devices.
• Labeling of devices to determine owner, contact information and purpose.   I take issue with identifying purpose of the device as that, in my opinion, could just make someone ever more curious as to what is on the device.  However, identifying the owner, giving their general business address and general voice and facsimile telephone numbers for their location is what I would recommend.
• Acceptable uses of the technology.   Email is one thing, doing something that involves cardholder data is another.  I think if companies think this through, there is probably little, if any, reason for BYOD to be even near the cardholder data environment (CDE).  However, if for some bizarre reason you can come up with a valid reason, then all requirements for remote access of cardholder data apply such as personal firewall, no way to disable the firewall, strong passwords, two-factor authentication, encrypted connection, etc.  And remember that these devices have keyboard loggers, so all data input is recorded so keep that fact in mind when designing your information security requirements for BYOD.
• Acceptable network locations for the technologies.  In the case of BYOD, this is anywhere outside of the organization’s network perimeter.
• List of company-approved products.  For those organizations issuing the BYOD, this is not a problem.  For those of you that allow anyone with anything to connect as long as it runs iOS, Android or Windows 8 RT/Pro, your list is going to say “Any iOS, Android or Windows 8 RT/Pro device.”
• Automatic disconnect of sessions for remote-access technologies after a specific period of inactivity.  With BYOD, this can be problematic as most BYOD do not use traditional remote access connectivity and inactivity can all be in the eye of the beholder, so to speak.  As a result, inactivity timeouts can be difficult, if not impossible, to enforce in some instances.  As a result, you may have to be either creative or use a compensating control to comply with this requirement.

Going through the above, I think most organizations would see BYOD for what it is – a fad.  Do not get me wrong, BYOD has its uses in the corporate environment, but they are fairly limited and likely does not include cardholder data.

However, if you have come up with a business justification for processing, storing or transmitting cardholder data using BYOD, there are number of other requirements you are going to have to address.  I know there are other potential requirements that could be involved, but these are the most well known that will need to be complied with under the PCI DSS.

• Requirement 1.4 – Install personal firewall software on any mobile and/or employee-owned computers with direct connectivity to the Internet (for example, laptops used by employees), which are used to access the organization’s network.   While this can be accomplished by a number of security vendors, it is the enterprise management of those solutions that is currently lacking and the ability to globally enforce policies.
• Requirement 3.4 – Render PAN unreadable anywhere it is stored (including on portable digital media, backup media, and in logs).  Most cell phones and tablets do not support device encryption.  As a result, how will you protect any cardholder information stored on the device?  Remember that these devices have keyboard loggers, so any cardholder data input on the device is collected and stored by the device whether you like it or not.  The bottom line is that you will have to restrict BYOD to only those devices that can support whole device encryption.
• Requirement 4.1 – Use strong cryptography and security protocols (for example, SSL/TLS, IPSEC, SSH, etc.) to safeguard sensitive cardholder data during transmission over open, public networks.  You would not think that this would be a significant problem, but it can turn out to be very significant and I will speak about this later on.
• Requirement 8.3 – Incorporate two-factor authentication for remote access (network-level access originating from outside the network) to the network by employees, administrators, and third parties.  This capability is available, but if you are using any of the solutions that run on the BYOD, your users will likely have issues trying to connect and get their token value from that device.
• Requirement 8.5.10 – Require a minimum password length of at least seven characters.  This is likely a deal breaker for users.  Most like their PIN or swipe and will not want to give them up for a seven character, strong password.  In addition, some security solutions may create a situation where the phone cannot be answered without unlocking the device and, a seven character password may cause calls to be missed.
• Requirement 8.5.11 – Use passwords containing both numeric and alphabetic characters.  This can be problematic when some virtual keyboards require flipping between three or more screens to get to certain special characters.  With phones with physical keyboards, there may be limitations to the number of special characters available that could create problems with the password uniqueness requirement in 8.5.12.
• Requirement 8.5.13 – Limit repeated access attempts by locking out the user ID after not more than six attempts.  Some security systems may not be able to enforce this without wiping the device.
• Requirement 8.5.14 – Set the lockout duration to a minimum of 30 minutes or until administrator enables the user ID.  I am not aware of a security solution currently available that can enforce this on a smartphone or tablet.  You will have to meet this with a compensating control.
• Requirement 8.5.15 – If a session has been idle for more than 15 minutes; require the user to re-authenticate to re-activate the terminal or session.  As with 8.5.14, I am not aware of a security solution that can enforce this on a smartphone or tablet.  You will have to meet this with a compensating control.
• Requirement 10.2 – Implement automated audit trails for all system components to reconstruct events.  This is not an issue with Android and may be an issue with Windows 8 RT, but is definitely an issue with Apple iOS.  There is no utility I am aware, outside of forensic utilities, which can meet this requirement in an Apple iOS environment.  In addition, I am not certain how you get this log data back for any sort of analysis without chewing up a tremendous amount of data bandwidth or device memory.  At the end of the day, this will also likely be satisfied by a compensating control, if you can actually come up with enough controls that go above and beyond the PCI requirements.
• Requirement 10.3 – Record audit trail entries for all system components for each event.  If meeting requirement 10.2 is a gymnastic event, then there is the configuration of the log data to ensure that all of the necessary log information is collected.  I am sure that under Android and Windows there is probably some way to ensure that the necessary log data is required.  But with Apple, iOS, Apple will have to be able to provide this capability.  And knowing how stubborn Apple can be about having their hand forced in these matters, getting access to configuration of log data let alone log data will likely be a battle.  Again this will also likely be satisfied by a compensating control, if you can actually come up with enough controls that go above and beyond the PCI requirements.
• Requirement 10.4 – Using time-synchronization technology, synchronize all critical system clocks and times and ensure that the following is implemented for acquiring, distributing, and storing time.  For smartphones, they get their time synchronization from the carrier.  However, that time does not likely correlate to the time used by enterprise systems.  It will be close, but not necessarily the same thus complicating any forensic examination if one is required.  This can obviously dealt with in a compensating control.
• Requirement 10.5 – Secure audit trails so they cannot be altered.  Given that users have administrator access on their BYOD; this could be a problem that cannot be easily solved.  Yes, there are security solutions available that can lock down a device, but they can also lock them down so far that users can begin to wonder what the point of having the device is.  As a result, you will find you have a very fine line to tread in this area.

In the end, you start to understand why BYOD is a difficult thing to justify when you need to comply with all of the aforementioned PCI requirements.  But there are a few other considerations that you will still need to address.

The first situation that should concern any organization considering BYOD is the loss of that BYOD.  It is virtually guaranteed that BYOD will result in lost devices and you will need policies, standards and procedures to address that eventuality.  The way most organizations address this issue is providing a remote device wipe capability that can be invoked whenever a device is reported lost.  Not that such capability ensures that every lost device is wiped, but it is better than nothing, but not by much.  This is usually backed up by the bad password entry policy that wipes the device after six incorrectly entered passwords.  So even if the remote wipe does not destroy the data, the bad logon attempts will.

BYOD brings up an information ownership issue as, according to the latest statistics, 70% of all BYOD are owned by the employee, not the organization.  As a result, you are allowing the organization’s information to be processed or stored on a device not owned or necessarily even controlled by the organization.  While you can have an employee sign an agreement regarding the organization’s ownership of the information and the employee’s responsibilities for protecting that information, the issue of enforcement of such an agreement can be very problematic depending on platforms and other technology issues.  You can have a remote wipe capability, but that brings up the potential legal issue of can you also wipe an employee’s personal information such as contacts, music and pictures as well?  Just remember the old saying, “Possession is nine tenths of the law.”  When you finally get through court proceedings, does it really matter if you won when the data has already likely been disseminated?

Then there is the question about whether your applications actually capable of dealing with BYOD?  Most internally facing applications that employees desire access are not engineered for secure, remote access from BYOD.  As a result, a lot of organizations are turning to VPN technology to solve the remote access issue.  However, organizations using VPN are finding out that, while VPN clients are free for download to the BYOD, licensing for the BYOD VPN client is over and above the licensing the organization has already purchased for notebooks.  In addition, depending on the device, running the VPN client can make devices run very slow to the point of being worthless when connected.

When VPN is not a solution a lot of organizations are using remote desktop (RDP), Citrix virtual desktop or similar remote control environments to provide secure access to internal applications.  Having worked with a few of these in a smartphone and tablet environment, I can tell you their use is haphazard at best due primarily to screen size and lack of a mouse and a real keyboard.  In addition, we also find that some of these secure remote desktop solutions are not using secure communication methods.

In addition to VPN and remote control, a lot of organizations are implementing HTTPS for secure connectivity to their applications.  However, this creates all sorts of new security issues related to authentication and protecting the applications which are typically not engineered to be externally facing.  We are finding that in the rush to enable applications for HTTPS, there are numerous security vulnerabilities that are being introduced.  We also see vulnerabilities as well with applications developed specifically for mobile devices.  In the haste to get BYOD up and running, the security vulnerabilities are not corrected before the applications are put into use (a violation of PCI DSS requirement 6.6) which puts the applications and, potentially, the internal networks at significant risk of compromise.

And finally, data input from smartphone and tablets can be highly erroneous not just because of human typing errors but also because of auto-correction systems that are implemented on these devices.  Anyone considering significant data entry through BYOD is just asking for trouble as this erroneous data input could result in legal issues later on due to mis-spellings and mistakes in addresses and other information.

Most executives do not understand the security and privacy issues of BYOD because they have not encountered them and are not aware of them even when time is taken to educate them.  Unfortunately it usually takes an executive losing their BYOD to help management appreciate the issues with BYOD and to slow down the drive to integrate BYOD until their concerns regarding security and privacy are addressed.

As you can see, using BYOD is not as simple a process as your end users might think.  This is even truer when that BYOD will be part of your cardholder data environment (CDE).  There are a number of innovative solutions for BYOD that are secure, but those solutions are expensive and make the BYOD only a display device.  However, if you want to be able to sleep at night, I would highly recommend looking at those purpose built solutions.

2012 Database Threats

I attended a Webinar recently put on by Application Security Inc. regarding the threats to databases for the coming year.  If you did not attend it, you missed a good session.  But the most disturbing thing brought up was their top 10 list database vulnerabilities and misconfigurations.  Their top 10 list is:

1. Default or weak passwords
2. SQL injection
3. Excessive user and group privileges
4. Unnecessary DBMS features enabled
5. Broken configuration management
6. Buffer overflows
7. Privilege escalation
8. Denial of Service
9. Unpatched RDBMS
10. Unencrypted data

If you look at my post a while back on the 2011 Verizon Business Services’ reasons for why organizations were breached, there is a great correlation between Verizon’s report and what Application Security Inc. is saying.

Their first point about weak or default passwords is very clear and should not need to be discussed.  In this day and age, we should all be ashamed that this is even on the list, let alone the first item on the list.  The bottom line here is that, if you use default or weak passwords, you deserve to be breached.

They brought up and interesting point about SQL injection attacks that a lot of organizations do miss or underestimate and that is the internal SQL injection.  Most organizations are so focused on the external threat that they forget about the threat from the inside.  Worse yet, most security professionals and DBAs are unaware of the threat SQL injection poses even without the Web.  Since most of today’s attacks are perpetrated once past the perimeter, the protection from the inside attack is very relevant and very important.  Because once an attacker is on the inside, it is relatively trivial to use SQL injection or other techniques to obtain data.  More and more organizations are beginning to understand the insider threat and are firewalling all of their database servers away from the general user community as well as minimizing the number of users that have direct SQL access to those servers.

Excessive privileges cannot always be addressed at the DBMS level.  In today’s packaged software world, a lot of the rights are managed and maintained at the application level and that security matrix is maintained in a database table.  The granularity that can be granted is usually where things go awry because the application’s security system only provides an “all or nothing” approach.  Application vendors are getting better with this because of SOX, HIPAA, PCI and the like.  However, organizations typically need to be on the most current releases to have access to such enhanced security granularity.  Unfortunately, there are very few organizations that can afford the most current release or can implement the most current release due to their extensive modifications.  The simplest way to address this issue is the periodic reviews of database privileges and minimizing those users that have excessive privileges.  In the longer term, I expect we’ll see the return of the data dictionary with the addition of user rights and roles that will manage this problem.

Unnecessary features enabled are a vendor and DBA issue.  In some cases, vendors make changing features impossible or near to impossible once the RDBMS is installed.  In some cases, there are physical reasons as to why a feature must be enabled at installation.  However, there are also instances where features could be enabled or even disabled at anytime, but because the vendor only wanted to do that at installation, that is the only time you can deal with the feature.  This results in a lot of DBAs installing the RDBMS with every feature/function available, whether it is needed or not, just in case they might need it later on.  Do not get me wrong as I understand the drivers for this practice.  In today’s “I needed it yesterday” world, it is tough to be responsive when something will require an entire re-install of the RDBMS and migration of existing data in order to get something done.  It is time for IT people as a whole to start explaining to non-IT people that there are just some tasks that take time to do properly no matter how quickly anyone needs them completed.  Our infrastructure has become susceptible to attack in large part because of this rapid response desire.  If we intend to make things secure, we need to stop and think things through before creating even larger issues.

The pervious issue feeds directly into the next; broken configuration management.  Configuration management is broken because the vendors make it virtually impossible not to break it.  And even when configuration and changing configuration is easy and manageable, DBAs are not always as structured as other IT operational disciplines.  As a result, whether talking about the configuration of the RBDMS or the client that loads on the workstation, configurations are too broad because of the “just in case” factor.  I know it is a pain to only enable what needs to be enabled and then six months later have to reinstall everything just for a particular feature, but that is the correct way to do things if you intend to be secure.

Buffer overflows, privilege escalations and denial of service are all common vulnerabilities that organizations will have differing levels of success in mitigating.  I will tackle the easiest to address first, privilege escalation.  If there is any area where security can always be addressed it is with privilege escalation.  The reason privilege escalation exists is because someone, usually a developer, created the issue because they decided to allow users to perform a task that the user should not be allowed to perform.  Because if they were allowed to perform the function, then they would not need their privileges escalated to perform it.  The easiest thing to do is to disable those functions that require privilege escalation.  However, in some cases, that approach will create operational issues that will be unacceptable.  In those cases, monitor the daylights out of things so that you can be sure that the privilege escalation did not result in a different outcome.

In a lot of cases, there can be little done to address a denial of service (DoS) attack short of blocking the offender(s).  Denial of service does not compromise information; it just makes the information stored in the database unavailable.  And for most organizations, that is an important distinction.  If no information has been or can be compromised, then DoS is an annoyance and should be treated as such.  However, some DoS attacks can be used to defeat security measures in the RDBMS by causing the RDBMS to fallback to a basic operational state.  It is in these situations that one has to be careful because information can be lost.  The easy fix is to put a firewall in front of the database and enable DoS attack protections.

Buffer overflows are the most insidious attacks because, in some cases, there is little that can be done to stop them.  A lot of security professionals make the success of buffer overflow attacks sound like they are all the result of sloppy coding practices.  And while there is some truth to that view, the amount of which depends on the skills of your programmers, the success of buffer overflow attacks is also the result of embedding too much flexibility into our applications and leveraging the capabilities of the RDBMS.  In today’s world of open constructs such as SQL and RegEx, we have effectively made everyone a potential database programmer all in the sake of expediency.  Yes, customer service is highly improved, but at what cost?  Web application firewalls can minimize buffer overflows by “learning” how your SQL calls are typically structured, but they are not a complete answer nor do they completely remove the risks.  The way to fix the problem is to reduce functionality and make applications more complicated and difficult to use.  For most organizations that is not an option.  As a result, we must minimize the risks but be willing to accept the risks that remain as a result of our desire for ease of use and flexibility.  Minimizing the risk may mean implementing that Web application firewall internally as well as externally.

While I was glad to see that unpatched RDBMS software low on the top 10 list, I was very disappointed that it was still in the top 10.  One would think with all of the discussions about the importance of patching software, this would not occur in the top 10.  I understand the issues of compatibility and testing that make patching difficult, but really?  Maybe you need to invest in more than one or two instances of the RDBMS.  This is the cost of doing business the correct way.  If you are not doing things the correct way, then do not complain when you have a breach.  So while you saved yourself money on licensing costs on the front end, you likely paid for that cost savings a hundredfold on the back end.

I also understand the issues and fears with encryption.  For a lot of people, encryption is this mystical science that only certain “geeks” practice and practice well.  For others, the problem with encryption is the perceived loss of ready access to their data.  As time goes on, I would say that unencrypted data will rise to the top of the top 10 list.  Why?  Because the information age is all about the control of information.  The more information you control and can use to your advantage, the more power and control.  If your information can be readily obtained through public sources or the lax security surrounding your information systems, then you have little, if any, power or control.  The next 10 years will likely be spent by most organizations figuring out what information is critical to their business model and implementing the necessary protections around that information.  Critical information will be protected like the gold at Fort Know because, to that organization, that is their “gold” and it must be protected accordingly.  And that protection will likely involve encryption for some or all of it.

I know that people have a lot on their plates these days.  However, if you are a security person or a DBA, you need to leverage these surveys to your advantage and address the top 10 issues.  If more companies did this, the less data that would be breached.

Cannot Say It Better

If you read nothing else this week, you need to read this posting by Daniel E. Geer, Jr., Sc.D.

People in the Loop: Are They a Failsafe or a Liability?

The MPLS Privacy Debate Continues

At this year’s PCI Community Meeting the issue of whether or not MPLS is private came up again.  I was in one of the Open Forums when the topic of MPLS and whether it was private came up.  This is not a new issue as it has come up before and I have also discussed it in two previous postings.

The reason it came up was that a former network engineer wanted to understand from the PCI SSC technical representatives how they justify MPLS being private.  What ensued was an excellent discussion regarding the architecture of MPLS and the PCI SSC’s rationale for considering it private.  For those of you not familiar with MPLS, in a nutshell, MPLS is just a larger IP network used to route customers’ network traffic over an IP network.

What the network engineer brought up was the fact that an MPLS network is no different from any other IP network with spanning tree and other architectural issues that hardly make MPLS private.  They also brought up the fact that even with Frame Relay and other older telephony technologies, those circuits are also being sent over MPLS by the carriers.  Given that at some point MPLS traffic has to technically co-mingle with other customers’ network traffic, how can the PCI SSC stick to its claim that MPLS is private?  The answer provided was a bit disconcerting to some in the room.  But for those of us with an understanding of the engineering issues related to MPLS, it was expected.

The group present was told that MPLS is considered private because the carriers consider it private and it is sold as a private network service.  A lot of people in the room gasped and the next question asked was, “Isn’t that a lot like saying trust me?”  As the PCI SSC representative continued to explain, there really is not another way to work with MPLS.

Is it possible to breach data in an MPLS network?  Yes.  Can it be easily accomplished?  Not really.  The attacker would have to have access to a carrier’s core switch and have a port or two in promiscuous mode to gather all of the packets flowing through that switch.  As a result, organizations need to accept the risk presented by MPLS.  The unfortunate fact is that most organizations do not even know there is a risk however slight it might be.  At the end of this discussion, the PCI SSC person recommended that, if an organization is concerned about the privacy of MPLS, then they should encrypt their data over the MPLS network.

So, there you are.  If you think MPLS is not private, then encrypt your data.  Hopefully this issue is resolved.

Other relevant posts:

The ‘MPLS Is A Private Network’ Debate

An Update On The MPLS Privacy Debate

Visa Is Upset

It seems that I ruffled some feathers at Visa Inc. with my post regarding their program to incentivize adoption of EMV in the United States.  Since I irritated another vendor today, I thought why not make the day complete and irritate another vendor?

As a result of my “A Carrot for Chip and PIN” post, I was contacted by Visa’s public relations firm requesting that I correct my post to properly characterize the program.

“My client, Visa Inc., requests a correction to a factual error on your PCI Guru blog: “A Carrot for Chip and PIN” (http://pciguru.wordpress.com/2011/08/13/a-carrot-for-chip-and-pin/).
While the initiative is certainly aimed at promoting the use of EMV chip, it is not aimed at promoting PIN, per se.  Hopefully, the following post on the Visa corporate website will provide clarification, but please feel free to contact me if you have questions: http://blog.visa.com/2011/08/26/pin-largely-unaffected-in-u-s-migration-to-emv-chip-2/
Many thanks in advance for correcting the story!”

As requested, I went and read the Visa blog entry.  This blog entry is regarding the fact that PIN usage was not being affected or required by the new program.  Apparently a major industry media outlet had implied that Visa was pushing for not using PINs which is not the case.  However, if you read my posting, I do not reference anything regarding PIN usage.  As a result, I asked the PR person to clarify what the problem was with the post.

“I guess I’m a bit confused about your request for a correction
EMV is known as “Chip and PIN” everywhere around the world.  My post does not discuss PIN usage only that Visa is promoting “Chip and PIN” as a card format as well as the RFID contactless card.
I’m always willing to make corrections, but is what Visa is requesting is that I not use the terminology “Chip and PIN” and refer to it only as EMV?”

To which, I received the following reply.

“Yes, it would be correct if you just removed the references to PIN. While signature is the most common form of authentication uses with chip around the world, some regions such as the UK have so popularized the term chip and PIN that it has virtually become one word.
So yes, it can correctly be referred to as a move to “EMV chip” or just “chip” if you prefer.
Many thanks!”

At first blush, this seems to be a very petty argument as to why I need to change my blog post.

But whoa!  Signature is the most common form of authentication with EMV cards around the world?  So, what is the point of having EMV if signature verification is still used?  I have always been told that the whole point of EMV was the coupling of the chip technology with the personal identification number (PIN).  The only reason signature is the most common authentication method is because, outside of Europe, Ireland and the UK, no one has the infrastructure on a large enough scale to process EMV with a PIN.  That is the whole reason Visa is trying to push EMV and contactless is to broaden its use.

Basically, from my interpretation of this response, I was accurate in my original post when I stated that Visa thinks that removing the PCI ROC requirement is enough to drive merchants to implement EMV or contactless terminals.  How could that be when it would take most merchants 10, 20 or even more years of ROC cost to equal the cost of replacing terminals?  Just how does an organization justify such an expense?  Particularly since the other card brands have not agreed to support this program.

But the other thing that disturbs me about this response is that Visa is upset with the use of the term Chip and PIN.  Never mind the fact that Visa uses the term Chip and PIN on their own Web sites around the world as a reference to EMV.  As well as the fact that Chip and PIN is essentially being synonymous with EMV.

So I respond to the PR person.

“I have reviewed my post (http://pciguru.wordpress.com/2011/08/13/a-carrot-for-chip-and-pin/) against the post on Visa USA’s Web site (http://blog.visa.com/2011/08/26/pin-largely-unaffected-in-u-s-migration-to-emv-chip-2/) and I fail to see why any correction is necessary.
The post from the Visa blog references the fact the [media outlet] stated that the PIN was being dropped in the move announced in http://usa.visa.com/download/merchants/bulletin-us-adopt-dynamic-authentication-080911.pdf.  The Visa blog post goes on to further clarify and define the fact that PINs will still be used.
My blog post says nothing about the PIN being used or not used.  My blog post is about business reasons why such a program are not going to be a reason for US banks or US merchants to move to EMV.  As I reread my post, other than the fact that I used the term “Chip and PIN” in the title and then as a “aka” reference for EMV in the first paragraph, the remainder of the entry refers to the card by EMV or the dual chip terminal.  As a result, I fail to see the need to make any changes to the post as the post has no relevance to the Visa USA blog post other than they both reference the aforementioned Visa program to promote EMV in the US.
If Visa USA does not like the use of the term “Chip and PIN” then I suggest that Visa USA take that matter up with the UK and Irish banks that created it more than a decade ago.  The fact that EMV and “Chip and PIN” are now synonymous with each other is also an issue that I am not responsible for nor will making any change to my blog entry effect.
If there is anything else I can assist you with, please let me know.”

The PR person responds.

“EMV is not synonymous with chip and PIN. The EMV standard specifies a number of cardholder verification methods including signature, offline PIN, online PIN, and no verification. Also, while you may possibly be most familiar with chip and PIN implementations in the UK and Ireland, in fact the majority of global implementations of EMV chip have been with signature. Citing chip and PIN in the headline implies that every chip transaction would be verified with a PIN (as they are in the UK and Ireland), which in the U.S. is incorrect, and I know you want to avoid factual errors.
Thanks again for your consideration of this request. Please consider me a helpful resource on future security matters in which Visa Inc. may be a good fit for your story.”

While I understand the PR person’s point, let us face facts.  Google Chip and PIN or EMV and the other term comes up in the results.  If that is not the definition of synonymous, I do not know what is.  Visa’s beef with my post really is the implied connotation by using the term ‘Chip and PIN’ in the title that a PIN would be required.  Whereas, all I was trying to do was to provide an easily Google-able term for people interested in EMV since EMV is usually referred to as Chip and PIN.  Such a complaint is laughable if it were not so sad.

Then to bring up offline PIN entry when it has been repeatedly shown to be the biggest reason why EMV and contactless with PIN can result in card present fraud is amazing and just shows the limited knowledge this individual has regarding their client’s products and services.  But to add insult to injury, they then bring up the wonderful fact that EMV and contactless can also be used with no authentication.  Not that I think anyone would actually do this, but it is an option.

However, the issue of not using the PIN along with the chip truly comes through in this response.  In my very humble opinion, the fact that Visa actually believes that pushing EMV without the PIN is just hysterical.  What is the point?  And this response actually confirms that I was correct in what I stated in my original post and is why I wrote the original post in the first place.  Given the current state of affairs, there is no business reason for EMV or contactless if PIN is not part of the equation.

But this incentive program does nothing to address the even larger issue that merchants and banks face which is the one of card not present fraud.  Card not present fraud is growing at a 20% to 35% clip depending on the survey you read from wherever in the world and comprises more than 50% of total card fraud.  If Visa really wanted to make a difference and give merchants and banks a reason to push for EMV and contactless adoption in the United States, they would gather the various stakeholders together in e-Commerce and come up with a common API that would allow EMV and contactless work online.  That would rein in card not present fraud and would truly create a business reason for investing in EMV and contactless capability.

As it is now, EMV and contactless are solutions looking for a problem.

PCI SSC Releases Virtualization Guidelines

On Tuesday, June 14, 2011, the PCI SSC released an Information Supplement regarding Virtualization Guidelines.  Not only does this Information Supplement cover virtualization from a VMware and Hyper-V perspective, but also goes into cloud computing.

The supplement is broken into six sections:

• Introduction
• Virtualization Overview
• Risks for Virtualized Environments
• Recommendations
• Conclusion
• Virtualization Considerations for PCI DSS

The Introduction and Overview sections are good foundations.  But if you have a good knowledge of the concepts of virtualization, I would not waste time reading these sections.  The Risks section is a very good discussion of the risks presented by virtualization.  However a lot of readers of this supplement are likely going to be disappointed as there is little new material covered in this section that has not been discussed before in other information sources or even my blog entries.  In my opinion, the Recommendations section presents what would be expected.

The real gem in this supplement is the Appendix that provides the Virtualization Considerations for PCI DSS.  This supplement takes the relevant PCI DSS requirements and provides a lot of guidance regarding what QSAs should consider when assessing virtual environments.  In a number of these, there are also some additional best practices and recommendations made by the writers of the supplement.  In reading these best practices and recommendations, one would think these would be common sense, but I guess you just cannot assume that any more.

Page 23 has the other great gem in a diagram that graphically represents the responsibilities of cloud customers and cloud providers regarding who is responsible for data, software, user applications, operating systems, databases, virtual infrastructure, physical infrastructure and the data center where everything resides across the three types of cloud services; IaaS, PaaS and SaaS.  If you are explaining cloud computing to non-technical people, this is probably one of the best diagrams I have seen to explain responsibilities.

If I had to take the PCI SSC to task on anything, I would argue that cloud computing does not necessarily have anything to do with virtualization.  Yes, a lot of cloud computing solution providers are using virtualized systems to provide their services, but not every cloud provider uses virtualization.  And even if the cloud provider does use virtualization, why is that the customer’s concern?  In my opinion, cloud computing should be an entirely separate document.

I have included below links to all of my prior posts on virtualization for reference.

• PCI and Virtualization
• Server Virtualization and PCI
• Cloud Computing and PCI Compliance
• More on “The Cloud” and PCI Compliance

VoIP And PCI Compliance

I have had some interesting conversations with people lately regarding voice over IP (VoIP).  It fascinates me as to how little people really know and understand how this technology works.  But what really scares me is how this lack of information is putting organizations at risk.

The most obvious problem with VoIP is segmenting it away from the cardholder data environment (CDE).  I am really disturbed by the number of organizations that have no security around their VoIP.  Yes a lot of organizations have segmented the VoIP from the rest of their network, but there are no controls that stop anyone from getting into that network segment.  As a result, anyone with the right set of tools can gain access to the traffic in the VoIP network segment.

The next thing that scares me is the lack of security surrounding the VoIP servers including any call recording servers.  People treat these VoIP servers just like their traditional PBX.  Unlike their PBX that likely ran a proprietary version of UNIX, VoIP servers are just Windows or Linux servers running a VoIP application.  As a result, they are susceptible to all of the viruses, malware and everything else any other server is susceptible.  However, these servers typically do not run anti-virus (performance issues) or are they hardened to any rational security standard.  When they get infected or hacked, it seems to be a shock to the system administrator.

And what about the call recording technology?  We keep hearing from vendors of call recording solutions that they use proprietary recording methods requiring special CODECs.  While in some instances the proprietary claims are true, what we are finding more and more is that vendors are just manipulating file header information such that Windows Media Player, iTunes and the like do not recognize the file as being in a valid format.  However, tools such as VLC Media Player are able to see past the header changes and recognize these files for what they are, WAV, MP3 and the like.  Thus some proprietary formatting claims are all a bunch of smoke and cannot necessarily be relied upon for security or privacy.  Another tell on the proprietary nature of call recordings is that, when you “convert” a recording, if the conversion software seems to be copying the recording more than actually converting it, it is likely that the header is being fixed to WAV, MP3, etc.  Real audio conversions typically take more time than just copying because the file has to be fully processed.

But the final insult in this whole scenario is the lack of understanding security personnel have regarding the VoIP protocols.  While VoIP call setup and teardown are usually conducted over TCP/IP (a stateful protocol), the actual call itself is conducted via streaming protocols over UDP/IP (a non-stateful protocol).  As a result, when you start talking to security people about VoIP security, their knee-jerk response is to tell you that VoIP is secured by the corporate firewall.  However, given that the VoIP protocols are stateless, even behind a firewall really does not provide any protection.

So you have a VoIP solution in place.  What should you be doing to ensure its security if it is in-scope for PCI compliance?  Here are my thoughts.

• Properly segment your VoIP from the rest of your network.  This means either physically or logically separating the VoIP from the rest of your network.  This also means implementing access control rules so that only those devices and people that need access to the VoIP network have access.  If you are also using your VoIP phones as the network jack for a PC, make sure to VLAN that jack to something other than the VoIP VLAN.
• If you can, implement the VoIP segment without DNS and DHCP and use MAC filtering to avoid the accidental or deliberate plugging in of a PC into a network jack that is VoIP only.  At a minimum, use MAC filtering to at least control what gets plugged into the LAN jack.
• Closely monitor your VoIP segment and generate alerts on any devices that are unplugged or plugged in.  Also monitor for any protocols other than the VoIP protocols that your VoIP system uses.
• Do not use the last octet or any other portion of the phone’s IP address as the extension number.  Yes, I know this is an easy way for the help desk to identify and troubleshoot phones, but it is also easy for an attacker to locate targets of interest, so keep that in mind when you are implementing your VoIP solution.
• Never, ever connect your VoIP network to another VoIP network outside of your explicit control.  Given that VoIP primarily uses UDP/IP, you cannot expect any firewall to protect your VoIP system from anything outside of your control.  Always use plain old telephone system (POTS) circuits to connect to any foreign network.  I know that is not as sexy as VoIP, but how else can you protect your VoIP system from outside influences?
• Work with your VoIP vendor to harden all servers that manage the VoIP system.  These are just Windows, Linux, etc. systems.  Obviously you will need to do some testing of this and you may not be able to use all of the hardening items in your server hardening standard, but you would be surprised at what you can do that the vendor says will not work.  Remember, they are just trying to cover their butts should a problem crop up.
• Be careful implementing VoIP on traditional PBXs.  A lot of these solutions are just PCs or servers and can be easily hacked once on your network just like their full VoIP brethren.
• Get a hold of VLC Media Player or similar tool and see if you can play a recording off of your call recording system.  We are getting about a 25% hit rate using VLC to play recordings.  A lot of the success of this approach depends on the age of the call recording system.  The newer the systems, the more likely it is that you will find that the recordings are just tweaks of existing standards.

Anatomy Of A Breach

People are always asking me why complying with the PCI standards is important as in, “What’s in it for my company?”  So I thought I would take a known, documented breach and walk through where PCI compliance would have made a difference.  And for those naysayers that point to the PCI DSS and say that compliance does not matter, I intend to show that compliance does lead to security.

The breach I am going to use is the Wal-Mart breach which was documented in an article in Wired magazine back in October 2009.  Wal-Mart has what most professionals would consider a robust control environment.  However, what this breach shows is that even with such an environment, a breach can still occur.  That is not to say that Wal-Mart did not make mistakes and it is those mistakes that I want to point out so that we can all learn.

For some background, the Wall-Mart breach occurred sometime between 2005 and November 2006 when it was discovered by Wal-Mart.  The good news, at least as far as Wal-Mart has ever publicly shared, was that no cardholder data was ever released as a result of the breach.  However, the final report issued internally by Wal-Mart was never shared outside the company, so it is anyone’s guess as to whether the claim that no cardholder data was ever released is accurate.  This was not Wal-Mart’s first cardholder data breach.  In the Fall of 2005, a small number of Sam’s Clubs gas station systems were accessed by intruders and around 600 credit card accounts were believed to have been compromised.

The breach was discovered by accident when a server crashed.  During the investigation to figure out what had happened to the server, one of the investigators found that L0phtcrack had been installed on the failed server and that it was L0phtcrack that had caused the server to fail.  Obviously, L0phtcrack was not an approved application to have installed.  As a result, this information caused an even larger investigation to be launched.

Before we discuss L0phtcrack, let us discuss file integrity monitoring.  This incident points out why the PCI DSS mandates file integrity monitoring in requirement 11.5.  But just monitoring known files is not enough.  This is where an organization needs to be going above and beyond in order to better ensure their security.  While monitoring critical files, you also need to be monitoring any new files that might be added to a system.  And alerts generated by your file integrity monitoring system need to be reconciled to all changes being made to the systems.  Any file addition, change or deletion not documented in changes needs to be investigated to determine its cause.  Based on the timeline, while Wal-Mart may have had critical file monitoring going on, it either was monitoring only a limited number of files and directories, not monitoring for new files and/or any alerts were not being followed up in a timely manner.

Then there is a topic not even mentioned in the PCI DSS but just as important.  Root Cause Analysis (RCA) is something that everyone should conduct in the event of a failure and needs to be an activity conducted as part of an organization’s incident response process.  Because of their RCA process, Wal-Mart found that L0phtcrack was the cause of the server failure.  Since L0phtcrack was not an approved program and was likely installed by an attacker, Wal-Mart personnel broadened their investigation to determine if L0phtcrack was installed on other systems.

While L0phtcrack should be an obvious program that should not be installed, it is not always that easy.  This is why requirements 2.2 and 12.3.7 are important, so that when doing an investigation, the investigators know what to expect to see installed as well as what was approved so that they can quickly determine if the server was running approved software.  Again, I am certain that L0phtcrack would not have been part of those standards.

That even larger investigation led to Wal-Mart to determining that over 800 systems and servers had been compromised or attempted to be compromised.  The compromise was traced back to a remote access VPN account that was used by a former Wal-Mart employee in Canada.  That account had been used by the intruder to enter Wal-Mart’s network and begin the compromise of their systems.  While investigating the breach, Wal-Mart personnel suspended that account and the intruder moved over to another terminated employee’s account.  When they disabled the second account, the intruder moved over to a third terminated employee account.

Requirement 8.5.4 states that accounts for terminate employees should be disabled or removed immediately and this was obviously not followed in this case.  Requirement 8.5.5 states that inactive accounts should be removed if not used for 90 days or more.  Unfortunately, we do not know if any of the accounts had been inactive for more than 90 days.  We also do not know if any of these accounts were disabled.  However, in such a breach, if the attacker has any sort of administrative access, it takes almost no time to activate a disabled account.  That is why an organization needs to remove those accounts as soon as possible, particularly any account that might have administrative privileges.

The investigation quickly focused on one particular Wal-Mart system, point-of-sale (POS).  Documentation from the investigation indicates that the intruder(s) were very focused on POS source code, executables, databases and documentation.  The intruder(s) were so focused on POS, that they even downloaded the latest technical specifications for Wal-Mart’s POS system.  As a result, investigators focused much of their efforts on POS systems at store locations and at corporate.

If not already obvious, investigators inspected log files to determine that the compromise went at least as far back as June 2005.  If you want to have a concrete example of why log information and proper time keeping are important and requirement 10 is so focused on log data and time setting, there is no better example than this breach.  Thanks to an obviously large retention of log data, Wal-Mart was able to at least figure out when and where the breach started as well as trace the actions of the intruder(s) through their network and systems.  It is implied that the time settings on servers and network devices must have been fairly closely synchronized as it is never mentioned if there were time correlation issues in the log data.  Had Wal-Mart had to rely on system and event logs that were contained only on the network devices and servers, the when, where and how of this breach might have never been known.

Unfortunately, the log data was not as complete as it could have been.  As a result, the Wal-Mart investigators were somewhat stymied in their efforts to better understand the breach.  Server logs were only configured to log unsuccessful logon attempts.  As a result, they were not able to track the successful logon attempts of the disabled accounts that were being used by the intruder(s) and therefore trace the actions of the intruder(s) through their network and systems.  A lot of administrators save log space on internal systems by not logging all activities.  I am also guilty of doing this as I also used to believe that successful attempts internally were not a big deal to miss.  However, as the internal threat has become more and more prevalent, I have changed my opinion and now I log everything I possibly can on all systems.  Requirement 10.2.5 implies that logging of all authorization and identification mechanisms are logged, but it does not specifically call out that successful and unsuccessful attempts are to be logged.

The saddest fact of all was that none of this should have been a shock to Wal-Mart IT and security personnel.  Almost six months prior to discovering the breach, Wal-Mart’s QSA had completed their PCI assessment and had found numerous areas where Wal-Mart was not compliant.  A lot of the areas of non-compliance were the direct result of how the breach occurred.

So what are the lessons that should be learned from this incident?

• Compliance does matter and does result in security.  I do not care whether you follow the PCI DSS, FISMA or any other well known security standard.  The purpose of all security standards is to provide guidance on how to secure hardware and software so that it is difficult to compromise.  If you comply with any of these standards, you greatly enhance your security posture.  However, the best security comes down to more than just complying with a standard.  If an organization really wants to be secure it will have to go beyond just what the standard requires.
• Security is not perfect.  The purpose of any security programs is to limit the damage of incidents when they occur so that they do not get out of control.  All we can expect to gain out of a security programs is minimizing the potential risk that an incident results in a breach of sensitive information.  A good friend of mine has a great quote on this point.  He always likes to say, “I just want my security program to be sufficient enough that it makes everyone else an easier target than my company.”  What security standards let you have is the information you need to know where the bar is set so that you can make investments to do that little bit more.
• Most breaches are discovered by accident.  It has been my experience that even with great tools and instrumentation, the discovery of a breach or compromise all comes down to the uncovering of information that results in someone becoming curious and digging further into the incident and discovering that systems and/or data have been compromised.  This is not to say that monitoring and alerting is not worthwhile.  It is just that it is very rare that a breach or compromise is uncovered when the initial alert was issued.  It takes follow up on all of the alerts to actually uncover the breach or compromise.
• Follow up should be the standard for all alerts and a documented Root Cause Analysis (RCA) process should be followed as part of an organization’s incident response plan.  This is where most organizations get sloppy and miss the signs of a breach or compromise.  They do not treat all alerts consistently,  do not perform the RCA process every time and therefore earlier warnings go undiscovered until the situation gets truly serious such as when a production server crashes.
• If you do not have at least a year’s worth of log data, you are probably going to be in the dark about how, when and where the compromise occurred.  There is a lot of push back from organizations about hanging onto log data, particularly more than three months worth.  A lot of this comes down to the cost of storing such a huge amount of data.  However, had Wal-Mart only had three months worth of log data, they never would have known when they had been breached nor the focus of the breach.
• What gets logged is also very important.  Wal-Mart’s breach would have been a bit easier to investigate had the log data been complete.  Just because you are on the inside of the network is not an excuse to not log everything.  As I have pointed out before, log data is IT’s version of a commercial airliner’s flight data recorder.  Without all of the data, it can be almost impossible to isolate the cause of a compromise.
• As soon as employees and contractors are terminated, they need to be removed from the access control system.  I know that this can cause issues with some operating environments, but there are work arounds to avoid those complications.
• And finally, there are no easy ways to ensure security.  Security requires diligence.  Extended diligence typically results in tedium which then results in diligence faltering.  As a result, organizations interested in maintaining their security need to combat tedium by rotating security and operations personnel through positions so that tedium does not set in.  This has an added benefit in improving cross training of personnel.

Can You Trust Your QSA?

A great question that gets asked often these days.

It is also the title of a great article in the November 2010 issue of Digital Transactions that documents the dilemma faced by QSAs and merchants regarding PCI compliance.  Give it a read and take a walk in a QSA’s shoes.