Compliance, Compliance Testing and Security

I was recently on a Webinar presented by a major security vendor and one of their points was that executive management is finally starting to realize that compliance does not equal security.  If you read this blog regularly, you know I really do not like the phrase “compliance does not equal security” and I view it as a convenient dodge by those who use it as a way to weasel out of their responsibilities.

But during this Webinar I had an epiphany regarding this topic.  It is the confusion between security, compliance testing and reporting and the act of compliance by your technology, employees and business partners with your organization’s security policies, standards and procedures that is the problem.

I know I am just asking for flame mail with this post, but I am so tired of people looking to blame everyone but themselves about their inadequacies surrounding information security.  As I have done before, to paraphrase Tom Hank’s character in ‘A League of Their Own’, “There’s a reason security is hard.  If it wasn’t hard, everyone would do it.”

Security is not always easy, particularly when upper management does not have buy in.  But even when upper management supports security efforts, I have seen security personnel not take advantage of that fact and get the job done.  Security does not have to be hard, but it does take more than just slamming some firewalls and intrusion prevention gear down, tossing a SIEM into the mix and thinking you are done.  Security is a never ending journey because someone is always coming up with new ways to attack you.

Anyway, to start off, let us take a look at some definitions first so we are all on the same page.

Compliance is defined as:

“Conformity in fulfilling official requirements.”

“Official requirements?”  Could that possible mean your organization’s security policies, standards and procedures?  You bet.  In this instance, we are talking about those that correspond to the PCI DSS, but this also applies to ISO 27K, FISMA, HIPAA, GLBA or any multitude of frameworks and regulatory requirements.

Conformity is defined as:

“Compliance with standards, rules, or laws.”

Based on these definitions, security is all predicated on complying with what are deemed an adequate set of security policies, standards and procedures.  Conversely, if you are not complying with an adequate set of security policies, standards and procedures, then your organization cannot be as secure as it could be.  As a result, compliance has to equal security as long as the security policies, standards and procedures are considered adequate.  Therefore security professionals that quote the mantra, “compliance does not equal security” either have a problem with the compliance side of the equation (most likely) or with the standards/frameworks (the dodge).

Over the years there have been a lot of discussions about the PCI DSS, ISO 27K, FISMA and other security frameworks and whether or not they are adequate.  The important thing to remember is that all of these standards or frameworks are merely ante into the information security game.  They are the bare minimum or a baseline to get to a basic level of security.  Should you being doing more?  Definitely, but what those efforts beyond the standard/framework are depends on what you are trying to secure, your network and application architectures and a multitude of other factors related to your computing environment and how it is used.  Those are factors that cannot be taken into account by any standard/framework because they would start to become impossible for others to follow and implement.  The bottom line here is that if you want someone to tell you exactly what to do to secure your networks and applications, go hire a consultant you trust and they will tell you everything you want to know.

The rub in all of this is that, based on the breach reports from Verizon Business Services, Trustwave, et.al. as well as compliance testing reports I have reviewed, none of you out there are 100% compliant to begin with, let alone even close.  Every organization I am aware has problems complying with the basics, let alone with any advanced security requirements in the published standards/frameworks.  So if you cannot comply with what you already have, explain to me how a different framework is going to change that fact unless it is less stringent than the framework you are already trying to use?  And if that other framework is less stringent, while that may solve the compliance issue (which I seriously doubt), exactly how is a less stringent framework going to make you secure?  The answer is that it will not make you secure.

What security professionals struggle with is that compliance is a never ending, 24x7x365 effort.  Drop your guard for an instant and it can be game over.  But provided your security policies, standards and procedures are appropriate and detailed (the reason why you want to use an appropriate standard/framework), your organization is not as secure as it can be unless your personnel and devices comply 100% of the time with every defined security policy, standard and procedure.  If you want confirmation of these facts, again, just look at the breach analysis reports year after year.  The reason there are breaches is because of non-compliance with one, but usually more, of an organization’s security policies, standards and/or procedures.

This brings me to the rumblings of late regarding a rethinking of defense in depth.  Defense in depth is predicated on using layers of security devices and controls to minimize the risk that a security incident occurs not to completely prevent an incident although you might get lucky.  For example, firewalls are the sledge hammer of security tools.  However, because we need to have ports open for outsiders to access applications, we follow our firewalls with intrusion detection/prevention devices to ensure that no one abuses the protocols used by the ports.  We follow that up with monitoring of log data from the firewalls, IDS/IPS, routers, switches and servers to identify any “sneaky” attacks using the protocols we allow.  The layers are there to cover the various holes we need to have in order to make our networks and applications function.  The tighter and smaller we can make those holes, the more secure we will be, but there will still be some amount of risk.  So we bring in more layers to cover those risks until it is more expensive to address the risk than to accept the risk.  That remaining risk is the residual risk that we therefore manage and control through detection and correction.

The other thing defense in depth relies on is the control triad.  The idea being that, because you cannot entirely prevent every security incident, you need a way to detect the incident so that you can take action to stop or minimize the impact of the incident.  You follow that up with periodic assessments of your control environment to identify and correct any deficiencies or improve your program based on new information regarding security.  The follow up assessments can be activities such as a root cause analysis (RCA) of an incident, an internal audit of user accounts and user rights or brining in a network security team to assess your security architecture and controls.  All of these activities will result in findings and recommendations to make your security systems and controls better.

And that brings us full circle to the PCI assessment.  It is merely a tool used by the acquiring banks, card brands, processors and others to obtain reasonable assurance that your organization is doing what it can to minimize the possibility of a breach of cardholder data.  It is not meant to be, nor could it ever be, an absolute complete assessment of an organization’s security posture and therefore provide absolute assurance that a breach will not occur (even though the PCI SSC and card brands tend to imply that fact).  Compliance assessments are only a snapshot of personnel and device compliance at the time the reports were written.  This is no different than going to the doctor for your annual physical which results in a snapshot of your health at that point in time.  It is not that those compliance reports are worthless; they just need to be referenced and used properly based on the fact that they are a snapshot.  Just as your doctor will tell you to lose weight or stop smoking, compliance reports provide recommendations on where you can make improvements or adjustments in your policies, standards and procedures based on what compliance evidence was found, or not found, during the assessment.

So, what are the lessons to be learned?

• Security is not and never will be perfect; there will always be residual risk that must be managed and controlled.
• Compliance does equal security, at least as best as your preferred standard or framework defines it plus whatever enhancements you have made.
• Compliance assessments and reports point out where your organization was not compliant and needs to do better, not to prove your organization is secure.

Use the tools at your disposal correctly, stay current on threats and monitor your security posture and you will likely live a long, prosperous and secure life.

Keep hiding behind “compliance does not equal security” and you will forever be living off of your “luck” until it runs out (usually sooner rather than later).

Security And Compliance

I have written a lot about this topic over the years and was recently reviewing my Compliance Is Not Security – Busted! post and the comments that came in regarding it.

A theme of a number of the comments was that compliance does not equal security.  DUH!  I have never once said or even implied that compliance equaled security as – yes, here it comes – security is not perfect!  However, if you are complying with any security program/framework such as the PCI DSS, ISO 27K, etc., then you are likely more secure than those who are not.

Security technology such as firewalls, routers, servers, applications, etc. can all be set up with rules that are complied with 100% of the time, day in and day out, no exceptions.  The problem comes down to people who are fallible.  Their compliance is never 100% and you are probably lucky to have anyone above 90%, no matter how much security awareness training you do.  As a result, in organizations that are truly complying with the PCI standards, this is where the security breach starts, with people for one reason or another.

No, I am not necessarily talking about social engineering, although social engineering is growing because of the fact that organizations have invested a lot in security technologies yet people are fallible.  People can be the root cause because of any or all of the following.

• How dare you do that to me!  This is the most obvious of the people issues that comes to mind.  Face it, when backed into a corner, people lash out just like a trapped animal.  The supposedly wronged party wants their proverbial “pound of flesh.”  They get that pound of flesh by hurting the organization that has just hurt them.  This can be as minimal as taking office supplies to downloading databases to a USB drive as they empty their desk.  Obviously, a database, network or system administrator’s access is much different than a clerk’s.  However, if your security is minimal on the inside as it is in most organizations, the clerk may actually have better access than the administrators when it comes to sensitive information.  Such a situation may not be the fault of the administrators, that old version of POS or ERP may not have the ability to be more granular regarding access to information.
• Over inundated with alerts and cannot identify real alerts from false positives.  This typically occurs when an automated tool is implemented but never tuned to the organization’s environment.  In this sort of an environment, finding real alerts can be like finding a needle in a haystack when there are thousands of alerts an hour scrolling by on the screen.  This usually makes management wonder why the tool was needed in the first place.
• Saw an alert and ignored it.  We see this most often coupled with the aforementioned inundation issue.  The other most common version of this issue is with internally used SSL certificates that were generated incorrectly or use a default certificate supplied by the application.  Users then see the “There is a problem with this Website’s security certificate” or similar error message in their browser whenever these flawed certificates are encountered and become conditioned to ignore the error message.  Over time, they become conditioned to ignore all of these sorts of messages, including those for malware infected Web sites and, surprise, you have been compromised.  I have lost count how many people have said to me, “We just ignore those alerts because we know they are false positives.”
• Saw the alert but got side tracked and never came back to it.  This is a problem we see all of the time.  For example, the person that monitors the network is also the person that manages the network and configures the network.  An alert comes in and the person begins a root cause analysis (RCA) only to get pulled away because a remote facility is offline.  The offline issue gets resolved, but other issues come up as well as meetings and telephone calls and the person never gets back to the RCA for the alert because there is no “tickler” to remind them to go back and complete the RCA.  In the meantime, the attacker has gained their beachhead and is probing the network for whatever value it may contain.
• Just did not put together all of the pieces to know they were compromised.  Like the reasons 9/11 occurred, most organizations do not correlate all of the potential incidents occurring in their networks and therefore do not understand that there is an active effort to compromise their network or that they have already been compromised until well after the incident has caused damage.  The reason this is important is that once an attacker is inside your organization’s security perimeter, it is typically game over because there are few controls to prevent access and identify that data is being taken.

If you have read the Verizon Business Services Data Breach Investigations Reports (DBIR) over the years you know how the bulk of attacks get inside, they are the result of people.  For the last two years, the DBIR has used the VERIS Event Threat Grid to show how breaches occur.  Across the top of the grid are the categories; Malware, Hacking, Social, Misuse, Physical, Error and Environmental.  The Social, Misuse and Error categories imply mistakes or deliberate acts of people.  If you read the definitions on the VERIS Web site, Malware is also very people centric as is hacking.  Surprisingly to some will be that the Physical and Environmental categories also have a good number of people errors.  Based on just a quick read, it looks to be that about 60% to even 70% of all of the incidents categorized by VERIS has some form of people error component.

Since we are not going to get rid of people in our organizations any time soon, what are you to do?

• Admit that people are the problem and focus your security measures accordingly.  Every 12 step program says the first step is to admit the problem which, in this case, is that people are fallible.  As a result, we need to construct our security measures such that this fallibility is minimized as much as possible.  One of the best solutions is to integrate alerts into your help desk or change management system so that a ticket is generated.  Those tickets need to have an escalation process behind them so that if they are not investigated within a period of time, they are bumped up to the next higher rung of management and that escalation continues until the tickets are finally addressed.  This way there is visibility for the alerts should they slip through the cracks.  As a side benefit of this approach, you gain statistics to reinforce why you need more staff and/or more/better tools.
• Strengthen your internal security measures.  As things stand, once inside most organization’s security perimeter, there is very little that stands in the way of an experienced attacker getting the data they desire.  Regardless of whether it is an insider attack or an attacker has managed to get inside, there is already justification for organizations to beef up their internal security measures.  To address this problem, I would recommend the security architectures as documented in my Fort Knox approach, Forrester’s Zero Trust Model or McGladrey’s Ultra Secure Network.  But most organizations do not have the infrastructure architecture, the application architecture or even the will to take such approaches.  But that does not excuse an organization from just saying they cannot do anything.  If anything, most organizations could vastly improve the monitoring they do on their internal networks.  Monitoring needs to be coupled with reducing the total number of ports that are open between network segments.  Most internal networks do a terrible job of this because of a variety of factors including applications people that cannot tell what ports need to be open to avoiding operational issues by just leaving things open.  Another area of improvement is reviewing user access rights on all systems and applications, not just those in-scope for PCI compliance.
• Constantly tune your alerting system(s).  Just as attack methods are not static, neither are networks, systems and applications.  Changes are occurring all of the time in an organization’s IT environment, yet if you ask the people running the SIEM about changes, nine times out of ten, nothing seems to be changing other than requests to look for a new signature or anomaly.  There is a belief in the SIEM user community that a SIEM’s update process is making the necessary changes in the policies that ship with the SIEM.  To a certain extent SIEM solutions are similar to anti-virus and malware solutions.  However, because a SIEM monitors log data and the log data provided varies greatly from organization to organization, each organization needs to periodically review and adjust their alerting criteria to make sure that it reflects the organization’s operating environment and not just some template from the SIEM vendor.  If an organization is not reviewing its SIEM alerting rules based on the changes made, at least quarterly, then it is highly likely that the SIEM is not alerting properly.
• Establish separate consoles from your SIEM for network, system, security and application administrators.  What a network administrator is looking for is vastly different from what an application administrator is looking for and what any particular group might be looking for to generate an alert.  As a result, to have only one console is really silly and non-productive.  Yet time and again, we see SIEM implementations with just that, one console and everyone being driven by email or SMS alerts.  The people alerted then have to get to the SIEM to find out what exactly triggered the alert and then determine what to do about it.  Having your own console view simplified things by only listing that viewer’s alerts and no one else’s alerts.  This allows people to focus on their problems and not the whole organizations problems.  The idea behind the single console is that if everyone knows what is going on overall, then correlation would occur because everyone sees everything.  While you would think that would be the case, in reality, people just want to fix their problem and move on, not the entire organization.  Which leads to my last point.
• Watch the overall alerting picture so that correlations can be made.  According to most sources, today’s attacks are becoming more sophisticated and multi-pronged in their approach.  For example, while most DDoS attacks are just to be a pain in the posterior to the target and disrupt access to the target’s Web site, there are those DDoS attacks that are used as cover so that people inside are blinded to the real attack(s).  Whether or not the DDoS was a decoy depends on what other events or incidents occurred during the DDoS attack, if your alerting system did its work.  Higher end SIEM solutions can provide basic correlation rules, but most SIEM solutions require the end user to develop those correlation rules.  It is these correlation rules that help organization identify these more sophisticated attacks.  That said, these correlation rules do not have to be very sophisticated.  For example, during a DDoS attack, you really only need to look for malware attacks, failed authentication attempts and other anomalies that would be likely indicators of the DDoS attack being used to mask the real attack.

Is all of this going to address your security issues?  Sorry, not a chance.  None of the above stops all breaches, it merely minimizes the possibility that a breach goes on for months or years.  Hopefully it minimizes a breach down to weeks, days, maybe even hours in some cases but it will never totally eliminate them.  Security is not perfect.

There is a side benefit to all of this and that is it will assist you in doing RCA.  RCA is very effective in getting rid of those nagging operation issues that occur from time to time and mess up the delivery of your organization’s goods and services.  All of the information you collect for security purposes can also be used to find the needle in the haystack that is causing a database to corrupt, a network connection to drop or a server to fail because now you have information as to what was going on that led up to the problem.

The reason an organization is not secure is that there are so many areas of improvement needed that the full control triad is no longer functioning and holes exist that will allow an attacker to operate without the knowledge of the organization.  Until the controls are implemented and operating properly, it will be impossible to determine if they are secure or not.  The recommendations I have made will hopefully give you a better picture of what you face and reacting to issues that need attention before your organization is the next one to be breached.

What To Do About Insiders

The first posting I did on this subject was to provide an understanding that, despite the news stories, the insider threat is a very real threat and needs to be addressed.  However, what is an organization to do?  Employees and others need to have access to certain information in order to get their jobs done.  What steps should an organization take to minimize the insider threat?

First, I need to be very clear about this.  Even when you do all of what I recommend, you are only minimizing the insider threat.  The insider threat can never be totally mitigated.  Insiders must have access to information that the general public or even you business partners do not have access.  As a result, should an employee get sloppy with controls or go “rogue,” you can expect to lose whatever information that person had access.  Remember my mantra – security is not perfect.

I posted some ideas a while back on controls for automation.  Here are my minimum recommendations for manual controls to put into place to minimize the insider threat.

• Management needs to recognize the importance of management controls.  The “tone at the top” really does mean something when it comes to controls.  However, management needs to understand that these sorts of controls are no absolute guarantee of avoiding issues.  Properly implemented, monitored and adjusted as necessary, such a control environment will confirm to the rest of the organization that management believes that controls are important.  If management does not know what to do regarding management controls, then they should consult with a public accounting firm as they are very aware of control environments and can assist in the design of a control environment.

• Preventive controls.  Preventative controls, as their name implies, put in place something to prevent a problem.  A prime example of a manual preventive control is requiring a minimum of two signatures on checks.  The larger the amount on the check, the more people that have to sign off on the check.  Under such an approach multiple people have to collude to defraud the system.  This sort of approach can also be taken for report reviews of inventory, cash on hand and any other metrics that are important to the survival of the organization.  The idea is to ensure that at least two people are involved in these reviews and that they physically sign off on their review and document and start an investigation into any irregularities.
• Detective controls.  As the name implies, detective controls are controls used to detect problems.  Following the example in preventative controls, the other people signing off on a check or reviewing a critical metric report is a detective control.  If the reviewer feels that something is not right with what they are reviewing, they are obligated to notify their immediate supervisor of the issue and ask the submitter to physically document the situation.  Once documented, the reviewer can then either sign off and accept the explanation, or refuse and further investigate.
• Corrective controls.  Corrective controls are those controls used to ensure that the preventative and detective controls are focused on the right problems and are going to be able to be relied upon going forward.  Keeping to the theme, in the event of an irregularity being identified, management should then institute a root cause analysis and determine what caused the situation and make the necessary changes to the preventative and detective controls to ensure that people do not try to circumvent the control environment.
• Hold employees responsible for the control environment.  Management may be responsible for establishing controls, but it is the employees that make the control environment actually work.  Employees should have their key controls evaluated at least annually to reinforce the importance of controls.  In our check example, the people signing off on checks should be evaluated on how many checks with problems are issued by the organization that they were required to sign.
• Solicit control improvement ideas from employees.  The problem most organizations have with management controls is keeping them relevant.  A common example we see is a problem that occurred ten years ago has been addressed by automated controls in a new computer system, yet management continues to require the manual control to be followed.  Most of the time, employees know exactly what needs to be done, but management does not want to recognize that fact.
• Have a third party periodically assess your controls.  In addition to employees providing ideas, organizations should periodically invite a third party, such as their accounting firm, to assess the control environment and recommend changes.  A number of years ago I worked with a large organization where we discovered that the way one of their computer systems had recently been modified, checks could be generated and bypass approvals and oversight.

For those of you that are going to recommend these minimum controls, my heart goes out to you.  The road ahead is likely to be very bumpy and contentious if your organization has a mediocre control environment.

Something to share with management as you push this sort of project is that there are very measureable benefits to implementing controls.  Every organization that I have worked with over the years has found that a byproduct of their controls projects has been fewer customer complaints and fewer employee screw ups.  Avoiding problems or making them smaller and less impactful on customers can add up to serious savings in time and money.

If you have a mature control environment, take a look at how you can make them better, more effective and more relevant.  If you do not have a mature control environment, then take baby steps.  Look to your accounting area as they will likely have the most robust control environment.  Grab one of those accountants and use them to help you look at other areas that may have problems that controls can address.

Best of luck to all of you on your journey.

Compliance Is Not Security – Busted!

You hear this as a common lament from security professionals, “Compliance is not security.”  This remark has always sounded like a dodge to me.  I suppose the reason I think this is that a majority of the people who seem to utter this phrase always seem to have questionable security practices.  But since I hear this complaint so much, I thought I should see if these people have a point.  So like Jamie and Adam on Mythbusters, I decided to see if this saying really held water.

The first thing I did was to get the definition for the term ‘compliance’.  From Webster’s dictionary, ‘compliance’ is defined as:

“Conformity in fulfilling official requirements.”

That definition seems to say it all.  If you are conforming to defined requirements, then you are compliant.  Therefore, whether or not an organization is secure comes down to whether or not the requirements they are complying with are deemed adequate to ensure the security of the organization.  So, it is the standard that is the problem, not the compliance with the standard.  Let us examine the PCI DSS for its completeness in creating a secure environment.

First, the PCI DSS is organized into six domains and twelve requirements.  The six domains are:

• Build and Maintain a Secure Network
• Protect Cardholder Data
• Maintain a Vulnerability Management Program
• Implement Strong Access Control Measures
• Regularly Monitor and Test Networks
• Maintain an Information Security Policy

The naysayers in the group will point to the fact that the PCI DSS seems to be exclusively focused on the security of cardholder data.  While true, I would argue that the following domains are required across an organization’s network to ensure the security of cardholder data.

• Build and Maintain a Secure Network
• Maintain a Vulnerability Management Program
• Implement Strong Access Control Measures
• Regularly Monitor and Test Networks
• Maintain an Information Security Policy

So we only lose the second domain of “Protect Cardholder Data” when we focus on the cardholder data aspects.  There are still five of the domains remaining that are relevant to an organization’s entire network.  And even if those are focused on the cardholder data environment, in order to ensure the security of the cardholder data environment, security would have to exist on the other areas of the network not part of the cardholder data environment.  However, change cardholder data to employee data, contract data, board of director meeting minutes, customer data, product engineering data, and you begin to see that the PCI DSS can provide a framework for all of those sensitive data types as well.  As a result, I would argue that the PCI DSS does offer a reasonable framework for providing security.

Now, just so you know I have not taken a big gulp of the “PCI Kool-Aid.”  Is the PCI DSS the perfect security framework?  Not at all.  For that matter, there is no security framework that has been developed that is 100% perfect.  They all have their own flaws and problems, but for the most part they do an admirable job in securing an organization if properly implemented, monitored and tweaked.  But let us be clear, there is no such thing as a perfect security framework because as I have said time and again – wait for it – security is not perfect.  For those of you that are implicitly selling security to your management as perfect need to stop it.  You are doing the security profession a disservice and are likely cutting your career short as well.

The real reason I think a lot of security professionals regurgitate the saying; “compliance is not security” is that they are confusing compliance with consistency.  True security requires 100% consistency in the execution of procedures.  However, not all security can be totally automated and some security practices must involve people.  Unfortunately, people are fallible, so 100% consistency is not possible when people are involved.  Because we know people are fallible, we develop our security practices such that there are layers to protect us from the occasional missteps of people.  Those layers are structured around the control triad of protective controls, detective controls and corrective controls.  I have written on the control triad before, so I will not bother to explain these principles.  Needless to say, if your controls do not have these three components, then you are likely not as secure as you might think.

Because at some point people are involved in any control environment, compliance programs are developed to test an organization’s controls over a period of time to ensure that all standards and procedures are executed all of the time.  Compliance programs use frameworks such as the PCI DSS, SOX, HIPAA and the like to determine what areas to assess and to ensure that the requirements documented in these frameworks are being consistently followed.  But the key is that controls are assessed over a period of time, usually six to twelve months.  Assessees are required to provide proof over the assessment period to prove that controls are being enforced 100% of the time.

This is an area where the PCI DSS falls short as a lot of the requirements are only assessed as of a given point of time.  Unfortunately, that given point in time can be different for each requirement.  As a result, it is very easy for an organization to be compliant on paper, but not compliant consistently.  If the PCI SSC could make one change that would improve the PCI assessment process it would be to require assessing organizations over a period of twelve months.  I know of a few QSACs that follow such an approach, but they are few and far between.

The bottom line is that security professionals need drop the “compliance is not security” mantra as long as the framework used addresses all aspects of security.  All of you out there that are hiding behind this saying need to find some other excuse for not securing your organizations.  “Compliance is not security” is an excuse, and a lame one at that.  Because if you are truly doing your job, then you are complying with some security framework and, therefore, compliance is security.

Myth busted!

Network Segmentation – One Last Discussion

Based on feedback I am getting, my previous posts regarding network segmentation are still not getting the point across regarding proper network segmentation.  So, this time I am going to use my post regarding the Control Triad and hopefully everyone will now understand what constitutes appropriate network segmentation.

As a quick refresher, the control triad is compromised of preventative controls, detective controls and corrective controls.  All of these control types are required to ensure a secure environment.  The more individual controls you have under each of the three control types, the less likely an incident will occur and the more coverage you should be able to afford your organization should a control go temporarily out of compliance.  However, an individual control can really only appear under one of the control types otherwise that control is diluted as it becomes a single point of failure causing the control triad to not function properly.  With that explanation, let us look at proper network segmentation from the control triad perspective.

Preventative Controls

The following would be considered the minimum preventative controls when talking about network segmentation.

• Firewall(s) with rules that restrict traffic to a limited number of ports to/from the cardholder data environment.
• Router(s) with ACLs that restrict traffic to a limited number of ports to/from the cardholder data environment.
• VLAN(s) with ACLs that restrict traffic to a limited number of ports to/from the cardholder data environment.
• Private wireless network(s) use a separate VLAN(s) from the cardholder data environment with access controls enforced for any access to the cardholder data environment from wireless.  Private wireless access points are configured with WPA2 using Enterprise authentication and AES 128-bit or greater encryption.
• Software firewall on server(s) in the cardholder data environment that restricts traffic to a limited number of ports/services to/from the server(s).
• Restricted administrative access to infrastructure devices in or controlling access to the cardholder data environment.
• Access controls that restrict administrative and end-user access to applications in the cardholder data environment or that access the cardholder data environment.

Remember, when I say. “limited number of ports to/from” I mean a very limited number of ports.  Yes, there may be instances where you might have 100 ports open to/from your cardholder data environment, but you better have a valid business reason for every one of those 100 ports.  And just so we are all clear, a valid business reason documents the reason why the port needs to be open, the risk presented to the cardholder data environment that the port is open, actions that have been taken to minimize the risks, and management approval of the port being open.  And the business reason for opening a port needs to be more than just “it needs to be open” or “the application will not function unless it is open.”  You need to document why it has to be open so that in the event of a breach you can quickly rule out the ports that might have been the cause based on the type of attack.

When we talk about restricting access, you need to be restricting access.  In small and mid-sized organizations, restricting access might not be feasible.  In those cases, forcing personnel to go to management to gain access is the way to properly provide control.  In large organizations, what we are talking about is restricting access to fewer personnel than everyone that has access to normal production.  The idea is that not everyone in support or business users should have access to the cardholder data environment.  The rule here is the fewer the better but do not make it so few that you create issues.

If you want to go the extra mile, the following controls can further enhance your security.  However, for some organizations, they come at a cost in operational efficiency that is unacceptable.

• Disable all unused physical jack connections on all infrastructure devices.  Any activation of a jack requires a service ticket and standard management approvals.
• Disable dynamic host configuration protocol (DHCP) in all retail locations.
• Public wireless in retail facilities provided by a separate third party and on a separate circuit that connects to the Internet.
• Required use of encrypted, two-factor authenticated virtual private network (VPN) connections from any wireless network to gain access to any internal network.
• Access to the cardholder data environment is not allowed for users connecting through any remote access connection.

Detective Controls

The following would be considered the minimum detective controls when talking about network segmentation.

• Network and host intrusion detection/prevention systems that monitors the aforementioned firewalls, routers, VLANs and servers that are protecting the cardholder data environment and generate alerts to appropriate personnel when an intrusion or incident is detected.
• Daily analysis of infrastructure device configurations to ensure that only approved configuration changes are made to these devices.
• Daily monitoring of devices to alert on any foreign devices that are added or when devices are removed from the network.
• Daily analysis of log data from the preventative controls to find potentially anomalous log entries that indicate a variance in the preventative controls or a potential incident.
• Change management records for all infrastructure devices, servers and applications in-scope for PCI compliance.

The key here is to generate alerts should any anomalous activity be detected.  But that is the rub.  What is anomalous?  Anomalies are not always the easiest things to identify or define.  As a result, your detective controls may take a while to fine tune.  However, the organizations that do the best job of managing their detective controls organize their anomalies by the PCI DSS requirements they are trying to meet.  This allows them to tweak their anomaly detection capabilities by PCI DSS requirement.

Then there is the issue of what do you do if you detect an anomaly?  Most of the time, an anomaly is not dealt with because of one of two reasons.  The first reason is because the detection solutions are new and are not functioning properly because no one has taken the time to tune them.  The second reason is that because of changes in the environment, the detective controls need to be re-tuned to reflect the changes.  Regardless of why, the detective controls need to be adjusted so that they are not generating excess false positives resulting in people chasing phantom issues.

If you want to go the extra mile, the following controls can further enhance your security.  While these sorts of tools are available as open-source solutions, there are also many commercial solutions as well.  Regardless of whether they are commercial or open-source solutions, tools that will perform these functions typically take a significant amount of time and effort to tune so that they provide the right amount of information for the right incidents.

• Real-time analysis of infrastructure device configurations to ensure that only approved configuration changes are made to these devices.
• Real-time monitoring of devices to alert on any foreign devices that are added or when devices are removed from the network.
• Real-time analysis of log data from the preventative controls to find potentially anomalous log entries that indicate a variance in the preventative controls or potential incident.

All real-time does is provide you with instantaneous alerting.  Most small and even mid-sized merchants do not need real-time analysis and alerting.  Not that they cannot use it, it is likely overkill for their environments given the threat of attack.  However for governmental agencies/departments, financial institutions, health care organizations and most large merchants; real-time analysis and alerting is mandatory.

And if you think tuning for daily reviews was painful, tuning real-time analysis and alerting systems is at least twice as painful.

Corrective Controls

The following would be considered the minimum corrective controls when talking about network segmentation.

• Change management procedures.
• Incident response plan(s) for addressing any issues identified by the detective controls.
• Root Cause Analysis (RCA) procedures.
• Action plans that result from the incident response process that require changes to the preventative and/or detective controls.  At a minimum, the action plans must document the correction needed, the person(s) responsible for getting the correction completed and the timeframe for the correction to occur.
• Internal audit review of the preventative and detective controls.
• QSA review of the preventative and detective controls.

Here is where a lot of organizations miss the boat.  You have detected an anomaly, you dealt with the anomaly, but you do not analyze why the anomaly occurred or you do an analysis but then you do nothing to correct any issues that might have been identified.  As a result, the anomaly continues to be encountered but actions are not taken to minimize or even eliminate occurrences.  This is why the advanced persistent threat (APT) is successful.  APT relies on the fact that eventually all organizations get sloppy and do not take corrective actions to maintain or even improve their controls.

There may be a number of preventative, detective and corrective controls that I may have missed or did not consider since everyone has unique environments.  At a minimum, if your organization has implemented these controls and they are all operating effectively, you are going to better than the majority of organizations out there and much less likely to have a serious incident that could result in a breach.

And that is the problem all organizations face, keeping these controls functioning effectively every day without missing a beat.  That is why we have defense in depth.  If one control is not functioning properly, there are other controls that will cover in the interim until the control is back functioning properly.

Finally, as I always like to remind people, just because you implement all of these recommendations does make you invincible.  All these recommendations do is just making the likelihood of an incident and the potential damage resulting from an incident lower than if you had little or no controls in place.  How much lower depends on a number of factors, but the risk will be lower.  And after all, it is all about lower risk.

Hopefully the issue of what constitutes appropriate network segmentation has now been put to rest.

Security Is Not Perfect

I am really getting tired of people complaining about data breaches, the lack of security and how it is all the fault of security initiatives such as the PCI DSS, the QSA involved or the carelessness of the organization that was breached.  It is not that QSAs, the PCI DSS and the like do not have a responsibility in all of this, but that responsibility seems to be over allocated to QSAs and the PCI DSS rather than focused on the real cause.  According to a recent Verizon Business Services report, the real cause is usually human error.

What are worse are the comments from credit card company executives that seem to portray the PCI standards as ‘THE’ standards for securing cardholder data.  What these people fail to either understand or acknowledge is that even if every security measure in the PCI DSS were put in place and worked exactly as they should, breaches would still occur.  Albeit at a much slower pace and with less information released.  Why?  Because, security is not perfect.  Never was and never will be.

But, but, but …

If you have told people that your security systems are absolutely, positively protecting your organization’s information assets and that nothing can go wrong, shame on you.  You need to better manage people’s expectations regarding security.  This expectation by people that security is perfect is something that needs to be debunked immediately.  Yes, the security measures put in place are protecting the organization’s information assets, BUT …   Those measures are not perfect, never will be perfect and at some point will break down resulting in some sort of incident.  Your job is to make sure that your security measures ensure that any incident that occurs is as small as possible.

But, but, but …

No ‘buts’ about it.  Any security professional knows that security only reduces or minimizes risks; it does not remove them all.  Yes, some risks may be eliminated when proper security practices are implemented.  Nevertheless, for the most part, some security risks will always remain, regardless of the security measures put in place.  Proper management of these remaining security risks should minimize the risks as much as possible.  However, minimized risk does not imply that the threat or vulnerability cannot be leveraged.  Minimized risk means that controls are in place to make the likelihood of a compromise using any remaining vulnerability as low as possible.  As a result, security programs such as the PCI DSS are focused on keeping all but the most dedicated attackers at bay.  It is what I call the “99-1 rule.”  Security focuses on protecting your organization from 99% of attackers.  However, the remaining 1% of attackers will cost you too much time, resources and effort to keep them out.  Your hope is to at least be notified when that 1% attacks.

A prime example of security not being perfect is the banking industry.  Banks have invested heavily in a variety of security measures including vaults, silent alarms, video recording, die packs and other deterrents.  Vaults have definitely reduced the amount of money lost in a robbery.  However, one would argue that silent alarms, video recording and die packs have not had a significant impact on the number of bank robberies.  Where silent alarms, video recording and die packs come into play is in reducing the number of successful robberies.

That is what the PCI DSS and similar security standards are all about.  They were developed to make the successful breach as difficult a thing to accomplish as possible.

However, there are people out there that are dedicated to breaching security for whatever reason.  Whether or not you are in their sights is why you rely on the control triad and defense in depth.  The idea being that this will give you a “heads up” if your organization is being attacked.  But be careful and do not become complacent.  Dedicated attackers are like hunters.  They research their prey so that they know about the defenses of their target and they develop sophisticated plans to defeat those defenses or at least keep them at bay.  These are people skilled in their craft.  They take a part-time job as part of the cleaning staff at a building where their prey is located so that they can scope out their quarry and determine where the weaknesses are located.  If they need other expertise, either they will acquire that expertise through training or they will team with someone that has that expertise.  In the end, if there is a way, these people will find it and exploit it.

If you want to see these concepts in action, watch any of the Ocean’s 11, 12 or 13 movies.  They may be campy but the planning concepts used in these movies mimics just how an attacker goes about planning to obtain your databases or other information.  And if you do not believe that such people or threats really exist, read my post regarding the Advanced  Persistent Threat (APT).

The key to remember is that you are never going to remove all of the risk, you are only going to manage it.  With proper controls in place, threats can be managed and the risk to your assets minimized.  However, do not let your guard down because that is when attackers will take advantage.

Network Segmentation – Take 2

I have had a couple of discussions recently regarding what constitutes good network segmentation.  Apparently, my original post was just too cryptic, so I’m going to use some examples in this post to hopefully clarify where people are going wrong.

The PCI DSS gives very little guidance on network segmentation.  In fact, the only statement near a definition says. “Network segmentation can be achieved through internal network firewalls, routers with strong access control lists or other technology that restricts access to a particular segment of a network.”  But those are the mechanics of network segmentation.  This definition does not specify or illuminate the additional controls required to ensure segmentation which is why I wrote the original post.

In my first example, the network in question is segmented by VLANs.  The cardholder data environment (CDE) is contained in one VLAN and there are another eight VLANs defined.  All VLANs are internal and none face the Internet.  Access control lists (ACL) have been implemented to control the communications between the various VLANs and the CDE VLAN.  Monitoring of all of the VLANs has been implemented through a variety of methods including network monitors, host monitors and system/event log monitoring and analysis.  Network administrators monitor consoles that bring up any critical alerts that could indicate a potential attack or compromise.  This definition sounds pretty good does it not?  The problem is that it is all in the details and the details tell a different story.

In reviewing the VLANs’ ACLs we determined that two of the VLANs have TCP and UDP ports 1 through 65535 open to the CDE VLAN.  Whoa!  Every port is open to the CDE VLAN from these two VLANs?  Yes, that is correct.  This is not what the PCI SSC thought was “strong access control lists.”  In digging further, we inquire as to why this condition exists.  We are told that, ”We were unable to determine what the applications needed to have open between these VLANs, so rather than break anything, we just opened everything to be safe.”  To be safe?  Safe is a term that has different meanings relative to each person’s view that uses it.  In this case, because the two VLANs were internal, apparently the other VLANs were considered also ‘safe’.

But a lot of network administrators would point to the monitoring as the way they control things.  Are you serious?  I do not care how much monitoring you do.  With every port open, that monitoring is going to likely generate enough false positives to make identifying the real threats like finding a needle in a haystack.  And this was confirmed later on when we observed the network administrators that monitor the network.  They were ignoring almost everything that came up on their screens.  When we questioned them about this, they said, “We have tried to tune the alerts, but have not been able to significantly reduce the false positives.  We get around 10,000 to 25,000 alerts a day.  So we do the best we can to find the real threats.”  The best we can?  Security is not forgiving, let alone for people that are doing ‘the best they can’.

The moral of this example is that if you have every port or close to every port open, you cannot consider your network properly segmented.  I do not care what the other controls are that you believe are in place.  You have to be realistic.  And justifying having all of those ports open has to be more than implying you were too lazy and did not want to make the effort to find the real answers.

My other example involves a network that does have a limited number of ports open between their CDE VLAN and their other VLANs, albeit there are quite a few open ports.  They also have monitoring in place and their network administrators are very diligent in ensuring that alerts are addressed as quickly as possible.  Unlike my first example, these folks are seeing around 300 to 500 alerts of which 10% to 15% are false positives.  The problem is with their documentation.  In reviewing the firewall rules that segment the VLANs we documented all of the ports open to/from the CDE VLAN to the other VLANs.  We interviewed the Manager of their network management and administration department and inquired as to the business reason for each of the open ports.  Of the 100 or so ports defined in the ACLs, they can only give us business reasons for about 20% of them.  Heaven forbid they should document the reason in the configuration file, but there is no other documentation available.  The Manager even tries to find documentation in the help desk system where they log all of their changes, but even after refining the search criteria, there are just too many records to sift through in our one hour meeting to find what we need.  Not even proof that management knows that these ports are open, the risks that are involved with these ports being open and that management approved that these ports be opened.

The moral here is that documentation is the foundation from which you build.  If you have a shaky foundation, you will have shaky security and are likely a candidate for a compromise and breach.  This is why documentation is important.  If you cannot remember why ports were opened, users were allowed access to data and other security relevant issues, how can you even think you are secure?  The answer is you cannot be secure if you cannot answer basic questions.

But it gets better.  This same individual earlier in our meeting had confirmed that they were the one that reviewed the firewall rules quarterly and showed us emails to prove that fact.  Then as we are going through the CDE ACLs, they say, “Oh, that rule should be removed.  It was for a business partner that we have not done business with in more than four years.”  Now, do you think I seriously believe that you are really reviewing these firewall rules quarterly when you admit that a given rule should have been removed four years ago?  We document four more firewall rules that should have been changed or removed.  It is situations like this that cause a QSA to shudder and then wonder what other ugly things are under the rocks and just how far you need or want to dig to find them.

Our moral here is telling the QSA what they want to hear when you know you will have to contradict yourself later on.  All it does is make you look incompetent.  But this situation also points out a good point regarding the duties of a QSA in conducting their assessment.  QSAs not only rely on interviews and documentation, they also rely on observations to ensure that organizations not only talk the talk but also walk the walk.

So what then is proper network segmentation?  A properly segmented network is much more than just technology.

The foundation of a properly segmented network starts with the control triad of preventative, detective and corrective controls.  Preventative network controls are going to be firewall rules and VLAN ACLs and any other controls that prevent or control access.  Detective network controls are going to be related to the monitoring you implement.  Monitoring can be real time and/or log analysis after the fact, but it should not be limited to just access to/from the CDE.  Monitoring also needs to include monitoring the network traffic for anomalous traffic.  Finally, you need corrective controls to ensure that any issues discovered with the preventative and detective controls are addressed as soon as possible.  Corrective controls are usually generated as action items created from such things as the lessons learned from an incident response plan or findings from an audit.

Once you have decided on the controls you will implement, you then need to create documentation that supports those controls.  For networks, the documentation that is key is to document every port that is open inbound to or outbound from the CDE environment.  Each of those ports will have been formally approved by management with the risk presented by having the port open.  And that risk analysis needs to include not just the port in question, but any other relevant ports, if necessary, as certain combinations of ports may increase or decrease the risk.  This risk analysis is important for a number of reasons.  First, it documents the basic analysis of risk and provides the rationale for having made a decision at that time.  That documentation can also save you if a breach occurs as you can understand what the people were thinking when they originally opened the port and also understand potential methods that might have been used to cause the breach.  This documentation is also important for the quarterly reviews as you can use the documentation to refresh your memory as well as assisting you in making changes to the rules if business conditions change.  Yes, I know firsthand that documentation is the last thing anyone wants to do.  But without it I will guarantee you will not remember six months or more down the road why you did what you did and for whom.  And in the security business, it is that sort of knowledge that can mean the difference between being secure and being a target.

The next item that needs to be documented is the users, programs, services and organizations that have access to the CDE.  In the case of programs and services, this should be tied to the aforementioned list of ports open.  In a breach, this documentation will reduce the number of likely suspects of where the breach came from.  As a result, you can see why it is important to limit the number of people, programs and organizations that have access to the CDE.

The final piece of documentation that needs to exist is what should be done in the event a problem or an alert is generated.  If people do not know what their responsibilities are in regards to providing feedback, then alerts will be missed or ignored and problems may not be addressed as quickly as they should.  Responses to problems or alerts should include detail regarding the conditions that created the problem or alert, the steps take to address the problem or alert and any issues that may have resulted from addressing the problem or alert.  If the problem or alert is not addressed in the timeframe required, there needs to be an escalation process so that the problem or alert receive the necessary visibility of management should they go unaddressed.

I hope this provides the additional examples of network segmentation.

‘Defense In Depth’ And The Control Triad

A lot has been written recently regarding that the PCI DSS relies on the concept of ‘defense in depth’.  However, very little has been written about how ‘defense in depth’ is actually implemented by the PCI DSS.

Before going into the PCI side of defense in depth, let us discuss the concept of defense in depth.  Defense in depth relies on the triad of preventative, detective and corrective controls.

• Preventative controls are those controls put in place to prevent, as best possible, a security incident.
• As I have repeatedly pointed out, security is not perfect.  Detective controls are designed to back up preventative controls by detecting incidents that may occur due to a shortcoming in or failure of the preventative control(s).
• Corrective controls are those controls that back up both the preventative and detective controls by providing feedback to ensure that any lapses in those controls are corrected or those controls are redesigned to better prevent or detect their respective conditions.

With that as background, let us discuss an example of the preventative, detective and corrective controls that are part of the PCI DSS.

One of the primary preventative controls in the PCI DSS is network segmentation.  Properly implemented, network segmentation physically or logically separates PCI network traffic from all other network traffic.  By separating PCI network traffic from all other network traffic, you minimize the potential that PCI traffic is corrupted by non-PCI traffic.  Network segmentation comes in many forms.  It can be the firewall based on the rules that are implemented to segregate PCI network traffic from the Internet to the DMZ, the internal network to the DMZ or the general internal network to the PCI internal network.  It can also be the VLANs that are implemented across your internal network to segregate PCI traffic from all other internal network traffic.

On the detective side, logging is probably one of the biggest detective controls.  Every device can generate a log and, if properly configured, logging can provide a plethora of information regarding an organization’s network.  However, in order to detect an incident, log information must be reviewed and analyzed.  Not only must it be reviewed, but also with today’s sophisticated attacks, the log information must be correlated to other devices’ log information.  This requires a security information and event management (SIEM) system that centrally collects all log information from all devices and then conducts real-time or almost real-time analysis of that information looking for indications of any potentially anomalous behavior.  When potentially anomalous behavior is believed to be detected, the SIEM alerts the appropriate personnel to further investigate the behavior.

Detective controls can also be corrective controls.  The analysis of the log information is an example of just such a control.  Not only is it a detective control, it is also a corrective control.  That is because the analysis of the log information typically results in corrective actions to address the anomalous condition that is detected.

The Report On Compliance or Self-Assessment Questionnaire process is also an example of both detective and corrective controls.  Either process asks your organization to examine the triad of controls and detect any shortcomings in how these controls function.  If the controls are not functioning properly, the process identifies those controls that need to be changed and allows for the development of plans to correct those controls.

An incident occurs when the triad breaks down.  The preventative control is not properly designed to prevent all likely incidents.  A detective control does not detect all possible flaws in the preventative control(s).  And the most common offense?  When issues are identified with preventative or detective controls, action is not taken to correct the shortcomings of the preventative or detective controls.

Trust me; regardless of how good your organization is at following its policies, standards and procedures, you will have areas where the control triad does not function properly.  A good organization seeks out, identifies control issues, and addresses them as soon as they are identified.  A really good organization does not make their efforts to seek out and identify failings as a ‘witch hunt’ because they recognize that people are only human and always need to improve.

I could go on and on and on regarding the control triad and how it applies to the PCI DSS requirements.  However, I think you get the general idea.  So, the next time you are complaining about why a particular requirement exists, think about what it prevents, detects and/or corrects.  All of the PCI DSS requirements are there for a reason and provide cover for a significant number of the other requirements.  And that is what ‘defense in depth’ is all about.