Based on some of the comments I get from blog readers and articles I see espousing the “compliance is not security” mantra, I think it is time for a refresher on where compliance fits into an organization’s security program or any other audit/assessment program.
As a reminder, Miriam Webster defines compliance as:
“Conformity in fulfilling official requirements”
Therefore, for PCI, an organization is conforming to the PCI DSS when they assess. For ISO 27K, the organization is conforming to the ISO 27K standard. For a financial audit, an organization conforms to the Generally Accepted Audit Principles (GAAP). The list can go on and on.
Where things get confusing and messy is whether or not the compliance program is detailed and tight enough to ensure security. Programs such as the PCI DSS, ISO 27K and others have to be written in such a way that any organization (regardless of technology, architecture, applications, etc.) can assess to it. Whether we are talking about the PCI DSS, ISO 27K, COBIT or any other assessment program, there must be flexibility to allow for all sorts of solutions, not just a preferred or “perfect” one. As a result, there is a lot of flexibility built in that produces a not so tight security program. This therefore means that being secure requires going beyond what the compliance program requires.
But compliance is only a part of an overall security program. I have discussed the control triad in prior posts, but it bears repeating here. There are three parts to any effective security program: protection, detection and correction.
A good compliance program is part detection and part correction. The detection side of compliance relates to detecting controls that are not functioning as designed or are missing altogether. Changes to the environment are always the culprit in controls not functioning properly or missing. The correction side of compliance is defining what needs to be corrected in order to address the shortcomings determined during an audit/assessment.
Where the “compliance does not equal security” naysayers first go wrong is with ignoring execution. A compliance program is all about assessing the execution of an organization’s security program and ensuring that an organization is executing that program flawlessly. But an annual assessment such as with the current PCI process is only part of the overall compliance process. Compliance must be assessed constantly which is why the PCI SSC is introducing the concept of business as usual (BAU). BAU is meant to drive organizations to constantly assess compliance with requirements based on their risk assessment. Some requirements may have to be assessed daily, whereas other requirements might only be assessed weekly, monthly or even semi-annually
A working compliance program points out any execution issues with the security program and recommends solutions. To paraphrase the numerous security professionals that has pointed this out over the years, “To ensure the security of our organization, we have to get it right every minute of every hour of every day. Attackers only have to get it right once.” Therefore a compliance program is the mechanism that keeps the security program functioning as close to “right” as is humanly possible. But as I always point out, security is not perfect. So even with a good compliance program, security issues will still occur.
But that is where security has its biggest issues are with execution. If organizations could execute any security program, even the PCI DSS, with almost 100% accuracy every day, the number and seriousness of data breaches would be virtually non-existent. But that is where most organizations get burned is with execution. I have yet to encounter any organization that can execute all parts of any security program above 90% and that is on a good day. In most cases organizations are well below that percentage. As a result, regardless of the security measures in place, the controls are not functioning as defined and therefore security holes abound resulting in a myriad of ways to defeat security. Add in the human errors that occur and I think you start to understand the challenges of keeping a security program working effectively.
The second place that the naysayers go wrong is not recognizing the inherent limitations of the assessment programs. Every security program I have ever encountered has a multitude of limitations due to the necessity that they fit all situations. Then there is the fact that they are, after all, mere baselines for security, not a “be all to end all”. To truly be secure an organization must go well beyond what the PCI DSS, ISO 27K or any other security programs call out. That is not to say that a company cannot be reasonably secure following only these programs. But then we go back to my earlier statement that the program must be executed close to 100% compliance every day which is where organizations get burned. It is not the program that causes the security issues; it’s the inability to execute the security program consistently that is the cause.
As I said earlier, all security programs are the bare minimum for security. As a result, in order to be secure an organization needs to go beyond the requirements specified by any security program. Unfortunately, it is the rare organization that goes beyond any of the recognized security programs. Why? For most organizations, the security personnel just do not have the time to develop their security program beyond what is already called out by ISO 27K or PCI DSS. Some of this is due to staffing issues, but more often than not, it is due to a lack of upper management recognition and understanding of what true security really takes in personnel, time and other resources. A lot of that lack of understanding comes from the organization’s risk assessment or more likely the lack of one. Without a good understanding of the information security risks to the organization, it is very hard to determine what measures need to be taken, why they need to be taken and what resources are necessary. Once everyone can understand the risks, then a true security program can be developed for the organization taking into account the PCI DSS, ISO 27K or any other security standards that need to be followed.
Finally, compliance programs are never ever static. They need to be constantly adjusted to reflect the introduction of new technologies, changes to the existing environment and the removal of technologies.
For example, if an organization implements a new security information and event management (SIEM) solution, the compliance program needs to be changed to reflect that new SIEM. The most likely changes are the addition of new conditions that will be monitored for and alerts generated. The compliance program needs to add those new conditions to their testing and make sure that alerts are truly generated when the conditions are encountered. In addition, if the new SIEM introduces any changes to existing alerts, those changes also need to be adjusted in the compliance testing. Finally, if some SIEM conditions are being removed, the compliance program needs to reflect that removal as well.
Changes to the environment also need to be reflected in the compliance program. Application changes can result in changes to the compliance program particularly if the application controls its own authentication processes, encryption of sensitive data and other critical controls. Another area is network changes that also need to be reflected in a compliance program. While adding new physical locations is typically only a minor change to a compliance program, changing firewalls, routers or switches can result in significant changes to the compliance program.
One of the biggest things neglected with compliance is when technology is removed from the environment. Nine times out of 10, the compliance team is never notified of the decommissioning of equipment. Security professionals might think that compliance is not affected if something is removed. However the control environment might be significantly affected with the removal of technology because security is unaware of other areas that are relying on a technology for controls. Then once the compliance team comes out for their assessment they find that, with the removal of certain technology, there are now numerous controls that are no longer performing as designed and security gaps exist as a result.
An example of a change that can adversely affect compliance but is typically overlooked is staff reductions. One of my clients a number of years ago terminated a number of personnel in an effort to save costs. A couple of the staff let go were performing manual processes that were critical to monitoring the security environment and identifying security issues that were critical to their PCI compliance efforts. Management believed that these people were superfluous to their operation, i.e., a nice way of saying they were unnecessary overhead, so they were terminated. Obviously when these people were terminated, those critical controls were no longer functioning. To add insult to injury, this situation was not identified until almost 10 months later during their PCI compliance assessment. Then the organization had only two months to somehow put these critical controls back into place without the necessary resources to do them manually. What eventually happened was that people had to step up and add these tasks to their own already heavy workloads. While that got the organization through their PCI assessment for the current period, the stress of keeping those controls operating as designed proved to be too much and some people quit creating new control failure situations. Ultimately management had to admit that more people were needed as well as some new tools were necessary to minimize control failures as well as minimize the number of people required.
The bottom line is that the compliance assessment process is the check and balance to make sure that an organization’s security program is designed appropriately and is functioning effectively. Hopefully you now have an appreciation as to the purpose of an effective compliance program and why it is important to the overall security process.
So the next time you say that “compliance is not security” think about this post and understand the implications of your statement. Without the compliance process, you have no idea as to whether your security program is effective or not.
PCI Guru 
JJ: These are very valid points about the checklist/checkbox mentality and compliance being too often used as a mean to push the bare minimum. That being said, it doesn’t mean that compliance itself is the limiting factor as it has more to do with how you approach your compliance evaluation. We all had experiences with a CISO/CPA like the one you brilliantly described but we also have to acknowledge that at the other end of the spectrum, the information security field is flooded with technical “box-pushers”. Whatever your problem is, they’ll find a box (ie. an appliance) to fix it all. The box is the center of it all and they generally have multiple colored blinking led and fancy names. They’re all Next Gen and they will make all your problems go away.
But to go back to the original discussion, the biggest problem lies with the fact that there’s no clear definition of what being secure means. Therefore, security is like a destination that we’ll never reach even though we’re all doing our best !
Well said.
About a year ago we had a General Counsel at a local infosec chapter meeting. He is GC for a company that just about anybody in the world would instantly recognize. He was asked what worries him the most. His reply was that he worries the most about being found negligent in a court of law. That significantly raises penalties in the USA and it could invalidate insurance coverage.
He said that being found negligent in relation to your peers is what is really bad. When asked if that meant his company could be found to be “totally sucks” as long as his peer companies also “totally sucked”. He laughed and said “Kind of.” He said you don’t get a lot of credit for being a lot better than your peers because, after all, you did get breached and they didn’t. But it will really hurt if you’re found significantly worse.
I found it quite enlightening. I think that is what drives the compliance mindset. Not meeting the compliance minimums probably would mean you were negligent. And if you don’t get a lot of credit for being a lot better, why do it other than for your own peace of mind? And if you’re high enough up in the company and you do become the sacrificial scapegoat, your employment contract probably guarantees you a very nice separation package.
Very good points. Unfortunately, lawyers and courts have significantly more influence than the PCI SSC, PCI standards and the QSA/ISA.
I have always explained the difference between compliance a security as a function of time. Compliance is largely focused on a point in time. Security has to have time dependent functions built into it. Such as continuous improvement. Or agility to handle new threats, or acquisitions, etc. I think compliance is like running the numbers on a company’s P&L at the end of the year. The company might look great according to audit, but fail a few months later. Security has to look deeper to analyze trends, similar to a financial analyst. Audit doesn’t take into account the changing market conditions (changing threat landscape).
While the PCI DSS, ISO 27K and other programs are a point in time, they do not have to be. This is what is driving the business as usual (BAU) approach through the PCI SSC. Like having to have four quarters of vulnerability scanning, BAU will add additional requirements that will be assessed over the period of a year such as change control, log reviews and the like. That point is to get organizations to begin embedding security into their day-to-day operations rather than an annual event.
Fred, treating compliance as a point in time is one way to manage your compliance posture but it is not an intrinsic caracteristic of compliance. There are processes where you can easily achieve real-time compliance. An example would be a technical compliance monitoring solution that automatically prevent any attempt to configure a device in a way that is not consistant with an existing hardening guide. There’s also many situations where security is treated as a point in time. I’ve seen many cases where the hardening guide is configured during the project integration phase and never to be looked at again.
Therefore, the ability to maintain security/compliance over time is an attribute of your security/compliance program and not the difference between security and compliance that you alluded to.
Well said Jeff!
> Where the “compliance does not equal security” naysayers first go wrong is with ignoring execution.
For me it’s the opposite. Execution usually ends with checking the compliance box and that’s why I say “compliance does not equal security”. Because it doesn’t. Being compliant never means you’re secure. What it does mean is that you have met the bare minimum requirements of some program and that’s it. Being compliant can help you bleat “But we were compliant!” when you get breached and you could use it as a weak defense to a lawsuit that alleges you were negligent. But that can only work if you benchmark your industry peers and can show you were no less secure than the industry.
> As I said earlier, all security programs are the bear minimum for security.
I see “compliant” as the “less than minimum” for security. But it can be a good baseline for a security program because it helps a security team get traction within a company.
BTW, I would expect to see the phrase “bare minimum ” and not “bear minimum”, but yes, even compliance can be a “bear”! 🙂
One place where a good security team differs from the audit/compliance/IT group is in pen test scoping. I used to be in conflicts because I scope pen tests to reveal our weaknesses, known and unknown. Audit/compliance/IT want a good report so they look good and have no remediation items. While a solid pen test and good remediation will eventually lead to a clean pen test report, the opposite is not true. A clean report derived from a bad pen test gives a false sense of security.
PCI ASV scans and PCI pen tests are a good case in point. They are designed to help prevent card loss without caring about other weaknesses. On one ASV scan they detected that an Internet-facing router was running as an NTP server and the security team realized it could have been used as a source in a DDoS attack against other companies. It was rated a “low/pass” because it could not affect the security of card data. It actually was a “critical/fail” to the security team.
I used to work for an Information Security Officer who was a CPA. I was always in conflicts with him. A CPA is at their best when their customer meets all compliance and regulatory requirements and expends no resources on getting better in those areas. One of our biggest arguments was over me requiring 15-character minimum passwords/passphrases for administrative level accounts (to assure LANMAN hashes were not generated and sitting in memory). His position was that our regulator only required 8-character passwords and did not differentiate between user and administrative accounts so we shouldn’t either. His world and training had him doing the minimum to “check the box” and what he could never understand was that our true adversaries were not the regulators. The real adversaries know our regulatory minimums and would seek to exploit them as gaps.
That is the heart of “compliance does not equal security.”
How did I overcome that conflict? The final decision on scoping was made by the COO. That person asked a simple question: “Why would our company not want to do this?” Since the only answer that audit/compliance/IT could use was “Because we’d look bad and we’d have a lot of extra work fixing things.”, they backed off. And they had a lot of work that year. And nowadays they don’t. And nowadays we take absolutely nothing out of scope for internal or external pen tests. Not the mainframes, SQL servers, nothing. Because now they’re kept updated and secured and yes, we still have solid tests but now we get clean reports as well.
This is the problem, you keep equating compliance with the PCI DSS or ISO 27K. Compliance is ensuring that you meet YOUR ORGANIZATION’S security program. Complying with the PCI DSS should therefore be a byproduct of your compliance efforts not the entire focus. Therefore when you imply “compliance is not security” you are implying that you organization’s security program is just as basic/lame/stupid/etc. as the PCI DSS. I don’t think that is what you think of your security program, but that is exactly what you are implying.
I know that is not the case based on your later comments, but that is what a lot of security professionals are basically doing when they talk about “compliance is not security”. If you are not doing continuous improvement of which the compliance process is a large part, then you never know how good/bad your security program is operating.
I liked your example of your CPA CISO. However, hopefully this example is from very, very long ago. If not, then why was your organization still doing generating LANman hashes? LANman hashes have been a bad practice since NTLM authentication was introduced.
Two years ago, actually.
Many people don’t know it but the registry key you set to disable LANman hashes only keeps them from being stored on disk. They are still generated and in memory unless the password is 15 characters or longer. That was true through at least Server 2003.
As late as last year we still had several pieces of critical software that only worked with LANman. It was still a supported and current version from the vendor. You don’t want to know the h*** we went through with Dell to get that thing remediated. And they did not see why there was a problem. Nobody else was complaining. It was something they bought for SharePoint work and just stuck their name on. Don’t even get me started on mainframes. Our multi-million dollar wonders still require NetBIOS enabled to share files. And only last year began to support something other than FTP for file transfers. (No, not IBM.)
The question is who controls the security posture of a company. It’s not the security function if they even have one. It’s senior management and to them the bare minimum is all that needs done because that’s all the regulators demand. To them, and to audit, compliance == what the regulators demand. And if those requirements did not make the company secure, the regulators would change the requirements, right?
This is why we’re finally seeing so many 30+ year C*O’s lose their jobs over breaches. They don’t understand IT and they got blindsided by how much it could affect the company despite how well their paperwork was compliant.
Sorry to hear about your current situation and environment.
I have to say, I too have encountered a lot of CIO/CSO/CISO that are great politicians, project managers or accountants and wouldn’t know a firewall from a switch or how to secure a paper bag let alone a network. They work out great until technical decisions need to be made and they trust the widget vendors’ sales personnel more than their own technical staff.
In regards to who controls the security posture, that falls to information security until it impacts the business. Then you run into non-technical personnel making technical decisions based solely on business reasons and not on the risks involved because they don’t like or understand the risks. Then when they get breached or some other adverse situation occurs, they shoot the CIO/CSO/CISO for not being more forceful in stopping their stupidity.
That is the risk inherent in today’s workplace & marketplace. Extremely well stated professional hazards.
Happens in and to much smaller organizations also. Their own lack of understanding combined with their own incorrect assumptions BASED on that lack of understanding often result in the breaches they often pass the buck for.