Based on some of the mail I am getting these days, there is a lot of confusion regarding secure coding standards and application vulnerability scanning, that is, requirements 6.5 and 6.6.
First, let us talk about the intent of these requirements. The overall intent of both of these standards is to stop insecure applications from being placed in production. The intent of requirement 6.5 is to ensure that secure coding techniques are part of the system development lifecycle (SDLC) and that the most obvious errors, at the moment those are the OWASP Top 10, have been addressed during development. The intent of requirement 6.6 is to ensure that either code reviews are conducted or an application firewall is used to protect applications.
The most common question I get regarding requirement 6.6 is that since it does not specify what should be tested, does that imply that only the OWASP Top 10 needs to be looked for when conducting the code review?
When will you people learn? When the PCI DSS does not specify something, you always assume that you need to test everything. In the case of requirement 6.6, you need to conduct application vulnerability scanning for all potential vulnerabilities, not just the OWASP Top 10. This will become more important under PCI DSS v2.0 when they add other application vulnerability standards into the mix. The bottom line is that if all you are testing for is the OWASP Top 10, you are not doing enough testing.
Another area where people get things wrong is that they conduct application vulnerability testing just like they do network vulnerability testing, which is after the application is in production. Wrong! Unfortunately, the PCI SSC has only trained the QSAs to understand this fact and only merchants and service providers that have been through ISA training likely know about this requirement. Because of this, QSAs get beat up all of the time by merchants and service providers when they mandate application vulnerability testing and remediation before an application goes into production. However, if you think about it, it has always been implicit in these requirements. Remember, the intent of these requirements is to avoid putting vulnerable applications in production. That is why you need to conduct your scanning as part of your QA processes before the application goes into production. If any high, critical or severe vulnerabilities are discovered as part of the testing, those need to be either remediated or compensated for before the application is placed into production.
The final issue we consistently see is that secure coding techniques and code reviews are nowhere to be found in the SDLC. A lot of organizations point QSAs to various coding Web sites for their SDLC. They assume that these sites have already embedded secure coding techniques in their SDLC and that may or may not be the case. A lot of SDLCs document how to create application security, but say little or nothing regarding secure coding techniques. As a result, they are shocked when the QSA comes back and says that secure coding techniques are not in place. But what this points out is that the organization does not use the SDLC because had they used it, they would have known that the SDLC did not address secure coding and code reviews.
The lessons you should have learned are as follows.
- While requirement 6.5 only calls out the OWASP Top10, you need to also be worrying about all the other application vulnerabilities that could exist.
- SDLCs are meant to be used, not just offered as a way to meet a requirement.
- Secure coding techniques need to be documented as part of the SDLC and need to be followed.
- Requirement 6.6 requires you to scan for all application vulnerabilities, not just the OWASP Top 10.
- Application vulnerability scanning is performed before an application goes into production.
- If high, critical or severe application vulnerabilities are identified by scanning, those vulnerabilities must be fixed before the application goes into production.
PCI Guru 
In 6.5.a, it was mentioned that “training in secure coding techniques for developers”. What is the frequency of training expected?
I have clients that do it annually and I have others that only do it every few years. My preference is to have at least annual training on secure coding techniques just so it stays at the forefront. This can be reinforced by ensuring your code review process sends back code that is not secure. This may require your code reviewers to have more than annual training.
For a company going for PCI compliance, secure coding was not followed for previously developed applications. How to address this?
The past is the past. There is nothing you can do about the “sins” of the past.
You need to start anew and introduce secure coding techniques training and code reviews to your application development processes. You also need to make sure that you introduce the documentation of those activities into your change management system. And if you don’t have a formal change management process, you need to get one implemented and use it religiously. A lot of help desk systems have change management processes embedded in them, so you may already have such a system and not know it or are just not using those features.
Where most organizations get tripped up on PCI compliance is change management – application and/or network changes. They have poor change management processes and do not track every change, changes are not consistently reviewed or approved, changes are not closed out after implemented, there are no backout processes documented, etc.
The other area that trips up organizations that do application development is secure application development and code reviews. Secure application development techniques and code reviews need to be integrated into the development process, not an “after the fact” process as with networks. The biggest mistake that organizations make is not testing application security BEFORE they are placed into production. Unlike networks, the PCI DSS expects that applications will be assessed for security before they are allowed into production. If security vulnerabilities are found, they must either be remediated or mitigated and these activities must be formally documented and approved by management before the application can be placed into production.
I have a query with regards to PCI Penetration Testing. As per my understanding, Vulnerability scanning for systems (within PCI Scope) should be performed before the system goes into the production so that Vulnerabilities can be fixed. Also, Vul Scanning for Web Application should also be performed before application is exposed to Internet (Production). But I am little confused about Pen Testing for Application as well as Infrastructure also. Should we perform App and Infra Pen Testing in Production Environment or it can be done in QA also? Please suggest me. Also, please guide me if there is any Procedure/ Guideline stating this from PCI Security Council. Thanks.
In simple words, I want to know if Pen Testing and Vul Scanning for Infra/ App can be performed in Production or it can be performed in QA environment only?
Where you perform your vulnerability scanning and penetration testing depends on if you have both a production and QA environment. If you do have production and QA environments, then your production and QA environments need to be equivalent, that is servers are configured the same, infrastructure is configured the same, etc. but there does not need to be the same numbers of devices. If they are not equivalent, then those tests must be performed against your production environment. A lot of companies do not want vulnerability scanning and penetration testing conducted against their production environment since it has a high likelihood of causing problems with production, so they maintain an equivalent QA environment that can be used for such testing.
Where we typically run into issues is when the production environment is comprised of physical servers and the QA environment is virtualized. In those instances, we have to test the production environment because we cannot get comfortable that the virtualized systems in QA will respond the same as the physical systems in production.
Thanks PCIGuru. However, I have small doubt, when you say that there does not need to be the same numbers of devices, what does that mean? I assume that when you say that both QA and Production needs to be equivalent, it means that QA Web Server and Prod Web Server should have same configuration and same for DB or any other server which is in scope, but for number of devices, does that mean that instead of 1 Web Server in QA/ Production, I can have 2 servers and that should also be fine. So there can be 2 servers in QA while only one in Prod but all three will have similar configuration. Am I right. Please advise. Thanks.
What we typically see in the QA environment is a lack of a redundant configuration, So where production would have a fully meshed set of firewalls, load balancers, etc. along with duplication of Web and database servers, QA will not have this full redundancy.
I also wanted to know if this is applicable for all aspects like Code Review, Vulnerability Scanning (using Automated scanner) and Penetration Testing of the application?
You are correct.
However, just so we’re all clear. The first key thing to remember about using an application code scanner is that it must be part of the code review process, but not the entire code review process. The other key point is that any vulnerabilities identified by the vulnerability and pen testing must either be corrected prior to be placed in production or they need to be mitigated.
Vulnerability scanning and penetration testing of infrastructure (i.e., firewalls, routers, load balancers, switches, servers, etc.) can be done before or after it is placed in production and is usually performed after it is in production. However, vulnerability scanning and penetration testing of applications MUST be done before the application is placed into production as part of the application development process. This is all from the Guidance and Reporting Instructions (currently in Draft) documents as well as the QSA training I have received over the years.
Thanks PCIGuru. Let me summaries it to avoid any confusion or to make sure that I understood the terms and conditions of PCI assessment fully.
1) VA, PT and Code Review are necessary requirement of PCI for Application and Servers.
2) VA, PT and Code Review of the application MUST be done on QA and issues should be fixed before the application placed in the Production.
3) In case QA is not available, assessment can be performed in Production.
4) QA and Production should be of same configuration but need not to be same number of Servers/ Infrastructure. There may be 2 Servers in QA and 3 in Production or vice-versa but exactly same configuration.
5) Guidance and Reporting Instructions for the requirement that VA, PT and Code Review of the Application MUST be done in QA is in Draft stage and will be releases soon.
Hope I understood the terms correctly. However, I have one more small query, as you mentioned that QA does not normally has same redundant configuration like lack of fully functional Firewall, Load balancer etc, will it not affect the process of PCI assessment and all assessment will be considered as valid as it would have been done with all these infra in QA as well?
Thanks & Regards,
Akash…
Just a couple of comments.
Your point 5 is not correct. The four previous points have always been part of the PCI requirements. There has just been a lot of confusion around them until the last year or two. There is no further clarification coming from the PCI SSC on this topic because they feel they have more than clarified what is expected by the code review and testing processes.
In order for your vulnerability and penetration testing be valid, wherever you conduct them should be equivalent to the production environment. While the test environment may not have the same number of devices (i.e., lack of redundancy, virtualization, etc.), all of the devices and applications involved must be exactly as they are configured in production. If not, then your testing results will not be equivalent to the results you would expect from production.
Let me preface this with I agree with what you are saying because I think you are focusing on security and not compliance, but this topic does mix the two.
To me, your point “When the PCI DSS does not specify something, you always assume that you need to test everything.” emphasizes the biggest flaw with PCI as far as merchants and vendors are concerned. When Bob Russo states “Everybody that has been breached has been noncompliant with the standard…”, there is a reason — it’s impossible to be compliant with all encompassing assumptions like “test everything.” If and when a breach happens, the conversation goes something like this:
*** Scenario #1 ***
Forensic Vendor – What secure coding practices were used?
Application Vendor – Our SDLC is based around OWASP.
Forensic Vendor – You didn’t incorporate NIST standards as well?
Application Vendor – Nope.
Forensic Vendor – Failed, wrong answer. You’re out of compliance.
Bob Russo proved right, PCI intact.
*** Scenario #2 ***
Forensic Vendor – What secure coding practices were used?
Application Vendor – Our SDLC is based around OWASP and NIST.
Forensic Vendor – You didn’t incorporate ABC or XYZ standards as well?
Application Vendor – Nope.
Forensic Vendor – Failed, wrong answer. You’re out of compliance.
Bob Russo proved right again, PCI intact.
And so on…
Do your best to be secure; security should be the goal. If you are ever breached, you will be out-of-compliance so compliance should not be your goal.
This is my biggest problem with the PCI approach. Security is not perfect, never was and never will be. However, the card brands and the PCI SSC treat security as though it is perfect. I still love the quote from one of the forensic examiners for PCI who said, “When you are on a witch hunt, you are always going to find witches.” This is why I’ve always said that the card brands need to get off their high horse and stop treating every breached merchant like a criminal. Yes, there will always be the merchant that was a fool and those should be treated accordingly. However, most of my clients are doing the right things, but since security is not perfect, could end up being breached at some point. And even though they are trying to comply, if they are breached, they will be stocked and pilloried just like the merchants that were not diligent. And that’s just plain wrong and not deserved.