Remind me again how far down the road we are with the current practices of information security? Has it really been almost three decades?
Yes, it really has been about that long. Yet, it continues to fascinate me how those practices are so misunderstood in the information security community. Particularly when a significant portion of that community holds one or more certifications in the topic. In my very humble opinion, we should not be having such basic understanding conversations and yet these topics keep constantly coming back up.
One of those practices that is heavily misunderstood is penetration testing. I have written numerous posts on the subject and yet the understanding of penetration testing continues to be a challenge. This is particularly true when it comes to PCI compliance. As a result, I decided I would share some important points about PCI compliance and penetration testing as is required in requirements under 11.3.
A lot of these misunderstandings are clarified in the PCI SSC’s Information Supplement on Penetration Testing. If you have not read this document, I highly recommend it as it explains a lot about the process and why it is important.
Yet, sadly, it too has some flaws in it that have created more problems than the document solved. The biggest of which is on page 7 where it discusses the scope of a penetration test. The only thing those of us can figure is that the group that developed the information supplement assumed that the cardholder data environment (CDE) is implicitly secured by the security measures taken because of the PCI DSS requirements. Therefore, the actual CDE is not required to be tested. How any information security professional could think this is a good practice is beyond a lot of us in the profession, but the information supplement only says that the CDE MAY BE tested.
But then on page 8, comes this quote.
“When access to the CDE is obtained as a result of the testing, the scope of the penetration test may allow the tester to continue exploring inside the network and further the attack against other systems within the CDE, and may also include testing any data-exfiltration prevention (data-loss prevention) controls that are in place.”
“May allow” further exploration? Wait! What? Since when does an attacker stop looking around? Are you kidding us?
In my very humble opinion (and the opinion of those of a LOT of others in the profession), everything that is in scope for PCI compliance needs to be penetration tested. No exceptions! But all QSAs and penetration testers that take this approach have received push back (sometimes significant) from clients and then we usually back off and do exactly as the information supplement states. I would like to tell you that it ends well, but I have had numerous clients come back later (sometimes years later) and complain (some very loudly) to me that I should have stuck to my guns and tested everything as they hold me responsible for why they got hacked.
The next big misunderstanding is what is considered a “passing” or “clean” penetration test? Section 4.1.6 on page 16 of the Penetration Testing information supplement discusses what constitutes a successful penetration test.
“Defining the success criteria for the penetration test allows the entity to set limits on the depth of the penetration test. Without agreeing upon the point at which the penetration test is complete, there is a possibility of the tester exceeding the boundaries and expectations of the target entity. This should be documented in the rules of engagement.”
Sorry, but this is not an excuse for the assessed entity to avoid penetration testing by setting the scope of the test too small. It is up to the client and the penetration tester to agree on the scope based on network and dataflow diagrams. Going back to my earlier statement, a penetration test should test every device/system that is in scope for PCI compliance. When I say everything, that does not mean that every “Connected To” system needs to be penetration tested such as with Domain Controllers or other devices where the penetration tester can confirm that standardized configuration practices are in place and can be proven to be standard. It also does not mean that containers or servers that are spawned to increase capacity in a virtual or cloud environment need to be individually tested as long as the penetration tester can document that all instances are the same.
Another misunderstanding is when exploitable issues are found is that only critical, high or severe exploits need to be addressed in order to get a “passing” test. Sorry to rain on your parade, but penetration testing is not like vulnerability testing where critical, high or severe vulnerabilities need to be fixed within 30 days and the others can be addressed within 90 days. An exploit found by a penetration test MUST BE remediated or mitigated and then retested to get a “passing” test. All someone needs to do is to read the PCI DSS requirements under 11.3 and see that nowhere in those requirements is there ever a reference to critical, high or severe exploits, nor any other remediation criteria. ALL exploits documented must be addressed, regardless of any documented criticality, and either be remediated or mitigated and then retested to get a “passing” test.
A key point about retesting is that the penetration tester does not have to conduct a complete penetration test as a retest. The penetration tester only needs to retest those exploits that were found in the original testing exercise.
Another critical point is when an exploit is mitigated and not remediated. When mitigated, that means that the organization is relying on a variety of controls to detect and alert on the exploit being used. When testing an exploit that is being mitigated, the penetration tester needs to have a clear understanding of those mitigating controls so that when they review their testing results they can attest to the fact that all of the mitigating controls functioned to identify, alert and then allow personnel to stop an attack. If those mitigating controls cannot be confirmed then the mitigation is not considered successful.
Another point of confusion regarding penetration testing is network segmentation testing. Why segmentation testing is bundled with penetration testing is unknown, but it has always been that way. A lot of us would prefer it was a separate requirement in section 11, since it does not require a penetration tester to conduct this testing which is a surprise to most. The person conducting the segmentation testing only needs to be deemed as “qualified” to conduct the testing.
People are also typically surprised that segmentation testing does not require anything more than Nmap installed on a laptop. The key though to a successful segmentation test is that every segment deemed “out of scope” must be tested to ensure that they have no direct communication capability with systems contained in the CDE – inbound to the CDE OR outbound from the CDE. This testing requires having Nmap test all 65,535 TCP and UDP ports to/from every network segment which takes time – a lot of time depending on the number of active IP addresses in the network segment.
For large organizations with tens of thousands of network segments, the idea of testing EVERY segment that is out of scope against the CDE is not realistic. There are two options to address this situation. The first option is to use a tool such as AlgoSec, FireMon, Tufin or the like. Most large organizations have such a tool installed. Using the tool’s database, queries can be run against the CDE and all segments to find any “holes” in those rules. To use this approach though the entire network needs to be in the tool’s database because you are going to test ALL segments and you need to test all the segments. Sadly, most large organization do NOT have all their network segments in the tool, so it is not usable for segmentation testing.
The second option requires an analysis of the firewall and routing rules to determine if there are ways to “sample” network segments that are covered under the same rules in the firewalls and routers. If the segmentation tester can document that say one thousand network segments are all governed by the same set of firewall/router rules, then testing of five randomly selected segments of those thousand segments and getting the same results, those results can be extended to the remaining 995 segments. But the key to this second option is making sure that the rules are exactly the same for all one thousand segments which is sometimes not as easy as it sounds.
Hopefully these clarifications will assist you in conducting and evaluating penetration testing.
PCI Guru 