This question comes up more than I would like so it is probably a good discussion topic. And it pairs up nicely with my previous post regarding passing vulnerability scans.
First, let us get the obvious out of the way. A penetration test is not a vulnerability scan and a vulnerability scan is not a penetration test. Yes, I know that the lines are blurring between vulnerability scanning and penetration testing with certain tools. However, in the true definition of a penetration test, a penetration test is one where you take the output of a vulnerability scan and using the vulnerabilities identified, you attempt to leverage those vulnerabilities to obtain control of the device and ultimately gain control of your objective. In the case of PCI compliance, the objective is to obtain cardholder data.
Next, penetration testing is not all about the tools. I do not care whether you use tools like Metasploit, SAINTexploit or Core Impact. The best penetration testers use very few tools. But the one tool that all good penetration testers share in common and rely on most is the world’s most high powered computer on the face of the Earth, their brain. With penetration testing it is not always about the tools, but tools can help. The key to successful penetration testing is being able to connect the dots based on what vulnerabilities you have available. You have to change your mindset from one of being a “good guy” to one of “what kind of opportunity do I have and how far can I take it?” In most cases, the target has very, very few or even no vulnerabilities, but other devices around the target may have vulnerabilities that ultimately may lead to the target being compromised. It is the ability of the penetration tester to put the path together that is important as the path to a compromise is never a straight line.
Here is a real world example of what I am talking about.
I was working with a company and one of the things that they had not performed was an internal penetration test. As their QSA, I obviously asked them to conduct a penetration test. The first question their Director of Security asked was if it was legal for his group to conduct such a test. This is the best first question to ask and something you should always ask whether you are a consultant or an employee. Regardless of whether you are a consultant or employee, you should always have a letter signed by an officer of the organization that states you are allowed to conduct the penetration test. Under federal law, it is a felony to conduct such testing without such permission. There are a number of examples where well meaning people have been arrested and put in jail because they did not have such approval.
The next question from the Director was what tool they should use.
I stopped the Director right there and said, “Your next question should have been, is my staff qualified to conduct a penetration test?” Just because people have certifications such as CISSP or CISM does not mean that a person can qualify as a penetration tester. If a person has a current GIAC Certified Penetration Tester (GPEN) certification, I would consider that person qualified to be a penetration tester. By the same token, just because someone is a great hacker also does not necessarily qualify them to be a penetration tester. A good penetration tester needs to not only have the skills, but also needs to document everything about how they got in. I worked with a great hacker a number of years ago that was probably one of the best at getting into whatever system he put his mind to get into. However, he was a lousy penetration tester because he failed to document how he did what he did to compromise systems. As a result, once he was done, he had no documentation to show for all of his work other than the compromised target and a very vague memory of how he got there. It is the documentation of the compromise that is worth its weight in gold and what you need as a result of a penetration test. Without such documentation, there is no way to address the shortcomings in security that was used to compromise the target. In reviewing the Director’s staff, he had a couple of people that I deemed qualified, so we moved forward.
Back to the tool question. The first tool out of the Director’s mouth was Nessus. Nessus is a great vulnerability scanner and can do some penetration-like testing, but it is not a true penetration testing tool. So we used Nessus to get a list of potential vulnerabilities of the in-scope PCI devices and systems and they downloaded a copy of Metasploit to use as their penetration testing tool. One of the things I do not like about Metasploit is that not all exploits are necessarily available under Metasploit. Another thing about Metasploit that troubles me is that a lot of the Metasploit exploits are “live” exploits and if successful, compromise the target. If you want to “neuter” Metasploit exploits, it is up to you and your programming expertise to identify the problem areas and then remove them and still have a valid test. Regardless, once that target gets compromised, the only option to correct the problem is rebuild that device. As a result, another set of vulnerability scans and penetration testing have to be done that could cause the whole process to start over again. Some of the commercial penetration testing tools use “widgets” that get installed in memory for conducting their compromises. Since these “widgets” are only memory resident, the target only needs to be rebooted to clear them out of the system. The key thing to note though is that regardless of approach, once a penetration test is done, there is clean up afterwards that must be done in order to ensure security.
Planning an attack is very important. Anyone can use a tool and get nowhere. The art in penetration testing is how an attack is constructed. There needs to be a good analysis of the vulnerability scan to see what opportunities are available. In the case of my client, there were a very limited number of vulnerabilities with which to work. There were a couple of low rated vulnerabilities that showed some promise. One of the penetration testers asked, “Low rated vulnerabilities, how can those be used?” It all depends on what those low rated vulnerabilities are. In this case, there were a couple of SMB and NetBIOS vulnerabilities that while rated low, could be used to escalate privileges. And that is exactly where I suggested they start. It took the penetration testers a couple of days, but ultimately they were able to leverage those original vulnerabilities to the point that they were able to escalate their privileges to where they were able to penetrate a server that stored cardholder data.
The first piece of good news is that the data stored in the compromised server is encrypted and they could not get to the keys, so the data remained secure even though the server was compromised. Another piece of good news is that these penetration testers kept good notes on everything they did and had plenty of information on what needed to be fixed to improve their security posture. The final piece of good news was that no devices were harmed (i.e., crashed) during the conducting of the penetration test. There were two devices that required replacement after the test because the escalation of privileges left them in an unsecure state. This was not too onerous since the environment is redundant and the backups were used for testing.
But there was also bad news during the penetration test. The worst piece of bad news was that even though the penetration test set off a number of alerts, those alerts were not addressed by the network administration group. The common refrain we heard during the debriefing was that they knew it was a penetration test and therefore just ignored them. When asked how they knew it was the penetration test and not a valid alert, we got looks of incredulity as though it was just painfully obvious. However, given that the attack was launched outside of the organization, management found it hard to believe that these people knew this was not a real attack. As a result, management is taking a hard look at this group as well as looking at ways to make the alerting more effective and require that all alerts be addressed.
So, what are the lessons learned from this exercise?
- Vulnerability scanning is not penetration testing. Even if the vulnerability scanner allows for the use of credentials and can correlate internal and external vulnerabilities, it does not replace a properly conducted penetration test.
- Not everyone can be a penetration tester. Certifications do not necessarily matter.
- Penetration testing is not a license to crash every device you test. Penetration testing is to prove that a compromise can occur and that an objective achieved, not that devices can be crashed. Crashing devices only proves that a denial of service can be conducted and anyone can prove that.
- Regardless of the risk rating on vulnerabilities, you cannot discount their value in a penetration test.
- Planning of the penetration test is like planning a campaign during a war. Not only does the path to the ultimate objective need to be plotted, but contingencies planned in the event the path to the object is blocked at any point. Planning includes considering any likely points where the compromise might be noticed.
- Penetration testing will likely result in service outages and those outages should be taken into consideration during the planning process. If possible, the penetration test should be conducted in a replica of the production environment. If the penetration test is conducted in production, then management needs to understand and approve of the likely service outages that will occur as a result of this testing.
- Penetration testing is not just running some tools and producing a report. Even with great tools, penetration testing takes time and can take a lot of time compared to vulnerability scanning. Patience is required. However, if you are spending more than five days conducting a penetration test, you are likely taking too much time.
- Regardless of whether you inform others in the organization that a penetration test is being conducted or they are not informed, you should expect that any alerts that are generated are addressed and notification of management occurs just as it should based on your incident response plan.
UPDATE: Here is a great article on 10 tips for successful penetration testing. http://www.csoonline.com/article/636040/penetration-tests-10-tips-for-a-successful-program
PCI Guru 