21
Feb
10

What Is Penetration Testing?

This question comes up more than I would like so it is probably a good discussion topic.  And it pairs up nicely with my previous post regarding passing vulnerability scans.

First, let us get the obvious out of the way.  A penetration test is not a vulnerability scan and a vulnerability scan is not a penetration test.  Yes, I know that the lines are blurring between vulnerability scanning and penetration testing with certain tools.  However, in the true definition of a penetration test, a penetration test is one where you take the output of a vulnerability scan and using the vulnerabilities identified, you attempt to leverage those vulnerabilities to obtain control of the device and ultimately gain control of your objective.  In the case of PCI compliance, the objective is to obtain cardholder data.

Next, penetration testing is not all about the tools.  I do not care whether you use tools like Metasploit, SAINTexploit or Core Impact.  The best penetration testers use very few tools.  But the one tool that all good penetration testers share in common and rely on most is the world’s most high powered computer on the face of the Earth, their brain.  With penetration testing it is not always about the tools, but tools can help.  The key to successful penetration testing is being able to connect the dots based on what vulnerabilities you have available.  You have to change your mindset from one of being a “good guy” to one of “what kind of opportunity do I have and how far can I take it?”  In most cases, the target has very, very few or even no vulnerabilities, but other devices around the target may have vulnerabilities that ultimately may lead to the target being compromised.  It is the ability of the penetration tester to put the path together that is important as the path to a compromise is never a straight line.

Here is a real world example of what I am talking about.

I was working with a company and one of the things that they had not performed was an internal penetration test.  As their QSA, I obviously asked them to conduct a penetration test.  The first question their Director of Security asked was if it was legal for his group to conduct such a test.  This is the best first question to ask and something you should always ask whether you are a consultant or an employee.  Regardless of whether you are a consultant or employee, you should always have a letter signed by an officer of the organization that states you are allowed to conduct the penetration test.  Under federal law, it is a felony to conduct such testing without such permission.  There are a number of examples where well meaning people have been arrested and put in jail because they did not have such approval.

The next question from the Director was what tool they should use.

I stopped the Director right there and said, “Your next question should have been, is my staff qualified to conduct a penetration test?”  Just because people have certifications such as CISSP or CISM does not mean that a person can qualify as a penetration tester.  If a person has a current GIAC Certified Penetration Tester (GPEN) certification, I would consider that person qualified to be a penetration tester.  By the same token, just because someone is a great hacker also does not necessarily qualify them to be a penetration tester.  A good penetration tester needs to not only have the skills, but also needs to document everything about how they got in.  I worked with a great hacker a number of years ago that was probably one of the best at getting into whatever system he put his mind to get into.  However, he was a lousy penetration tester because he failed to document how he did what he did to compromise systems.  As a result, once he was done, he had no documentation to show for all of his work other than the compromised target and a very vague memory of how he got there.  It is the documentation of the compromise that is worth its weight in gold and what you need as a result of a penetration test.  Without such documentation, there is no way to address the shortcomings in security that was used to compromise the target.  In reviewing the Director’s staff, he had a couple of people that I deemed qualified, so we moved forward.

Back to the tool question.  The first tool out of the Director’s mouth was Nessus.  Nessus is a great vulnerability scanner and can do some penetration-like testing, but it is not a true penetration testing tool.  So we used Nessus to get a list of potential vulnerabilities of the in-scope PCI devices and systems and they downloaded a copy of Metasploit to use as their penetration testing tool.  One of the things I do not like about Metasploit is that not all exploits are necessarily available under Metasploit.  Another thing about Metasploit that troubles me is that a lot of the Metasploit exploits are “live” exploits and if successful, compromise the target.  If you want to “neuter” Metasploit exploits, it is up to you and your programming expertise to identify the problem areas and then remove them and still have a valid test.  Regardless, once that target gets compromised, the only option to correct the problem is rebuild that device.  As a result, another set of vulnerability scans and penetration testing have to be done that could cause the whole process to start over again.  Some of the commercial penetration testing tools use “widgets” that get installed in memory for conducting their compromises.  Since these “widgets” are only memory resident, the target only needs to be rebooted to clear them out of the system.  The key thing to note though is that regardless of approach, once a penetration test is done, there is clean up afterwards that must be done in order to ensure security.

Planning an attack is very important.  Anyone can use a tool and get nowhere.  The art in penetration testing is how an attack is constructed.  There needs to be a good analysis of the vulnerability scan to see what opportunities are available.  In the case of my client, there were a very limited number of vulnerabilities with which to work.  There were a couple of low rated vulnerabilities that showed some promise.  One of the penetration testers asked, “Low rated vulnerabilities, how can those be used?”  It all depends on what those low rated vulnerabilities are.  In this case, there were a couple of SMB and NetBIOS vulnerabilities that while rated low, could be used to escalate privileges.  And that is exactly where I suggested they start.  It took the penetration testers a couple of days, but ultimately they were able to leverage those original vulnerabilities to the point that they were able to escalate their privileges to where they were able to penetrate a server that stored cardholder data.

The first piece of good news is that the data stored in the compromised server is encrypted and they could not get to the keys, so the data remained secure even though the server was compromised.  Another piece of good news is that these penetration testers kept good notes on everything they did and had plenty of information on what needed to be fixed to improve their security posture.  The final piece of good news was that no devices were harmed (i.e., crashed) during the conducting of the penetration test.  There were two devices that required replacement after the test because the escalation of privileges left them in an unsecure state.  This was not too onerous since the environment is redundant and the backups were used for testing.

But there was also bad news during the penetration test.  The worst piece of bad news was that even though the penetration test set off a number of alerts, those alerts were not addressed by the network administration group.  The common refrain we heard during the debriefing was that they knew it was a penetration test and therefore just ignored them.  When asked how they knew it was the penetration test and not a valid alert, we got looks of incredulity as though it was just painfully obvious.  However, given that the attack was launched outside of the organization, management found it hard to believe that these people knew this was not a real attack.  As a result, management is taking a hard look at this group as well as looking at ways to make the alerting more effective and require that all alerts be addressed.

So, what are the lessons learned from this exercise?

  • Vulnerability scanning is not penetration testing.  Even if the vulnerability scanner allows for the use of credentials and can correlate internal and external vulnerabilities, it does not replace a properly conducted penetration test.
  • Not everyone can be a penetration tester.  Certifications do not necessarily matter.
  • Penetration testing is not a license to crash every device you test.  Penetration testing is to prove that a compromise can occur and that an objective achieved, not that devices can be crashed.  Crashing devices only proves that a denial of service can be conducted and anyone can prove that.
  • Regardless of the risk rating on vulnerabilities, you cannot discount their value in a penetration test.
  • Planning of the penetration test is like planning a campaign during a war.  Not only does the path to the ultimate objective need to be plotted, but contingencies planned in the event the path to the object is blocked at any point.  Planning includes considering any likely points where the compromise might be noticed.
  • Penetration testing will likely result in service outages and those outages should be taken into consideration during the planning process.  If possible, the penetration test should be conducted in a replica of the production environment.  If the penetration test is conducted in production, then management needs to understand and approve of the likely service outages that will occur as a result of this testing.
  • Penetration testing is not just running some tools and producing a report.  Even with great tools, penetration testing takes time and can take a lot of time compared to vulnerability scanning.  Patience is required.  However, if you are spending more than five days conducting a penetration test, you are likely taking too much time.
  • Regardless of whether you inform others in the organization that a penetration test is being conducted or they are not informed, you should expect that any alerts that are generated are addressed and notification of management occurs just as it should based on your incident response plan.

UPDATE: Here is a great article on 10 tips for successful penetration testing.  http://www.csoonline.com/article/636040/penetration-tests-10-tips-for-a-successful-program

About these ads

4 Responses to “What Is Penetration Testing?”


  1. 1 GWJ
    February 25, 2010 at 4:44 AM

    As a penetration tester of some 10 years experience, I can say this is clearly written by someone who has read a lot about pen-testing but never actually worked week-in-week-out as one. It’s just very wide of the mark in many areas.

    Either that or they do things differently in the states. Which I guess may account for all the massive breaches that keep occurring over there.

    • February 25, 2010 at 7:03 AM

      This is not a “how to” column nor is it a “techie” column. It is meant for people that are involved in complying with PCI and need to have an appreciation of the effort involved as well as ensure they are not getting “smoke blown up their skirt.”

      The post was not meant to be a be all to end all for penetration testing. It was meant to be instructive to non-technical people in what penetration testing is and the fact that it is nothing near a vulnerability scan.

      You have 10 years of experience and you know better. However there are too many people out there that do not have any experience and are accepting all sorts of things as penetration testing when it is not even close.

  2. 3 Mike Williams
    February 21, 2010 at 8:46 PM

    HI PCI Guru,

    I like the topic for this post; this is an issue I run into as well. In the beginning of the post you made the comment, “However, in the true definition of a penetration test, a penetration test is one where you take the output of a vulnerability scan and using the vulnerabilities identified, you attempt to leverage those vulnerabilities to obtain control of the device and ultimately gain control of your objective.”

    In my experience when evaluating pen test teams the good ones use far more for “input” than just the restults of a vulnerability scan (in fact the really good ones don’t use vulnerability scanners at all because they’re too easily detected). My difference of opinion stems from the fact that there aren’t any network vulnerability scanners that are going to tell you that there’s an open wireless access point on the network. Likewise, there isn’t a vulnerability scanner that’s going to tell you that one the DBAs has his life’s history on facebook, responds to every friend request he receives and opens PDF attachments in unsolicited email. Lastly, there isn’t a vulnerability scanner that will tell you that the secretary regularly forgets to lock the front door when she leaves at night. These are just a few simple examples. In my opinion, a pen test is a tactical exercise to test an organization’s security measures. All of them. I’m interested in your thoughts.

    Thanks,
    Mike


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Announcements

FishNet Security is looking for experienced QSAs for their PCI practice. If you are an experienced QSA and are looking for a change, go to the Web site (http://www.fishnetsecurity.com/company/careers), search for 'PCI' and apply.

If you are posting a comment, be patient, as the comments will not be published until they are approved.

If your organization has a PCI opportunity, is in need of assistance with a PCI issue or if you would like the PCI Guru to speak at your meeting, you can contact the PCI Guru at pciguru AT gmail DOT com.

I do allow vendors to post potential solutions in response to issues that I bring up in posts. However, the PCI Guru does not endorse any specific products, so "Caveat Emptor" - let the buyer beware. Also, if I feel that the response is too "sales-ee", I reserve the right to edit or not even authorize the response.

Calendar

February 2010
M T W T F S S
« Jan   Mar »
1234567
891011121314
15161718192021
22232425262728

Enter your email address to subscribe to the PCI Guru blog and receive notifications of new posts by email.

Join 970 other followers


Follow

Get every new post delivered to your Inbox.

Join 970 other followers

%d bloggers like this: