On Monday, March 7, the United States Federal Trade Commission (FTC) issued a news release that I am sure got a lot of notice by practice leaders of the PCI qualified security assessor companies (QSAC). On Friday, March 4, the FTC commissioners decided in a 4-0 vote to compel the following QSACs to respond to a 6(b) Special Report order.
- Foresite MSP, LLC;
- Freed Maxick CPAs, P.C.;
- GuidePoint Security, LLC;
- Mandiant;
- NDB LLP;
- PricewaterhouseCoopers LLP;
- SecurityMetrics;
- Sword and Shield Enterprise Security, Inc.; and
- Verizon Enterprise Solutions (also known as CyberTrust)
The first thing that is notable in my mind is that some of the big players in the PCI assessment business are absent from this QSAC list. I am not sure how the FTC arrived at this QSAC list, but it would be interesting to know their methodology.
But even more interesting and concerning is the information the FTC is requesting. From their request, here is a sample of some of the questions they are asking and the information they are seeking.
- For each year of the Applicable Time Period, state the number and percentage of clients for which You completed a Compliance Assessment and for which You declined to provide: a “Compliant” designation on the Attestation of Compliance (“AOC”); or an “In place” designation on the final Report on Compliance (“ROC”).
- For each year of the Applicable Time Period, state the number and percentage of clients for which You completed a Compliance Assessment and for which You provided: a “Non-compliant” designation on the AOC; or a “Not in place” designation on the ROC.
- The extent to which the Company communicates with clients in determining the adequacy of any compensating control. As part of Your response, provide all documents related to a representative Compliance Assessment that considered a compensating control, including all communications between the Company and the client or any third party such as PCI SSC, a Payment Card Network, an Issuing Bank or an Acquiring Bank.
- The policies and procedures for completing a Report on Compliance (“ROC”), including, but not limited to a discussion of whether a draft report is created, whether that draft is shared with the client or any third party such as PCI SSC, a Payment Card Network, an Issuing Bank or an Acquiring Bank, whether the Company accepts input on the draft from the client or any third party, and whether the Company ever makes changes to the draft report based upon the client or other third parties’ input. As part of Your response, provide all documents relating to a representative Compliance Assessment in which You provided a draft of the report to the client and/or any third parties, including a copy of the draft report, any communications with the client or third parties about the draft report, and the final ROC.
- Provide: a copy of the Compliance Assessment with the completion date closest to January 31, 2015; and a copy of a Compliance Assessment completed in 2015 that is representative of the Compliance Assessment that the Company performs. For each Compliance Assessment provided in response to this specification, the Company shall also include a copy of any contract with the client for which the Compliance Assessment was performed, all notes, test results, bidding materials, communications with the client and any other third parties, such as the PCI SSC, a Payment Card Network, an Issuing Bank or an Acquiring Bank, draft reports, the final ROC, and the AOC.
- State whether the Company ever identifies deficiencies in a client’s network during a Compliance Assessment and gives the client the opportunity to remediate the deficiency before the Company completes its final ROC. If so, provide all documents relating to a representative Assessment where the Company gave the client an opportunity to remediate before completing the ROC, including any communications between the Company and the client or any third parties such as PCI SSC, a Payment Card Network, an Issuing Bank or an Acquiring Bank, and the final ROC and AOC.
- State whether the Company ever identifies deficiencies in a client’s network during a Compliance Assessment and issues a final ROC before the deficiencies are remedied based on assurances that the client will remedy the deficiencies in the future. As part of Your response, provide copies of all policies and procedure related to remedying deficiencies.
- State whether the Company has any policies or procedures relating to potential conflicts of interest, including, but not limited to, any policies that prevent the Company from providing Compliance Assessments to clients to which it has also provided another type of service, or that concern the marketing or provision of other services to clients for which You have provided a Compliance Assessment. As part of Your response, provide copies of all relevant policies and procedures.
- State the annual number of the Company’s Compliance Assessment clients that have suffered a Breach in the year following the Company’s completion of the Assessment for each year of the Applicable Time Period. For each such client, state whether it was subsequently determined not to be PCI compliant and provide the date of the initial Compliance Assessment and any communications between the Company and client or any third parties such as PCI SSC, a Payment Card Network, an Issuing Bank or an Acquiring Bank related to the Breach.
All of these questions lead one to believe that the FTC is looking to confirm that the PCI assessment process is a sham.
It will be very interesting to see how the FTC interprets the results of this effort. However, based on these questions and how I know they will end up being answered, I would venture to say that the result will be the government getting into the data security game with regulations.
PCI Guru 
I think this is the reason: The FTC also opened investigations into nine companies identified by Tiversa, according to a 2015 staff report from the House Committee on Oversight & Government Reform, which did not name the companies. The status of those cases is not clear.
http://www.reuters.com/article/us-tiversa-doj-probe-exclusive-idUSKCN0WK027?mod=djemRiskCompliance
The nine companies: Foresite MSP, LLC; Freed Maxick CPAs, P.C.; GuidePoint Security, LLC; Mandiant; NDB LLP; PricewaterhouseCoopers LLP; SecurityMetrics; Sword and Shield Enterprise Security, Inc.; and Verizon Enterprise Solutions (also known as CyberTrust).
https://www.ftc.gov/news-events/press-releases/2016/03/ftc-study-credit-card-industry-data-security-auditing
There’s one very big QSAC in particular that I would put at the top of the list. One which has been involved as the responsible QSA for several of the huge US retail companies that suffered huge breaches in recent years. I’m sure you know which one I mean…
Very curious to why it didn’t make the list.
I think the FTC is very well aware of the QSAC you and I both are thinking. 😉
As they are asking for RoCs they seem to be going for how the big merchants and service providers get assessed. My first experience as a QSA was with a large service provider. We obtained the previous RoC but it quickly became clear that a lot of the ‘evidence’ cited by the previous QSA (a large QSAC) was very dodgy. After a long and painful process we got to a point where we could sign off but we weren’t invited back. What made it even more annoying was that the PCI SSC called the report for QA and took an extraordinarily pedantic and nit picking approach to it. It was especially galling when we knew what the previous QSA had got away with.
I guess what I am saying is that I think there is some bad practice that should be stopped but I hope it doesn’t just make life even more difficult for QSAs trying to do a decent job.
The Council’s AQM process back during the v1 and v2 days were very poor because the person running the process treated assessments like they were AICPA SSAE 16 reports, not the simple assessments at a point in time. The AQM process for v1 was essentially being graded after the fact against a grading scale the QSAC had never, ever seen before. That is why a lot of QSACs went into remediation because we were never told how the Council wanted ROCs written until that very first AQM review. Let alone the fact that the Council at the time was recommending that QSACs develop ROC templates that would contain the correct wording to pass the AQM process. The idea being that all the QSA had to do was a mass replacement of keywords and they would have a ROC completed. One QSAC took that approach to the extreme and created what we in the industry called the “ROC-O-Matic” that was the penultimate in ROC automation. Fortunately, v3 of the standard took that template approach away and QSAs have no choice but to perform and document their work. However, v3 ROCs now take a lot of time to write as a result. Something I don’t think the Council intended to have happen.
This is interesting, they seem to be getting at the conflict of interest ingrained in the whole auditee paying the auditor and the resultant dubious practices.
However having had involvement with more reputable players in the industry, I can say that there are many times when organisations are deemed not compliant at the start of engagements. In fact this is not the exception but the norm for these companies.
Although in saying this I have heard of players in the field that are much more “flexible” with their definition of compliance.
I was particularly troubled by the FTC’s questioning around compliant and non-compliant assessments for that very reason. Many times a QSA starts out an assessment and then it gets turned into a Gap Assessment because of the amount/significance of the compliance issues identified. The QSA then goes away and comes back to conduct the ROC assessment once the gaps have been all remediated. As a result, the original ROC was not issued because of the gaps found and only a compliant ROC was issued later.
Hmm… I am not even sure how I feel about “…the government getting into the data security game with regulations”. On one hand, it could give PCI DSS (and other standards under the umbrella) the pull and the teeth it requires to be taken seriously (as opposed to the tick-boxes exercise).
The FTC should be asking the same questions of the Big 4 accounting firms for SSAE16 SOC2s…
As a recovering Big 4 alum, I can tell you that will NOT happen. LOL!
There’s no such as SSAE16 SOC2 as SSAE16 are reported as SOC1.
But I agree, if the FTC was serious, they’d also look at shady practices from these accounting firms.
Yes actually there are a SSAE16 SOC 2 and a SOC 3 reports. The SOC 3 is the replacement for the old Webtrust/Systrust reports. The SOC 2 is a specialized version of the SOC 3 that allows the organization to pick the domains they wish to have assessed. The SOC 1 is the old SAS 70 and has two varieties, the Type 1 (no testing of controls) and the Type 2 (testing of controls over a reporting period).
No there isn’t an SSAE 16 SOC 2. SSAE 16 is only used as a SOC 1 report for controls relevant to internal controls over financial reporting, leveraging AT 801 (Reporting on Controls at a Service. Organization) while SOC 2 reports are for controls at a service organization that are relevant to security, availability, processing integrity confidentiality, or privacy, leveraging AT 101 (Attest Engagements). And SOC 3 is the publicly-available report linked to a SOC 2 report.
You’re also incorrect with your types. You’re right that Type 2 is for the testing of controls over a reporting period but there’s also testing of controls in a Type 1 but just as a point in time. So you would test it as of today instead of covering a period of 6 months.
SOC 1 is used to support the production of financial statements while SOC 2 is used by idiots who haven’t read the underlying standard (Trust Services Principles) which has to be the worst standard (if we can call it that) ever created.
Having done SAS 70 and SSAE 16 reports for over 25 years, I respectfully disagree with you. Per the AICPA SSAE 16 standard, there are three Service Organization reports defined – SOC 1, SOC 2 and SOC 3.
The SOC 1 report is the replacement for the old SAS 70 standard and comes in two types as did the SAS 70, type (1) and type (2). A type 1 report describes the control environment, but no testing is done to confirm that those controls are in fact operating as designed. A type 2 report tests the documented controls over a period of time, at least for a minimum of 6 months or more typically over a year. The key to the SSAE 16 SOC 1 is that it is for financial reporting purposes only and has only tangential use for any other purpose even though a lot of IT and internal audit organizations used them for other purposes.
The second report is the SOC 2 which is a test of the controls over a number of security domains. The service organization can select the number of domains they wish to have audited. But once selected, those domains must be fully assessed. The service organization may not “cherry pick” the audit areas documented in the domains selected.
The final report is the SOC 3 which is replaces the AICPA’s WebTrust and SysTrust reports. The SOC 3 is also a security oriented audit. But unlike the SOC 2, the service organization must be audited over all of the domains, not just ones they select.
Guru, you probably have the best website for real-life PCI issues but please educate yourself over SOC reports as you’re wrong, a SOC 2 report cannot be audited using SSAE 16.
SSAE 16 reports (SOC 1) are leveraging AT 801
SOC 2 reports are leveraging AT 101 as the attestation standard while using the Trust Services Principles as a source of criterias.
SOC 3 reports are just summary of SOC 2 reports that can be publicly displayed (ex. on an organization website)
So to go back to your comment, there’s NO such thing as a SSAE 16-based SOC 2 report but please don’t trust me, see for yourself at the following locations:
http://www.aicpa.org/Research/Standards/AuditAttest/Pages/SSAE.aspx
http://www.ssae16.org/white-papers/aicpa-service-organization-control-soc-reports.html
I am not arguing over what AT standard is being relied upon as that only truly matters to CPAs. So while the SSAE 16 reports use different auditing standards for evidence and sampling, the SOC 2 and SOC 3 reports are still defined under and part of SSAE 16. That is all I was trying to get across.
Your post #16 is still incorrect. SOC 2/3 reports are not defined under and part of SSAE 16. Go read AT sec. 801 and tell me where you find references to SOC 2/3.
That methodology issue aside, these reports are mostly useless and it would be nice for the FTC to include them in their scope.
Learn something new every day. But I do not feel quite so bad as I feel the AICPA has done a very poor job disassociating SSAE 16 and the “Service Organization Controls” auditing process properly given they use the SOC terminology across the board for the reports. Thanks for the clarification.
No problem … they are subtleties I live with everyday since I’m working for a large service provider and for 99% of the contracts we signed, we have tons of education to do since they all ask for SOC1 stuff and we systematically refuse since none of our services are remotely relevant to our customers ICFR.
PCI assessments have their strengths and weaknesses but as least they’re based on a proper, but imperfect, security standard. Something that can’t be said of all these SOC reports. Even SOC2 can”t be considered as a real security standard since it’s only built around principles and criterias and it has nothing specific about controls which should be derived from existing standards like ISO 27002, PCI DSS, NIST 800-53 or others.
I hear you. I had a client years ago that we did an Agreed Upon Procedures assessment for their customers because their customers’ auditors demanded such a report for SOX compliance. We argued ad-nausium that there was no such SOX requirement for the telecommunications services being provided to their customers, but the other audit firms refused to back down. We were at least able to talk them into an Agreed Upon Procedures report from a SAS 70 Type II they wanted. It was great money for a project that provided little to no value from a financial audit perspective. But this is what happens when auditors run amok.
When SOC 2 came out, there was a lot of discussion regarding the testing within the domains. Even from the AICPA there was limited clarity regarding what an auditor could do. Some of our CPAs argued that we could use ISO, PCI, NIST, HIPAA and the like for testing. Others argued that was not the case and that only the testing in the SOC 2 program were allowed. I know in our firm, the SOC 2 “purists” won out, but I have seen SOC 2 reports from other firms that do seem to follow the high levels of ISO and PCI. I say high levels because if AT 101 testing were applied at lower levels clients would not pay for that level of testing nor would they likely pass testing.
Your SOC2 purists may have won it but at last, the AICPA had the wisdom, for the 2014 revision, to clarify that and they were quite clear in stating the following:
Appendix B — Illustrative Risks and Controls
.19 The illustrative risks and controls presented in this appendix are for illustrative purposes only. They are based on a hypothetical entity in a hypothetical industry. They are not intended to be a comprehensive set of risks and controls or applicable to any particular entity. Accordingly, they should not be used as a checklist of risks and controls for the criteria. Practitioners should consider using other frameworks such as, NIST 800-53, Cloud Controls Matrix (CCM) for such guidance.
Unfortunately this is probably a good thing and overdue
In my experience with Securitymetrics, they are just a money making factory churning out PCI “certificates” based on answers given to them in a web form by someone that doesn’t really understand what they are typing….which is why I’ve seen so many compliant companies that have actually filled in the wrong SAQ but have been approved anyway, as nobody checks.
Are the FTC trying to weed out these companies as providing no value rather than PCI itself?
Don’t really know much about the FTC not being American…but kind of sound like our Office of Fair Trading or Information Commissioner’s Office.
Office of Fair Trading would be more in line with our FTC.
All of the certifications targeted at small online businesses are just a crappy Q&A. Think anyone partnered with PayPal, Braintree, Stripe, etc.
Companies choosing the wrong SAQs are as much the fault of the council as the assessors. They are meant to be “self” assessments, right? The SAQs use such confusing wording even for experienced sysadmins that it’s clearly a lawyer-esque money making scheme.
What SAQ a merchant should use is up to the acquiring bank to determine – not the Council or the QSA. Unfortunately, most acquiring banks are ill equipped to make such a decision let alone arbitrate PCI compliance issues. QSAs have complained to the Council since day one about the acquiring banks shirking their responsibilities in the equation, but little has been done to correct that situation.