This is a common question that QSAs encounter from clients. The client has an SSAE 18 Controls at a Service Organization (SOC) report from one of their service providers and they want to know if they can use it to satisfy any or all of the requirements in 12.8, 12.9 and 12.11 related to vendor management?
The biggest caveat in this discussion is that the PCI SSC does not sanction the use of any report other than a PCI Attestation Of Compliance (AOC) and/or a PCI Report On Compliance (ROC) in addition to any other PCI reports. The Council has repeatedly stated that if a QSA chooses to rely on an SSAE 18 SOC 2 report (or any other compliance report for that matter), the QSAC and their client accepts the risk if the SSAE 18 SOC 2 does not cover what the QSA claims it covers and therefore relies upon it for fulfilling PCI ROC requirements. As a result, most QSAs will not accept an SSAE 18 SOC 2 report (or any other non-PCI compliance reports) for any reason.
For those of us “recovering” certified public accountant (CPA) types that have conducted SSAE18 audits, we know how to read and interpret these reports. As a result, when we are asked about SSAE 18 SOC 2 reports being relevant, our answer is that, “It depends on what the SOC 2 covers and how it was tested.”
Before we get too deep into this discussion though, we need to define the terminology surrounding this topic. The first thing is that SSAE 18 replaced SSAE 16 as of 2017 even though nothing else appears to have changed. The next key thing anyone needs to know about SSAE 18 is that there are three reports that can come from this reporting series: SOC 1, SOC 2 and SOC 3.
The first, SOC 1, is for financial auditors only. It used to be called a SAS 70 years ago. It is a report focused on financial controls that an external auditor needs to ensure that the financial numbers coming from the third party can be relied upon in their annual audit of their client. Yes, these SOC 1 reports can cover security controls, but that is only in regard to financial systems, not necessarily the third party’s entire environment. In addition, the control coverage is typically not as deep as required for PCI compliance. The bottom line is that any reliance on a SOC 1 report outside of financial systems should never be assumed.
I am going to cover the SOC 3 report next because it covers all of the security domains. The SOC 3 report (also sometimes referred to as the ‘SysTrust’ report) covers the following domains:
- Organization and Management – The criteria relevant to how the organization is structured and the processes the organization has implemented to manage and support people within its operating units.
- Communications – The criteria relevant to how the organization communicates its policies, processes, procedures, commitments, and requirements to authorized users and other parties of the system and the obligations of those parties and users to the effective operation of the system.
- Risk Management and Design and Implementation of Controls – The criteria relevant to how the entity (i) identifies potential risks that would affect the entity’s ability to achieve its objectives, (ii) analyzes those risks, (iii) develops responses to those risks including the design and implementation of controls and other risk mitigating actions, and (iv) conducts ongoing monitoring of risks and the risk management process.
- Monitoring of Controls – The criteria relevant to how the entity monitors the system, including the suitability, and design and operating effectiveness of the controls, and takes action to address deficiencies identified.
- Logical and Physical Access Controls – The criteria relevant to how the organization restricts logical and physical access to the system, provides and removes that access, and prevents unauthorized access to meet the criteria for the principle(s) addressed in the engagement.
- System Operations – The criteria relevant to how the organization manages the execution of system procedures and detects and mitigates processing deviations, including logical and physical security deviations, to meet the objective(s) of the principle(s) addressed in the engagement.
- Change Management – The criteria relevant to how the organization identifies the need for changes to the system, makes the changes following a controlled change management process, and prevents unauthorized changes from being made to meet the criteria for the principle(s) addressed in the engagement.
There are also some additional considerations that are related to Confidentiality specified in the Trust Services Principals and Criteria (TSP), but those are not required to be covered in the SOC 3 report.
Finally, there is the SOC 2 report. The SOC 2 report uses the same TSP as the SOC 3 but with a twist. The third party can select any or all of the seven domains to be assessed. Think of it as a “cafeteria style” assessment. With the SOC 2, the AICPA does not require that all domains be covered (as with the SOC 3), the assessed entity can select only those domains they wish audited. As a result, a third party could select only the ‘Organization and Management’ domain to be assessed and nothing else in their SOC 2 report. Therefore, just because you have a SOC 2 does not mean it covers the domains necessary for your PCI assessment. Like the SOC 3, in addition to the seven domains, the SOC 2 can also cover none, any or all of the additional considerations documented in the TSP.
Within each of these SOC reports there is a Type I and a Type II report. A Type I report is basically worthless from a reliance perspective because no testing of the controls is ever performed. With a Type I report, the auditor is signing off on the fact that the third party has controls defined and formally documented. But without testing, there really is no point to this report. Yet every now and then, I encounter a Type I report that an organization has relied upon for years.
The only report worth anything is a Type II report which tests the control environment to ensure that the controls are functioning as designed. So, when you get that SOC 2 report, you need to make sure you have a Type II report where testing has been performed by the auditor. Even then though, the report might not be as useful as you might think.
I Have A SOC 2 Type II Report From A Service Provider
While you want to read the whole report in detail, when I am pressed for time and cannot read it in its entirety, here is where I focus so that I can get a quick view of what I have. Some CPA firms provide a one-page Executive Summary that gives the reader a quick overview of the report, provides the timeframe the report covers, opinion, exceptions and other useful information. But that is not required by the AICPA so you cannot always rely on such an overview being in every report you receive. When they are available, they can help you focus your quick review efforts even better.
The first thing to do is to read the auditor’s opinion which should be the first section of the report. It is in the form of a letter on the auditor’s letterhead and signed by the auditing firm. The opinion the auditor provides will be either:
- Unqualified – no material control weaknesses or failures were identified.
- Qualified – some material control weaknesses or failures were identified.
- Adverse – significant control weaknesses or failures were identified.
An unqualified opinion is what all organizations desire and what most reports document. But do not be fooled by an unqualified opinion. There still could have been control weaknesses or failures identified but they did not rise to the level of being considered “material”. I have seen some unqualified reports with control weaknesses that I would have considered material as their auditor, so you might still want to contact the organization to get clarification on any weaknesses identified.
A report with a qualified opinion is not the end of the world, but that will all depend upon what control weaknesses or failures created the qualification. Someone misusing their access can be minor compared to not performing backups of servers for months. As a result, you need to read each control weakness to determine the criticality of the control failure as well as review management’s responses to how they addressed or will address the failure. Again, you may find yourself contacting the organization to clarify weaknesses documented.
In my experience, reports with an adverse opinion never get issued to the public. Management sees all of the control failures and weaknesses and then embarks on the long arduous task of cleaning up their control environment.
The next section to look at is the one labeled ‘Information Provided by Independent Service Auditor’ or similar. This is the section that will contain the testing results and will define which of the domains were covered as well as the timeframe the report covers. Most organizations issue SOC reports annually, so you always want to make sure that you have the most current report. If the coverage end date is getting within three months of a year old or more, you should contact the third party and ask them when the next report will be issued. They should inform you that the new report is in progress and give you an estimated date the report will be issued. If they do not give you a succinct answer, I would be concerned.
You need to go through this section looking at a couple of things. The first is to determine which of the domains were covered. While documenting those domains, you also need to review the testing that was performed and at what level of detail those tests were conducted. For example, it is not unusual to see tests for change control cover five random changes but not test those changes for having appropriate documentation, backout instructions and testing, only that the changes were approved. At some point you will need to read this section carefully to determine what, if anything, will cover the testing required by the PCI DSS. But a quick perusal will usually give you an idea of what you are likely going to get out of the SOC 2 for PCI compliance, if you are going to get anything at all.
This leads to the next section of the report you should read. The last section of all SOC reports is usually titled ‘Supplemental Information Provided By [Organization Name]’. This section contains information that was provided by the entity being audited but is not covered by the auditor’s opinion. There can be all sorts of information presented here but the important point to remember is that the auditor did not test or assess the accuracy of that information. So, you need to take any information provided in this section with a bit of skepticism.
It is in the Supplemental Information section that you want to look for a sub-section titled ‘Management’s Response to Control Exceptions’ or similar. Even when an organization has an unqualified opinion, there can still be items listed in this section. If there are items listed, you want to carefully read what those items were and how management addressed or corrected the condition. If you find any control issues and responses that concern you, you should contact the entity and get those discussed so that you are comfortable with the situation. If you cannot get comfortable with the situation, then you may want to consider additional controls at your end to compensate for the control weakness with the third party.
In the next post I will take you through a more thorough review of the SOC report.