COPYRIGHT NOTICE — This document is intended for internal use. It cannot be distributed to or reproduced by third parties without prior written permission from the Copyright Coordinator for the Office of the Auditor General of Canada. This includes email, fax, mail and hand delivery, or use of any other method of distribution or reproduction. CPA Canada Handbook sections and excerpts are reproduced herein for your non-commercial use with the permission of The Chartered Professional Accountants of Canada (“CPA Canada”). These may not be modified, copied or distributed in any form as this would infringe CPA Canada’s copyright. Reproduced, with permission, from the CPA Canada Handbook, The Chartered Professional Accountants of Canada, Toronto, Canada.
7040 Audit Conclusion
Jul-2020
Overview
This section presents the requirements pertaining to the audit conclusion. It provides guidance on the four types of conclusion that are possible to reach as well as guidance on what to consider when forming a conclusion. There is also a specific reference to how to form a special examination opinion, including the concept of significant deficiency.
CSAE 3001 Requirements
21. If an objective in this CSAE or a relevant subject-matter-specific CSAE cannot be achieved, the practitioner shall evaluate whether this requires the practitioner to modify the practitioner’s conclusion or withdraw from the engagement (where withdrawal is possible under applicable law or regulation). Failure to achieve an objective in a relevant CSAE represents a significant matter requiring documentation in accordance with paragraph 82 of this CSAE.
28. If the engaging party imposes a limitation on the scope of the practitioner’s work in the terms of a proposed direct engagement such that the practitioner believes the limitation will result in the practitioner disclaiming a conclusion on the underlying subject matter, the practitioner shall not accept such an engagement as an assurance engagement, unless required by law or regulation to do so. (Ref: Para. A156(c))
34. In some cases, law or regulation of the relevant jurisdiction prescribe the layout or wording of the assurance report. In these circumstances, the practitioner shall evaluate:
(a) Whether intended users might misunderstand the assurance conclusion; and
(b) If so, whether additional explanation in the assurance report can mitigate possible misunderstanding.
If the practitioner concludes that additional explanation in the assurance report cannot mitigate possible misunderstanding, the practitioner shall not accept the engagement, unless required by law or regulation to do so. An engagement conducted in accordance with such law or regulation does not comply with CSAEs. Accordingly, the practitioner shall not include any reference within the assurance report to the engagement having been conducted in accordance with this CSAE or any other CSAE(s) (see also paragraph 75).
48. If it is discovered after the engagement has been accepted that some or all of the underlying subject matter is not appropriate for an assurance engagement, the practitioner shall consider withdrawing from the engagement, if withdrawal is possible under applicable law or regulation. If the practitioner continues with the engagement, the practitioner shall express a qualified conclusion or disclaimer of conclusion, as appropriate in the circumstances. (Ref: Para. A89)
49. The practitioner shall consider significance when: (Ref: Para. A90-A98)
[...]
(b) Evaluating whether the underlying subject matter is free from significant deviation.
56. The practitioner shall consider whether individual deviations identified during the engagement (other than those that are clearly trivial) have characteristics, for example a root cause or a problematic pattern, that indicate the aggregate effect of individual deviations is likely to be significant. (Ref: Para A120)
65. If one or more of the requested written representations are not provided or the practitioner concludes that there is sufficient doubt about the competence, integrity, ethical values, or diligence of those providing the written representations, or that the written representations are otherwise not reliable, the practitioner shall: (Ref: Para. A140)
(a) Discuss the matter with the appropriate party(ies);
(b) Reevaluate the integrity of those from whom the representations were requested or received and evaluate the effect that this may have on the reliability of representations (oral or written) and evidence in general; and
(c) Take appropriate actions, including determining the possible effect on the conclusion in the assurance report.
68. The practitioner shall evaluate the sufficiency and appropriateness of the evidence obtained in the context of the engagement and, if necessary in the circumstances, attempt to obtain further evidence. The practitioner shall consider all relevant evidence, regardless of whether it appears to corroborate or to contradict the measurement or evaluation of the underlying subject matter against the applicable criteria. If the practitioner is unable to obtain necessary further evidence, the practitioner shall consider the implications for the practitioner’s conclusion in paragraph 69. (Ref: Para. A147-A153)
69. The practitioner shall form a conclusion about whether the underlying subject matter is free from significant deviation. In forming that conclusion, the practitioner shall consider the practitioner’s conclusion in paragraph 68 regarding the sufficiency and appropriateness of evidence obtained and an evaluation of whether identified deviations are significant, individually or in the aggregate. (Ref: Para. A5, A120 and A154-A155)
70. If the practitioner is unable to obtain sufficient appropriate evidence, a scope limitation exists and the practitioner shall express a qualified conclusion, disclaim a conclusion, or withdraw from the engagement, where withdrawal is possible under applicable law or regulation, as appropriate. (Ref: Para. A156-A158)
71. The assurance report shall be in writing and shall contain a clear expression of the practitioner’s conclusion about the underlying subject matter. (Ref: Para. A4, A159-A161)
72. The practitioner’s conclusion shall be clearly separated from information or explanations that are not intended to affect the practitioner’s conclusion, including any findings related to particular aspects of the engagements, recommendations or additional information included in the assurance report. The wording used shall make it clear that findings, recommendations or additional information is not intended to detract from the practitioner’s conclusion. (Ref: Para. A159-A161)
73. The assurance report shall include at a minimum the following basic elements:
[...]
(l) The practitioner's conclusion on the objective of the engagement: (Ref: Para. A2-A4, A176-A181)
(i) When appropriate, the conclusion shall inform the intended users of the context in which the practitioner's conclusion is to be read. (Ref: Para. A178)
(ii) In a reasonable assurance engagement, the conclusion shall be expressed in a positive form. (Ref: Para. A177)
[...]
(iv) The conclusion in (ii) or (iii) shall be phrased using appropriate words for the underlying subject matter and applicable criteria given the engagement circumstances.
(v) When the practitioner expresses a modified conclusion, the report shall contain:
a. A section that provides a description of the matter(s) giving rise to the modification; and
b. A section that contains the practitioner's modified conclusion. (Ref: Para. A181)
[...]
75. If the practitioner is required by law or regulation to use a specific layout or wording of the assurance report, the assurance report shall refer to this or other CSAEs only if the assurance report includes, at a minimum, each of the elements identified in paragraph 73.
76. The practitioner shall express an unmodified conclusion when the practitioner concludes:
(a) In the case of a reasonable assurance engagement, that the underlying subject matter complies, in all significant respects, with the applicable criteria; or
[...]
77. If the practitioner considers it necessary to communicate a matter other than those specifically related to the underlying the subject matter that, in the practitioner’s judgment, is relevant to intended users’ understanding of the engagement, the practitioner’s responsibilities or the assurance report, and this is not prohibited by law or regulation, the practitioner shall do so in a paragraph in the assurance report, with an appropriate heading, that clearly indicates the practitioner’s conclusion is not modified in respect of the matter.
78. The practitioner shall express a modified conclusion in the following circumstances:
(a) When, in the practitioner’s professional judgment, a scope limitation exists and the effect of the matter could be significant (see paragraph 70). In such cases, the practitioner shall express a qualified conclusion or a disclaimer of conclusion.
(b) When, in the practitioner’s professional judgment, there is a significant deviation in the underlying subject matter. In such cases, the practitioner shall express a qualified conclusion or adverse conclusion. (Ref: Para. A190)
79. The practitioner shall express a qualified conclusion when, in the practitioner’s professional judgment, the effects, or possible effects, of a matter are not so significant and pervasive as to require an adverse conclusion or a disclaimer of conclusion. A qualified conclusion shall be phrased to inform the intended users of the effects, or possible effects, of the matter to which the qualification relates. (Ref: Para. A187-A190)
80. If the practitioner expresses a modified conclusion because of a scope limitation but is also aware of a matter(s) that causes a significant deviation in the underlying subject matter, the practitioner shall include in the assurance report a clear description of both the scope limitation and the matter(s) that causes the significant deviation.
CSAE 3001 Application Material
A2. The practitioner in a performance audit describes in the report the objective of the engagement and the underlying subject matter so that the reader can understand and properly interpret the results. The wording of the objective would be determined by the circumstances of the engagement. For example, the objective for a performance audit may be to conclude whether the entity being audited has adequately managed a program so that the entity’s key responsibilities under that program have been met. The practitioner’s conclusion relates to the objective and scope of the engagement and follows logically from the description of the criteria and findings. If the engagement has more than one objective, the assurance report provides a conclusion on each objective.
A4. Where the underlying subject matter is made up of a number of aspects, separate conclusions may be provided on each aspect. All such separate conclusions do not need to relate to the same level of assurance. Rather, each conclusion is expressed in the form that is appropriate to either a reasonable assurance engagement or a limited assurance engagement. References in this CSAE to the conclusion in the assurance report include each conclusion when separate conclusions are provided.
A98. Concluding on the significance of the deviations identified as a result of the procedures performed requires professional judgment. For example:
- The applicable criteria for a performance audit for a hospital’s emergency department may include the speed of the services provided, the quality of the services, the number of patients treated during a shift, and benchmarking the cost of the services against other similar hospitals. If three of these applicable criteria are satisfied but one applicable criterion is not satisfied by a small margin, then professional judgment is needed to conclude whether the hospital’s emergency department represents value for money as a whole.
- In a compliance engagement, the entity may have complied with nine provisions of the relevant law or regulation, but did not comply with one provision. Professional judgment is needed to conclude whether the entity complied with the relevant law or regulation as a whole. For example, the practitioner may consider the importance of the provision with which the entity did not comply, as well as the relationship of that provision to the remaining provisions of the relevant law or regulation.
A120. “Clearly trivial” is not another expression for “not significant.” Matters that are clearly trivial will be of a wholly different (smaller) order of importance than significance determined in accordance with paragraph 49, and will be matters that are clearly inconsequential, whether taken individually or in aggregate and whether judged by any criteria of size, nature or circumstances. When there is any uncertainty about whether one or more items are clearly trivial, the matter is considered not to be clearly trivial.
A152. Whether sufficient appropriate evidence has been obtained on which to base the practitioner’s conclusion is a matter of professional judgment.
A155. The practitioner’s professional judgment as to what constitutes sufficient appropriate evidence is influenced by such factors as the following:
- Importance of a potential deviation and the likelihood of its having a significant effect, individually or when aggregated with other potential deviations, on the practitioner’s report.
- Effectiveness of the appropriate party(ies)’s responses to address the known risk of significant deviation.
- Experience gained during previous assurance engagements with respect to similar potential deviations.
- Results of procedures performed, including whether such procedures identified specific deviations.
- Source and reliability of the available information.
- Persuasiveness of the evidence.
- Understanding of the appropriate party(ies) and its environment.
A156. A scope limitation may arise from:
(a) Circumstances beyond the control of the appropriate party(ies). For example, documentation the practitioner considers to be necessary to inspect may have been accidentally destroyed;
(b) Circumstances relating to the nature or timing of the practitioner’s work. For example, a physical process the practitioner considers to be necessary to observe may have occurred before the practitioner’s engagement; or
(c) Limitations imposed by the responsible party or the engaging party on the practitioner that, for example, may prevent the practitioner from performing a procedure the practitioner considers to be necessary in the circumstances. Limitations of this kind may have other implications for the engagement, such as for the practitioner’s consideration of engagement risk and engagement acceptance and continuance.
A157. An inability to perform a specific procedure does not constitute a scope limitation if the practitioner is able to obtain sufficient appropriate evidence by performing alternative procedures.
A177. An example of a conclusion expressed in a form appropriate for a reasonable assurance engagement is: “In our opinion, the entity has complied, in all significant respects, with XYZ law.”
A178. It may be appropriate to inform the intended users of the context in which the practitioner’s conclusion is to be read when the assurance report includes an explanation of particular characteristics of the underlying subject matter of which the intended users should be aware. The practitioner’s conclusion may, for example, include wording such as: “This conclusion has been formed on the basis of the matters outlined elsewhere in this independent assurance report.”
A180. Forms of expression which may be useful for underlying subject matters include, for example, “in compliance with” or “in accordance with.”
A181. Inclusion of a heading above paragraphs containing modified conclusions, and the matter(s) giving rise to the modification, aids the understandability of the practitioner’s report. Examples of appropriate heading include “Qualified Conclusion,” “Adverse Conclusion,” or “Disclaimer of Conclusion” and “Basis for Qualified Conclusion,” “Basis for Adverse Conclusion,” as appropriate.
A187. The words “except for” are commonly used to indicate the matter(s) to which a qualification relates. However, other wording may be used to clearly indicate those matter(s).
A188. The term “pervasive” describes the effects on the underlying subject matter of deviations or the possible effects on the underlying subject matter of deviations, if any, that are undetected due to an inability to obtain sufficient appropriate evidence. Pervasive effects on the underlying subject matter are those that, in the practitioner’s professional judgment:
(a) Are not confined to specific aspects of the underlying subject matter; or
(b) If so confined, represent or could represent a substantial proportion of the underlying subject matter.
A189. The nature of the matter, and the practitioner’s judgment about the pervasiveness of the effects or possible effects on the underlying subject matter, affects the type of conclusion to be expressed.
A190. Examples of qualified and adverse conclusions and a disclaimer of conclusion are:
- Qualified conclusion (an example for limited assurance engagements with a significant deviation) – “Based on the procedures performed and the evidence obtained, except for the effect of the matter described in the Basis for Qualified Conclusion section of our report, nothing has come to our attention that causes us to believe that the entity has not complied, in all significant respects, with XYZ law.”
- Qualified conclusion (an example for reasonable assurance engagements with a significant deviation) – “We conclude that the entity increased the capacity of its facilities in a manner that meets its needs in the short term. However, the entity did not develop a long-term plan to ensure its capacity needs will be met in the future.”
- Adverse conclusion (an example for a significant and pervasive deviation for both reasonable assurance and limited assurance engagements) – “Because of the importance of the matter described in the Basis for Adverse Conclusion section of our report, the entity has not complied, in all significant respects, with XYZ law.”
- Disclaimer of conclusion (an example for a significant and pervasive limitation of scope for both reasonable assurance and limited assurance engagements) – “Because of the importance of the matter described in the Basis for Disclaimer of Conclusion section of our report, we have not been able to obtain sufficient appropriate evidence to form a conclusion on the on whether the entity has complied, in all significant respects, with XYZ law. Accordingly, we do not express a conclusion on such compliance.”
Financial Administration Act Requirements for Special Examinations
Section 139(1) An examiner shall, on completion of the special examination, submit a report on his findings to the board of directors of the corporation examined.
(2) The report of an examiner under subsection (1) shall include
(a) a statement whether in the examiner’s opinion, with respect to the criteria established pursuant to subsection 138(3), there is reasonable assurance that there are no significant deficiencies in the systems and practices examined; and
(b) a statement of the extent to which the examiner relied on internal audits.
OAG Policy
Audits shall have a clear conclusion against the audit objective. [Nov-2016]
OAG Guidance
What the CSAE 3001 means for the audit conclusion
The standard requires that the audit concludes against the audit objective and that the conclusion is clearly indicated in the audit report.
The standard also requires that the engagement leader supports the audit conclusion with sufficient appropriate evidence, and that he/she forms a conclusion about whether the subject matter is free from significant deviation (see OAG Audit 2020 Significance).
If the evidence shows that one or more of the audit criteria have not been met, then the engagement leader must use professional judgment to decide whether to express a “reservation” in the form of a qualified or an adverse conclusion, and explain the reason for the reservation in the audit report.
If the team has been unable to obtain sufficient appropriate evidence regarding an entity’s conformity with any of the audit criteria, we have a scope limitation. The standards require that the engagement leader expresses a qualified conclusion, disclaims a conclusion, or withdraws from the audit. A disclaimer of conclusion rarely occurs in the OAG’s performance audits and could not occur in special examinations.
The following diagram illustrate the steps set out in CSAE 3001 in forming the conclusion, with relevant paragraph numbers referenced.
How to form a conclusion
In forming conclusions, using their professional judgment, the auditors evaluate the sufficiency and appropriateness of the evidence obtained (see OAG Audit 7021 Evaluate sufficiency and appropriateness of audit evidence, and OAG Audit 1051 Sufficient appropriate audit evidence). They also assess the significance (see OAG Audit 2020 Significance) of the findings in relation to the audit objective. The conclusion should not be a summary of findings, but rather be a clear conclusion against the audit objective.
The conclusion has to be expressed using a positive form; for example, “The entity has complied, in all significant respects, with xyz . . .”
There are essentially four different types of conclusions that can be used in direct engagements:
- unmodified (clean) conclusion (“yes”);
- qualified conclusion (“yes, but” or “no, but”);
- adverse conclusion (“no”); and
- disclaimer of conclusion (when the audit team is unable to conclude due to lack of sufficient appropriate evidence).
Audit team members need to clearly communicate the type of conclusion in the audit report. They also need to use professional judgment in forming a conclusion (OAG Audit 1042 Applying professional judgment). For example, the team may decide to qualify the conclusion when some parts of an entity’s performance are satisfactory while others are unsatisfactory. The conclusion can then contain an "except for" statement to disclose the deviations from satisfactory performance.
If it is an unmodified (clean) conclusion (“yes”) or an adverse conclusion (“no”), the audit team should echo the words of the audit objective(s) in the conclusion. In an adverse conclusion, the audit team should also ensure to have a very clear “no.” An adverse conclusion is used when the significance and extent of the deviations from satisfactory performance are pervasive. When performance is fully satisfactory or highly unsatisfactory, concluding against the overall objective may be straightforward and the audit report will reflect a completely positive or adverse conclusion, as appropriate.
A qualified conclusion (“yes, but” / “no, but”) should contain the following essential elements: (1) clear announcement of the conclusion with specific reference to the audit objective, (2) clear “placement” (“yes, but / no, but”), and (3) reason for the (“yes, but / no, but”). For example:
(1) We conclude that the Department of Widget Affairs administered its Widget program in accordance with the Widgets Act, (2) with some improvement required in communications. (3) In particular, Widget status reports need to be communicated from headquarters to regional offices sooner so that Widget officers in the field have up-to-date lists of fees to be charged.
The core of a clear overall audit report conclusion is a single key paragraph with a clear overall conclusion. In straightforward cases, the entire conclusion may only be one paragraph. In more complex cases (multiple auditees, not a clean conclusion, etc.), further explanatory paragraphs can be added if the conclusion is not self-evident. The conclusion should be in a separate section of the written audit report in order to make it clear to the readers that this is the conclusion of the audit, not a finding on a specific aspect of the audit or a recommendation (see OAG Audit 7030 Drafting the audit report).
If, despite best efforts, an audit team is unable to obtain sufficient appropriate evidence, it may report the available evidence and its limitations, but it cannot draw findings and conclusions from the evidence. If the OAG decides to report the matter, it would state it as a qualification to the conclusion—that the auditors could not evaluate part of the subject matter because of lack of evidence (i.e. scope limitation). When the lack of evidence is significant, the audit report will express a “disclaimer of conclusion” due to incapacity to obtain sufficient appropriate audit evidence.
If the audit team determines there is a scope limitation (which occurs when the team is unable to obtain sufficient appropriate evidence), it would either explain it in the conclusion using a qualified conclusion or a disclaimer of conclusion (as explained above).
Special examinations
Special examination opinion statement and conclusion. Consistent with sub-paragraph 139(2)(a) of the Financial Administration Act (FAA), each special examination report must include a statement on whether in the examiner’s opinion and based on the established criteria, reasonable assurance exists that there are no significant deficiencies in the Crown corporation’s systems and practices examined.
The special examination opinion statement serves as a basis for the conclusion against the audit objective and is included in the conclusion paragraph in the audit report.
The special examination conclusion can take one of the following forms:
-
Unmodified (clean) conclusion. In our opinion, based on the criteria established, there was reasonable assurance there were no significant deficiencies in the Corporation’s systems and practices that we examined. We concluded that the Corporation maintained its systems and practices during the period covered by the audit in a manner that provided the reasonable assurance required under section 138 of the Financial Administration Act.
-
Qualified conclusion (one significant deficiency). In our opinion, based on the criteria established, there was a significant deficiency in the Corporation’s [identify specific systems and practices named in report], but there was reasonable assurance there were no significant deficiencies in the other systems and practices that we examined. We concluded that, except for this significant deficiency, the Corporation maintained its systems and practices during the period covered by the audit in a manner that provided the reasonable assurance required under section 138 of the Financial Administration Act.
-
Qualified conclusion (two significant deficiencies). In our opinion, based on the criteria established, there were significant deficiencies in the Corporation’s [identify the two systems and practices named in report], but there was reasonable assurance there were no significant deficiencies in the other systems and practices that we examined. We concluded that, except for these significant deficiencies, the Corporation maintained its systems and practices during the period covered by the audit in a manner that provided the reasonable assurance required under section 138 of the Financial Administration Act.
-
Adverse conclusion. In our opinion, based on the criteria established, there were significant deficiencies in the Corporation’s systems and practices that we examined for corporate management and management of operations. As a result of the pervasiveness of these significant deficiencies, we concluded that the Corporation had not maintained its systems and practices during the period covered by the audit in a manner that provided the reasonable assurance required under section 138 of the Financial Administration Act.
Because the FAA prescribes the need to provide a statement of opinion on the systems and practices selected for examination, a disclaimer of conclusion is not an option for special examinations. In the special examination practice, the ability to report on a significant deficiency replaces the need to disclaim a conclusion. If, for example, there was an inability to obtain sufficient appropriate evidence related to one of the areas selected for examination, the engagement team would either opine that there is a significant deficiency in the specific area (i.e. it would issue a qualified conclusion) or, where the inability affects multiple areas, that there is no reasonable assurance that the corporation’s systems and practices selected for examination achieved the statutory control objectives (i.e. the team would issue an adverse conclusion).
The special examination report template prescribes where the opinion should be located in the audit report as well as the exact wording of the statement/conclusion to be used in the possible scenarios (unmodified, qualified, or adverse) (see OAG Audit 7030 Drafting the audit report).
What is a significant deficiency? The FAA prescribes the use of the words “significant deficiency” to be used in the opinion, but does not define what it is. The OAG considers a significant deficiency to have occurred when there is a significant deviation from criteria. A significant deficiency is reported when the systems and practices examined did not meet the criteria established, resulting in a finding that the Corporation could be prevented from having reasonable assurance that its assets are safeguarded and controlled, its resources are managed economically and efficiently, and its operations are carried out effectively.
Significance is judged in relation to the reasonable prospect of a matter influencing the judgments or decisions of a user of a special examination report. For example, factors that may influence the engagement team’s judgment on what is significant in a particular circumstance might include the potential public, legislative, economic, or environmental impact.
Clearly, the definition of “significant” is a matter of judgment and depends on the circumstances. Ultimately, one of the major deciding factors is the identified or potential impact of a deficiency.
Factors considered to determine if a significant deficiency exists. The engagement team may take the following factors into account when determining whether a finding constitutes a significant deficiency:
-
Extent of deviation from criteria. A finding should be clearly linked to criteria and, for it to be significant, there should be substantial deviation from criteria. Where there are deviations, the engagement team needs to establish whether there are compensating systems or practices to help achieve the desired result.
-
Impact of the deficiency. To be significant, the deficiency’s impact on achieving the corporation’s statutory control objectives should be clear, serious, and consequential. When selecting key systems and practices and developing criteria, considering the corporation’s exposure to risk will help trace the impact of any deficiencies subsequently identified. The impact may be potential; that is, the consequences may not have materialized yet.
-
Relevance to the board, the Minister, Parliament, or other users of the report. The engagement team should consider what is of interest and relevance to report users. If a finding is of little or no consequence to users, it may not be significant. What is relevant to report users are a finding’s impact (what it will mean to them) and cause (why it happened). Of course, there may be a difference of opinion between what the engagement team believes is relevant to users and the corporation’s views on the issue, in which case the examiner would report the deficiency as significant if convinced of its consequence to users.
-
Practicality of the solution. If the likely cost of correcting the deficiency is greater than the benefit to be derived, the deficiency’s significance may be questionable.
-
Number of reported deficiencies. Minor deviations from several criteria may signal minor problems, or may be symptoms of a problem (or theme) of greater significance that should be reported as a significant deficiency.
- Planned corrective actions. If the corporation has action plans in place or even in process to correct deficiencies that have been classified as significant, these deficiencies should still be included in the report as significant because they existed during the examination period and because there is no assurance that the planned actions will correct the problem or that the actions will continue after the report date.
How is a significant deficiency formulated? A significant deficiency must have a clear evidentiary link to a criterion. Problems often occur when the wording used to describe the deficiency is too general; for example, if the significant deficiency does not discuss the impact(s).
In order to be clear and meaningful, a significant deficiency should identify the problem, its cause, and its effect.
All reasoning behind the decision on the significance of a given deficiency must be documented in the examination file.