4020 Risk Assessment
Jul-2020

Overview

Canadian Standards on Assurance Engagements (CSAE) 3001 requires auditors to consider significance and identify and assess risk when planning and conducting an audit. While significance is discussed in section OAG Audit 2020 Significance, this section addresses the identification and assessment of risk.

CSAE 3001 Requirements

51R. The practitioner shall obtain an understanding of the underlying subject matter and other engagement circumstances sufficient to:

(a) Enable the practitioner to identify and assess the risks of significant deviation; and

(b) Thereby, provide a basis for designing and performing procedures to respond to the assessed risks and to obtain reasonable assurance to support the practitioner’s conclusion. (Ref: Para. A99-A103, A105-A109)

53R. Based on the practitioner’s understanding (see paragraph 51R) the practitioner shall: (Ref: Para. A110-A114)

(a) Identify and assess the risks of significant deviation; and

(b) Design and perform procedures to respond to the assessed risks and to obtain reasonable assurance to support the practitioner’s conclusion. In addition to any other procedures on the underlying subject matter that are appropriate in the engagement circumstances, the practitioner’s procedures shall include obtaining sufficient appropriate evidence as to the operating effectiveness of relevant controls over the underlying subject matter when:

(i) The practitioner intends to rely on the operating effectiveness of those controls in determining the nature, timing and extent of other procedures, or

(ii) Procedures other than testing of controls cannot alone provide sufficient appropriate evidence.

Revision of Risk Assessment in a Reasonable Assurance Engagement

54R. The practitioner’s assessment of the risks of significant deviation may change during the course of the engagement as additional evidence is obtained. In circumstances where the practitioner obtains evidence which is inconsistent with the evidence on which the practitioner originally based the assessment of the risks of significant deviation, the practitioner shall revise the assessment and modify the planned procedures accordingly. (Ref: Para. A114)

CSAE 3001 Application Material

A13. Engagement risk does not refer to, or include, the practitioner’s business risks, such as loss from litigation, adverse publicity, or other events arising in connection with the underlying subject matter.

A14. In general, engagement risk can be represented by the following components, although not all of these components will necessarily be present or important for all assurance engagements:

(a) Risks that the practitioner does not directly influence, which in turn consist of:

(i) The susceptibility of the underlying subject matter to a significant deviation before consideration of any related controls applied by the appropriate party(ies) (inherent risk); and

(ii) The risk that a significant deviation that occurs in the underlying subject matter will not be prevented, or detected and corrected, on a timely basis by the appropriate party(ies)’s internal control (control risk); and

(b) The risk that the practitioner does directly influence, which is the risk that the procedures performed by the practitioner will not detect a significant deviation (detection risk).

A15. The degree to which each of these components is relevant to the engagement is affected by the engagement circumstances, in particular:

  • The nature of the underlying subject matter. For example, the concept of control risk may be more useful when the underlying subject matter relates to an entity’s performance than when it relates to information about the effectiveness of a control or the existence of a physical condition.
  • Whether a reasonable assurance or a limited assurance engagement is being performed. [...]

The consideration of risks is a matter of professional judgment, rather than a matter capable of precise measurement.

A16. For some performance audits in the public sector, there may be a higher risk of the auditor concluding that a matter detected during the audit is a significant deviation when that is not the case. This may result, for example:

  • From the complexity of the underlying subject matter being audited;
  • The extensive use of professional judgment in dealing with highly qualitative matters that are open to various interpretations; and
  • The breadth and depth of the scope of the engagement.

The risk may be mitigated by, for example, in-depth investigations of all matters detected, and the application of review and other quality control procedures targeted at reducing this risk to an acceptable level in the circumstances of the engagement.

A17. Reducing engagement risk to zero is very rarely attainable or cost beneficial and, therefore, “reasonable assurance” is less than absolute assurance, as a result of factors such as the following:

  • The use of selective testing.
  • The inherent limitations of internal control.
  • The fact that much of the evidence available to the practitioner is persuasive rather than conclusive.
  • The use of professional judgment in gathering and evaluating evidence and forming conclusions based on that evidence.
  • In some cases, the characteristics of the underlying subject matter when evaluated or measured against the criteria.

A85. [...]The nature and extent of planning activities will vary with the engagement circumstances, for example the complexity of the underlying subject matter and criteria. Examples of the main matters that may be considered include:

[...]

  • The practitioner’s understanding of the appropriate party(ies) and its environment, including the risks of significant deviation.
  • Identification of intended users and their information needs, and consideration of significance and the components of engagement risk.
  • The extent to which the risk of fraud is relevant to the engagement.

[...]

A99. Discussions between the engagement partner and other key members of the engagement team, and any key practitioner’s external experts, about the susceptibility of the underlying subject matter to significant deviation, and the application of the applicable criteria to the facts and circumstances of the engagement, may assist the engagement team in planning and performing the engagement. It is also useful to communicate relevant matters to members of the engagement team, and to any practitioner’s external experts not involved in the discussion.

A100. The practitioner may have additional responsibilities under law, regulation or relevant ethical requirements regarding an entity’s non-compliance with laws and regulations, which may differ from or go beyond the practitioner’s responsibilities under this CSAE, such as:

(a) Responding to identified or suspected non-compliance with laws and regulations, including requirements in relation to specific communications with management and those charged with governance and considering whether further action is needed;

(b) Communicating identified or suspected non-compliance with laws and regulations to an auditor; 6(7) and

(c) Documentation requirements regarding identified or suspected non-compliance with laws and regulations.

Complying with any additional responsibilities may provide further information that is relevant to the practitioner’s work in accordance with this and any other CSAE (e.g., regarding the integrity of the responsible party or those charged with governance). Paragraphs A192-A197 further address the practitioner’s responsibilities under law, regulation or relevant ethical requirements regarding communicating and reporting identified or suspected non-compliance with laws and regulations.

A101. Obtaining an understanding of the underlying subject matter and other engagement circumstances provides the practitioner with a frame of reference for exercising professional judgment throughout the engagement, for example, when:

  • Considering the characteristics of the underlying subject matter;
  • Assessing the suitability of criteria;
  • Considering the factors that, in the practitioner’s professional judgment, are important in directing the engagement team’s efforts, including where special consideration may be necessary (for example, the need for specialized skills or the work of an expert);
  • Establishing and evaluating the continued appropriateness of quantitative and qualitative factors that are significant;
  • Developing expectations for use when performing analytical procedures;
  • Designing and performing procedures; and
  • Evaluating evidence, including the reasonableness of the oral and written representations received by the practitioner.

A109. In both a reasonable assurance and a limited assurance engagement, the results of the entity’s risk assessment process may also assist the practitioner in obtaining an understanding of the underlying subject matter and other engagement circumstances.

A114. An assurance engagement is an iterative process, and information may come to the practitioner’s attention that differs significantly from that on which the determination of planned procedures was based. As the practitioner performs planned procedures, the evidence obtained may cause the practitioner to perform additional procedures.

OAG Policy

The audit team shall perform a risk-based planning exercise to determine the scope and approach of the audit. [Nov-2015]

The audit team shall consult an internal expert (i.e. an internal specialist), as deemed necessary according to the risk assessment that is conducted during the planning phase of the audit. [Nov-2015]

The audit team shall reassess risk when planning and performing the assurance engagement to respond to changing circumstances. [Nov-2015]

OAG Guidance

Risk is defined here as the likelihood of an event influencing the achievement of an objective. Risk assessment is a continuous, dynamic process of gathering, updating, and analyzing information throughout the audit.

What CSAE 3001 Means for Conducting the Audit

In a direct assurance engagement, the audit team decides on the nature and scope of the underlying subject matter to be reported on. This decision is based on an understanding of the subject matter, including risks and relevant internal controls (OAG Audit 4010 Understanding the subject matter in planning an audit; OAG Audit 4025 Internal controls). In understanding the subject matter, the audit team is required by standards to identify and assess, in the planning phase, the risk that the underlying subject matter contains significant deviations (i.e., negative audit findings) with applicable criteria. The audit team uses this information to form the basis of its approach to conducting the audit and obtaining reasonable assurance to support its conclusion. Furthermore, the audit team is required by standards to reduce the engagement risk to an acceptably low level to achieve reasonable assurance (which is standard practice for OAG audits).

Although significant risks are initially considered in the planning phase of the audit, they may be identified at any stage of the audit.

Audit standards require that the team stay alert to changing circumstances and reconsider the impact of new and conflicting information on its risk assessments and modify its response accordingly.

The assessment of risks is a matter of professional judgment, rather than a matter capable of precise measurement (see OAG Audit 1042 Applying professional judgement). Maintaining professional skepticism (see OAG Audit 1041 Applying professional skepticism) throughout the engagement is also necessary if the audit team is, for example, to reduce the risks of

  • overlooking unusual circumstances;

  • overgeneralizing when drawing conclusions from observations; and

  • using inappropriate assumptions in determining the nature, timing, and extent of the procedures, and evaluating the results thereof.

Also, to properly document the nature and extent of the audit work performed, auditors need to demonstrate how evidence gathered through risk assessment and the work performed on internal control links to the scope and approach of the audit. To facilitate this process and document its decisions, the audit team completes the relevant audit procedures along with the following tools:

  • Engagement Risk Assessment Template (ERAT)
  • Functional Risk Identification Template (FRIT)
  • Risks and Controls Assessment Template (RCAT)
  • Audit Logic Matrix (ALM)

For more information, refer to the Guide on risk-based planning and scoping for direct engagements listed below under Audit Guidance.

Assessing Subject Matter Risks

In planning, the audit team identifies and assesses the significant risks that may affect the achievement of an entity’s objectives as they relate to the subject matter being audited.

The audit team identifies and assesses these risks in order to identify areas where there is a higher risk of significant deviation in the audit (i.e., negative audit findings).

Because risk is an event influencing the achievement of an objective, the first step in identifying subject matter risks is to understand and identify the entity’s objective(s) as they pertain to the subject matter of the audit and then think about what could go wrong that could prevent the achievement of those objectives.

To identify and assess subject matter risks, two templates have been developed to assist audit teams:

  • Functional Risk Identification Template (FRIT)
  • Risk and Control Assessment Template (RCAT)

The Office has identified several key functional areas (for example, human resources or information technology) that may be susceptible to risk and jeopardize the achievement of the entity’s objectives. The risks associated with these functional areas are to be considered by all audit teams. Other risks pertinent to the subject matter under audit must also be identified and assessed. Taken together, any functional risks identified and any specific subject matter risks identified will allow the audit team to understand what may prevent the achievement of the entity’s objectives and therefore help to focus the audit work.

Functional Risk Identification Template. The audit team considers the relevance of key functional areas based on its preliminary understanding of subject matter objectives, and identifies and assesses these risks by completing the Functional Risk Identification Template. This process triggers the team to consider consulting with relevant internal specialists (see OAG Audit 3081 Consultations). Consultation serves two purposes. It allows the audit team to discuss the significant risks identified with the Internal Specialists and how these risks may affect the approach and scope of the audit. It also signals to the internal specialists that the audit team may need to obtain additional guidance and assistance during the audit, given the risk identified.

While most consultations are optional, based on the professional judgment of the engagement leader, the audit team must consult with the Internal Specialist for the Environment and Sustainable Development and obtain sign-off in the audit file. The Internal Specialist, Environment and Sustainable Development, may also require the team to complete a further assessment. If consultation takes place with other internal specialists, the audit team should document the results of the consultation.

Risk and Controls Assessment Template. The audit team uses the Risk and Controls Assessment Template to identify any risks pertinent to the subject matter of the audit and also brings forward any functional risks identified on the FRIT. Taken together, these risks are now considered the subject matter risks. The audit team then assesses all risks identified, and based on the assessment, determines and documents its preliminary scoping decisions.

The audit team may assess the importance of, and risks associated with, the entity’s activities by considering factors such as:

  1. Economic, social, and environmental impact—Programs, activities, or processes affecting a large segment of the population or vulnerable populations, or which impact environmental sustainability, may be considered to be more important to the entity. The significance of the subject matter can also be assessed by the high economic, health, security, or safety implications; the size of the relevant entity or program; the number of entities scoped into the engagement; and the Office’s past experience reporting on the subject matter.

  2. Relevance to stakeholders—The interest shown by the legislature or other governing bodies, by management of the entity or by the public may indicate the importance of the activity to stakeholders. Factors such as the sensitivity and visibility of the subject matter (e.g., the accountability relationship of the entity, the importance of its mandate, the degree the entity or program is visible in the public eye and recent parliamentary interest in the subject matter) should also be considered.

  3. Diversity, consistency and clarity of the entity's objectives and goals—Diverse or inconsistent objectives increase the risk that the entity’s activities or programs are not operating with due regard to one or more of the principles of economy, efficiency, effectiveness and environment and sustainable development. Entity objectives and goals that are not clearly defined may increase the risk that they will not be achieved because they are not understood by employees.

  4. Complexity of operations—An increase in the complexity of an entity's operations, through increased variety and type of programs, functions and activities may increase the risk that the entity does not achieve its objectives and goals or that they are not achieved efficiently, economically or with due regard to the environment or sustainability. The nature of the entity’s operations, such as the existence of complex and specialized transactions or issues that are highly technical and require significant professional judgment to evaluate; and multiple entities with shared and overlapping responsibilities, may increase the risk.

  5. Complexity and quality of management information and control systems—Complex systems may be more difficult to develop, enhance and maintain. When adequate management information systems are not maintained, proper control may not be exercised.

  6. Impact of environmental or organizational change—Changes in an entity’s environment or organization can impact the continuity of operations and the understanding of priorities and processes by employees. This may increase the risk that the entity’s goals and objectives will not be achieved. Environmental changes include new government priorities, significant budget amendments and changes to enabling legislation. Organizational changes include changes in leadership, reorganization, new initiatives and staff turnover.

  7. Financial magnitude and nature of transactions—Large dollar amounts, high transaction volumes and transaction complexity and flow may create increased risks to the entity.

  8. Management response to previously identified deficiencies – Areas where management has not made adequate improvements to address important issues raised in prior performance audits or other studies may be more important and higher risk.

  9. Organizational structure—Centralization and decentralization of key activities such as budgeting, payroll, disbursements, human resources management and facilities management each create their own operational risks. Similarly, program delivery through agents carries different risks than those associated with direct program delivery.

  10. Program delivery method—Programs in the public sector may be delivered by policy instruments such as expenditure, regulation and revenue-raising; may provide goods or services directly or may redistribute income; and may be delivered directly or by using agents. The amount of associated risk may vary depending on the delivery method.

The audit team assesses identified risks as either normal, elevated, or high. The default level is normal unless a risk is considered to be higher. In assessing the risks, the team should consider the likelihood (or probability) of the risk occurring and the impact (or significance) if it were to occur. Assessing risk requires professional judgment. The following matrix presents factors to help the audit team determine the likelihood and impact when assessing the subject matter risk. The audit team then uses the assessment to rank the risks identified. The audit team documents and summarizes briefly its rationale for the risk rating of each risk. How each risk stands in relation to the other risks is often a key consideration.

Subject matter risk assessment matrix

Impact: If the risk were to occur it . . .

Low

Medium

High

. . . may affect subject matter operations, but not in a way that would jeopardize outcomes and objectives. Limited impact on subject matter results or clients.

. . . may jeopardize operational, but not strategic subject matter objectives. Moderate impact on subject matter results or clients.

. . . significant impact on meeting strategic subject matter objectives. Significant impact on subject matter results or clients.

Likelihood

Based on judgment, . . .

High

. . . the risk is occurring or will probably occur in routine circumstances or is likely to be pervasive

Elevated

Elevated to High

High

Medium

. . . it is not unusual for the risk to occur from time to time

Normal to Elevated

Elevated

Elevated to High

Low

. . . it may occur in very rare circumstances

Normal

Normal to Elevated

Elevated

In identifying and assessing risk, the audit team also needs to understand the internal controls relevant to the subject matter. This understanding helps the auditor identify factors that may reduce the risk and provides a basis for designing procedures. OAG Audit 4025 Internal Controls discusses the requirements related to the assessment of internal controls in the planning phase.

Special Examinations

As for all direct engagements, special examination teams must assess the subject matter risks related to the Crown corporation under audit. In addition to the team’s risk assessment, the OAG has developed a set of core systems and practices and related standard criteria that must be examined in every special examination. The final audit scope and approach must respond to the assessed risks. From a risk perspective, the ‘core’ systems and practices mitigate ‘portfolio-wide’ risk—that is, significant risks that are common to all Crown corporations subject to special examinations. Core represent the key systems and practices where, if there are significant deficiencies, individually or collectively, the deficiency(ies) could prevent the Crown corporation from having reasonable assurance that its assets are safeguarded and controlled, its resources are managed economically and efficiently, and its operations are carried out effectively.

Subject Matter Risks of a specific Crown corporation may justify adding to the Core scope

Notwithstanding the use of core systems and practice and standard criteria for every special examination, the audit team should assess subject matter risk in the planning stage. Based on the team’s risk and internal control assessment for the Crown corporation under audit, the Engagement Leader determines whether to expand the scope of the examination. Where there is an elevated risk of a significant deficiency that could prevent the corporation from achieving its statutory control objectives, engagement leaders have two options for proceeding:

  • If the risk is covered by the Core systems and practices, engagement leaders can consider increasing the audit effort in the area of elevated risk;

  • If the risk is not covered by the Core systems and practices, engagement leaders can justify adding to the scope due to an elevated risk of a significant deficiency.

Subject matter risk and considerations for incorporating separate lines of enquiry for Information Technology, Human Resource Management and Environment

Special examinations can include separate sections for Information Technology (IT), Human Resource Management (HR) and Environment. But often, the messages regarding these systems and practice have fit well into the management of operations, because generally these functional areas are the means by which the Corporation achieves its mandate. They may also fit into strategic planning and corporate risk management. However, if the team’s risk and control assessment reveals an elevated risk of a significant deficiency in IT, HR, or Environmental systems and practices that could prevent the Crown from achieving its statutory control objectives, the Engagement Leader may expand the scope of the examination to include coverage of these areas.

Managing Engagement Risk

Engagement risk. Engagement risk is the risk that the auditor expresses an inappropriate conclusion in the audit report. It is the risk that the auditor identifies a significant deviation (i.e., a negative audit finding) when one does not exist, or the risk that the auditor does not identify a significant deviation when one does exist.

Audit standards describe engagement risk as being made up of three components, although the extent of these and their importance will vary with the circumstances of the audit.

  • Inherent risk: risk related to the susceptibly of the subject matter and its environment before the consideration of controls; for example, susceptibility to the current economic or political environment.

  • Control risk: risk that a significant weakness exists related to the subject matter that will not be prevented, detected, or corrected on a timely basis by the entity management’s internal controls. This risk is a function of the design and effectiveness of the internal controls relevant to the subject matter. The auditor does not directly influence this risk.

  • Detection risk: risk that audit procedures will not result in sufficient appropriate audit evidence that is interpreted appropriately by the auditor (e.g. using inappropriate auditing procedures, misapplying procedures, and misinterpreting audit results). It is a function of the effectiveness of the auditors’ procedures.

The audit team does not directly influence the first two components (inherent and control risk), but seeks to understand them when assessing the subject matter risks. This helps the team properly inform its assessment of detection risk and develop audit procedures to minimize it.

Managing engagement risk. Engagement risk is unavoidable because auditors cannot obtain absolute assurance that their audit conclusion is appropriate without examining 100 percent of all relevant information or activities relevant to the subject matter and the audit objective. However, audit teams plan and perform the audit so that engagement risk will be limited to an acceptably low level that is, in their professional judgment, appropriate for expressing an opinion against the audit objectives and managing the risk of an inappropriate conclusion.

Factors affecting engagement risk include

  • management integrity;

  • the availability of enough quality information;

  • complexity of the subject matter, as well as the specific audit approach and testing strategies proposed (e.g., complicated modelling, quantitative analysis);

  • audit team (people);

  • the Office’s past experience and relationship with the entity(ies) (e.g., recent history of difficult or contentious engagement issues, including disagreements with management, negative conclusions in previous reports, reservations in previous opinions and reports, or access issues);

  • team availability, capacity, and competence (e.g., the composition of the engagement team, such as a newly appointed audit principal, significant turnover in the engagement team, or the team’s limited experience with the objectives or subject matter, and/or competence in required assurance skills and techniques);

  • timeframes, schedules, and resources (e.g., shorter timeframes, smaller budgets, lack of dollars for contracting external expertise, production conflicts; extensive travel can increase audit risk);

  • the extent of professional judgment and expertise required by the engagement team in order to assess risks of significant deviations in the subject matter;

  • sensitivity, visibility, or significance of the subject matter; and

  • any other specified risk factors, such as known and potential threats to the Office’s independence.

Engagement Risk Assessment Template. At the start of the audit, the engagement leader is responsible for identifying and assessing engagement risk and identifying any specific mitigation strategies required. The Engagement Risk Assessment Template is used to document this assessment. Engagement risks are assessed as normal, elevated, or high. The default level is normal until a risk is assessed as higher. In assessing the risk, the engagement leader considers the likelihood (or probability) of the risk occurring and the impact (or significance) on his/her ability to provide a reasonable assurance conclusion. Assessing risk requires professional judgment. The rationale for the risk ratings should be documented.

Mitigating the risk of an incorrect conclusion. The Office’s standard quality control and procedures for planning, executing, and reporting an audit are designed to ensure that auditors gather sufficient appropriate evidence and, hence, reduce the likelihood of inappropriate conclusions.

Reducing engagement risk to an acceptable level is achieved by proper planning to develop an appropriate audit approach (see OAG Audit 4042 Audit scope and approach). It is also managed by ensuring adequate supervision and reviewing audit work, as well as implementing various quality control processes.

Mitigation strategies are tailored to the engagement circumstances and may include

  • involving various experts either internal or external to the Office (see OAG Audit 2070 Use of experts);

  • enhancing the involvement of senior management (i.e., the assistant auditor general and/or the Auditor General. See also OAG Audit 2040 Discussions with the Auditor General);

  • convening advisory committees (see OAG Audit 2050 Advisory Committee meetings); and/or;

  • implementing other strategies such as specialized training, enhanced team supervision and review, or specific entity relation protocols.

If the risk of an inappropriate conclusion is assessed as higher than normal, the engagement leader may also recommend appointing a quality reviewer (see OAG Audit 1081 Selection of engagement quality control reviews and appointment of quality reviewers).

Some controls may offset the need for others. For example, if the audit team is experienced and the audit subject matter is well defined, there may be no need for external advisors at the planning phase.

The audit team considers the impact of engagement risk in making their audit scope and approach decisions as well as in developing the nature and extent of the audit procedures required. The audit team describes how audit work is designed to address the engagement risks in the audit logic matrix (ALM). In the ALM, the audit team sets out the audit design including evidence-gathering techniques, the information and sources required along with associated limitations. Consideration of the limitations is done to help the audit team ensure it gathers sufficient appropriate evidence to minimize the risk of forming an incorrect conclusion. The quantity of evidence needed is affected by the risks of the underlying subject matter containing a significant deviation and the engagement risk. The higher the risks, the more evidence is likely to be required. The quantity of evidence is also affected by the quality of such evidence (the higher the quality, the less may be required). The ALM is discussed in OAG Audit 4044 Developing the audit strategy: audit logic matrix. Evidence-gathering methods are discussed in OAG Audit 4045 Evidence-gathering methods.

Revision of Risk Assessment

The audit team monitors the risks affecting the subject matter and the engagement throughout the audit. The audit team is required to re-assess the risks if there is a significant change in circumstances that could alter the original assessment. The engagement risk assessment and team’s mitigation strategies are updated at the end of the planning stage to support the engagement leader’s assertion at the examination approval stage (OAG Audit 4080 Examination approval) that significant engagement risks have been assessed and updated, and that  ways to mitigate them have been identified, implemented, and documented. Engagement risk is also considered as the audit team forms its audit conclusion and evaluates the sufficiency and appropriateness of audit evidence in support of its report. This assessment specifically informs the content and publication approval of the report (see OAG Audit 8017 Report content approval and date of the report).

Next Steps

The audit team uses the risk assessment to identify and assesses relevant key controls that mitigate the risks identified (OAG Audit 4025 Internal controls) which then helps the audit team inform its scope and approach decisions (OAG Audit 4042 Audit scope and approach).