Performance Audit Manual

Due care

The audit team should exercise due care.

3.1 Due care requires the auditors to carry out their audit work diligently, conscientiously and with rigour. It requires that the audit be performed in accordance with professional standards and OAG policies. Following professional standards and OAG policies means that auditors exercise sound judgment when deciding the audit objective, what and when to audit, the basis for measuring performance, the audit approach and methodology, the extent of audit, the issues to be reported and the overall audit conclusions. Due care also requires that those supervising the audit work and providing review and challenge on the major audit decisions exercise similar vigilance.

Objectivity and Independence

The audit team should be made up of individuals who have an objective state of mind and are independent.

3.2 Auditors must maintain an objective state of mind. This means that the auditor does not direct the audit toward areas of personal interest or prejudge findings. The findings and report can be influenced only by evidence obtained and assembled in accordance with the other audit policies and guidance contained in this manual. The auditor needs an unbiased point of view when making decisions about scope, criteria, audit evidence, significance of observations, and conclusions.

3.3 Independence requires that the Office and members of the audit team, whether staff or contract personnel, be free of any hindrances to their independence that could impair (or be seen to impair) their impartiality in carrying out their work, making judgments, forming opinions and conclusions or making recommendations.

3.4 Audit staff are encouraged to develop and maintain good relations with officials and staff in the audited organization. The audit policies require the auditor to recommend corrective actions when reporting deficiencies. This conduct is to be carried out in a way that does not impair the independence of the Office or the auditor.

Competence of the audit team

The audit team should have collective knowledge of their subject matter and auditing proficiency necessary to fulfil the requirements of the audit.

3.5 The audit team leader should identify at an early stage in the planning process if specialized or technical skills, not available on the audit team, are required to complete the audit. The early identification will allow the necessary lead time to acquire suitable staff from within the Office or to obtain persons under contract.

3.6 The quality of a performance audit is directly related to the people assigned to the audit. An audit procedure that requires the exercise of judgment beyond the ability of the person expected to make the judgment will likely end in failure.

3.7 The Office has an obligation to Parliament, the audited organizations, and the other stakeholders to ensure that competent personnel conduct audits. This requires the audit team to possess, or collectively possess, the knowledge, disciplines, skills and experience to carry out the audit effectively. This includes any specialists that are contracted to assist the team in any way.

3.8 The audit team should have:

• knowledge of performance audit concepts and techniques and the ability to apply the knowledge;
• experience and technical skills to effectively deal with the subject matter of the audit;
• knowledge of the audit entity; and
• a general knowledge of the government environment.

3.9 The audit team should consult Functional Responsibility Leaders (FRLs) and other support groups such as Product Leaders (PLs) and Subject Matter Experts (SMEs) where necessary to obtain expert advice particularly when the audit team lacks the necessary specialized knowledge.

3.10 Audit advisors should have the appropriate background and knowledge to effectively review and challenge the key decisions of the audit.

3.11 Where appropriate competence is not available, the audit should be redefined or deferred until appropriate personnel are available.

Supervision

The audit team should ensure proper supervision of all its members.

3.12 Supervision involves directing audit staff and monitoring their work to ensure that the audit objectives are met. Supervision is an essential and continuous process that requires that the audit principal, directors and other supervisors should:

• ensure that all team members fully understand the audit objective(s);
• delegate audit projects to team members with a clear outline of what is expected from the project;
• provide appropriate counsel, advice and on-the-job training based on the experience of the team members;
• ensure that audit procedures are adequate and properly carried out;
• ensure that the performance audit policies and the audit reporting process are followed;
• ensure that audit evidence is appropriate, sufficient and documented and that it supports audit observations and conclusions; and
• ensure that only necessary audit work is carried out and that budgets, timetables and schedules are met.

Entity management's input to the audit

The audit team should seek entity management's views about critical elements of the audit.

3.13 Good relations between audit staffs and entity management is built on the basis of respect and trust. Where this type of relationship exists, both management and the Office can benefit when the audit team seeks input throughout the course of the audit. As noted earlier under the section dealing with Objectivity and Independence, such relationships do not compromise the auditors' independence or the quality of the audit report.

3.14 For larger entities, principals should provide annually their usual contact in the department with a five-year audit plan including details on audit topics and timing for the first two years, and less precision for the next three years, as well as a list of potential government-wide audits, and offer to meet with the Audit and Evaluation Committee of the Department or other pertinent senior level committees to discuss the audit plan. AAGs/CESD should communicate this plan to the Deputy Minister.

3.15 The audit team should seek entity management's input when:

• planning the audit to obtain views on the critical success factors for the activity being audited, management's responsibility for the activity, sources of criteria, risks, management concerns, and other audits or studies carried out in the area;
• finalizing the audit plan to obtain views on the approach and the criteria selected for the examination phase;
• developing findings to agree on the facts, or to obtain alternative sources of evidence;
• developing recommendations to obtain management's views on the best ways to correct the problem;
• obtaining agreement on the facts, observations, issues, and recommendations contained in the audit chapter, or to point out any disagreements; and
• finalizing the draft chapter to obtain the deputy head's comments and planned departmental actions to correct any deficiencies, and any disagreement with the report.

3.15.1 The audit team should seek management's acknowledgement of responsibility for the activity as it relates to the objective of the audit, where practicable. If the audit team does not obtain management's acknowledgement, the audit team should:

1. obtain other evidence that an accountability relationship with management exists, such as a reference to legislation or regulations;
2. consider how the lack of management's acknowledgement might affect their work and conclusions; and
3. disclose in the report that acknowledgement of responsibility has not been obtained where it was practicable to do so.

Consultation and advice

The audit team should obtain sufficient, appropriate consultation and advice throughout the audit.

3.16 Performance audits are often complex undertakings requiring a wide range of skills, expertise and experience to be completed cost-effectively. As noted throughout this manual, considerable judgment is required at all stages of the audit. The requirement to have an Audit Advisory Committee, a Quality Reviewer named to each audit, and the support of internal leaders/specialists (Functional Responsibility Leaders (FRLs), Product Leaders (PLs) and Subject Matter Experts (SMEs)) ensures that appropriate advice and assistance are available to the audit teams. Audit teams should consult with a Quality Reviewer, the Audit Advisory Committee, FRLs, PLs and SMEs as appropriate.

When an audit is deemed of high risk as it relates to a FRL area, FRL consultation is considered mandatory from the outset of the audit. Audit risk as it relates to FRL consultation is determined by such factors as auditability (e.g. audit team capacity to deal effectively with the FRL area of expertise) or credibility (e.g. the Office has previously taken a reported position as it relates to this area).

The assessment of an audit as high risk as it relates to an FRL area is generally determined as a result of the One-Pass Planning exercise.

The Performance Audit Quality Management Framework also requires that Audit Principals account for sufficient consultation with FRLs (see Quality Assessment/Help Checklist).

3.17 Following is an outline of the key responsibilities of these advisory bodies.

The Audit Advisory Committee

3.18 Audit advisory committees are established for all performance audits. Members of a committee, from both inside and outside the Office, are selected on the basis of their skills, insights, relevant knowledge and experience. Outside advisors are recognized as leaders in their fields of expertise.

3.19 The committee is designed to primarily provide a forum where the audit team can seek advice on the objectives of the audit, the general approach, and the significant matters and issues that are to be reported. The team also presents information to the committee at the critical decision points of the audit and normally meets two to four times during the course of an audit. The audit team consults with committee members on the following aspects of the audit:

• the preliminary audit objectives, background and rationale for the audit, initial lines of inquiry, and the relevance of the planned audit to the Office's mandate;
• the scope, general approach and criteria, and emerging issues;
• proposed observations, recommendations, conclusions and reporting strategy; and
• the report chapter to assure that it addresses the right message, is fair, significant and clearly presented.

3.20 The role of the committee is to:

• advise on planned coverage, matters of potential significance and audit approach in the early stages of the audit;
• provide expert counsel on the significance of issues;
• review the avenues for quantification being pursued and whether they will be achieved;
• provide independent review, challenge and counsel at the critical control points of the audit; and
• advise on whether the report "message is right" and the issues are significant, and on the tone, fairness and reasonableness of the presentation.

3.21 The audit Principal could also use individual members of the committee with expert knowledge as special advisors to the audit team.

Functional responsibility leaders (FRLs)

3.22 The Office has established three categories of internal leaders/specialists available to performance audit teams to provide consultation and expert advice: Functional Responsibility Leaders (FRLs); Product Leaders (PLs); and Subject Matter Experts (SMEs). These internal leaders and specialists are expected to establish and maintain personal expertise in their subject matter areas; current knowledge of related government policies and developments; an up-to-date inventory of outside consultants with appropriate skills for potential use by audit teams; the latest methodologies for auditing in their areas; and the means to communicate with and train audit staff in their area of expertise.

• Functional Responsibility Leaders (FRLs) have been identified for those areas directly related to the Office mandate in areas where it has a long and consistent history of reporting. When an audit is identified as high risk from an auditability and credibility perspective as it relates to the FRL area, FRL consultation throughout the audit engagement is considered mandatory. The identification of an audit as high risk in an FRL area is made as a result of the One Pass Planning process or in the case of an unplanned audit engagement by the Audit Principal prior to the outset of the audit with input and agreement of his/her AAG and the FRL involved.
• Product Leader (PL) -Performance Audit is one of six product leaders that the Office has identified to maintain and update current audit policies and standards for each audit product. OAG audit practitioners should seek advice and consult with this leader when questions arise of a technical and methodological nature during the conduct of their audit work.
• Subject Matter Experts (SMEs) are internal Office specialists who maintain current state-of-the-art knowledge/skills concerning specific subjects often essential to the successful completion of our audit work. SMEs assist and advise audit teams and also conduct audits in their areas of expertise. When required, audit practitioners should seek advice, guidance, techniques and resource references to enhance the quality of their audits. Where areas such as Fraud, Surveys and Quantitative Analysis, Access to Information and Mandate Issues arise in the audit, the Audit Principal should consult the SME as early as possible.

3.23 Audit teams consult appropriate FRLs, PLs and SMEs to seek their advice on scoping decisions, methodology, and contracting for external consultants with specialized skills. When an audit is deemed of high risk as it relates to a FRL area, FRL consultation is considered mandatory from the outset of the audit. It is the audit Principal's responsibility to obtain approval from all mandatory FRLs consulted during the performance audit and other FRLs/SMEs identified in the Chapter Proposal or recommended for consultation. These consultations should be accurately reflected the Summary Examination Plan and on the Final Approval Form.

Audit risk as it relates to FRL consultation is determined by such factors as auditability (e.g. audit team capacity to deal effectively with the FRL area of expertise) or credibility (e.g. the Office has previously taken a reported position as it relates to this area). (October 2004)

Mandatory consultation includes a FRL review of relevant sections of the PX draft chapter including advice on the consistency of observations and recommendations with previous Office positions taken publicly. Consultation with PL-Performance Audit and SMEs is voluntary based on the team's need for advice and assistance at any time during the conduct of the audit. (October 2004)

Legal Services

3.24 Audit teams seek advice from Legal Services on potential legal issues arising during the audit, possible recommendations to change legislation, engagement of outside legal counsel, mandate and third party references in the audit report. Further guidance is provided under Consultation with Legal Services in Chapter 8.

Regional offices

3.25 Audit teams consult with and seek the advice of the regional offices early in the planning stage when the audit has significant regional implications. Further guidance is provided under Co-ordination with Regional Offices in Chapter 8.

Quality Reviewer

3.26 A Quality Reviewer is named for each performance audit, study or audit note. S/he acts as an advisor to the audit Principal and is responsible for providing reasonable assurance to the AAG responsible for the audit on the quality of the report. (The role of the Quality Reviewer is explained in more detail in Chapter 2 under Roles and Responsibilities.)

Through discussion with the team and through the review of key documents, s/he will focus her/his attention on the following matters:

• the risks associated with the audit, how best to deal with these risks and the adequacy of other consultations in relation to these risks;
• the survey report and examination plan particularly in relation to the adequacy (for performance audits and studies) of the proposed level of work/methodology, of resources and independence of the team;
• the appropriateness and sufficiency of evidence related to high risk findings;
• the compliance of the report with previous Office positions and Performance Audit Reporting Principles.

The results of the review are documented in a memo to file to outline what review work was done and its results.

In cases where the Quality Reviewer and the Assistant Auditor General are in disagreement and where in the opinion of the Quality Reviewer, the Office is placed at risk, the Auditor General would be consulted.

It is the audit Principal's responsibility to ensure that the Quality Reviewer is consulted on a timely basis, and receives the information needed to perform his/her review.

Other support groups

3.27 Other support groups in the Office provide advice on media relations, report style and use of graphics, needs-related training, and audit methodology.

Documentation

The audit team should maintain appropriate documentation and files.

3.28 Audit working papers and files are used to document key audit decisions and work. Audit documentation is relevant, complete and understandable, and structured for easy access. In carrying out its audit activities, the Office is required to comply with the National Archives of Canada Act (the Act) in its management of records. In addition, it is Office policy to conduct our performance audits in accordance with the standards for assurance engagements set by the CICA and to draw upon standards and practices of other disciplines. In relation to CICA standards, this includes the standards regarding documentation in audit files, especially in documenting matters the auditor considers important in providing evidence to support the conclusion expressed in the report. Further details are provided in the Guidance for Managing Audit Records in the OAG and the Audit Files Subject Classification and Numbering Guidelines.

3.29 Complete, indexed and cross-referenced working papers are of critical importance when reviewing findings with management, briefing the Auditor General, providing support at Public Accounts Committee hearings, answering subsequent queries from the client and others, and planning future assignments. Clear indexing and cross-referencing is key to ensuring the evidence is readily accessible.

3.30 Teams should maintain a Performance Audit Control File that contains the most significant reports, approvals and decisions throughout the life cycle of the audit. This includes:

• approvals resulting from the Office planning process, including the chapter proposal;
• approved survey plan if the team conducted an overview, otherwise, the chapter proposal;
• overview (if applicable) and audit survey reports;
• approved examination plan and any modifications;
• Executive Committee and PAMC decisions;
• management's views on the criteria and other elements of the audit;
• audit programs (if any) specifying the work to be carried out and work completed;
• comments and advice from advisors, Legal Services, FRLs, PL-Performance Audit, SMEs and the Quality Reviewer, and the significant audit decisions taken by the audit team and AAG based on this advice;
• reporting phase signoffs;
• significant correspondence with departmental management; and
• management's comments on the project reports and draft chapters and steps taken to resolve any differences.

3.31 The audit team should prepare substantiation binders that contain audit evidence most pertinent to the report content. We produce substantiation binders as a means to provide assurance as to the quality of the audit. Gathering together the evidence specific to a report for easy access also allows the Office to respond to internal or external enquiries (e.g. a hearing by the Public Accounts committee).

3.31.1 The goal is to ensure observations, conclusions and recommendations flow logically from available evidence and are well supported. The evidence in the substantiation binders should be persuasive so that a review of it by a reasonably knowledgeable person will result in similar observations, conclusions and recommendations.

3.31.2 Substantiation covers all aspects of the report. In addition to the evidence needed to support factual statements, the substantiation binders include support for the judgments, assumptions and audit conclusions made by the auditor, for example, in the form of a working paper setting out the logical arguments and supporting evidence for the auditor's decisions. Usually only some small part of a document or a working paper summary is needed as proof for a particular statement.

3.31.3 The auditors use their professional judgments in deciding what to include in the substantiation binder to support the report. They need to ensure that sufficient appropriate evidence for the more contentious, sensitive and highly visible issues are included in the binder. For other matters, such as the background information on the entity, the audit team can choose to include a cross-reference to the evidence found in other audit working paper files rather than putting a copy of the evidence itself in the substantiation binder. The binders are to be carefully indexed and cross-referenced to supporting details. Further guidance on audit evidence is included in Chapter 4.

3.31.4 Before issuing the PX Draft, the audit team gathers together the documentation to enable the PX to determine that sufficient appropriate evidence was obtained to support the major observations, recommendations and conclusions. The substantiation binders should be completed and reviewed prior to the issuance of the Transmission Draft.

3.31.5 By convention and practice, all working papers are confidential documents belonging to the Office. Audited organizations, Parliament and the public do not have automatic right of access to working papers. All requests for working papers from the media or public should be forwarded to the SME for Access to Information. Further guidance on matters of access is provided under Access to Information in Chapter 8.

Communications with Parliament and others

The audit team should deliver clear, persuasive and effective communications to Parliament and other stakeholders.

3.32 The primary means of communicating audit results is through the reports of the Auditor General. The reporting policies and related guidance for performance audits are covered in depth in Chapter 5 and some are posted on the INTRAnet.

3.33 Reports of the Auditor General, when tabled in the House of Commons, are automatically referred to the Standing Committee on Public Accounts (PAC). The Report of the Commissioner of the Environment and Sustainable Development is automatically referred to the Standing Committee on the Environment and Sustainable Development.

3.34 The audit AAG/CESD, Principal or team members may be called upon to communicate audit findings to members of Parliament. Further guidance is provided under External Communications in Chapter 8.