Performance Audit Manual

Follow-up Audit

Issues or recommendations that were presented in previous reports and that are of continuing interest to Parliament and/or that pose a significant risk should be re-audited.

6.1 The Status Report presents the results of audits of selected issues and recommendations presented in previous reports. This report provides a high level of assurance. Consequently, performance audit methodology should be applied throughout the planning, examination, and reporting phases of the follow-up audit. Since the decision to go ahead with a follow-up audit depends on parliamentary interest and/or significant risk, the decision about the timing of the follow-up audit will vary accordingly. The One-pass-plan should take the timing into consideration.

6.2 The focus of the follow-up audit is to determine whether the problems or issues originally identified have been resolved or not. Issues may evolve with time; focussing strictly on specific actions taken to address recommendations may not provide an accurate picture, as recommendations may not be applicable to the new circumstances or the evolution of the issue. The purpose of the follow-up audit is to determine the progress achieved in addressing the original recommendations and resolving the issues.

Planning the follow-up audit

6.3 Planning the follow-up audit is essential to the success of the undertaking. Some initial planning is required, as not all recommendations or issues raised in the original audit report should be re-audited, and as new issues may have arisen. Many questions arise when planning the follow-up audit. The main ones are as follows:

• Did the entity have enough time to address the issues raised originally?
• Are the recommendations still relevant?
• Should the audit only address the implementation of the recommendations, or should the issues themselves be re-audited?
• Have the initial problems or issues identified evolved with time?
• From a risk perspective, what are the key issues for re-audit?
• What does the audit team know, in relation to the previous audit?

Selecting issues to be re-audited

6.4 Concerns raised by parliamentarians following the original audit can provide guidance to the audit team about what issues should be selected for re-audit. The audit team should review debates of the Public Accounts Committee and the relevant standing committees of the House and the Senate, and questions raised in the House of Commons to determine key concerns and to determine what should be re-audited.

6.5 As the audit team reviews the situation related to the issues to be re-audited to develop a new audit plan, the team may find that issues have evolved; if this is the case, the team may need to redefine the issues. The audit team may also identify new issues and may decide that it is important that they be assessed and reported to Parliament. Issues from more than one chapter could also be included. Under these circumstances, the audit team may need to amend or prepare new objective(s) and criteria.

Seeking the views of the audited entity

6.6 When amended or new objective(s) and criteria are required, the audit team must seek the views of the audited entity (see paragraph 3.15).

6.7 The audit team should use the Guide to Assess Entity Progress, which the Office regularly uses to annually assess progress in implementing recommendations. This Guide will assist in determining whether or not we are satisfied with the progress achieved (see paragraphs 6.16 to 6.28). The audit team should discuss the criteria in the Guide with the entity early in the audit. The representatives of the entity should be aware of the criteria used to assess progress; so discussions with the entity are important at this stage.

Resourcing the audit

6.8 Ideally, the original audit team is responsible to carry out the follow-up audit. However this may not always be possible.

6.9 The need to involve members of the audit team that conducted the original audit may depend on the complexity of the issues to be re-audited. If the issues that will be re-audited are complex, the audit Principal should consider making efforts to include members from the original performance audit in the current audit team.

6.10 Under certain conditions, it may not be possible for any of the previous team members to participate in the follow-up audit. In the event that the audit team is comprised primarily of new members, the audit Principal and/or Director can organize a meeting with members of the previous audit team, if they are available, when the follow-up audit is about to begin. This meeting can provide a comprehensive briefing/orientation for new team members on the audit issues and the previous audit approach.

Advisory committee

6.11 When the issues being re-audited are the same as those of the original audit, the audit Principal and/or Director may decide, at the planning stage, to consult the advisory committee members on the timing and the approach proposed by the audit team. However, when the approach is essentially the same as for the original audit, this consultation may not be required.

6.12 A risk-based assessment of the original issues audited may identify a reduced number of key issues for re-audit. If this situation occurs, it should be an agenda item for consultation with advisory committee members prior to the examination phase of the follow-up audit.

6.13 At the reporting stage of the re-audit, the focus of the consultation may be different. If new issues have not been included in the scope of the re-audit, the consultation will concentrate on whether our assessment of the entity's actions with regard to the recommendations made in the original audit is reasonable.

Recommendations

6.14 In order to take the Office annual monitoring policy into account, the audit team should not reiterate previous recommendations in the Status Report chapter. Reiterating the recommendation would provide the entity with an additional five to six years to implement it. In most instances, this is not desirable. When reporting on new issues, recommendations can be made to address these issues (see paragraphs 4.86 to 4.97). As for all performance audits, an entity response to the recommendation may be included in the report.

Chapter Main Points

6.15 The chapter Main Points are essentially the same as for all performance audits (see paragraphs 5.19 to 5.22). However, the first paragraph of the follow-up audit should provide an overall opinion as to whether we are satisfied or not with the progress achieved. To that end, the Guide to Assess Entity Progress (see paragraph 6.7) can be used for each recommendation audited to arrive at an overall conclusion as to satisfaction.

Annual Monitoring of Progress in Implementing Recommendations

Progress in implementing all recommendations from previous reports should be assessed for a maximum period of five years, or until the issue is resolved, or obsolete.

6.16 Monitoring work is neither an audit nor a review engagement, since it provides no assurance. The primary purpose of monitoring progress is to keep departments focussed on the need for corrective action in response to previous Office recommendations; to allow the audit team to maintain knowledge of the entity's business; and to provide information for the Office Departmental Performance Report. It also allows the Office to provide additional information to parliamentarians on request.

6.17 Monitoring work should be based on self-reporting by the audit entity and subsequent review by the audit Principal. This review is to assess whether an entity's claims are consistent with our existing knowledge of its business; it also assesses whether or not, given complexity and resources assigned, the entity has made reasonable achievements.

Applicability

6.18 This policy applies to the monitoring of all recommendations from performance audits, and audit notes including those of the Commissioner of the Environment and Sustainable Development (CESD). If a chapter does not contain recommendations, monitoring may still be warranted. The audit Principal is expected to decide whether observations raised in the original chapter require monitoring at the time the audit is entered into the recommendations database.

6.19 When some recommendations or issues are the subject of a follow-up audit, an update from the entity about what stage it is at in implementing the recommendations may not be necessary. If the follow-up audit work is almost completed, it is usually not necessary to ask the entity for an update about what stage it is at in implementing the particular recommendations that are part of the follow-up audit. The audit Principal can then exclude those specific recommendations from the ones that are sent to the entity for an update.

Reporting media

6.20 The results of monitoring are reported in the Office Departmental Performance Report, which is tabled in the House of Commons in the fall of each year.

Responsibilities

Recommendations database

6.21 The recommendations database holds all monitoring data. It consists of the following:

• recommendations from all audits;
• data classifying each recommendation such as recommendation identifier, difficulty rating, impact area, focus area;
• whether the entity accepted the recommendation;
• the level of Parliamentary endorsement of recommendations;
• self-reporting returns from each entity;
• the audit Principal's assessment of entity progress; and
• the rationale for the assessment.

6.22 Strategic Planning and Professional Practices is responsible for the database, specifically for

• creating the database and its maintenance as a data system;
• providing audit Principals with annual call letters to entities to update the database. The annual call letter includes chapters that were tabled one to six years ago.
• providing the Executive with any required reports and returns from the database;
• monitoring whether the database is up-to-date and complete, and informing the Executive of any shortcomings.

6.23 The entity Principal is responsible for obtaining self-reporting returns from entities and, where applicable, communicating the entity's responses to the audit Principal.

6.24 The audit Principal is responsible for updating the database in a timely fashion in response to the call letter from Strategic Planning and Professional Practices.

Tabling

6.25 After an audit report is tabled, Strategic Planning and Professional Practices is responsible for the input of the recommendations into the recommendations database.

6.26 The audit Principal is responsible for identifying the following:

• an assessment of whether or not the entity accepted the recommendation. This is not merely whether the entity claims to have accepted the recommendation, but also the Principal's full assessment of whether the response constitutes a substantive acceptance;
• observations where the audit Principal would like to monitor progress;
• the impact area and the Auditor General focus area of each recommendation;
• the expected difficulty for the entity of implementing the recommendation as outlined in the Guide to Assess Entity Progress (see paragraph 6.7); and,
• any existing recommendations that should be coded as "obsolete" in the recommendations database because new audit work or circumstances have rendered them redundant or out-of-date.

Hearings

6.27 Parliamentary Affairs is responsible for reviewing Public Accounts Committee reports that result from the hearings on audit reports, requesting input from the audit Principals, assessing the level of endorsement of each recommendation by the Committee, and entering their assessment into the recommendations database.

Annual monitoring steps

6.28 To ensure uniformity, the annual monitoring exercise should be carried out in a consistent manner. Detailed guidance on the steps to be followed is provided in the recommendations database.