This Web page has been archived on the Web.

1981 Report of the Auditor General of Canada

Introduction

5.1 This chapter is our second report on the monitoring of the Office of the Comptroller General's Improvement in Management Practices and Controls (IMPAC) initiative in government departments and agencies. The monitoring of the IMPAC process and its progress is carried out in accordance with the request of the Public Accounts Committee.

5.2 IMPAC was designed to identify the current state of management practices in departments and agencies, to support the development of individual departmental Action Plans for improvement of existing practices, and to follow through with the implementation of these plans. This is a large undertaking, initially involving 30 departments and agencies that account for about 85 per cent of government expenditures (excluding Public Debt) and about 300,000 person-years.

5.3 This chapter covers our monitoring of 18 IMPAC Action Plans approved as of 31 March 1981. We defined our IMPAC review process in Chapter 4 of our 1980 Report; our work is designed to provide the House of Commons with an independent, on-going review of the progress of IMPAC.

5.4 Since our 1980 Report, the Office of the Comptroller General (OCG) has been formulating and issuing internal policy guidelines regarding the IMPAC process. The policy guidelines formalize the approach to IMPAC described in our 1980 Report and are summarized below. There are three broad stages in the process.

5.5 Stage I involves the preparation of an IMPAC Survey Report by the OCG that outlines the state of the management practices at the time of the survey and identifies the improvements required to meet a standard of management practice agreed to by the Comptroller General and the deputy head. The objective is to bring about benefits in the organization that outweigh the cost of the improvement effort. Improvements required are determined in the areas of Planning, Organizing and Implementing, Controlling, Financial Administration, Internal Audit, Program Evaluation, and other areas which may be added to reflect better the needs of the organization surveyed.

5.6 Stage II involves the development of an Action Plan by the department that spells out how the surveyed organization will implement agreed improvements. The Comptroller General and the appropriate deputy head jointly endorse the Action Plan. Agreement between the department and the OCG on the Action Government Initiative for Improved Management Practices Plan is indicated by formal communications between the Comptroller General and the deputy head. The communications will also state, when appropriate, the implementation resource assistance to be provided to the department.

5.7 Stage III involves the implementation of the Action Plan by the department and monitoring by the OCG.

5.8 The Management Practices Branch of the Office of the Comptroller General has prime responsibility for overall direction and monitoring of IMPAC. Assistance is provided by two other branches: the Policy Development Branch, responsible for Internal Audit, and the Program Evaluation Branch.

Scope of Review

- Office of the Comptroller General - IMPAC role: a review of the processes as defined and utilized by the Office of the Comptroller General to implement and monitor IMPAC.

- Previous Audit Observations: a review of the IMPAC response to previous audit observations and recommendations of this Office for all 18 Approved Action Plans.

- IMPAC Process for Operational Improvements: a full review covering approach to achieve change, contents of Action Plans, and the implementation process for the organizations subject to comprehensive audit in 1981 that had approved Action Plans (Royal Canadian Mounted Police, Post Office, Correctional Service of Canada, Consumer and Corporate Affairs Canada). Revenue Canada, Customs and Excise was also reviewed.

- Follow-up on other matters in our 1980 Report: a review of departmental and Office of the Comptroller General responses to our observations regarding IMPAC in the 1980 Report.

Summary of Observations

(Exhibit not available)

5.11 From our review of the IMPAC process, we ascertained that no provision has been made by the OCG for the formal evaluation and documentation of the achievement of IMPAC objectives. We believe that it is essential that evaluation procedures be in place to determine whether the agreed Action Plans have corrected deficiencies or achieved the agreed objectives.

5.12 We reported in 1980 that the implementation of IMPAC within departments and agencies had been slower than planned; this is still true. At the time of our review, delays in meeting planned milestones had occurred in approximately 20 per cent of the IMPAC tasks within the Action Plans, even though many of the Plans had been revised. The scope and complexity of IMPAC, combined with changing departmental organizational structures or the difficulty in obtaining adequate departmental staff resources to implement IMPAC have been among the contributing factors. The Action Plan projects experiencing delays were mainly in the areas of Planning, Management Information Systems and Program Evaluation.

5.13 The Comptroller General, when approving the IMPAC and Implementation Assistance Programs for departments and agencies, requested that organizations monitor resource utilization and significant benefits achieved as a result of IMPAC. In our 1980 Report, we also reported on the need for OCG guidelines for standard reporting of Action Plan costs and benefits. Our review in 1981 generally did not find evidence of potential benefits being identified either at the Action Plan approval stage or subsequently during the Plan's implementation. Guidelines had been drafted at the time of our review for the recording of costs and benefits, but were not yet in use.

5.14 Although the project management and monitoring procedures carried out by the IMPAC liaison officers and staff of the OCG generally followed their stated practices, we found that improvements could still be made. In particular, attention should be given to improving the reporting of Action Plan progress. Within departments, attention should be directed to improving the standards of project management and status reporting, which varies in approach and quality from department to department.

5.15 Our review of the 18 approved Action Plans indicated that they addressed previous audit observations made by our Office. The full review this year of five departments found that the Action Plans approved had given adequate attention to plan content and management.

5.16 Since our last report, IMPAC coverage of government organizations, as evidenced by approved Action Plans, has increased from 7 to 18. Given the complexity and the long time periods involved in the IMPAC programs, the major challenge for the Office of the Comptroller General will be to maintain the momentum of departmental commitment to the IMPAC process.

Office of the Comptroller General - IMPAC Role

5.18 Observation. Subsequent to our comments in our 1980 Report on the informality of project management processes and the need for their further development, monitoring processes have been documented by the Office of the Comptroller General.

5.19 A key process in the Comptroller General's IMPAC follow-up procedure is the "IMPAC Anniversary Meeting" held with entities with approved Action Plans. It was planned that, on or about the annual anniversary of the Action Plan approval date, a meeting would be held with departments to review progress and future plans, to encourage the staffs of departments in their commitment to IMPAC, to cause departments to examine and comment on progress and to demonstrate the commitment of the Comptroller General to helping the department and its IMPAC program. Of the seven departments with Action Plans approved prior to April 1980, two meetings have been delayed with no future date set. The remaining five have had their meetings, but after delays ranging from two to seven months from the anniversary date. Various explanations were given for such delays, including lack of Comptroller General staff resources, difficulties in arranging meetings, absence of appropriate Office of the Comptroller General and/or departmental senior management staff, and revision of Action Plans. Delays in holding Anniversary Meetings could be detrimental to the purpose of this monitoring process and could adversely affect the department's commitment to and momentum of the Action Plans.

5.20 The Office of the Comptroller General has been allocated resources to supplement departmental IMPAC efforts. To date, $14 million has been allocated to 11 departments in amounts ranging from $0.4 to $3.5 million. We found reasonable control procedures in place for the Implementation Assistance Program (IAP). A condition of IAP is that departments report annually "with a detailed assessment of past and projected resource utilization and benefits derived therefrom."

5.21 At the time of our review, there were no standard requirements for such reporting or for the methodology to be used to define costs or benefits. The lack of such guidelines for the use of IMPAC participants has resulted in the reported estimated costs of IMPAC being calculated in an inconsistent manner. For example, some departments have only included the salary costs of additional staff required for IMPAC, while others have included the salaries of all staff involved in IMPAC, whether existing or additional. It was also noted that the inclusion of administrative overhead costs was not applied in a consistent manner. Guidelines are currently being prepared, are in draft form and expected to be finalized by the end of 1981.

5.22 OCG liaison officers, through contacts with departments and in accordance with the agreed monitoring milestones, maintain status reports. Our review of this monitoring process indicated that, although the framework for an adequate system existed, its execution needed to be improved. At the time of our review, status reports were inadequate to provide explanations of project delays, auditable trails of IMPAC progress, records of decisions affecting Action Plans, and evidence of follow-up actions resulting from previous status reports. It was also noted that the OCG had not provided departments with guidelines for reporting progress.

5.23 The majority of IMPAC programs are in their early stages of development or implementation. We stated in our 1980 Report that it was important that the IMPAC process include appropriate procedures to determine whether matters agreed as being deficient or requiring improvement had been dealt with or improved as planned. From this year's review of the IMPAC process, we ascertained that only limited provision had been made by departments for a formal evaluation of the achievement of IMPAC objectives and benefits from completed IMPAC projects. This is important as a means of demonstrating that the IMPAC program is contributing to departmental objectives.

IMPAC Response to Previous Audit Observations

5.25 Observations. We ascertained that most of our outstanding observations and recommendations were either covered by the Action Plans or by improvement projects initiated by the entity, outside IMPAC. It must be stated at this point in time that coverage by IMPAC does not, in itself, assure that the problem identified will be corrected.

IMPAC Process for Operational Improvements

Framework for Change

5.27 Observation. The magnitude and complexity of the program being undertaken should not be underestimated. The implementation programs of the 18 entities with approved Action Plans involve some 430 major projects comprising some 2,400 tasks at an estimated cost, as reported in the approved Action Plans, of some $110 million. The estimated time required to complete IMPAC for these 18 organizations involves a period from late 1978 to 1985.

5.28 As the entities have proceeded further into the IMPAC process, it has become apparent that many of the original surveys and Action Plans require revision. Such revisions have involved redefining the Action Plan because of the unforeseen complexity of the program, and rescheduling of project milestone dates because of improved knowledge of project time requirements or delays due to the difficulty of obtaining appropriate staffing. Of the six departments whose Action Plans were reviewed in the IMPAC chapter of our 1980 Report, four have made or were making major revisions to their plans.

5.29 With programs that are of a long-term nature, there are many factors that may affect their progress. These may include:

- central agency administrative policies, such as The Policy and Expenditure Management System (PEMS), that may require in corporation into the IMPAC Action Plans;

- organizational changes within a department that may affect the planned progress of IMPAC;

- program staff changes within the Office of the Comptroller General and entity organizations that may affect the level of expertise available for the implementation and orderly continuity of the IMPAC program; and

- availability of appropriate staff in departments to implement IMPAC projects.

5.30 With the constant pressures on government organizations to do the same or more for less, management is being challenged to decide where to apply resources. Thus, a major task facing managers is to rank program expenditures in order of priority, taking into account both tangible and intangible benefits. There was only limited evidence that the expected benefits of the IMPAC program had been documented either prior to approval of the IMPAC Action Plan or as implemented. The articulation of benefits from IMPAC should aid in maintaining a long-term commitment to the IMPAC program, particularly when management is faced with competing demands for resources.

Contents of IMPAC Action plans

5.32 IMPAC attention to underlying cause. IMPAC projects, which, in many cases, have a very broad scope, were reasonably addressing the underlying causes of problems. In some cases, departments decided during the implementation of agreed Action Plans that the original IMPAC plan projects could be improved. In such cases, the Action Plan or the particular project within the Action Plan has been revised.

5.33 Priorities in the IMPAC plan. Generally, we found that setting priorities for IMPAC projects was an informal process that took into account such matters as the inter-relationship of projects and the availability of resources. In some cases, the results of the project planning were displayed on critical path charts to aid in planning and testing the viability of the scheduling of IMPAC projects. However, identifying individual IMPAC project costs and benefits, whenever practical, will aid management in setting priorities and making decisions. Although the results of priority setting appeared to be reasonable, there were no guidelines or formal processes for establishing such priorities. Guidelines would normally be expected as a part of good project management practices.

5.34 Timing of IMPAC tasks. The timing estimates for projects have been determined by departments in a subjective manner based largely on experience. Timing considerations will require further work; IMPAC Action Plans are 2 to 21 months behind schedule. Project timing estimates that are overly optimistic and out of line with actual requirements also raise doubt about the quality of the IMPAC planning and final completion dates of the Action Plans, thus creating a question of confidence in the Action Plans.

IMPAC Implementation Process

5.36 Interim requirements. In a program such as IMPAC, with project times spread over periods from 1978 to 1985, opportunities may arise to implement interim improvements before achievement of the full planned result. From our interviews we concluded that consideration had been given to implementation of interim improvements in preparing Action Plans.

5.37 Progress against milestone. We have been concerned about the ability of Action Plans to meet their indicated milestones. The plans under review were recently approved, but already there have been delays in meeting milestones.

5.38 The major areas of progress slippage were the Planning, Management Information Systems, and Program Evaluation components of the Action Plans. Departmental representatives told us that the reasons for slippage were:

- unanticipated delays in starting up the Action Plan programs due to staffing requirements and/or organizational changes;

- the complexity of the processes under review, requiring amendments to plans as the problems were further examined; and

- other demands on senior managers' time, competing with their need to participate in the IMPAC decision-making process.

5.40 Having established across five departments that delays in progress were general, we extended our review of progress to all 18 departments. We found that, as of 31 May 1981, about 20 per cent of the tasks were behind schedule, excluding one department that was involved in a major revision of its Action Plan, and therefore had not established revised tasks and milestone dates. These delays occurred subsequent to six departments revising their Action Plans during the year. The need for these revisions throws doubt on the possibility of completing the IMPAC plans as originally scheduled. We have not attempted to assess the implications of such delays on resource requirements and final completion dates of the Action Plans.

5.41 Project management of IMPAC. Although we found that the project management processes and status reporting methods for IMPAC as practiced by the departments varied, the IMPAC plans were formal documents with stated project objectives, responsibilities were indicated and acknowledged, resource requirements were costed and scheduled and progress milestones and completion dates were documented. Such processes have helped the Office of the Comptroller General and departments to identify areas in the implementation of IMPAC requiring attention and/or corrective action.

5.42 In regard to project management, the following matters require further attention:

- monitoring costs and benefits, as requested by the Office of the Comptroller General;

- providing for the review of completed projects to evaluate whether they have met the original objectives as stated in the IMPAC program; and

- establishment of IMPAC project management and status reporting guidelines by the Office of the Comptroller General for use by departments.

Follow-up on Other Matters in our 1980 IMPAC Report

5.44 An Office of the Comptroller General policy statement effective 1 December 1980 indicated that further government entities would be covered by IMPAC. This would result in a coverage of entities representing approximately 99 per cent of person-years and 97 per cent of planned government expenditures (excluding public debt) based on 1980-81 Estimates. In October 1980, the OCG began an IMPAC survey of such an entity, the Treasury Board Secretariat. Depending on the availability of OCG resources, these additional entities would include such organizations as the National Research Council, Department of Communications, National Energy Board, Canadian Radio-television and Tele- communications Commission, the Agricultural Stabilization Board and the Northern Pipeline Agency.

5.45 Our 1980 Report noted that the Canada Employment and Immigration Commission was preparing a workplan for improvements to the Unemployment Insurance benefits system in regard to overpayments and underpayments. This workplan was to be prepared for inclusion in the Commission's preliminary approved IMPAC Action Plan. During the current year, considerable effort has been devoted to completing the workplan for improving the benefit and over- payment system. A draft of the workplan was submitted to the OCG and received agreement in principle on the objectives, planned approach and specific steps required to improve the control over the benefit payments. The Commission requested of the Treasury Board additional resources to implement this program. Approval for some resources was received during 1981. At September 1981, the Commission was in the process of revising the workplan in line with resources available for allocation to these initiatives.

5.46 We also stated in 1980 that payroll costs management issues, which are an integral part of management practices and controls, were not incorporated into IMPAC. These issues include human resource planning, training, and human resource information systems. In our opinion, this is a significant weakness in the IMPAC program, and it has not been addressed since our 1980 Report.

5.47 In 1980, when we reviewed the Action Plans of six departments, we observed that the Energy Sector activities of the Energy, Mines and Resources Action Plan were not on schedule. Our follow-up review in 1981 indicated that this was still a matter for concern. We also noted that the IMPAC program for the Department had been revised but had not been approved.