1996 November Report of the Auditor General of Canada
Chapter 38—Follow-up of Recommendations in Previous Reports
38.2 Federal Management of the Food Safety System. The new Canadian Food Inspection Agency is being created to consolidate all federally mandated food inspection and animal and plant health programs. It is intended to make the federal food inspection programs more efficient and effective, at a lower cost to the taxpayer.
38.3 Safety and inspection practices are not yet risk-based and consistent across products and industries, the frequency of plant inspections remains inconsistent, and frequency targets are often not met. Although this situation does not compromise health and safety, it will persist until common standards are finalized and inspection programs are revised.
38.4 Atomic Energy Control Board (AECB) - Canada's Nuclear Regulator. Recent initiatives undertaken by the AECB represent both a commitment to revitalize the organization and a considerable effort to address our audit concerns. However, much remains to be done.
38.5 The resolution of more than two thirds of our audit concerns is dependent in whole or in part on the outcome of proposed new legislation currently before Parliament. In our opinion, without updated legislation, the AECB remains limited in its ability to fulfill its mandate. However, as noted in our 1994 audit, new legislation by itself will not correct the deficiencies identified in the management processes and practices.
38.7 Observations and recommendations made in our Reports are normally followed up and their status reported in the annual Report two years after publication of the original chapter. This year, follow-up on 12 chapters has been reported in other chapters of our Reports, 17 chapters have had their follow-up deferred to future years, and there will be no formal follow-up on 5 chapters. Exhibit 38.1 summarizes the status of those chapters whose follow-up report would normally have been included in this chapter.
Federal Management of the Food Safety System - 1994, Chapter 13Assistant Auditor General: David Roth
Responsible Auditor: Bill Rafuse
Background38.8 Our 1994 audit examined the arrangements and co-ordination among the federal departments directly involved in delivering the federal mandate for food safety.
38.9 Health Canada has primary responsibility for health, safety and nutritional aspects of domestic and imported food. Agriculture and Agri-Food Canada and Fisheries and Oceans Canada share the food safety responsibilities with Health Canada, and further regulate the marketability of food products traded interprovincially and internationally.
38.10 The Interdepartmental Committee on Food Regulation is responsible for planning and co-ordinating the federal food safety system. The Committee responded to our Report and recommendations in 1994 ( Exhibit 38.2 ). It has also reviewed and concurred with the findings of this follow-up report.
Conclusion38.11 Major organizational changes are taking place in the federal food inspection system. The new Canadian Food Inspection Agency is intended to make federal food inspection programs more efficient and effective, at a lower cost to the taxpayer. The Agency will assume the responsibility for implementing many of our recommendations, as outlined in this report.
38.12 For both domestically produced and imported food products, there has been little improvement in the consistency of safety and inspection standards and the basis on which they are applied. Although some progress has been made in developing common inspection approaches, food commodities posing similar risks to human health may still receive very different levels of safety inspection. While not compromising health and safety, this situation will persist until common standards are finalized and inspection programs are revised.
38.13 There are several areas where Health Canada has not made changes and achieved desired results as quickly as planned. For example, its audit of inspection practices at Agriculture and Agri-Food Canada and Fisheries and Oceans Canada has not fully achieved its objectives. This audit should be currently providing a critical check on the inspection activities of these departments and, soon, on the food safety activities of the new Agency.
38.14 Health Canada has made slow progress in obtaining inspection information from the provinces, in reviewing and rationalizing its laboratory facilities and in publishing data on food-borne illnesses on a more timely basis.
38.15 The federal food inspection system is in a period of great change. During this time, it will be necessary for Health Canada and the new Agency to focus increased attention on a number of areas to ensure that outstanding deficiencies are corrected, that planned economies and efficiencies are captured and that the checks and balances in the system are maintained and strengthened.
Interdepartmental Committee on Food Regulation38.16 Restructuring the Committee. Driven by the need to reflect the restructuring of some member departments and our recommendation that it re-examine its own role and structure, the Interdepartmental Committee on Food Regulation changed its membership and structure late in 1994. The Committee`s mandate was updated, membership was shifted from deputy minister to assistant deputy minister level and the subcommittee structure was collapsed under the Interdepartmental Committee on Food Inspection.
38.17 Now, with the realignment of certain roles and responsibilities and the creation of the new food inspection agency, new structures and mechanisms may be required for effective planning and co-ordination of the federal food safety system. The mandate, structure and future role of the Committee will require review to determine whether further changes are necessary. Proper co-ordination will be important as changes are implemented.
Canadian Food Inspection Agency38.18 The most significant achievement since the audit has been the realignment of roles and responsibilities leading to the consolidation of all federally mandated food inspection and animal and plant health programs into a single federal food inspection agency. Legislation to establish the Canadian Food Inspection Agency received first reading 19 September 1996. The Agency will report to Parliament through the Minister of Agriculture and Agri-Food Canada.
38.19 A single federal contact. The new Agency is to provide a single federal contact for consumers, industry and the provinces for food inspection policy and delivery activities. For example, importers of several different types of food products will be able to deal with one federal organization on issues related to import inspection. The creation of the Agency should also facilitate the work of the Canadian Food Inspection System initiative toward achieving harmonized food safety standards across the federal and provincial governments and more streamlined inspection delivery - in short, a more integrated national system of food inspection.
38.20 All inspection services that were provided by Agriculture and Agri-Food Canada, Health Canada and Fisheries and Oceans Canada, and related to food safety, economic fraud, trade-related requirements and animal and plant health programs, will be provided by the new Agency. The responsibility for food safety policy, standard setting, risk assessment, analytical testing research and audit of the Agency's food inspection activities will remain with Health Canada.
38.21 Increased efficiency and reduced costs. The reorganization of departments and responsibilities is to reduce costs, enhance efficiency and improve the effectiveness of inspection services. Specifically, it is to eliminate interdepartmental overlap and duplication in such areas as enforcement, risk management, laboratory services, informatics systems and communications, leading to savings of $44 million. Combined with other savings that are planned, the cost of all federal food standard-setting and inspection efforts will fall from $400 million in 1994-95 to roughly $300 million when implementation is complete in 1998.
38.22 Emergency operations. Our 1994 audit found that no evaluation process existed to determine if proper actions were taken in response to food emergencies. In November 1995, Health Canada issued standard operating procedures for responding to emergency situations that called for post-emergency reviews and assessments. It has been agreed that the Agency will take the lead in responding to food safety emergencies and, as well, will be responsible for the critical post-review function. It is important that Health Canada review these activities as part of its audit responsibilities.
38.23 Food inspection standards . Our audit recommended that specific attention be directed to making food safety standards and inspection approaches consistent. Since then, departments have agreed on new health hazard risk categories for foods, based on their inherent risk to human health and that associated with their processing. Agriculture and Agri-Food Canada has initiated a project to identify and measure the risks associated with specific food products and to then allocate its food safety inspection resources based on these risks.
38.24 The Agency will take the lead role in developing common inspection approaches and tests for food products in the three risk categories and will be responsible for applying them. Health Canada will remain responsible for establishing food safety policy and standards. These safety and inspection standards, when fully applied, will mean that foods posing similar risks, and the plants and industries producing them, will be subject to the same health and safety requirements and will receive the same level of inspection.
38.25 Progress in this area is being made, but the situation today is not much changed from 1994. Safety and inspection standards are not risk-based and consistent across products and industries, plant inspection frequencies remain inconsistent, and frequency targets are still not met.
38.26 Food import control. The interdepartmental subcommittee on imports has been working for the past two years on a number of initiatives to improve the co-ordination of food import control activities. For example, the subcommittee is identifying gaps and duplication in existing inspection programs and will recommend ways to eliminate these situations.
38.27 The import subcommittee has found that policies concerning importer registration and licensing are inconsistent. It has also found that the responsibility for inspection of certain food commodities is unclear and that the level of inspection activity aimed at different food products may not be consistent with their potential risk to human health.
38.28 The subcommittee has confirmed that improvement in the co-ordination and consistency of import controls has been modest and much remains to be done. Departments are confident, however, that the Canadian Food Inspection Agency will have a positive impact on these and other issues, as the Agency will be the single federal contact point for importers and all federal import inspection responsibilities will rest with it.
38.29 Cost recovery. Following our observation that only a very small portion of inspection costs was recovered, the 1995 federal Budget highlighted user fee increases for food, meat and fisheries inspections. Although significant cost recovery increases have occurred - to $19 million in 1995-96 and $28 million (forecast) in 1996-97 - the amounts still fall far short of the cost of providing these food inspection services. In the future, the Agency will be responsible for the consistent and equitable application of cost recovery policies to food processors and distributors. It will continue to emphasize the recovery of inspection costs that provide a direct benefit to industry. The Agency's first Business Plan will outline its financial strategies and cost recovery objectives.
38.30 Co-ordination with United States food inspection agencies. Our 1994 audit included a concurrent review, conducted jointly with the United States General Accounting Office. We assessed the exchange of information between the food inspection agencies of both countries with respect to controls over food imported from third countries. We found that improved co-ordination and information sharing between Canadian and United States food agencies could lead to better inspection coverage of foreign food-processing plants and greater assurance that unsafe food is not imported into our countries.
38.31 Some action has been taken by agencies in both countries, in part as a result of trilateral discussions between Canada, the United States and Mexico. The objective of sharing information on foreign plant inspections is to both co-ordinate review activities and share inspection results between the countries. However, legal concerns related to information exchange have slowed this initiative.
38.32 Progress on the sharing of information on rejected food shipments that are subsequently re-exported to or through the United States or Canada has been achieved largely through electronic means. For example, the United States Food and Drug Administration now posts its import inspection decisions on the Internet, where they are accessible to everyone. Canadian departments are exploring similar options. We have not assessed the completeness of this information or the consistency with which it is followed up by Canadian and United States food inspection agencies.
Health Canada38.33 The Minister of Health and the Food and Drugs Act have primacy in all matters relating to food health, safety and nutrition. The food safety provisions of the Food and Drugs Act apply to the entire Canadian food processing and distribution sector. Health Canada must maintain current and relevant standards to meet Canadian needs and see that there are reasonable enforcement strategies to safeguard the health and safety of the Canadian public.
38.34 In 1994, we concluded that Health Canada did not have all of the information needed to assess whether the food-related health and safety provisions of the Food and Drugs Act were applied fully and effectively to all food produced domestically or imported for sale in Canada. Gaps existed in the information received from inspection agencies - both federal and provincial - and in the coverage they achieved.
38.35 Food safety audit. Health Canada has developed a food safety audit program to provide assurance to the Minister of Health that the food safety provisions of the Food and Drugs Act are effectively applied. This is to be accomplished by examining all of the food inspection programs and activities of Agriculture and Agri-Food Canada and Fisheries and Oceans Canada that relate to the safety of the Canadian food supply.
38.36 Audits of various inspection activities in the two departments are currently under way. After two years, however, Health Canada cannot reach a conclusion on the food-processing industry's compliance with the food safety requirements of the Food and Drugs Act . The Department's audit process has not progressed to the point where the required level of assurance can be provided to the Minister of Health. This critical review of the effectiveness of federal food inspection programs must soon be directed to the activities of the Agency.
38.37 Provincial food inspection activities . In 1996, Health Canada initiated a project to gather information on provincial and territorial food inspection activities including, for example, whether:
- compliance with standards equivalent to those of the Food and Drugs Act is verified;
- gaps exist in the inspection coverage of plants and imports;
- inspection approaches are consistent; and
- inspection programs are based on an assessment of risk to human health.
38.38 Laboratory rationalization. Agriculture and Agri-Food Canada and Fisheries and Oceans Canada have conducted individual reviews of their laboratories. Fisheries and Oceans Canada subsequently closed a number of its facilities. Health Canada plans to complete its own laboratory review by December 1996. The task will then still remain to rationalize remaining facilities across the system before the problems of overlap, duplication and overcapacity are fully resolved. The Canadian Food Inspection System initiative offers the potential for further co-ordination and streamlining of food laboratory facilities at the federal-provincial level.
Program effectiveness and accountability38.39 Food-borne illnesses. At the time of our audit, the latest information published by Health Canada on food-borne illnesses was for 1987, a time lag of seven years. This year, summaries for 1988 and 1989 were published. Although Health Canada has recognized the timeliness of food-borne illness data to be of paramount importance, delays in reporting this information remain unacceptably long. This information provides one indicator of the effectiveness of the food safety system. Although it could assist managers in allocating resources and managing programs more efficiently, information seven years old is of little use for these purposes.
38.40 Program evaluation. Little progress has been made in acting on our recommendation to measure the effectiveness of the Canadian food safety system by conducting a program evaluation study in co-operation with the provinces. A proposal to conduct an evaluation within the Canadian Food Inspection System initiative was put on hold pending creation of the new Agency.
38.41 Federal food safety programs have not been formally evaluated for 10 years. Information on the results and effectiveness of changes made during this period is incomplete. An evaluation framework could now be developed based on the agreement concerning roles and responsibilities that was reached in creating the Agency. The framework could provide the basis for various evaluation projects, some of which could be designed and started right away.
38.42 Reporting to Parliament. Part III of Health Canada's Estimates now contains more and better information on the Department's food safety activities. Further improvements in the information available to members of Parliament are expected when the Canadian Food Inspection Agency produces its first Business Plan for 1997-98. Both the Agency and Health Canada have committed to developing indicators of their performance, and to measuring and then reporting their results against them.
Agriculture and Agri-Food Canada - Farm Income Protection - 1994, Chapter 14Assistant Auditor General: Don Young
Responsible Auditor: Douglas Timmins
Background38.43 Our 1994 chapter focussed on farm income support programs delivered by Agriculture and Agri-Food Canada. We assessed and reported the extent to which the Department had:
- designed programs with clear, consistent and commonly understood objectives to address farm income priorities;
- established systems to measure and report on their effectiveness; and
- obtained reasonable assurance that the programs were subject to proper financial management and control.
Conclusions38.45 The Department has made progress in response to most of our recommendations; however, further work will be required to fully address our concerns in some areas.
38.46 Progress is lacking in some areas where it is hard to make changes to existing agreements; therefore, we can expect that progress will be minimal until new programs or agreements are in place. In other areas, agreement by the provinces and producers is needed and obtaining it will not be easy.
Programs with clear, consistent and commonly understood objectives and performance measures38.47 In 1994, we were concerned that Agriculture and Agri-Food Canada continued to operate programs without a clear consensus on what they were expected to achieve, and thus program performance could not be measured. We concluded that the Department would have a difficult time determining whether the principles of social, environmental and production neutrality and equity as set out in the Farm Income Protection Act were being adhered to.
38.48 Since 1994, the Department has developed with the provinces, taking into account industry deliberations, a Framework Agreement on Agricultural Safety Nets. Four provinces have already signed the agreement, which includes a program objective statement and statements of principle on equitability, environmental sustainability, and minimization of interprovincial and intercommodity bias and trade vulnerability. The signed agreements include no principle statement on encouraging long-term social and economic stability of family farms and communities, a principle of the Farm Income Protection Act .
38.49 At the same time, the Department has developed a list of industry/farm-level performance indicators covering five departmental strategic goals in the areas of sustainable growth, rural opportunities, financial security, resource and environmental sustainability, and a safe and high-quality food supply.
38.50 From this list it is not clear which performance indicators are considered relevant in relation to the recently developed program objective and principle statements. Also, no plans are in place to obtain the provinces' or farmers' acceptance of those measures to be used to monitor program results. If program performance is to be reported against expected outcomes, there is still a need to clearly set out what is to be measured, to ensure that it is commonly understood and defined, and then to collect the data.
Proper financial management and control38.51 Up to 31 March 1994, the Saskatchewan Crop Insurance Program had received $536 million from the federal and provincial reinsurance programs to cover a cumulative negative fund balance, even though the Program is required to be self-sustaining. Between 1991 and 1994, there was a 58 percent increase in premium revenue to address this problem, but at the same time a 20 percent decline in producer enrolment occurred. Since 1994, two studies have been commissioned to determine the causes of withdrawal. They concluded that some factor other than price had resulted in declining participation, and recommended further research. In the two years following our audit, producer participation and premium income increased. These factors helped to reduce Saskatchewan's cumulative negative fund balance. Further design changes may still be required to ensure that the Program is self-sustaining.
38.52 The federal Crop Reinsurance Program provides a means for the federal government to share with provincial governments the financing of any large financial losses that may result from the Crop Insurance Program. The deficit in the federal reinsurance account for Saskatchewan due to such financing has been reduced from $355 million in 1994 to $285 million in 1996. Some progress is being made by the Department to revise the Crop Reinsurance Program in order to achieve financial self-sustainability.
38.53 In 1994, we identified a number of areas where the financial management and control of the Net Income Stabilization Account (NISA) needed to be strengthened. Reasonable progress has been made in addressing four of our five specific recommendations, while the fifth, the evaluation of the Net Income Stabilization Account Program, has just been initiated.
38.54 With the exception of our recommendation that it establish guidelines in future agreements for provincial program management and levels of service, the Department has made reasonable progress in response to the recommendations regarding deficiencies in its financial management and control.
Timely preparation and disclosure of financial statements, annual reports, and program evaluations38.55 The Department continues to make progress in improving the timeliness of its preparation of financial statements. However, despite a commitment to the Public Accounts Committee to have evaluations of the farm income protection programs completed in 1993, 1995 and 1996, none had been completed at the time of our follow-up audit.
Atomic Energy Control Board - Canada's Nuclear Regulator - 1994, Chapter 15Assistant Auditor General: Maria Barrados
Responsible Auditor: Ellen Shillabeer
Background38.56 The objective of our 1994 audit was to assess whether the Atomic Energy Control Board (AECB) was fulfilling its obligations as a regulatory body. We focussed our audit on certain aspects of the AECB's regulatory program that are important to the fulfilment of its mandate.
38.57 Since we issued our Report in 1994, two major developments have been put into motion that have the potential to significantly change the operations of the AECB as well as to address our audit observations and recommendations.
38.58 First, the AECB has undertaken a major project, Project 96 and Beyond, which when implemented is expected to re-create and restructure the way it does business. In the fall of 1994, an internal review was initiated to look at the strengths and weaknesses of the organization. Subsequently, in July 1995, the President of the AECB launched Project 96. This project touched on almost all aspects of AECB's operations and involved the active participation of AECB staff at all levels of the organization. Project 96 recommendations were submitted to the President in June 1996 and are currently being considered by the President and AECB senior management for adoption and implementation.
38.59 Second, Bill C-23 was tabled in the House of Commons on 21 March 1996 to replace the Atomic Energy Control Act of 1946. This Bill was given second reading on 12 June 1996. At the time of writing this report, the House of Commons Standing Committee on Natural Resources had started hearings on the Bill.
Scope38.60 We followed up on each of our recommendations and on certain significant observations of our 1994 Report. We reviewed the AECB's August 1995 status report to the Public Accounts Committee on action taken in response to the recommendations of our 1994 Report. In June 1996, AECB officials provided us with an update to this status report as well as to certain audit observations. We discussed reported progress with them and obtained and reviewed supporting documentation.
38.61 We also assessed the extent to which the recommendations of Project 96 and Bill C-23 would address the concerns raised in our 1994 audit.
Conclusion38.62 Recent initiatives undertaken by the AECB, most specifically Project 96, represent both a commitment to revitalize the organization and a considerable effort to address our audit concerns. These initiatives, coupled with the anticipated passage of new legislation, have the potential for providing the regulatory framework and key elements that are necessary to steer the AECB in a direction that will allow it to more fully meet its obligations as a regulatory body.
38.63 The AECB should be commended for its efforts to date and for taking steps in the right direction in responding to our audit observations and recommendations. However, much remains to be done to fully address our audit concerns and to achieve the desired results. We recognize that the AECB is facing an obvious constraint, in that resolution of more than two thirds of our audit concerns is dependent in whole or in part on the outcome of Bill C-23. Nevertheless, the AECB needs to ensure that there is a quick disposition and implementation of relevant Project 96 recommendations and other initiatives that are currently under way and are not dependent on new legislation.
38.64 Despite the AECB's progress to date, the organization continues to be hindered in its ability to meet its regulatory obligations. In our opinion, without updated legislation and continued improvements in its management processes and practices, the AECB remains limited in its ability to fulfill its mandate.
38.65 Given the impact of Bill C-23 and of Project 96 recommendations on the AECB's progress in addressing our audit concerns, we will continue to follow up its actions and report on them at a future date.
Legislation respecting the AECB38.66 We observed in our 1994 audit that the legislation respecting the AECB, the 1946 Atomic Energy Control Act , required updating. Bill C-23, as it currently reads, will provide the AECB with a more appropriate range of regulatory tools to enforce compliance with its regulations and licence conditions. The Bill also contains specific provisions allowing the AECB to require financial guarantees from licensees to ensure that there are sufficient financial resources to properly store or dispose of nuclear waste. The AECB expects that our 1994 audit observation respecting the need for changes to its legislation will be addressed if Bill C-23 is passed. However, as noted in our 1994 audit, new legislation by itself will not correct the deficiencies identified in the management processes and practices.
Documented regulatory framework and strategic plan38.67 There is still no clearly documented regulatory framework and strategic plan. Their development and finalization is tied, in part, to the disposition of Project 96 recommendations, and to the outcome of Bill C-23. However, the AECB has undertaken a number of initiatives that are steps in the direction toward responding to our concerns in this area. For example, it developed a new corporate documentation structure, and established a policy governing the regulatory process and a procedure for the production of consultative documents. The AECB is currently undertaking a review of all regulatory documents. Furthermore, various Project 96 tasks have examined documentation, standards and guides, philosophy of regulatory effort, scope of regulatory mandate, and long-range strategic planning.
Improvements to the compliance and enforcement function38.68 Much remains to be done to improve the compliance and enforcement function. Nevertheless, some initiatives are under way to begin addressing our concerns in this area. For example, a general compliance and enforcement policy for the AECB has been approved, and the specific policy for radioisotope licensees has been drafted and related procedures developed. As well, approximately 50 percent of the activities outlined in the Task Analysis Project, which deals with inspection requirements for nuclear power plants, have been implemented; additional steps have been taken to apply the AECB's regulatory policy on decommissioning in the area of uranium mining; and some improvements have been made in the follow-up of non-compliance issues related to prescribed substances and radioisotope licences. The final resolution of our audit concerns in this area is dependent, in part, on the outcome of Bill C-23 and to a lesser extent on the disposition of Project 96 recommendations. In the meantime, we encourage the AECB to continue addressing those issues not directly linked to new legislation.
Jurisdictional agreements38.69 The AECB is continuing initiatives relating to agreements with the provinces or other federal agencies to reduce unnecessary overlap and duplication. However, these efforts are dependent on the counterpart agency agreeing to terms and conditions and on factors beyond the AECB's control. More work remains to be done to negotiate and finalize interagency agreements. Bill C-23, if passed, may also necessitate the revisiting of existing agreements to make them compatible with the new legislation.
Review of human resource requirements38.70 The AECB has made an effort in Project 96 to address the issue of human resource requirements. The Project includes recommendations that could potentially form the basis for determining AECB's human resource requirements in a rational manner. However, the passage of Bill C-23 and subsequent negotiations with other jurisdictions would also have an impact on the AECB's human resource requirements. In the meantime, the AECB still has not established a structured process to review on a regular basis the sufficiency of resources to fulfill its mandate.
Other initiatives and concerns38.71 The AECB has made considerable effort to address our audit concerns regarding the program evaluation and internal audit functions. We intend to monitor these management oversight mechanisms to ensure their continued active operation.
38.72 While steps have been taken to improve the Power Reactor Staff Annual Assessment Reports, opportunities for further improvement still exist. Steps also need to be taken to improve the AECB Annual Report to provide more meaningful performance information. As indicated in our 1994 Report, information to Parliament and the public could be further improved by providing in Part III of AECB's Estimates a cross-reference to its Annual Report and to its Staff Annual Assessment Reports.
Environment Canada - Environmental Partners Fund - 1994, Chapter 19Commissioner of the Environment and Sustainable Development: Brian Emmett
Responsible Auditor: Wayne Cluskey
Background38.73 The Environmental Partners Fund was initiated in 1989-90 as a seven-year, $70 million contribution program administered by Environment Canada. It was designed to fund projects that stimulated the development of local environmental initiatives and to enhance existing environmental programs. In 1991 the program became part of the Department's Environmental Citizenship Initiative.
38.74 We reviewed this program in 1994 because it represented the government's new approach to funding environmental initiatives - working with other parties who also provide services and/or funds to accomplish project objectives. This type of program has the potential to be more significant in the future as governments face increasing pressure to do more with less. It is therefore important that appropriate management structures and processes be developed for these new funding mechanisms.
38.75 We found that the projects we reviewed appeared to contribute to creating awareness of environmental issues or to resolving them, or to support the concept of sustainable development. However, we noted a number of deficiencies in the management of the program, including:
- incomplete evaluation of the program;
- inaccurate reporting of the amount of funds leveraged from participants other than the federal government;
- lack of appropriate performance indicators;
- a management information system that was not being used for a number of reasons;
- no analysis of the financial impact of in-kind contributions by other project participants on the amount of federal contributions;
- no monitoring of the total value of educational projects to ensure that the authorized cap was not exceeded; and
- no resolution of a number of management deficiencies identified in a 1991 internal audit report.
Conclusion38.77 The Department is in the process of making a number of modifications to the program and has developed performance indicators to address most of the concerns raised in our 1994 Report. It is still too early to tell if these modifications and performance indicators will be sufficient for the Department to adequately control program activities and properly evaluate and report the results of the program. An evaluation of the revised program is not planned by the Department until fiscal year 1998-99.
Performance indicators38.78 The 1991 internal audit identified a lack of appropriate performance indicators used by the program for reporting on the results being achieved. This was still true when we performed our audit in 1994. The Department has recently developed an evaluation framework for the program. From that framework, performance indicators were developed in consultation with experts in the Department. We reviewed those indicators and the Department appears to be on the right track. However, a large number of indicators are proposed and it could be difficult to aggregate them in a meaningful way for management decision making. Based on experience with these indicators, changes and improvements may be required in the future.
Management information system38.79 Management is developing a new management information system to replace the system that was discontinued two years ago. The old system was not being used by the regions because it was difficult to use, and did not provide information that was useful to them. It did not include data on project results and could not aggregate information on project results. The new system will contain information on the projects, including performance information, to aid in management decision making and provide data for reporting results and evaluating the program. The system is being developed in consultation with the regions to ensure that it meets their needs and does not require excessive resources to enter data.
38.80 Because no common system was used by the program previously, the few statistics reported for the program concentrated on the approved amounts. The new system should allow the Department to gather and report information on the actual results of the projects. Accurate information should then be available on such things as the amount of funds leveraged from other participants, the amount of in-kind contributions by other participants, the amount of funds spent on different types of projects and the benefits being achieved by the program. The current 1996-97 Part III of the Estimates for Environment Canada contains only a brief description of the program and the amount of funds budgeted to be spent. It contains no activity or performance information.
Program evaluation38.81 The Department will soon have in place the performance indicators and the management information system it needs to do a comprehensive evaluation of this contribution program. Even though it has made only fairly minor changes, the Department is treating the program as new (most of the activities eligible under the old program are still eligible under the new program and the process for reviewing and approving the projects is roughly the same). Only projects approved since this program became part of the Action 21 program (October 1995) will be subject to the next program evaluation. Since the program is considered new, an evaluation cannot be done until there is sufficient activity to arrive at a proper conclusion on its operations. A departmental evaluation of this program has been scheduled for the 1998-99 fiscal year.
Operational guidelines38.82 The Department is developing a revised set of operational guidelines for the program. The revisions pertain chiefly to the criteria for eligibility and evaluation of projects, and to the process for reviewing and monitoring projects. The proposed changes appear to address previously identified deficiencies, but this cannot be verified until projects that have gone through the revised process are reviewed.
Organizational structure38.83 Changes have been made to the organizational structure of the program. Delivery of the program rests with the regions. The headquarters office serves a co-ordinating function to the regions. Previously there was a headquarters organization that guided the program and administered national projects. There are no longer national projects and the program is now the responsibility of the Director General of Communications and, on a rotational basis, one of the regional directors general. There is frequent communication between headquarters and the regions in the form of teleconferences. Projects are submitted to a departmental review committee to ensure that they are eligible and consistent among regions. Projects are approved by the regional directors general. This new structure puts more responsibility on regional management to ensure that the program is operating in accordance with its authority and guidelines. Given that these changes are quite recent, we are not in a position to comment on how well the new organizational structure is working.
Environment Canada - Ice Services - 1994, Chapter 20Commissioner of the Environment and Sustainable Development: Brian Emmett
Responsible Auditor: Wayne Cluskey
Background38.84 In 1994 we examined the Ice Services Program of the Atmospheric Environment Service of Environment Canada. We focussed on the Branch's strategic planning document and on the 1992 action plan developed by the Ice Services Branch in response to the recommendations of a 1991 program evaluation. It was an important period for the Ice Services Branch, which was preparing to change the way it collected and processed ice information by converting from an aircraft-based to a satellite-based system (RADARSAT), while planning and implementing its response to the government's cost-cutting requirements.
38.85 We noted the importance of defining the needs of its primary client, the Canadian Coast Guard, and those of other clients while involving them in the development and implementation of changes to ice data collection. We were concerned about the lack of a science and technology policy to clarify the national and international responsibilities of the Ice Services Branch and to provide guidance to operational managers at this difficult time. With respect to cost-cutting measures, we indicated the need to proceed cautiously in identifying commercial opportunities because of the uncertainties associated with future revenue generation.
38.86 The conversion to a satellite-based ice data collection program, made possible with the launching of RADARSAT in 1995, was expected to produce significant savings and provide the Ice Services Branch with opportunities for revenue generation. It was particularly important, therefore, that preparations be made for unforeseen emergencies caused by partial or complete failure of RADARSAT during or immediately after launch (short term), or later during its scheduled five-year lifetime (long term).
38.87 In this respect, we found a lack of formal agreements with other agencies to provide back-up ice data collection platforms. In addition, while a contingency plan for RADARSAT failure had been developed, it had not been completely tested to assure clients of a continuation in the ice information service should a catastrophe occur.
38.88 Finally, we were concerned about the lack of an ice data archiving policy within the Ice Services Branch, given its role as one of the world's centres of ice expertise and information. In the absence of such a policy, there was no direction to fully develop the new Climatological Ice Data Archiving System into a national registry, incorporating data from both government and private sources and including databases in danger of being lost to future researchers.
Conclusion38.89 The Ice Services Branch has taken action to address all of our observations and recommendations. However, some work remains to be done on the science and technology policy and on the formation and testing of a long-term contingency plan to ensure continuity of service should RADARSAT, now fully operational, fail at some point during its lifetime.
Observations38.90 In determining the progress of the Ice Services Branch since the 1994 audit, we reviewed two status reports prepared by Environment Canada, which provided information on action taken to address our observations and recommendations. The first report was prepared in August 1995 at the request of the Public Accounts Committee of Parliament, and the second was prepared in February 1996 at our request.
38.91 We reviewed documentation supporting statements made in those reports, together with other material prepared since their completion.
Science and technology policy development38.92 The Ice Services Branch still has no formal science and technology policy. Environment Canada indicated in its 1995 status report that a draft policy paper had been prepared and preliminary consultations with stakeholders had taken place. The Department was to finalize this document early in 1996 following further review by stakeholders.
38.93 At the time of our follow-up, the only document on file relating to a science and technology policy was a draft paper entitled "Strategic Directions for Science and Technology in Ice Services ' Environment Canada" (dated 26 April 1995 ). This paper discusses the role of Environment Canada in understanding ice and providing information on its physical properties and distribution. It stresses the need for partnerships and provides elements of an overall management strategy. This document is over 12 months old and, according to the Department's 1996 status report, work on the development of the science and technology policy has been suspended due to more pressing priorities.
38.94 At such a time of technological change - that is, the move to a satellite-based ice data collection system (RADARSAT) together with the Department's move to cost recovery for services and the development of revenue-generating projects - a science and technology policy would provide direction for departmental management in the development of business plans. In our view, the finalization of such a policy for the Ice Services Branch should remain a priority. It would supplement the Department's general statements in "Environment Canada's Science and Technology ' Leading to Solutions", which was published earlier this year. In that document, the Department recognizes that science and technology are "Canada's keys to the new century".
Conversion to RADARSAT38.95 RADARSAT was launched in November 1995, a few months later than originally predicted, and the Ice Services Branch began receiving images early in 1996. It formally converted to the primarily satellite-based system of ice data collection in April 1996.
38.96 In anticipation of the conversion, the Ice Services Branch developed a policy and a strategic approach to both the recovery of costs for many of its services and the commercialization of operations where feasible. Both depend on the continued provision of high-quality ice data by RADARSAT to the Ice Services Branch and its clients, at the present cost or a lower cost.
38.97 However, satellites can and do have problems, as evidenced by well-publicized failures in the last few years. Such problems are often costly to rectify and may result in loss of client confidence and in their consideration of alternative data delivery mechanisms. Thus the development and updating of short-term and long-term contingency plans and a regular testing program for back-up systems to be used in an emergency are important considerations for a service-oriented group like the Ice Services Branch.
38.98 We note that a short-term contingency plan was prepared and tested under contract. This covered emergencies that might have occurred during the launch of RADARSAT or immediately thereafter. Thus, adequate precautions were taken for this sensitive period.
38.99 However, a long-term contingency plan being prepared by the same contractor has not yet been finalized and tested. This plan will require third-party agreements that document administrative arrangements to supply ice data directly, to make available suitably equipped aircraft to collect data, or to provide satellite time for the use of the Ice Services Branch as needed. The completion date for the long-term contingency plan is late 1996.
38.100 In view of the difficulties experienced with satellites in the past several years, the absence of such a plan for a six-month period following start-up presents an element of risk. The plan should have been ready and tested before RADARSAT was launched, or at least by the end of its commissioning in late March 1996. It is important, therefore, that the Ice Services Branch proceed as quickly as possible to rectify the situation.
Indian and Northern Affairs Canada - Social Assistance - 1994, Chapter 23Assistant Auditor General: Don Young
Responsible Auditor: Grant Wilson
Background38.101 In 1994 we examined, on a selective basis, the administration by Indian and Northern Affairs Canada of its social assistance to First Nations.
38.102 We recommended that improvements be made in several areas, including the delivery of services, use of supportive legislation, initiatives to decrease dependency on social assistance, involvement of First Nations in social reforms, development of data for effective management, monitoring for compliance with social assistance requirements, and accountability.
38.103 The 1996 follow-up work included assessing the Department's reports on its progress in implementing remedial action and reviewing recent (June 1996) internal audit reports on social assistance. We did not audit the remedial action reported by the Department, nor did we attempt to identify any new issues. Those steps will be considered in our future audit plans.
Conclusion38.104 Although the Department has reported important progress in addressing our recommendations, more work is necessary to implement them fully.
38.105 We believe that the Department needs to intensify its efforts to improve accountability for its social programs and continue its development of data for effective management. In addition, monitoring for compliance with social assistance requirements needs to be strengthened.
38.106 Improvements are needed in these areas to help ensure that maximum benefits will be obtained for the expected substantial social assistance expenditures in the future.
Observations38.107 In its July 1996 progress report, the Department indicated that three of the seven 1994 audit recommendations had been fully implemented. These related to the involvement of First Nations in social reform and its impact on them, the consistency of funding and delivery of services within regions, and the review and finalization of social program arrangements with certain provinces.
38.108 The remaining four recommendations were reported as partially implemented, with work continuing where applicable. These areas include co-ordination of efforts to reduce dependency, the use of legislative authority, accountability and monitoring, and development of data for management.
38.109 In addition, the Department reported the results of two internal audits of social assistance in June 1996. They covered the social assistance database and methodology for compliance.
38.110 With respect to the database, which is needed for accountability and other important purposes, the Department has reported progress under its action plan to improve the accuracy and completeness of the information it collects and uses. The improvement initiative is expected to be completed by December 1996, although full improvements to the database may take additional time.
38.111 With respect to methodology for compliance, the Department's internal assessment was that the departmental compliance, monitoring and accountability framework was confused and weak. It did not adequately ensure that social assistance funds expended by First Nations were being used for their intended purpose as appropriated by Parliament.
38.112 Although the internal audit report on compliance methodology was released in June 1996, the Department's conclusions were based on reviews that it conducted in February and March 1995. We were advised that considerable improvements in monitoring had been made since then. However, the Department has yet to assess the results of such improvements. The Department's action plan for changes to compliance methodology calls for their completion by April 1997.
Department's comment: As described in this chapter, substantial progress has been made in addressing this very complex area; we have established a detailed work plan to address the outstanding issues, and are monitoring its progress closely.