This Web page has been archived on the Web.

2000 December Report of the Auditor General of Canada

December 2000 Report—Chapter 24

Insert 24.1—Implementing Health and Safety Regulatory Programs

1. We have used the following major elements of the government's regulatory policy to organize information on the development and implementation of health and safety regulatory programs:

• identification and management of risks to health and safety;
• consultation with citizens on regulatory approaches and with stakeholders on proposed regulatory initiatives and the manner of their implementation;
• adherence to the regulatory process management standards for making regulations;
• management of human resources;
• recovery of costs of services provided to industry that benefit private interests;
• compliance and enforcement of regulations; and
• timely and complete reporting on program effectiveness to Parliament.

Risk Identification and Management

2. Background. The federal regulatory policy requires that regulatory authorities demonstrate that a problem or risk exists and that federal government intervention is justified. Sound procedures to identify and manage risks to human health and safety are needed to implement this policy requirement. Such procedures allow regulatory authorities to ensure that their programs focus on priorities, that government intervention is required and that funds and human resources are allocated to the best advantage.

3. Managing health and safety risks to Canadians is a complex task. While risks can be reduced, they cannot be entirely eliminated. It also is difficult to separate Canadian and international aspects of risks. In addition, the scientific assessment and public perception of risks may differ. The extent of resources allocated to reducing a specific risk is often heavily influenced by the public's tolerance of loss of life or injury.

4. As well, the extent to which risks can be reduced depends on factors such as technology, human and financial resources, availability and dissemination of information, choices made by individuals, genetic and socio-economic factors, competing policy priorities, economic competitiveness and trade agreements.

5. Risk management process. In October 1997 the Canadian Standards Association published a national standard for Canada, Risk Management: Guideline for Decision-Makers. This standard is similar to the processes in the Treasury Board's 1994 risk management policy. The Association's approach combines scientific assessments of risks with extensive stakeholder consultations and public communication.

6. Risk management, broadly defined, includes the following key elements:

• identifying risks, the risk management team and potential stakeholders;
• risk communication;
• assigning responsibility, authority and resources;
• conducting a preliminary analysis to define the scope of required decisions and risk scenarios;
• estimating the frequency and consequences of the risks;
• estimating the level of stakeholder acceptance of risks and the benefits and costs of containing the risks;
• identifying and assessing feasible risk control options, evaluating options for dealing with residual risks and assessing stakeholder acceptance of residual risks; and
• developing an implementation plan, evaluating the effectiveness of the risk management decision process and monitoring the effectiveness of the risk reduction program.

7. Patterns of findings. Our audits have regularly raised concerns about significant risk management in health and safety regulatory programs. In certain cases, they have identified incidents that posed threats to public safety, government personnel and the environment.

8. Our previous audits have found that regulatory authorities were having major difficulties identifying and managing health and safety risks. We found examples of inadequacies in the following:

• risk identification;
• risk assessment methodology;
• information on the extent of risks to health and safety; and
• information on potential liabilities.

9. Our current audits continue to find examples of deficiencies in risk identification and management:

• there is a lack of important information on the incidence of food-borne illness in humans and the prevalence of pathogens in the food supply;
• Health Canada has encountered a number of problems with the surveillance of biological drugs after they have been approved for sale, including the surveillance of adverse reactions and events;
• the Canadian Nuclear Safety Commission needs to strengthen its risk analysis and assessment of licensee performance; and
• the National Energy Board has not assessed the health and safety risks associated with making regulations for onshore pipelines less prescriptive.

10. However, there have been improvements. For example, the Canadian Food Inspection Agency has enhanced its surveillance activities for animal and plant health and is incorporating information gathered from these activities into its risk assessment process. Health Canada has adopted a proactive approach to identifying health and safety risks arising from biologics. Overall, Transport Canada has made satisfactory progress in applying systematic risk analysis for its air navigation regulations and enforcement activities.

11. Government's assessment. In March 2000 the Treasury Board Secretariat consolidated the results of the assessments of comptrollership practices by five pilot departments. The Secretariat's findings are consistent with our own. For example, it concluded the following:

• although risk management is understood intuitively, only a few departments have begun to establish a formal risk assessment framework;
• managers need better tools to assess risks, and there is often little training available; and
• frameworks for the delegation of authority exist, but there is a need to review them in light of greater decentralization.

Consultation and Co-ordination

12. Background. The government has increasingly recognized that it cannot solve problems alone. It has also recognized that wide-ranging consultation is needed to gain the co-operation of affected parties, to develop the most effective regulatory approach and to foster compliance and the achievement of objectives.

13. The 1986 federal regulatory policy stated that Canadians were to be consulted in the making of regulations. This requirement has been reiterated in all subsequent versions of the policy. According to the current policy, regulatory authorities must ensure that "Canadians are consulted and that they have an opportunity to participate in developing or modifying regulations and regulatory programs." This statement sets a clear expectation that Canadians will be consulted on regulatory programs, not just on specific regulations.

14. Patterns of findings . The 1993 report of the Sub-Committee on Regulations and Competitiveness of the Standing Committee on Finance identified consultation as one of the areas that present problems. The subcommittee recommended that there be greater stakeholder involvement in setting regulatory goals and determining the means of achieving them.

15. Overall, we have found that regulatory authorities are investing a lot of time and resources in trying to meet increased public, industry and intergovernmental demands for consultation. In some cases, we have reported that consultation has worked well. For example, in the Auditor General's 1997 Report, Chapter 27, Ozone Layer Protection: The Unfinished Journey, we indicated that stakeholders had expressed a high degree of satisfaction with Environment Canada's consultations.

16. In other instances, consultation could have been significantly improved. Our Commissioner's 1999 Report, Chapter 2, Sustainable Development Strategy Consultations, reviewed the consultation process undertaken by 28 departments and agencies to develop sustainable development strategies. Our review of consultation plans for preparing departmental sustainable development strategies revealed that the objectives of the consultation were generally clear. However, the Commissioner's Report raised the following concerns:

• there were major differences in the quality and comprehensiveness of consultation plans among departments;
• fewer than half of the departments had internal policies or guidelines for consultation;
• departments often did not co-ordinate their consultations with other departments;
• most departments did not consult stakeholders on the design of their sustainable development strategy;
• consultation with other departments was often too late for their comments to be realistically incorporated into any report, other than in a cosmetic fashion;
• few departments evaluated their consultation process; and
• participants received uneven feedback on how their comments were considered.

17. Our current audits continue to find improvements in the consultation process. For example:

• there was extensive consultation on legislative changes relating to the Canadian Nuclear Safety Commission;
• Health Canada has taken steps to identify stakeholders and consult them on proposed changes to regulatory frameworks for some biologics; and
• Transport Canada developed the Canadian Aviation Regulations through a structured, national consultation process.

The Development of Regulations

18. Specific regulations are often used to protect health and safety. The development of regulations is subject to the government's regulatory process management standards. We looked at whether regulatory authorities were following existing standards in our 1989 Report chapter on the federal regulatory process and in our 1993 Report chapters on the development of pulp and paper regulations, the firearms control regulations and parliamentary control over user fees. We did not review these matters in other previous audits or in our current audits.

19. Government self-assessments. In 1995 the Treasury Board Secretariat directed seven major regulatory organizations to implement by December 1996 mandatory quality assurance standards for the regulatory process. These standards were contained in the 1995 Treasury Board's publication, Regulatory Process Management Standards.

20. These organizations were also required to use the standards to determine their compliance with the government's regulatory policy, and report back to the President of the Treasury Board by 31 December 1999. The Board had received self-assessments from the organizations, including four from organizations that focus on health and safety regulatory programs: Health Canada, Transport Canada, the Canadian Food Inspection Agency, and Environment Canada.

21. The Privy Council Office has had the results of the self-assessments consolidated to assess current capabilities on a government-wide basis (see exhibit below). Overall, the consolidation identifies four areas of strength. The areas demonstrate good to advanced management capability. They include monitoring the regulatory environment, identifying potential problems, consulting and communicating with stakeholders, and ensuring interdepartmental co-ordination.

22. Thirteen areas were identified as "opportunities for further development," including better mechanisms for assessing the need for regulation, better capabilities in cost-benefit analysis, more clarity on the approach to resourcing regulatory programs and better performance measurement.

23. Six of the areas where there are opportunities for further development are rated as being in the "early stages of development" (see exhibit below). These areas include performance measurement and accountability, the ranking of problems and issues in order of priority. The remaining areas fall in between the "early stages of development" and "good management capability."

24. Regulatory impact analysis statement. In August 2000 the Privy Council Office completed a review of the usefulness of the regulatory impact analysis for decision making and the development of regulations. The government's regulatory policy requires regulatory authorities to publish information on the proposed regulation in impact analyses. The information is expected to describe the purpose of the proposed regulation, present the results of reviews of a range of regulatory and non-regulatory alternatives, summarize the results of consultations, indicate the benefits and costs of regulating and describe proposed compliance and enforcement policies and measures.

25. The review focusses on six regulations initiated by major regulatory authorities. On the whole, the review concludes that the requirements of the regulatory impact analyses have changed the regulatory process for the better. In particular, the review indicates that for major regulations, a large amount of information is being provided to the public and government decision makers, and consultation is extensive.

26. However, the review cautions that the descriptions of regulations or problems need to be clearer, assessments of regulatory and non-regulatory alternatives and costs and benefits can be significantly improved, and descriptions of compliance policies and approaches need to be more complete. The review also makes recommendations to improve the relevance and usefulness of regulatory impact analysis statements.

27. We are concerned that the quality of cost-benefit analyses or impact assessments still needs major improvement. In 1989 and 1993 we expressed concerns about the quality of these analyses and assessments. These procedures are intended to provide an objective assessment of whether the social and economic benefits of adopting a regulation outweigh the costs. Without this information it is difficult for the public and decision makers to have as reliable a basis as possible for understanding what the regulation will achieve and how much it will cost.

28. Report of the Public Policy Forum. In February 2000 the Public Policy Forum released a report, Managing Regulation: Policy, Practice and Prognosis. The report presents the results of a multistakeholder roundtable in the fall of 1999 on the federal government's regulatory process and its implementation. The stakeholders included 15 industry associations, five federal departments and agencies, two provincial governments and the Consumers' Association of Canada.

29. According to the report, there is a general consensus that the regulatory policy is sound, but a "disconnect appears to exist between the Policy and its implementation across government departments and agencies."

30. In particular, stakeholders at the roundtable stated the following:

• the regulatory development process often lacks fairness;
• the techniques of regulatory impact assessment, such as cost-benefit analysis, tend to be misused or manipulated to support a position;
• the process is too slow, procedural and rigid;
• there is still a tendency for the government to regulate, not to find alternative solutions;
• the widespread practice of regulation by negotiation and bargaining raises concerns over the consistency of laws and rules.

Human Resources

31. Background. The regulatory policy requires that regulatory authorities "when managing risks on behalf of Canadians, ...must ensure that limited resources available to [the] government are used where they do the most good." This requirement is difficult to achieve when risk assessment is weak.

32. The primary human resource challenge facing health and safety regulatory programs is obtaining sufficient funds to hire staff and determining the type and number of staff and their competencies. The programs also need to maintain a sufficient number of inspectors who are well trained in the most up-to-date methods and technologies and led by people with a broad understanding of the work they do. About 5,000 inspectors work in health and safety regulatory programs. Most of them work at Health Canada, the Canadian Food Inspection Agency, and Transport Canada.

33. Pattern of findings. Our previous audits have often found instances of insufficient resources to carry out responsibilities. For example, we found that there were insufficient resources to do the following:

• to oversee in the long term the radiation safety program and nuclear safety compliance;
• to deal with new issues in the national energy sector; and
• to apply and maintain specialized knowledge and technical expertise in deep geological disposal of radioactive waste.

34. Our current audit findings indicate that the Canadian Nuclear Safety Commission, the regulatory regime for biologics of Health Canada and the Canadian Food Inspection Agency are experiencing or will experience some major staffing problems.

35. The situation in Health Canada illustrates the problems and their consequences. Following the 1997 report of the Commission of Inquiry on the Blood System in Canada (Krever Commission), the government announced in 1998 that it would invest $125 million over the next five years to strengthen Health Canada's blood safety program, including regulatory and surveillance programs for related biologics. The government also gave the Department the authority to start hiring 84 full-time employees in 1998-99 and up to a total of 133 by 2002-03.

36. The Department indicates that it has yet to fill 30 of 94 positions allocated to the Therapeutic Products Programme in 1999-2000. Vacant positions were identified as either "staffing in progress" or "staffing to be initiated" and pertained to compliance and enforcement investigations, pre-market and post-market reviews, post-market surveillance, regulatory research and policy development. According to the Department, these shortfalls were the result of a number of factors, including lengthy staffing processes, the unavailability of qualified candidates, non-competitive salaries and the unwillingness of potential candidates to work in biologics in a post-Krever environment.

37. The staffing problems have a significant impact on the Department's ability to manage the workload of pre-market reviews of new biological products and conduct post-market assessments. The result is a significant backlog of new biological drug submissions, failure to meet established performance targets and incomplete post-market assessments. These problems are expected to worsen with the implementation of proposed changes to the regulatory framework for clinical trials. The present 60-day default period for approval will decrease to only 30 days for many clinical trial submissions. While authority was given to increase staff levels, Health Canada expects that it will have difficulty obtaining additional qualified resources in the immediate future. At the current level of staffing, it is estimated that this change will further increase the backlog of new biological drug submissions.

38. Other regulatory authorities are also experiencing difficulties. The Canadian Food Inspection Agency estimates that by 2006, 734 of its indeterminate employees will be eligible to retire, including about a third of inspectors and veterinarians. The Agency is already having some difficulty recruiting for some positions.

39. While the Canadian Nuclear Safety Commission has set priorities and developed a number of important human resource policies and practices, it also faces difficulties in hiring scientific and technical staff. Despite management's initiatives, the present vacancy rate is high, and positions are vacant for long periods. In addition, the employee population is aging and could suffer from a loss of leadership and experience.

40. Government's assessment. The government's assessment of human resource issues in regulatory organizations is consistent with ours. The government has identified major risks in retaining and recruiting inspectors. The Committee of Senior Officials has assessed the government's capacity to carry out its core regulatory and inspection functions. The committee's November 1999 report, A Public Trust: Keeping Canadians Safe and Healthy, identifies a number of key problems facing these functions. It reveals an aging population, many vacancies, few recruits and high attrition rates in some key areas. The data shows that these problems will become more serious in the next few years.

41. The committee's report notes the following:

• The workload of inspectors has increased dramatically, and the cost of training inspectors is high. For example, Transport Canada can spend up to $150,000 and take up to two years to fully train an inspector.
• The government is competing for scarce human resources in a highly competitive market. Key specialists in some diseases command salaries of over $200,000 in private practices. In areas such as aviation and marine engineering, the wage gap between the public and private sectors now exceeds 40 percent. It is unlikely that the government will be able to bridge gaps of this magnitude.
• Career progression is done through promotions to management, but this path is not always the best one for scarce expertise.
• There is concern for the status of personal liability of inspectors and the damage that their decisions could cause. Uncertainty has made some inspectors adopt an extremely conservative approach to risk management.

42. The report notes that the regulatory and inspection community is much older than the public service as a whole and that it has a high departure rate due to high retirement eligibility. As well, based on historical trends, significant non-retirement departures can be expected. Recruits are almost entirely from outside the public service, which maximizes competition with industry. Due to unique accreditation requirements, there is also low mobility between regulatory programs.

43. The report notes that by 2008, about 40 percent of the regulatory and inspection community will be eligible to retire, compared with 33 percent for the public service as a whole. However, it points out that certain inspection groups have a higher rate of projected retirement eligibility. For example, the rate for the group that deals with the food standards regulation is about 48 percent.

44. The report also notes that the role of inspectors has changed significantly. Traditionally, government inspectors examined processes and products, assessed their safety and took required action or designed the rules that must be followed to reduce risks to acceptable levels. Now they educate industry, encourage change, monitor activities and enforce regulations. This current role aims to build expertise in industry and to increase the probability that industry will operate within regulations.

45. The report points out that the new educational and monitoring functions are far more complex for the inspector. The inspection community has found it difficult to interpret the meaning in their operational context of the client service vocabulary, which has been prevalent in the government. Competencies now required of inspectors also include a range of soft skills to teach, monitor activities, encourage changes in attitude and develop new competencies in the industry client.

Cost Recovery

46. Background. Health and safety regulatory programs recover a portion of their operating costs from regulated industries. Information indicating the proportion of program funding that is based on cost recovery is not readily available. We estimate that health and safety regulatory programs collected about $154.6 million in 1997-98. Depending on the legislation, these funds can be re-spent by regulatory authorities, or they become part of government general revenues.

47. In 1999-2000, the Canadian Food Inspection Agency collected about $54 million (or 13 percent) of its total expenditures of about $416 million. It is authorized to re-spend these funds. Health Canada's Therapeutic Products Programme, which regulates drugs and medical devices, recovered about $40 million (or 63.5 percent) of its total expenditures of $63 million. All of these funds are re-spendable by the department.

48. In 1997-98, the National Energy Board recovered about $24.7 million (or 88 percent) of its total expenditures of some $28 million. These funds became part of the government's general revenues. The Atomic Energy Control Board, now the Canadian Nuclear Safety Commission, collected $30.8 million (or about 70 percent) of its total expenditures of $43.8 million. These funds also became part of the government's general revenues.

49. The federal government states that it will only initiate cost recovery for an activity that is legitimate and necessary, that is not available in the private and voluntary sectors, and that offers identifiable recipients direct benefits beyond those received by the general public. In 1994 the government made cost recovery a priority for departments and agencies. It undertook this initiative, while cutting spending to eliminate the deficit. Fees were introduced for many services previously offered at no cost.

50. Ministers are responsible for implementing and amending user charges in their area of responsibilities. By linking service costs to the fees it charged, the government expected that industry could scrutinize fees to ensure that they were fair and that the services were being delivered efficiently. The President of the Treasury Board is the point of contact for clients who feel that departments have not given them a fair hearing in the fee-setting process.

51. The objectives of the cost-recovery policy are the following:

• to promote the efficient allocation of resources;
• to promote an equitable approach to financing government programs, mandatory or otherwise, by fairly charging clients or beneficiaries who benefit from services beyond those enjoyed by the general public; and
• to earn a fair return for the Canadian public for access to, or exploitation of, publicly owned or controlled resources.

52. The policy requires that departments and agencies do the following:

• undertake meaningful consultations with clients throughout the fee-setting process, including the conduct of impact assessments and establishing a dispute-resolution process;
• follow appropriate costing and pricing practices;
• treat all user charge revenues as public funds; and
• spend these revenues only with the prior approval of Parliament and the Treasury Board.

53. Pattern of findings. We have reported to Parliament a range of major concerns about the implementation of the cost recovery policy. We reviewed the implementation of user fees in our 1993 Report, Chapter 25, Parliamentary Control Over the Raising of Revenues by Fees. More recently, we comprehensively reviewed the management of user fees by Agriculture and Agri-Food Canada, the Canadian Grain Commission and the Canadian Food Inspection Agency. The findings in our September 1999 Report, Chapter 11, Agriculture Portfolio - User Charges, illustrate our concerns.

54. The concerns that we reported included instances of the following:

• inadequate legislative frameworks;
• inconsistent cost identification and allocation methods;
• a lack of specific and enforceable guidelines to price services;
• unclear definitions of private and public benefits;
• a lack of co-ordination by central federal agencies;
• a lack of consolidated information for planning and reporting;
• a lack of information on user charges to concerned parties, including fee revenues not published in the annual budget;
• inadequate impact assessments;
• a lack of an open, clear and independent appeal process for those affected by user charges; and
• a need for broader consultation on service charges to take into account the interests of the public and of those who pay fees.

55. As part of the current audit of Health Canada's regulatory regime for biologics, we examined cost recovery for reviewing biological drugs. Since 1994-95 the Department partially recovers from the manufacturer the costs of reviewing biological drugs for market approval. The drug industry expected that cost recovery would help the Department meet established performance targets of shorter duration. However, we did not find clear objectives linking cost recovery and shorter approval times. For new biological drugs approved in 1999, the Department took an average period of 328 days (compared with a performance target of 180 days) to review priority-status submissions and 545 days (compared with a performance target of 300 days) to review non-priority status submissions.

56. In its recent evaluation of the cost-recovery initiative, the Department found that it is difficult to separate public and private benefits of cost recovery for biological drugs.

57. Report of the Standing Committee on Finance. In June 2000 the House of Commons' Standing Committee on Finance published its review of cost-recovery in government, Challenge for Change: A Study of Cost-Recovery. The committee examined whether the Treasury Board's cost-recovery and charging policy is being implemented consistently in the government and whether the Board's policy is sound. The concerns of the committee are similar to ours.

58. The committee found that the policy generally conforms to sound economic practices. However, government departments are not implementing the policy consistently, and central supervision and dispute resolution is ineffective or absent. The committee raised concerns that user fees are perceived as a tax, rather than prices for services, and that departments do not seem to be following the guidelines originally set out in the cost-recovery program.

59. The committee also reported that some groups are concerned about the potential threat to the integrity of regulatory programs because of the reliance of regulatory authorities on cost recovery for funding.

60. The committee concluded that a government-wide assessment of the cost and benefits of user fees is needed. Treasury Board Secretariat officials told the committee that a review of the cost recovery policy is under way and should be completed by the winter of 2001.

Compliance and Enforcement

61. Background. Regulations set legal process or requirements for industries involved in activities that affect health and safety. To be in compliance, a company must meet these requirements. To ensure compliance, regulatory programs need to ensure that those who must comply with the laws understand what is expected of them and that the laws are enforced in a fair, predictable and consistent way. Regulations have to contain provisions that are consistent, understandable, measurable and enforceable.

62. The complexity of enforcement depends on the nature of a regulation. The age of a law or regulation often determines the quantity and quality of compliance information that is available. Some regulations cover an ongoing activity in a specific area, such as the production of pulp and paper. Other regulations may refer to more transient activities that are more difficult to monitor, like the transport and disposal of hazardous waste. Lastly, regulations can impose complex technical requirements; the procedures to verify compliance with these requirements can also be complicated.

63. Compliance and enforcement also are areas significantly affected by technological changes, pressures from economic competitiveness, insufficient human resources, public scrutiny and reliance on collaborative arrangements with the provinces and territories.

64. Inspection programs verify compliance with the laws and their regulations. A plan identifies the type and frequency of inspections and monitoring activities that have to be conducted. The plan usually considers priorities, historical problems, operational factors and regional variability. Inspections can also take place in response to specific incidents.

65. Departmental policies or procedures list a number of actions that can be taken to remedy situations of non-compliance. These actions range from verbal and written warnings to criminal prosecutions. According to the report of the Committee of Senior Officials, A Public Trust, regulatory authorities are increasingly focussing on achieving compliance by seeking conformity with the law through educational programs, the encouragement of technology transfers and self-evaluation.

66. Pattern of findings. Our previous audits revealed recurring weaknesses in compliance and enforcement, including the following:

• unclear objectives for compliance with legal and policy requirements;
• inadequate identification of the regulated community, an ineffective process for determining the companies to inspect, and inconsistent inspection coverage of the regulated community;
• a lack of using risk assessment to focus inspection activities;
• inadequate information on the incidence of non-compliance and related consequences;
• inadequate implementation of audit and inspection programs with results fed into performance measures;
• insufficient random inspections to detect and prevent incidents of non-compliance;
• over-reliance on voluntary compliance with limited verification to demonstrate that this reliance is justified; and
• an absence of national reviews to ensure that different regions use consistent procedures.

67. Our current audits continue to find deficiencies in ensuring compliance with health and safety regulatory requirements. For example, the Canadian Nuclear Safety Commission's compliance and enforcement framework is incomplete. It needs to promptly develop outstanding regulatory documents and implement its compliance and enforcement policy.

68. For problems of non-compliance, the Canadian Food Inspection Agency's policy requires that inspectors take action to ensure compliance in the shortest time frame and to prevent the problems from recurring. Our review of a sample of cases noted that the compliance actions were not sufficient to achieve the Agency's goal because of limitations in legislation or a failure by the inspector to take more serious compliance action.

69. Our audit of Health Canada's regulatory regime for biologics contains findings about establishments that process and distribute semen for assisted conception. These findings illustrate the risks posed by incomplete inspection approaches. In June 1996 Health Canada implemented regulations governing these establishments. However, it started to monitor and inspect them only in March 1999. Although an accreditation process and inspection regime was intended to be part of the 1996 regulatory framework, these activities did not occur. There were no plans established before 1999 for the inspection of semen processing facilities, nor were there any standard operating procedures for inspection.

70. In March 1999 problems identified by an Ontario human semen bank initiated a national investigative inspection by Health Canada of all known semen establishments that following summer. The Department found that 43 of the 51 establishments under investigation did not fully comply with the regulations. Of these 43, 17 did not perform the required tests for specific infectious diseases and more than half did not maintain sufficient records for Health Canada to determine that they had done the required tests. As a result, the Department detained much of the semen supply for assisted conception in Canada, which created considerable anxiety among recipients.

71. Although problems continue to exist in health and safety regulatory programs, there have been improvements. In April 1999 Health Canada established a program to inspect all known semen establishments, and it began to formulate and test standard operating procedures for investigative inspections. In addition, the Department has given formal training to compliance officers to ensure that they have the necessary knowledge and skills to do their work. At the Canadian Food Inspection Agency, four of the five alternative service delivery arrangements for animal and plant health have devoted considerable attention to monitoring, controlling and auditing compliance. Transport Canada's Air Navigation Services and Airspace Branch has completed over 150 audits and inspections to date and identified 130 findings and observations. The Branch's enforcement actions have included counselling, monetary penalties and license suspensions.

Reporting on Effectiveness

72. Background. The Treasury Board requires departments to prepare performance reports in the fall of every year. The reports, as part of the government's Main Estimates, are tabled in the House of Commons and referred to the appropriate standing committee. They are intended to provide information on achieved results. Some federal entities have legislated reporting requirements, such as the Canadian Food Inspection Agency which is required to report performance in its annual report.

73. Pattern of findings. Our previous audits of reporting on the effectiveness of health and safety regulatory programs identified serious weaknesses, such as the following:

• a lack of specific requirements for reporting;
• an inconsistency and incompleteness of data;
• an absence of common protocols to measure and estimate incidents to make results comparable;
• an insufficient verification of actual results; and
• a lack of performance indicators to support statements of achieved results.

74. Our audits also identified instances where Parliament was not adequately informed:

• On the results of national efforts on smog reduction, Parliament did not receive meaningful, comprehensive and timely information about action on the promises made to Canadians in 1990.
• On the results of agreements under the Canadian Environmental Protection Act, Parliament received incomplete and outdated information.
• On the achievement of program objectives for the disposal of hazardous waste, Parliament did not receive information on actual spending, achieved or likely achievable results and constraints.
• On federal food safety activities, Parliament did not receive a description of the objectives of the federal food safety system, the roles of involved departments, and the results achieved;
• On the animal and plant health inspection program, Parliament received inadequate information to understand and assess the program's performance and response to serious outbreaks of diseases and pests.
• On nuclear power plants and licenses for prescribed substances and radioisotopes, Parliament received limited information on licensing actions and the number of compliance inspections.

75. Our current audit findings continue to find deficiencies in reporting on effectiveness. For example, The Canadian Food Inspection Agency Act requires the Agency to prepare a corporate business plan and an annual report to Parliament. The business plan must specify objectives and expected performance against these objectives. The 1997-2000 plan contains only limited information on milestones, time frames, expected level of effort, and measures of goal achievement. The information provided is not adequate because it does not allow Parliament and the public to determine how well the Agency is performing.

76. The Canadian Nuclear Safety Commission, like other nuclear regulators, has had difficulty in reporting performance. While its annual report contains many measures, the Commission needs to develop meaningful performance expectations and report measures within a clear context and strategy.

77. Transport Canada has made progress in reporting on effectiveness by publishing annual reports of the Air Navigation Services and Airspace Branch. However, significant deficiencies exist in ensuring the quality of the safety data. In 1997 we indicated that Transport Canada should conduct formal reviews of the quality of the air safety data that it receives from NAV CANADA. The Department has still not conducted or planned any formal review of such data.

78. The incident rate for losses of separation (the spacing required between aircraft) is a primary indicator of safety performance for an air navigation service provider. To be useful for both management and accountability purposes, it is critical that this data be timely and correct. The Department has not yet reconciled the reports of the Civil Aviation Daily Occurrence Reporting System (CADORS) with data from the incident-reporting system operated by the Transportation Safety Board of Canada. Further, the Department's audit regime for NAV CANADA does not include tests of the completeness and accuracy of the CADORS reports by reference to primary sources (radar and voice tapes). An other jurisdiction that has carried out these tests has found that self-reporting systems tend to understate the occurrence of incidents.