1.1 The information that most departments provided in their second annual progress reports on sustainable development strategies continued to fall well short of the information specified in the Treasury Board's Guideline for the Preparation of Departmental Performance Reports to Parliament. Thus, parliamentarians, Canadians and other stakeholders will find it difficult to judge whether the strategies are on track or whether corrective action is required. We expect that the quality of reporting will improve as departments adopt a more systematic approach to managing strategy implementation.

1.2 Overall, strategy implementation seems to be progressing. Based on our assessment of the information reported by departments in 1999, they met about 20 percent of the commitments set out in the sustainable development strategies, compared with 11 percent in 1998. Departments are also working on their management practices for implementing the strategies. Last year we reported that, on average, departments had established about one third of the management practices reflected in the ISO 14001 management standard, which is a benchmark of good practice. This year, on average, the six departments we examined were applying about half of those practices.

1.3 However, in four of the six departments, the management practices do not yet provide reasonable assurance that their strategies will be implemented consistently and achieve the intended results. We continue to believe that departments should establish and apply a management systems approach to support the implementation of their strategies.

Background and other observations

1.4 In 1997, 28 federal government departments and agencies tabled their first sustainable development strategies in the House of Commons. Since then, they have been working to implement their strategies. Departments are required to report annually to the House on their progress. This chapter provides the Commissioner's second annual assessment of that progress.

1.5 In his 1998 report, the Commissioner recommended that departments establish clear and measurable targets that they, parliamentarians and the public could use to judge whether or not the strategies are being implemented successfully. This year we reviewed the departments' revised targets to determine whether they had clearly stated the criterion or measure of success for each target and indicated an expected completion date. About 50 percent of the revised targets include a clearly stated criterion or measure of success and an expected completion date. Only 45 percent of departments included targets and performance indicators in their performance reports.

1.6 Last year we examined the management practices that six departments were following to implement their sustainable development strategies. As a benchmark of good practice, we used the ISO 14001 standard for environmental management systems. This year, using the same benchmark, we assessed the management practices of another six departments. In four of them we found significant gaps between their practices and the ISO standard. These four departments are still in the early stages of establishing a systematic approach to strategy implementation. They have not yet adopted a systematic approach to identifying their priorities, defining management expectations, assigning accountability for results at lower levels in the organization and identifying related training needs, or performing the self-assessments that would facilitate steady improvement.

Introduction

1.7 The position of Commissioner of the Environment and Sustainable Development was created by amendments to the Auditor General Act in 1995. The Commissioner's principal duty under the Act is to monitor and report annually to Parliament on the extent to which departments have implemented their action plans for sustainable development and met their objectives.

1.8 In 1997, 28 federal departments had their first sustainable development strategies tabled in the House of Commons. The objective of the strategies was to put the concept of sustainable development into practice by articulating what each major department across the federal government needed to do.

1.9 The strategies contained each department's action plan, including the objectives and targets that the department and others would use as benchmarks for measuring progress. To ensure accountability for results, ministers were directed to report in their annual departmental performance reports to Parliament on progress toward their sustainable development commitments.

1.10 In October 1999, ministers tabled their second annual sustainable development strategy progress reports, based on progress to 31 March. The purpose of these reports is to inform parliamentarians whether the departmental strategy is on track and, if not, what is being done to get it back on track.

1.11 This is the Commissioner's second report on the progress being made by departments. Our objectives are to give parliamentarians information they need to oversee departments' progress in implementing their sustainable development strategies and to help departments understand their management obligations and good practices for meeting them.

Focus of the audit

1.12 In conducting the audit, we set out to answer two main questions: "Are federal departments doing what they said they would do in their strategies?" and "Have departments established the capacity to implement their strategies?"

1.13 To determine whether departments are doing what they said they would do, we compared the goals, objectives, targets and actions set out by each department in its 1997 strategy with the performance information each presented in its October 1999 progress report. We accepted at face value the departments' assertions about the progress they had made.

1.14 We reviewed the strategies and the departmental performance reports of 28 departments (see Exhibit 1.1). We requested copies of any additional, more detailed sustainable development progress reports that departments had prepared, and we also reviewed these.

1.15 Last year, to examine departments' capacity to implement their sustainable development strategies, we compared six departments' related management practices with recognized standards of good management practice. This year we chose another six departments and repeated the process.

1.16 The departments we selected represent a cross-section of policy, program and operational mandates: Solicitor General Canada, Economic Development Agency of Canada for Quebec Regions, Department of Fisheries and Oceans, Canadian Heritage, Canada Customs and Revenue Agency and Western Economic Diversification Canada.

1.17 We believe it is results that matter to parliamentarians and Canadians, not the systems or procedures that produce them. However, the Commissioner has regularly observed that there is a persistent gap between the commitments made by departments and the results they achieve. He has referred to this as "the implementation gap". Our current emphasis on good management practices is founded on the premise that they go hand in hand with good results. Our approach is intended to reinforce good management practices, which we believe will contribute to achieving expected results over the long term. We expect that the implementation gap will begin to close as federal departments apply a management systems approach to implementing their strategies and reporting results.

1.18 The criteria we used to assess the management capacity of the six departments reflect the principles set out by the International Organization for Standardization (ISO) in its environmental management system standard 14001. The ISO 14001 standard is a widely accepted benchmark of good management practice and due diligence. The standard is consistent with the Treasury Board Secretariat's principles of good management for planning, reporting and accountability structures.

1.19 Additional details on the audit can be found in About the Audit at the end of this chapter.

Observations and Recommendations

What Ministers and Their Departments Were Asked to Do

1.20 The annual departmental performance reports are the key means of documenting and communicating the progress of federal departments in implementing their sustainable development strategies. The process of preparing them informs management where the department stands in relation to its objectives and identifies opportunities for improvement. The reports are thus the key mechanism for keeping the sustainable development strategies on track. They are also an important tool for the Commissioner of the Environment and Sustainable Development and others interested in monitoring the progress of the 28 departments.

1.21 To help departments prepare their performance reports, the Treasury Board Secretariat published its Guideline for the Preparation of Departmental Performance Reports to Parliament (see Exhibit 1.2). The Guideline clearly specifies the sustainable development performance information that departments should include in their reports for parliamentarians and others interested in monitoring the government's progress toward sustainable development.

1.22 Specifically, the annual progress reports are expected to contain five pieces of information: the key commitments set out by departments in their strategies; the indicators or measures that departments and others can use to gauge progress toward those commitments; the targets the departments expected to achieve during the reporting period; a summary of accomplishments related to each target; and corrective actions being taken to ensure that commitments are met. The summary information provided in the departmental performance reports is also to be cross-referenced to other documents to allow access to more detailed information.

1.23 In his 1998 Report, the Commissioner concluded that almost all departments had failed to establish the clear and measurable targets that are key to the success of the sustainable development strategy process. To rectify this, he asked departments to review their strategies and to establish a clear set of targets that they, parliamentarians and the public could use to judge whether or not the strategies are being implemented successfully.

What Departments Did

1.24 All departments that tabled a sustainable development strategy in 1997 also tabled a sustainable development progress report as a subsection of their October 1999 departmental performance reports. We assessed the extent to which the reports provided the information requested by the Treasury Board Secretariat in its Guideline. We found that the extent to which departments followed the Guideline varied, and so did the quality of the information in the reports (see Exhibit 1.3).

1.25 We also reviewed the revised targets that departments presented in response to the Commissioner's 1998 recommendation. Performance targets provide a point of reference against which progress can be measured. They need to be specific about the expected results, including measures or indicators of success as well as completion dates.

1.26 This year about 45 percent of departments included targets and performance indicators in their progress reports, compared with 35 percent last year. About half of the targets we reviewed had a completion date and a clear measure or indicator of success.

1.27 Departments frequently described the activities they had undertaken but generally did not indicate whether they had met the targets. Very few departments mentioned difficulties they had encountered in implementing their strategies, shortcomings, or corrective actions they were undertaking to keep the strategies on track. As a result, readers are often left wondering whether or not a department has done what it said it would do and whether or not it is moving forward with its strategy.

1.28 In many cases, the targets we reviewed describe ongoing or routine practices. For example, one department's strategy includes a target to "develop and implement policies and programs that consider broad horizontal policy issues and are consistent with Canada's social, economic and environmental well-being." In its progress report, the department stated about this item that a "legislative review exercise supported by new research . . . is allowing the department to look closely at the social and economic consequences of current and proposed new policy directions."

1.29 In this case, the department has stated both its expected results and its reported accomplishments in terms too vague to allow for an objective assessment of progress. The department indicates in its performance information that its legislative review is allowing it to "look closely" at the issues. However, it does not say whether it has actually met its target to "develop and implement" sustainable development policies or programs. The summary progress report for this department was not cross-referenced to other documents to allow access to more detailed information. Indeed, 23 of the 28 departments did not cross-reference additional information, although this is specified in the Treasury Board Guideline.

1.30 Several of the more detailed progress reports that departments provided in response to our request referred to key objectives and summarized related activities the departments had undertaken during the reporting period. The best of these reports clearly indicated each action item as either completed or in progress, thereby giving readers a better sense of the extent to which the department had implemented the actions set out in its strategy.

1.31 As was the case last year, information on the status of each activity made it easier to monitor progress on the action plans and conveyed a greater sense of assurance about the department's capacity to manage activities and track progress. Exhibit 1.4 presents the reporting format used by Natural Resources Canada. In contrast to most of the reports, it clearly indicates the extent to which the department has met its targets.

1.32 Departments, however, have continued to use terms like "encourage," "enhance," "improve," "promote," "assist," and "facilitate," without defining parameters for action, such as measurable benchmarks or expected completion dates. Departments have generally not specified the results they expect to achieve or when they expect to achieve them. They have still not been clear enough about how we will know when they have succeeded.

1.33 We recognize that it will take time for departments to establish the management practices they will need to implement their strategies. However, establishing clear performance expectations and adopting a straightforward, logical reporting framework are essential steps toward producing useful performance information for Parliament. The progress reports tabled in the House of Commons need to provide enough information for members of Parliament to judge whether action plans for sustainable development have been fully implemented and the objectives achieved.

1.34 Overall, the performance information in the 1999 progress reports continues to fall short of expectations. It thus limits the ability of parliamentarians and others to determine whether or not the strategies are on track and what corrective actions departments are taking.

Did Departments Do What They Said They Would Do?

1.35 The 1997 sustainable development strategies were intended to cover the three-year period ending in December 2000. In their second annual progress reports to Parliament, departments reported having accomplished on average about 20 percent of what their strategies said they would do (see Exhibit 1.5).

1.36 Most departments referred to some of the objectives of their sustainable development strategies (Exhibit 1.3) and reported activities they had undertaken for sustainable development during the reporting period. However, since only half of the departments included targets in their progress reports and only about half the performance targets are clear and measurable, there is little objective basis for judging progress.

1.37 Like last year, departments generally reported the activities they had undertaken. The large number of activities they reported often caused the reader to lose sight of the departments' sustainable development objectives. Departments generally did not indicate whether or not they had achieved their objectives, and in many cases the links between actions and objectives were too abstract to allow for an assessment of progress.

1.38 Some departments continued to report on activities that were not mentioned in their strategies. Ten departments provided no information on progress toward their strategic objectives. For these departments, there was no information available to assess the extent to which they had implemented their action plans for sustainable development or achieved their objectives. This represents a significant gap. The sustainable development commitments of these departments represent more than 25 percent of the total number of action items set out by the 28 departments.

1.39 For most departments, beyond tallying the percentage of activities they reported having accomplished, we were unable to conclude from their progress reports whether the strategies are on track or whether the departments are progressing toward sustainable development. Until departments report against clear targets using a logical, concise format such as the one specified in the Treasury Board Guideline, those interested in monitoring progress will lack the information they need to do so.

Have Departments Established the Capacity to Implement Their Strategies?

1.40 To gain a better perspective on departments' capability to carry out their strategies, we looked at the management practices the six departments are using to implement their action plans. We compared those management practices with the requirements of the ISO 14001 environmental management systems standard.

1.41 The ISO 14001 standard is the product of broad stakeholder consultation and consensus on the elements of good management practice. It has received unanimous approval from the standards bodies of 67 countries, including the Standards Council of Canada. It is considered to be consistent with sustainable development and compatible with diverse cultural, social and organizational frameworks.

1.42 All six departments we chose for this component of our monitoring work had indicated in their 1997 strategies that they were developing management systems to address their environmental issues. Two of the six indicated that their management systems would conform to ISO 14001.

A well-functioning management system is a strong indicator that intended results will be accomplished

1.43 A good management system is a cyclical process that links an organization's objectives, action plans and results. Exhibit 1.6 illustrates this management cycle. The purpose of a management system is to provide an organization with reasonable assurance that its work is conducted in accordance with applicable regulatory requirements, professional standards and the organization's own policies and procedures.

1.44 A well-designed management system provides a structured process for achieving continual improvement. It is a strong indicator that intended results will be accomplished. As a result, ISO 14001 is becoming widely established as the standard of due diligence for managing environmental issues and sustainable development.

1.45 Where senior management is committed to achieving results, such a system will enhance an organization's capacity to anticipate key issues and to achieve its performance objectives. A well-functioning management system helps to "operationalize" a sustainable development strategy, provide reasonable assurance that the action plan will be implemented consistently and reliably, and improve confidence that intended results will be achieved.

1.46 Exhibit 1.7 summarizes the type of practices and procedures one sees where environmental issues and sustainable development are being managed effectively. Documenting practices and procedures is not always necessary for effective management, but it does serve a number of important purposes. It helps to ensure that the organization's policies are implemented consistently over time. It helps to reduce the risk of, for example, the loss of corporate memory that can result from staff turnover. Documented policies, practices and procedures can also prove helpful in demonstrating that an organization has exercised due diligence in addressing an issue or preventing a problem - a key test in determining legal liability.

1.47 In 1994, the government directed departments to establish environmental management systems and to emulate the best practices followed in other sectors. Departments were further directed to make their management systems and operational practices more consistent with sustainable development.

1.48 Accordingly, in fulfilling his annual monitoring duties the Commissioner relies on recognized standards of practice for environmental management and sustainable development. This is much the same as financial auditors' reliance on accepted principles and criteria of control as part of monitoring and providing assurance in their audit work.

Our More In-Depth Look Revealed Progress but Also Gaps in Capacity

1.49 The Economic Development Agency of Canada for Quebec Regions has a highly developed management system for strategy implementation. Its general management system conforms to the ISO 9000 quality management system standard, and it has adapted that system to manage the implementation of its sustainable development strategy.

1.50 The Agency's current practices satisfy almost 90 percent of the requirements of the ISO 14001 environmental management systems standard. This supports our contention that good general management practices, like those of the ISO 9000 standard, are generic and transferable. The Appendix to this chapter summarizes how the Agency applied its existing management system to implementing its strategy.

1.51 Solicitor General Canada provides policy and administrative support to several arm's-length agencies that fall within the Minister's portfolio. However, the programs and activities of those agencies, many of which have the potential to affect the environment and sustainable development, are outside the scope of the Department's sustainable development strategy. The Department relies on the agencies to address these issues themselves.

1.52 Given those limitations, we found that the Department's planning practices are thorough and it is implementing its strategy through its existing Planning, Reporting and Accountability Structure. We concluded that while it could improve the management system, particularly documentation, its practices for implementing its strategy show no significant deficiencies.

1.53 It is noteworthy that two of the Department's sub-agencies tabled their own sustainable development strategies in 1997, recognizing their significant potential impact on the environment and sustainable development. We plan to audit the management systems of those agencies next year.

1.54 The four other departments we looked at in depth are working to establish processes and procedures to implement their strategies. Their current management and control practices are not yet developed enough to provide reasonable assurance that their action plans will be achieved. Exhibit 1.8 presents the results of our comparison of current management practices in the six departments with the major elements of ISO 14001. Results for each department we looked at this year are shown in Exhibit 1.9.

1.55 Last year we reported that departmental practices were most developed at the planning stage of the management cycle. They became weaker as the departments moved from planning to implementation, and weakest at the checking/corrective action and management review stages. In four of the six departments discussed in this chapter, our observations are consistent with those of last year (see the Commissioner's 1999 Report, Chapter 1, paragraphs 1.51 to 1.67). With the exception of the Economic Development Agency of Canada for Quebec Regions and Solicitor General Canada, we found that:

departments have yet to itemize or prioritize regulatory and other potentially applicable environmental and sustainable development obligations;
departments have yet to establish and apply systematic practices, procedures and work instructions for strategy implementation, monitoring and control;
departments have yet to assess their training requirements;
most departments have yet to perform an internal audit of their management practices for environmental issues and sustainable development;
top management has generally not reviewed the adequacy of the departments' management practices for strategy implementation;
performance targets and performance reporting are non-existent or vague, and consequently departments (and independent stakeholders) lack the information necessary to track progress or take corrective action;
most departments have no procedures to ensure that corrective action is taken when performance is not meeting expectations; and
although we were told that top management has periodically reviewed progress toward strategy objectives, we found no documentation showing the results of the reviews or management recommendations for corrective action.

Closing the Gaps

1.56 Two of the departments we looked at this year are well on the way to establishing good management and control practices for strategy implementation. Several departments produced clear, concise performance information for Parliament, in accordance with the Guideline they were provided.

1.57 However, most departments' management and reporting practices for their sustainable development strategies remain in the early stages of development. We observed many of the same gaps in management practices as last year. Accordingly, our recommendations from our work this year are consistent with those of last year.

1.58 In their sustainable development strategy progress reports, departments need to clearly communicate to members of Parliament and Canadians the results they have achieved in relation to key strategy goals, targets and indicators, and they need to explain variances and corrective actions required. Departments need to be specific about the extent to which they have implemented their action plans for sustainable development and achieved the objectives. They need to report on a consistent basis, allowing for some comparability from one year to the next, and they need to cross-reference the reports to more detailed information.

1.59 Departments should use the reporting format presented in the Treasury Board Guideline for the Preparation of Departmental Performance Reports to Parliament.

1.60 The individuals responsible for implementing the strategies, including those responsible for internal audit and management review, need appropriate training on the purpose and key requirements of a management system.

1.61 Departments need to perform regular assessments of their management systems to assess the extent to which they support the achievement of objectives and to identify gaps between their systems and good management practices. Senior management needs to review the findings and recommendations of those assessments and ensure that necessary corrective action is taken promptly.

1.62 Departments should establish and apply management systems to support implementation of their strategies. In doing so, they should give priority to assessing and meeting training needs, establishing the monitoring and reporting practices necessary to provide clear performance information to Parliament, and adopting the periodic self-assessment and review practices necessary to identify and close gaps between their current practices and good management practices.

1.63 We believe that it is feasible to expand the scope of general management systems to encompass strategy implementation. Departments ought to consider adapting their existing Planning, Reporting and Accountability Structures or apply similar good management practices such as those reflected in the ISO 9000 and 14001 standards.

Conclusion

1.64 This chapter provides our second annual assessment of departmental progress in implementing sustainable development strategies. Overall, departments are progressing. In 1999 they reported having met on average about 20 percent of the commitments set out in their strategies, compared with 11 percent in 1998. As Exhibit 1.5 indicated, the efforts of some departments are building momentum.

1.65 As we found last year, the quality of information that departments provided in their progress reports varied widely. Several departments followed the Treasury Board Guideline for reporting on progress and provided clear, understandable information on the status of the actions set out in their sustainable development strategies.

1.66 However, the performance information provided by most departments continued to fall well short of expectations. Thus, it remains difficult to judge whether the strategies are on track or whether corrective action is required. We expect that the quality of reporting will improve as departments adopt a more systematic approach to managing strategy implementation.

1.67 In our 1998 report we recommended that departments establish clear and measurable targets that they, parliamentarians and the public could use to judge whether or not they are successfully implementing their strategies. This year we reviewed the departments' revised targets and found that about half included a clear criterion or measure of success and a clearly stated completion date. In their departmental performance reports to Parliament, about 45 percent of the 28 departments reported progress toward their targets.

1.68 We also took our second annual look at the capacity of departments to implement their strategies. Using relevant sections of the ISO 14001 environmental management system standard, we examined six departments' practices and procedures for implementing their strategies. We found that four of the six departments are still in the early stages of establishing a systematic approach to strategy implementation. There are significant gaps between their practices and the ISO 14001 benchmark. These four departments have not yet adopted a systematic approach to identifying their priorities, defining management expectations, assigning accountability for results at lower levels of the organization, identifying related training needs, or performing the self-assessments that would facilitate steady improvement.

1.69 Thus, the current management control practices for strategy implementation in four of the six departments do not provide assurance that they will implement their action plans consistently or achieve the intended results of their strategies. To remedy that deficiency, we have recommended that departments establish management control systems, giving priority to training requirements, monitoring and reporting practices and the self-assessment and review practices necessary to facilitate improvement.

About the Audit

Objective

A key duty of the Commissioner of the Environment and Sustainable Development is to monitor the progress of departments in implementing their action plans and achieving their objectives for sustainable development. The long-term goal of this work is to promote understanding, accountability and best practices in the management of environmental and sustainable development issues in federal government departments.

The objectives of our audit were to influence departmental performance in the management of environmental and sustainable development issues, through better Parliamentary understanding and oversight of departmental performance; and to promote a better understanding among departments of their obligations and best practices for meeting them.

Scope and Approach

The Commissioner's second audit of sustainable development strategy implementation consisted of two complementary components:

1. An examination of departmental performance reports (DPRs) focussed on the extent to which the 28 departments and agencies that tabled sustainable development strategies in December 1997 had reported progress on the action plans and commitments set out in their strategies.

For all 28 departments, we reviewed the sustainable development content of the DPRs tabled in Parliament and supplementary documents that were referenced therein or that were provided to us in response to our request for information. To facilitate this component of our audit, we developed an electronic database containing all of the "commitments" made by each department in its strategy. The database allowed us to compare the accomplishments reported by the 28 departments in their progress reports with the goals, objectives, targets and actions contained in their sustainable development strategies. We did not audit departmental accomplishments to verify the accuracy of reported results; this will be the subject of future work.

Relying on the information contained in the DPRs and supplementary progress reports provided by the departments, we assessed the extent to which the departments had done what they said they would do in their strategies. We also assessed the extent to which departments provided the performance information specified in the Treasury Board Guideline for the Preparation of Departmental Performance Reports to Parliament.

2. The capacity audit component of our work focussed on the management practices being applied to implementation of the sustainable development strategies in six departments relative to established standards of effective management and control.

To facilitate this work, we developed an audit program based on the International Organization for Standardization (ISO) 14001 standard. ISO 14001 has received unanimous approval from the standards bodies of 67 countries, including the Standards Council of Canada, and is becoming established as the standard of due diligence for managing environmental and sustainable development issues.

We cross-referenced these criteria with the general criteria of good governance set out by the Canadian Institute of Chartered Accountants in its Criteria of Control, and with common principles of good management set out by the Treasury Board Secretariat in its guideline for departmental planning, reporting and accountability structures.

We provided our audit program, including a list of suggested documentary evidence, to the six departments approximately two months in advance of our examination. We scheduled and conducted on-site interviews and document reviews at each of the six departments to conclude whether the departments had established the capacity to consistently and reliably implement the action plans set out in their sustainable development strategies.

Audit Team

Acting Commissioner: Richard Smith
Director: Andrew Ferguson

Holly Shipton
Hilary Stedwill

For information, please contact Richard Smith.

2000 May Report of the Commissioner of the Environment and Sustainable Development

Chapter 1—Implementing Sustainable Development Strategies: Year Two—Work in Progress

Exhibits:

Main Points