This Web page has been archived on the Web.

1999 Report of the Commissioner of the Environment and Sustainable Development

Main Points

1.2 In their first annual progress reports to Parliament on sustainable development, departments reported on actions to date. However, the links between the large number of actions that departments reported and the objectives set out in their strategies are frequently too abstract to provide insights about progress. As a result, beyond tallying the activities reported accomplished by departments, we are unable to conclude whether the strategies are on track or whether corrective action is required.

1.3 Current management and control practices being applied to strategy implementation in the six departments that we audited in-depth do not yet provide assurance that their action plans will be consistently implemented or that the intended results of the strategies will be achieved.

Background and other observations

1.5 We compared the information provided by the departments in their progress reports with their strategy commitments and with what they were asked to provide to Parliament by the Treasury Board Secretariat in its Guideline for the Preparation of Departmental Performance Reports to Parliament . Most of the reports fell well short of the Secretariat's Guideline, making it difficult to judge whether progress is being made. We expect the quality of reporting to improve substantially as departments gain experience implementing their strategies and establish clear and measurable targets as we recommended last year.

1.6 We also took an in-depth look at the capacity of six departments to implement their strategies. Using relevant sections of the ISO 14001 environmental management system standard, we examined the practices and procedures being applied to the task.

1.7 Key gaps exist relative to the ISO 14001 benchmark. Departments have not yet adopted a systematic approach to identifying their priorities, defining management expectations, assigning accountability for results and identifying related training needs at lower levels in the organization; and they have not performed the self-assessments that would facilitate steady improvement.

Introduction

1.9 The strategies contained each department's action plan for sustainable development, including the objectives and targets that will be used by the department and others as benchmarks for measuring progress. To ensure accountability for results, ministers were directed to report annually to Parliament on progress against their sustainable development commitments in their Part III Estimates (now the Departmental Performance Reports).

1.10 In October 1998, ministers tabled their first annual sustainable development strategy progress reports based on progress to 31 March 1998. The purpose of these reports is to apprise parliamentarians of progress against commitments - that is, whether or not the departmental strategy is on track and, if not, what is being done to get it on track. Because only highlights were to be presented to Parliament, departments were also asked to provide references to other documents to allow interested parties access to more detailed information.

1.11 The 1995 amendments to the Auditor General Act , which required ministers to have sustainable development strategies prepared for their departments, also created the position of Commissioner of the Environment and Sustainable Development. Under the Act, the principal duty of the Commissioner is to monitor and report annually to Parliament on the extent to which departments have implemented their action plans and met their objectives for sustainable development.

1.12 This is our first report on progress. Our objectives are to help parliamentarians understand and exercise oversight over the departments' progress on sustainable development and to help departments understand their management obligations and best practices in meeting them.

Focus of the audit

1.14 To answer the first question, we compared the goals, objectives, targets and actions set out by each department in its 1997 strategy with the performance information each department presented to Parliament in its 1998 progress report.

1.15 We reviewed the strategies and Departmental Performance Reports of the 28 departments that tabled progress reports (see Exhibit 1.1) . In addition, we contacted these departments to request copies of any additional, more detailed sustainable development progress reports that had been prepared, and we reviewed those reports where available.

1.16 To answer the second question, we compared the management practices being applied to the implementation of sustainable development strategies in six departments with recognized standards of good management practice. The departments selected represent a cross-section of policy, program and operational mandates: Agriculture and Agri-Food Canada, the Atlantic Canada Opportunities Agency, Citizenship and Immigration Canada, Health Canada, the Department of Justice, and Veterans Affairs Canada.

1.17 The criteria we used to assess the capacity of these six departments reflect the management principles set out by the International Organization for Standardization (ISO) in its environmental management systems standard - 14001. This standard is consistent with the Canadian Institute of Chartered Accountants' criteria of control, the European Eco-Management and Audit Scheme (EMAS) and principles of good management set out by the Treasury Board Secretariat's Planning, Reporting and Accountability Structure.

1.18 Additional details on the audit can be found in About the Audit at the end of this chapter.

Observations and Recommendations

What Ministers and Their Departments Were Asked to Do

1.20 By informing management of where the department stands relative to its objectives and identifying opportunities for improvement, these documents are the key mechanism for keeping the sustainable development strategies alive and on track. They also serve as an important tool for the Commissioner of the Environment and Sustainable Development and others interested in monitoring the progress of the 28 departments.

1.21 To help departments present sustainable development performance information to Parliament, the Treasury Board Secretariat published the Guideline for the Preparation of Departmental Performance Reports to Parliament. The Guideline clearly specifies the information that departments should include in their progress reports (Exhibit 1.2) .

1.22 Specifically, the annual progress reports are expected to contain five pieces of information: the key commitments set out by departments in their strategies; the indicators or measures that departments and others can use to gauge progress against those commitments; the targets that departments expected to achieve during the reporting period; a summary of accomplishments relative to each target, including corrective actions being taken to ensure that commitments are met; and cross-references to other documents to allow readers access to more detailed information.

1.23 We recognize that departments are in the early stages of implementing their sustainable development strategies and that it will take time to accomplish all of the goals set out in those strategies. Moreover, the lack of clear measurable targets in the 1997 strategies means that there are few benchmarks to judge whether the strategies are being successfully implemented. However, we expect departments to have established accountability for the action commitments set out in their strategies, and the monitoring and reporting procedures necessary to provide annual performance information to Parliament as directed. Exhibit 1.3 presents the process used by Natural Resources Canada to monitor and report progress.

What Departments Did

All departments that produced a sustainable development strategy also produced a progress report

1.25 Only three departments provided all of the performance information specified in the Guideline, including reference to more detailed information. Six of the twenty-eight departments sent us a detailed progress report in response to our request for additional performance information.

1.26 The six detailed progress reports we received referenced each of the departments' key objectives and summarized related activities undertaken by them during the reporting period. The best of these reports clearly indicated the status of each action commitment as either complete or in progress, thereby allowing readers to understand the extent to which the department had implemented the actions set out in its strategy.

1.27 Information on the status of each action commitment made monitoring progress easier and conveyed a greater sense of assurance about the department's capacity to track its activities and manage its progress. Reports that presented information according to the Treasury Board Secretariat's Guideline greatly facilitated our monitoring work.

1.28 Adopting a straightforward, logical reporting framework is an essential first step toward producing useful performance information for Parliament. As Exhibit 1.4 indicates, most departments referenced some of the objectives from their sustainable development strategies and reported activities they had undertaken for sustainable development during the reporting period.

Most departments did not provide all of the information specified in the Guideline

1.30 Most departments did not provide references to other documents in their summary reports to allow readers access to more detailed performance information, nor did they provide such information in response to our request. Only 10 departments provided performance indicators that the department and others could use to gauge progress. Nine departments included performance targets for the reporting period.

Did Departments Do What They Said They Would Do in Their Strategies?

Reporting needs to be improved to allow a judgment to be made

1.33 In their first sustainable development strategy progress reports to Parliament, departments reported accomplishing about 11 percent of the actions set out in their strategies. The reports cover the initial three months of strategy implementation or about eight percent of the time horizon covered by the strategies. We did not audit the results reported to validate their accuracy; this will be the subject of future work.

1.34 For five departments, we found no performance information that allowed us to assess progress relative to the specific actions set out in their strategies. The sustainable development commitments of these five departments represent about 33 percent of the total number set forth by the 28 departments. This represents a significant gap.

1.35 Until departments present a clear set of targets as recommended in our 1998 Report, those interested in monitoring progress lack the broader benchmarks required to judge whether the strategies are being successfully implemented.

Have Departments Established the Capacity to Reliably Implement Their Strategies?

1.37 To gain a better perspective on the capacity of departments to successfully implement their strategies, we compared approaches to implementation under way in six departments with the ISO 14001 environmental management systems standard. The standard, developed under the auspices of the International Organization for Standardization, is a tool for managing environmental and sustainable development issues.

1.38 The ISO 14001 standard was the product of broad stakeholder consultation and consensus on the elements of good management practice. It has received unanimous approval from the standards bodies of 67 countries, including the Standards Council of Canada. ISO 14001 is considered to be consistent with sustainable development and compatible with diverse cultural, social and organizational frameworks.

1.39 Each of the six departments we chose for this component of our monitoring work indicated in their strategies that they were developing a management system to address their environmental issues. Three of the six indicated that their management systems would be based on ISO 14001.

A well-functioning management system is a strong indicator that intended results will be accomplished

1.41 Effective control embraces all of the elements of management including accountabilities, organizational resources and the processes and procedures that are required to achieve an intended result. Exhibit 1.6 indicates the type of practices and procedures one might expect to see where environmental and sustainable development issues are being effectively managed. The Canadian Institute of Chartered Accountants has developed a framework that organizations can use to develop and assess a management control system (Appendix A) .

1.42 A management system provides a structured process for continual improvement. It is a tool that enables an organization to achieve and control the level of performance it sets for itself. Although some improvement in performance may be anticipated when adopting a systematic approach, the rate and extent of improvement will be determined by the organization.

1.43 The existence of a well-functioning management control system is a strong indicator that intended results will be accomplished. Where senior management is committed to achieving results, such a system will enhance an organization's capacity to anticipate key issues and achieve its performance objectives. A well-functioning management system would help to "operationalize" a sustainable development strategy, provide reasonable assurance that the action plan will be consistently and reliably implemented and improve confidence that intended results will be achieved.

1.44 In 1994 the government directed departments to establish environmental management systems and to emulate best practices used in other sectors to manage these issues. Departments were further directed to improve their management systems and operational practices for greater consistency with sustainable development.

ISO 14001 is becoming the standard of due diligence

1.46 Accordingly, in performing his annual monitoring duties, the Commissioner would like to place reliance on recognized standards of practice for managing environmental issues and sustainable development in much the same way as financial auditors rely on accepted principles and criteria of control as part of their monitoring and assurance audit work.

A More In-depth Look Confirmed Capacity Gaps

1.48 Consistent with our expectation, we found that the management and control practices currently being applied to strategy implementation by these six departments are not yet sufficiently developed to provide reasonable assurance that their plans will be achieved.

1.49 Exhibit 1.7 compares current management practices being applied to strategy implementation with the ISO 14001 standard. On average, the six departments we audited have established about one third of the practices reflected in the ISO 14001 standard.

1.50 So far, departmental practices are most developed at the planning stage of the management cycle. They become weaker as the departments move from the planning stage to the implementation and operation stage, and weakest at the monitoring and corrective action stage.

Overall, we found that:

• corporate responsibility for strategy implementation has been assigned but no clear targets have been established;
• the sustainable development aspects of specific departmental policies, programs and activities have not been itemized or prioritized;
• regulatory and other potentially applicable environmental and sustainable development obligations have not been itemized or prioritized;
• key issues to be managed by lower levels in the departments have not been clearly identified or prioritized;
• the contribution and accountability of lower levels in the departments are largely undefined;
• training requirements have not been assessed; and
• practices, procedures and work instructions for strategy implementation, monitoring and control are lacking.

Planning

1.52 All of the six departments that we looked at established objectives for sustainable development in their 1997 strategies and have since assigned responsibility for strategy implementation at the corporate level.

1.53 However, the six departments did not present evidence to demonstrate that regulatory and other applicable requirements (such as past government policy directives on the environment, non-regulatory guidelines and codes of practice, and international environmental agreements) have been clearly identified, assessed or prioritized for management purposes. Similarly, we were not presented with evidence to demonstrate that the departments had formally assessed their key policies, programs and activities as a basis for establishing management objectives and accountabilities at various levels within the organization or with identified partner organizations.

1.54 Perhaps most significant, we saw no documentation to indicate that departments had assessed the degree of practical control they may have over the objectives set out in their strategies or established their current or baseline position with respect to those objectives. For example, many departments set far-reaching strategic objectives for sustainable development and noted that progress would require the co-operation of other organizations. But we saw no evidence that the co-operation of other organizations had been secured or that respective contributions to achieving shared objectives had been defined.

Implementation and operation

1.56 While responsibility for strategy implementation had been assigned at the corporate level, most of the six departments have yet to establish roles, responsibilities and accountability for contributing to the achievement of the strategy objectives at lower levels in the organizations. These departments have not yet "operationalized" their strategies. One department produced clear evidence that it had established accountability for results and the capacity to track and report progress toward its sustainable development strategy objectives. Exhibit 1.8 provides a description of the process used by Health Canada to monitor and report its sustainable development strategy progress.

1.57 In the six departments that we looked at in-depth, management processes and procedures required to achieve intended results are in very early stages of development. Most have not yet established a structured process with practices and procedures to ensure that their work is conducted in accordance with relevant professional standards, regulatory requirements and other responsibilities.

1.58 Documented practices and procedures are not always necessary for effective management but they do serve a number of important purposes. They help to ensure that the organization's policies are implemented consistently over time and to reduce risks such as the loss of corporate memory, associated with staff turnover. They may also prove helpful in demonstrating that due diligence was exercised in addressing an issue or preventing a problem - a key test in determining legal liability.

1.59 The documented practices and procedures we looked at had generally been established in response to past government policy direction (for example, procedures for environmental assessment of new policies and programs). Several departments described processes and procedures they were following to implement their strategies but they were unable to provide documentation for review.

1.60 A key aspect of good practice is continual improvement to processes, procedures and performance. Departments will find it difficult to systematically improve practices and procedures that have not been documented.

1.61 We noted that training needs relative to the environment and sustainable development have not been systematically assessed. None of the six departments presented evidence that key staff assigned to implementation have received training relative to the environmental aspects or the sustainable development objectives for which they are responsible. For example, while each of the six departments committed to implementing an environmental management system in its sustainable development strategy, they have not recognized a need for management systems training.

Checking and corrective action

1.63 In the six departments that we looked at in-depth, the management system elements required to monitor and report performance (relative to sustainable development objectives) and take corrective actions are in the early stages of development. None of the six departments has established a procedure for periodically assessing the adequacy of its management and control systems for sustainable development and determining the need for improvements.

Management review

1.65 Regular review of the adequacy of the departments' management system in supporting implementation efforts would also demonstrate the commitment of senior management to the departments' sustainable development strategy and diligence in managing these issues.

1.66 During our interviews with the six departments, we noted that senior management had been provided with briefings on various sustainable development strategy initiatives. However, the departments had not established a mechanism for gathering the information necessary to periodically review the adequacy of the management and control system in supporting sustainable development strategy implementation.

1.67 Existing management review processes were generally ad hoc and focussed on producing outputs in response to the Auditor General Act (1995) and other government directives rather than on opportunities for improved management practice.

Closing the Gaps

1.69 Departments should make better use of the reporting format presented in the Treasury Board Secretariat's Guideline for the Preparation of Departmental Performance Reports to Parliament.

1.70 The sustainable development strategy progress reports need to clearly communicate to members of Parliament and Canadians the results achieved relative to key strategy goals, targets and indicators, including an explanation of variances and necessary corrective actions. Departments also need to ensure that their reports are consistent, allowing for some comparability from one year to the next.

1.71 Departments should accelerate their plans to implement the management systems necessary to support implementation of their strategies. In implementing their management systems, departments should give priority to training needs and to adopting periodic self-assessment and management review practices.

1.72 Individuals responsible for strategy implementation, including those responsible for internal audit and management review, need to receive appropriate training on the issues for which they are responsible and on the purpose and key requirements of a management system.

1.73 With respect to self-assessment and management review, Departments need to perform regular self-assessments of their management system to identify gaps relative to good management practice. It is important that senior management regularly review the adequacy of the management system in supporting achievement of the departments' sustainable development objectives and ensure that necessary corrective actions are promptly implemented.

1.74 It should not be necessary to "re-invent the wheel." Departments ought to consider adapting their existing Planning, Reporting and Accountability Structure or applying similar good management practices such as those reflected in ISO 14001.

Conclusion

1.76 Departments are required to report annually to Parliament on strategy implementation. According to their first progress reports and related information, departments have so far completed about 11 percent of what they said they would do in their strategies. Departments are making progress on their action plans.

1.77 However, the information that departments provided varied widely. For most departments, it fell well short of the Treasury Board Secretariat's Guideline for the Preparation of Departmental Performance Reports to Parliament, making it difficult to judge whether the strategies are on track or whether corrective action is required. We expect that the quality of reporting will improve substantially as departments establish clear and measurable targets as we recommended last year and make better use of the Guideline.

1.78 Departments are just beginning to establish practices to support strategy implementation and key gaps exist relative to the ISO 14001 benchmark. Departments have not yet adopted a systematic approach to identifying their priorities, defining management expectations, assigning accountability for results, identifying related training needs at lower levels in the organization, and performing the self-assessments that would facilitate steady improvement.

1.79 As a consequence, the current management control practices that departments are applying to strategy implementation do not provide assurance that their action plans will be consistently implemented or that the intended results of the strategies will be achieved. To remedy that deficiency, we have recommended that departments accelerate their plans to put appropriate management systems in place, giving priority to training needs and implementing the self-assessment and management review practices required to facilitate corrective action.

1.80 A well-functioning management system can help to "operationalize" a strategy. It can provide reasonable assurance that the organization has established the capacity to meet and continue to meet its obligations and commitments and improve confidence among stakeholders that intended results will be achieved.

About the Audit

Objectives

The objectives of our audit are to influence departmental performance in the management of environmental and sustainable development issues, through better parliamentary understanding and oversight of departmental performance and better understanding among departments of their obligations and best practices in meeting them.

Approach and Scope

1. An examination of Departmental Performance Reports focussed on the extent to which the 28 departments and agencies that tabled sustainable development strategies in December 1997 had reported progress on the action plans and commitments set out in their strategies.

For all 28 departments and agencies, we reviewed the sustainable development content of their Performance Reports and supporting documentation referenced therein or that was provided in response to our request for information. To facilitate our examination, we developed an electronic database containing all of the "commitments" made by each department in its strategy. The database allowed us to compare the accomplishments reported by the 28 departments in their progress reports with the goals, objectives, targets and actions contained in their sustainable development strategies. We did not audit departmental accomplishments to validate the accuracy of reported results; this will be the subject of future work.

Relying on the information contained in the Performance Reports and supplementary progress reports provided by the departments, we assessed the extent to which the departments had done what they said they would do in their strategies. We also assessed the extent to which departments provided the performance information specified by the Treasury Board Secretariat's Guideline for the Preparation of Departmental Performance Reports to Parliament.

2. The capacity audit component of our work focussed on the management practices being applied to implementing sustainable development strategies in six departments relative to established standards for effective management and control.

To facilitate this work, we developed an audit program based on the International Organization for Standardization (ISO) 14001 standard. ISO 14001 has received unanimous approval from the standards bodies of 67 countries including the Standards Council of Canada and is becoming established as the standard of due diligence with respect to managing environmental and sustainable development issues. We cross-referenced criteria with the Canadian Institute of Chartered Accountants criteria of control and with Treasury Board Secretariat's draft Planning, Reporting and Accountability Structure.

We provided our audit program, including a list of suggested documentary evidence, to the six departments approximately two months before our examination. We scheduled and conducted on-site interviews and document reviews at each of the six departments to conclude whether they have established the capacity to consistently and reliably implement the action plans set out in their sustainable development strategies.

Audit Team

Joseph Odhiambo
Arthur Wong

For information, please contact Richard Smith.