This Web page has been archived on the Web.

1998 Report of the Commissioner of the Environment and Sustainable Development

Main Points

8.2 The sustainable development strategies are intended to establish the sustainable development objectives and action plans for each federal department subject to the Act. They are expected to be comprehensive, results-oriented and developed in consultation with the department's clients, partners and other stakeholders. Departments have committed to reporting annually on their progress toward sustainable development in Part III of the Estimates.

8.3 This chapter summarizes our work on performance measurement for sustainable development strategies. The purpose of this study is to advance the application of performance measurement to the management of environmental and sustainable development issues in federal government departments by summarizing key concepts and success factors and by describing the characteristics of good performance information. The study is also intended to help parliamentarians set their expectations for the strategies tabled by departments last year, and for the sustainable development aspects of departmental performance reports that will be issued annually. Parliamentarians are the main clients for the strategies and performance reports. Their visible interest in and use of this information will provide an important incentive for departments to make continual improvements to its quality.

8.4 Sustainable development involves the integration of social, economic and environmental considerations into how an organization defines its objectives and conducts its business. Government organizations have typically focussed on only one of these three dimensions - the one related to the public policy role for which each organization was established. Sustainable development also implies extending the planning horizon, consistent with considering the needs of both current and future generations. Federal government departments are now being challenged to broaden their perspective on the implications of what they do in order to protect and promote the well-being of both people and the environment that supports them.

8.5 There is no "master list" of generally applicable performance indicators for sustainable development. Rather, the starting points for departments are an assessment of their sustainable development impacts in the context of their existing public policy roles, and the goals they set to accentuate the positive aspects and mitigate the negative aspects. However, it is important that government departments work together to develop and use common indicators in areas where they are pursuing common objectives.

8.6 While the principles of sustainable development are relatively new, the principles of performance measurement and managing for results are well established and apply equally to sustainable development objectives.

Introduction

Good performance information is critical to the success of the government's sustainable development initiative

8.8 Establishing and using good performance indicators that reveal how well departments are doing in implementing their action plans and meeting their objectives is critical for the success of this government-wide effort. Therefore, an important requirement is that the strategies be results-oriented, identifying in sustainable development terms the key results departments expect to achieve, and how they will measure their performance.

8.9 This chapter is part of a much broader effort to promote results-based management in the public sector and to improve reporting to Parliament. The results orientation of the strategies is consistent with the government's proposed approach for reporting to Parliament, which is currently being tested through a pilot study. The new system is built on the idea that departments should set clear performance objectives, measure and report performance, and use that information to make programs work better.

8.10 Previous work on performance measurement by the Auditor General has found that progress is being made in reporting performance expectations and accomplishments. However, departments face a number of challenges in shifting the focus of management and measurement from inputs and activities toward the results achieved. These challenges include interpreting uncertain cause-and-effect relationships, setting clear objectives, translating those objectives into specific performance targets and indicators, and managing with incomplete information. These challenges are especially relevant to sustainable development given its broad scope.

8.11 Nevertheless, there has been sufficient progress to identify good practices and lessons learned. One of the key lessons learned from organizations that are adopting a focus on results is that results cannot be the concern of only the planners and evaluators. They must become the focal point for the organizations' senior executive leaders as well as the day-to-day actions of their managers.

Focus of this study

8.13 This chapter also summarizes our work in this area for parliamentarians. It is intended to help them set their performance measurement expectations for the sustainable development strategies and for the departmental performance reports that will be issued annually. Parliamentarians are the main clients for the strategies and performance reports. Their visible interest in and use of this information will provide an important incentive for departments to improve its quality.

8.14 As part of this study, we developed a workbook for establishing program-level performance objectives and indicators that support the achievement of a department's strategic objectives for sustainable development. The work steps are summarized in the Appendix . The entire procedure will be made available at the Internet site for the Commissioner of the Environment and Sustainable Development: http://www.oag-bvg.gc.ca. We invite comments and suggestions for improvement. Further details on the objective, scope and approach of the study are found at the end of the chapter in About the Study .

8.15 For related information, see: Report of the Commissioner of the Environment and Sustainable Development, May 1998, Chapter 1 - Greening the Government of Canada - Strategies for Sustainable Development, Chapter 5 - Expanding Horizons - A Strategic Approach to Sustainable Development, Chapter 7 - Counting the Environment In, and Report of the Auditor General of Canada 1997, Chapter 5 - Reporting Performance Information in the Expenditure Management System, Chapter 11 - Moving Toward Managing for Results, and Chapter 22 - Crown Corporations: Making Performance Measurement Work.

Study Findings

Performance Measurement for Sustainable Development Strategies

What departments are required to do

8.17 The government also provided departments with more specific guidance to assist them in preparing their sustainable development strategies. As stated in A Guide to Green Government , sustainable development strategies are to be:

• comprehensive, dealing with departmental policies and programs as well as with the management of the department's internal operations;
• results-oriented, identifying in sustainable development terms the main results the department will achieve, and how it will measure performance toward them; and
• developed in consultation with the department's clients, partners and other stakeholders.

8.19 The dual objectives of improving performance and accountability mean that establishing, using and reporting performance information are critical for the success of this government-wide effort. Performance will be improved by incorporating lessons learned into annual business plans and strategy updates; accountability will be enhanced through annual reporting of progress on implementing sustainable development strategies in departmental performance reports.

Sustainable development is about broadening an organization's perspective

8.21 The government-wide sustainable development strategy process is challenging departments to take a broader perspective on the implications of what they do in order to protect and promote the well-being of people and the environment that supports them.

8.22 To provide departments with a common starting point for the identification of performance indicators, A Guide to Green Government presents a series of goals and objectives linked to sustainable development to which departments might contribute. These are presented in Exhibit 8.1 .

8.23 There is no "master list" of generally applicable performance indicators for sustainable development. Rather, the starting point for federal departments is an assessment of their sustainable development impacts in the context of their existing public policy roles and the goals they set to accentuate the positive aspects and mitigate the negative aspects.

8.24 For example, the RCMP's mandate, under the Royal Canadian Mounted Police Act , allows it to enforce laws, prevent crime and maintain peace, order and security. The RCMP's vision pertaining to sustainable development is "to provide the highest quality police service in Canada, in the most environmentally responsible manner possible." In its strategy for sustainable development, the RCMP assesses its sustainable development impacts in the context of its existing role and establishes its priorities including:

• communications and training;
• contracting and procurement;
• fleet management;
• green building design;
• conserving natural resources; and
• hazardous materials management.

Beginning with strategic objectives

"Cascading" through the organization

Types of indicators

8.28 Three levels of environmental performance indicators identified by the International Organization for Standardization (ISO 14000) may be helpful to organizations in deciding what they should measure and report. For sustainable development, the following set is evolving to include certain "state of society" indicators as well:

• Operational indicators measure potential stresses on the environment (or society), as consequence of such things as the use of energy, the use of hazardous materials or the disposal of wastes.
• Management indicators measure the results of actions taken to eliminate or reduce potential stresses, such as an organization's programs for energy efficiency, waste reduction, pollution prevention, awareness building, participation or enforcement.
• State-of-the-environment indicators measure background conditions in the physical as well as the social environments, such as aspects of air or water quality, or factors related to poverty, the transfer of technology or public participation in decision making.

• At the most basic level, the Department will monitor and report on each commitment and action specified in its strategy. Performance will be assessed in terms of "did the Department do what it said it would do?"
• At a higher level, the Department will assess the extent to which its actions actually helped to accomplish the objectives identified in its strategy. For each of the strategy's objectives, the Department will report on a few indicators to help assess its achievements.
• At the broadest level, the Department will move beyond its individual contributions and work with partners and stakeholders to develop indicators of Canada's overall progress toward sustainable development.

Working together to develop common indicators

8.31 While departments differ substantially in terms of their policy mandates, all departments require operational support to fulfill them.

8.32 Initially, operational issues may present the most visible opportunities for co-ordination and harmonization of performance indicators across government departments. The "greening operations" component of departmental strategies is a good starting point for the development of common indicators.

8.33 Ongoing efforts to harmonize performance indicators related to operational issues demonstrate the potential for developing a generally accepted set of indicators for government operations.

8.34 An interdepartmental working group has developed a proposed set of common performance indicators for six operations-related issues. The participating departments have significant custodial responsibilities. The indicators were chosen to facilitate comparisons among facilities and across departments. These indicators are presented in Exhibit 8.4 and are described in more detail in Chapter 7 of the Commissioner's Report, entitled Counting the Environment In.

Sharing experience

8.36 We would anticipate that where departments are pursuing common objectives, consultation and consensus-seeking processes would result in an evolution toward common performance indicators.

8.37 A common set of performance indicators across federal departments would facilitate issue management, and reporting and oversight of the government's progress toward sustainable development. It could also assist in developing employee awareness and commitment.

The Performance Measurement Challenge

Prerequisites: a "plan-do-check- improve" management cycle

8.40 The government's expectations for such a management framework are outlined in Treasury Board Secretariat guidelines for departmental planning, reporting and accountability structures, which form part of the government's proposed Expenditure Management System - the process by which the federal government plans, budgets and seeks parliamentary approval for expenditures.

8.41 Under the present Expenditure Management System, departments produce strategic, three-year business plans that set out departmental goals, targets and performance indicators. These are submitted annually to the Treasury Board for review, and also form the basis for reporting to Parliament in the Estimates documents. Part III of the Estimates has now been split into two components: The spring Report on Plans and Priorities sets out performance expectations and general directions; the fall Performance Report provides information on results achieved.

Some key concepts

What is performance measurement?

• Did we do what we said we would do? (accountability)
• Did our actions achieve the specified results? (effectiveness)
• Did we achieve these results in an efficient manner? (efficiency)
• Can we do it better? (continual improvement)

8.45 Chapter 11 of the Auditor General's December 1997 Report, entitled Moving Toward Managing for Results, presents a case in point. A service line within a federal government department with a mandate to investigate potentially fraudulent activities shifted from measuring its performance on the basis of investigations completed to measuring and reporting on the results of those investigations in terms of savings achieved. The case study concludes that the activities of the service line have become more transparent and are potentially more justifiable as a result of the shift in emphasis toward outcomes achieved.

8.46 While the principles of sustainable development are relatively new, the principles of performance measurement and managing for results are well established and apply equally to sustainable development objectives.

Why measure performance?

8.48 To enhance program credibility and good will. In the absence of good performance information, stakeholders will draw their own conclusions about whether or not a government program has value. A performance measurement system is a useful tool for clarifying the scope and objectives of government programs for stakeholders, and for maintaining or restoring public confidence that government programs can make a difference in their quality of life.

8.49 To allocate and justify resources. In a similar vein, an important use of performance information is to allocate scarce resources among competing alternatives. Performance measurement can play a key role in informing Parliament and the public about the value, efficiency and cost-effectiveness of government programs.

8.50 To be accountable for public expenditures. Finally, and perhaps most important, public sector managers are entrusted with public funds to develop programs that maintain or improve a specific aspect of life. It is incumbent on them to identify the clients or stakeholders they intend to serve or target, how they intend to produce sustainable benefits or outcomes, and the indicators that can be used by stakeholders to evaluate their performance.

What is good performance information?

8.52 Performance indicators should help reveal the gap between an organization's actual level of performance and the level of performance that was specified as its performance target. Performance targets might be based on a benchmark best practice, a technical standard or some specified progression from the baseline value. A set of performance indicators should support a broader explanation of performance results - a " performance story" for managers and executives and for the organization's various audiences.

8.53 To assist in assessing how well a program is doing, performance information should explain:

• why the action was expected to be beneficial;
• who was affected by the action;
• what near-term changes the action produced that contributed to achieving the long-term outcome; and
• how specific products, services or outputs produced these changes.

Performance measurement pitfalls

8.55 Moving from measuring activities to measuring results. Much of the current reporting to Parliament consists of summarizing the activities of the department and the resources used. While probity and prudence in the use of resources are important elements of the government's management framework, it is the outcomes of activities - how they affect Canadians - that are most important for assessing whether the department is making a difference.

8.56 A program logic model can clarify the relationship between activities (how the department conducts its business), outputs (the products and services provided) and outcomes (impacts on clients, target groups or other stakeholders). Exhibit 8.7 illustrates this model.

8.57 Consultation and consensus building with stakeholders can help to establish the validity and credibility of the model and the outcome indicators.

8.58 Translating strategic objectives into performance targets. One of the first challenges in establishing a performance measurement system is to translate the organization's long-term strategic objectives into operational performance targets at the program level. The latter should relate clearly to the department's strategic objectives.

8.59 Clear, specific and time-bounded performance targets that are set forth in advance provide a basis for action and allow for an objective assessment of performance results.

8.60 For example, a "target level of satisfaction for 1997-98 - 80% " is documented as a target by a provincial government ministry in its business plan for 1997-1998. To monitor progress toward its objective, it will measure "client satisfaction with the level and quality of service provided in Northern Development Offices . " The ministry's objective is to "ensure access to government programs and services in northern communities." This objective is one of several intended to achieve the policy goal of "fair and equitable administration of the Mining Act . "

8.61 It is important that performance indicators relate to the performance targets. Where clear and measurable targets have not been established, departments need to identify the direction of change. Where performance indicators have not been developed, departments need to set out plans for developing them.

8.62 Interpreting cause-and-effect relationships. In some cases, the link between program-level activities/outputs and the organization's strategic objectives may not be transparent. Many programs seek to influence complex social, economic or environmental systems. Not all the factors affecting those systems fall within the control of any single program. Separating the influence of a program from the influence of other factors is a significant challenge.

8.63 However, to develop practical performance indicators for programs, management must define the specific results or influence that the programs can reasonably be expected to achieve. More detailed program evaluation can assist in setting those expectations. Consultations may also help to clarify the roles and respective contributions of other organizations.

8.64 For instance, in 1990, the Canadian Council of Ministers of the Environment, with representation from all provinces and the federal government, endorsed the National Packaging Protocol (NaPP). The NaPP contains a commitment to six packaging policies and three milestone targets intended to conserve natural resources through the use of recovered materials, and prevent pollution through source reduction. The success of NaPP is measured in terms of tonnes of packaging waste diverted from disposal. Using 1988 as a baseline, the National Packaging Protocol set the following targets for the reduction of packaging waste in Canada: 20 percent by 1992, 35 percent by 1996, and 50 percent by the year 2000. The National Packaging Monitoring System was established to track results.

8.65 Managing with incomplete information. A review of accessible data bases, information resources and analytical capabilities is useful to understand current limitations and potential investment requirements. An information inventory and a capabilities assessment need to be undertaken to understand these issues prior to formalizing performance indicators. It is important that the limitations and weaknesses of the current data be clearly explained to users of the performance information.

8.66 Where information is incomplete, performance objectives and performance indicators should be based on the best available information, while better information is being developed.

8.67 For instance, one organization that we interviewed has begun assessing the global warming potential, the acidification potential and the ozone-depleting potential of its air emissions - including those related to energy use in fleets and office buildings. The recent availability of conversion information for various gases and ozone-depleting substances made this possible. As a result, the organization's performance in terms of environmental effects will be more meaningful. However, prior to the availability of this information, the organization based its objectives and performance indicators on the best information available. It set specific targets for reducing releases of chlorofluorocarbons (CFCs) and other "air toxics" and measured its environmental performance on the basis of thousands of pounds of emissions per year.

8.68 Completeness versus overload. One of the key challenges an organization faces when establishing a set of performance indicators is how to strike an appropriate balance between providing a comprehensive picture of the organization's performance and overloading decision makers with information.

8.69 A complete set of indicators covers all aspects of performance that influence an organization's success. But if performance indicators are to be useful for decision makers and stakeholders, they must also be selective and focussed. Managers need to think in terms of the users' needs and their potential uses of performance indicators.

8.70 One way of balancing completeness and overload is to develop a set of key indicators linked to the organization's strategic objectives. Key indicators focus on the organization's success in fulfilling its public policy mandate.

8.71 For example, "emission levels (tonnes) of smog-causing pollutants" is documented as a key performance measure by a provincial government ministry in its business plan for 1997-1998. This key performance measure is linked to a specific, time-bounded and measurable target that in turn is linked to the strategic objective of "cleaner air". The objective of "cleaner air" is one of several objectives linked to the policy goal "to safeguard everyone's right to a healthy environment."

8.72 Key indicators can also be used by the organization's executive to help in directing the organization. In addition, they can provide the basis for reporting and accountability to ministers, Parliament and the public. We would expect key sustainable development indicators to be presented in the sustainable development strategies alongside the corresponding objectives and targets.

8.73 The key indicators would form the basis for more detailed objectives and indicators at each successive level in the organization. Managers at each level would establish indicators that reflect how their actions contribute to the achievement of the organization's objectives, and how that achievement will be measured. These indicators would be used for internal management purposes.

8.74 If there is a proliferation of "priorities" or if the organization's long-term objectives are not clear, it will be difficult to develop a manageable set of targets and key performance indicators. Establishing the organization's strategic priorities and setting clear objectives is therefore a prerequisite for an effective performance measurement system.

Keys to implementing a performance measurement system

8.76 Top management support. The commitment and leadership of senior management are the most commonly cited, and perhaps the most important, success factors for effective performance measurement. Clear, consistent and visible involvement by senior executives and managers at every level is needed to translate strategic objectives into operational performance targets and indicators.

8.77 It is also important to appoint a key person to be responsible for co-ordinating the department-wide performance measurement efforts, synthesizing the different inputs and maintaining contact with other individuals responsible for measurement across the organization (and in other departments). This institutional arrangement reinforces a performance measurement culture and facilitates documentation of overall results and the creation of an "institutional memory."

8.78 In the long term, performance measurement efforts will succeed only with the strong commitment of political and senior executive leaders. Only they can ensure that each department's strategic objectives and performance indicators become the basis for its day-to-day operations.

8.79 Alignment of objectives and indicators across the organization. Performance indicators can play a key role in focussing the efforts of managers on the organization's strategic priorities. To be effective in this regard, objectives and indicators created at successively lower points in the organizational hierarchy need to be built upon and aligned with those at higher levels. Managers at each level should take their cue from the managers above them and there should be direct linkages between objectives and key indicators at the strategic level and those at other levels throughout the organization.

8.80 The emphasis of sustainable development on policy integration implies that federal departments need to co-operate to achieve outcomes that individually they could only partially influence. Departments need to co-ordinate their efforts at the national level as well as between levels of government to ensure that programs are consistent with sustainable development objectives, that objectives and performance indicators are complementary and that program efforts are mutually reinforcing.

8.81 Internal communication. Internal communication provides the link between planning and practice. Managers at each level of the organization need to understand their role in achieving the strategic objectives in order to set and communicate program objectives that are aligned with them. The more clearly objectives are communicated at each level, the easier it is for employees to determine what needs to be done.

8.82 Use measurement to improve performance. Performance measurement should help managers identify what works and what does not, so that they may encourage and improve on what is working and discourage or replace what is not.

8.83 Performance measurement represents a significant investment of resources and will provide a return on that investment only if the resulting information is fed back into the decision-making process.

8.84 It is important that performance measurement facilitate learning, inspire confidence and be positive, not punitive. It has a significant role to play in encouraging creative and successful action and in ensuring that credit is given where credit is due.

8.85 Awards and recognition. The involvement of senior management in awards and recognition programs strengthens the perception that top management supports the initiatives and values performance. Good award and recognition programs can engender healthy competition between organizational units and reinforce both the level of effort and the use of indicators to demonstrate performance. They can have a significant impact on employee morale and, in turn, on motivation and performance.

Conclusion

8.87 Departmental sustainable development strategies are to be revised every three years to reflect new developments as well as past achievements. The development of performance objectives and indicators for sustainable development strategies will therefore be an ongoing activity.

8.88 There is no "master list" of generally applicable performance indicators for sustainable development. However, as in any area of public policy, parliamentarians expect good performance information, including both expectations and accomplishments, that allows a judgment to be made on how well things have been done.

8.89 A common set of performance indicators across the federal system would facilitate issue management, and reporting and oversight of the government's progress toward sustainable development. It would also assist in developing employee awareness and commitment to sustainable development objectives at all levels.

8.90 Good performance information is critical to the success of the government-wide sustainable development effort. To make progress toward their sustainable development objectives, managers need good information on what works and what does not, so that they may reinforce and improve on what is working and discourage or replace what is not. Central agencies and Parliament also need good performance information to fulfil their mandates and exercise their oversight function. In the absence of feedback provided by good performance indicators, progress toward objectives is difficult to manage, monitor or substantiate.

8.91 Parliamentarians are the main clients for the sustainable development strategies and performance reports. Through amendments to the Auditor General Act (1995), they established the requirement for departments to table sustainable development strategies, and created the position of Commissioner of the Environment and Sustainable Development to monitor them. The visible interest in and use of sustainable development performance information by parliamentarians will provide an important incentive for departments to improve its quality.

8.92 Improving the capacity of departments to produce clear, measurable and outcome-oriented performance information should help to improve sustainable development decision making, accountability and governance across federal departments.

About the Study

Objective

Scope and Approach

Each of the work steps was reviewed and refined through a series of five half-day interdepartmental workshops. Workshop participants included representation from a broad cross-section of federal departments. Participants were asked to test the validity of each work step between workshops, in the context of sustainable development work already proceeding in their departments. The complete workbook will be made available at the Office of the Auditor General of Canada web site: http://www.oag-bvg.gc.ca.

In addition to summarizing key concepts, success factors and criteria previously documented, the study also incorporated practical considerations raised through a series of interviews with six organizations outside the federal government, which are actively practising environmental management and sustainable development.

Study Team

For further information, please contact Richard Smith.