Developing Performance Measures for Sustainable Development Strategies

Getting Started

The reference point for beginning the work steps is the department’s sustainable development strategy. A process for developing sustainable development strategies, complete with strategic objectives and measures, is presented in A Guide to Green Government.

If this guide was followed, the department’s strategy will reflect the sustainable development aspects of its mission, the concerns of stakeholders, information on the key issues facing the department, and the main opportunities to manage its programs sustainably. The strategy should define the sustainable development objectives of the department.

This workbook is designed to assist work units within departments in developing objectives and measures that contribute to achieving the department's strategic objectives for sustainable development. The principle of "alignment" - the idea that there must be direct linkages between the strategic objectives set by the department and the objectives, action plans and measures of each of its work units - forms the basis of the approach.

There is no single model or process for developing performance objectives and measures, nor is there a process that will guarantee good results. We have attempted to synthesize lessons learned from the literature as well as the insights gained from our interviews and work with practitioners on applying performance measurement to the management of environmental and sustainable development issues.

Organization of the Workbook

This workbook is organized into two main parts:

Section 1 makes use of a performance framework that provides the logic for establishing program-level objectives and performance measures that support strategic objectives. The section begins by explaining the performance framework and contains five work steps (1 through 5) designed to assist users in working through the performance framework to identify key issues and objectives.
Section 2 sets out four work steps (6 through 9). These steps are intended to assist users in establishing sound performance measures to correspond with their objectives as well as accountability and resource requirements for implementation.

Each of the work steps was reviewed and refined through a series of five half-day interdepartmental workshops. Workshop participants included representation from a broad cross-section of federal departments and agencies. Participants were asked to test the validity of each work step between workshops, in the context of sustainable development work already proceeding in their departments.

For each step, short summary tables accompany a checklist of questions.

Section 1

Apply a Performance Framework

A performance framework brings structure to performance planning and clarifies the connection between activities, outputs and results. A good performance framework will address the following questions relative to the objectives specified in the department’s strategic plan:

Strategic Objective	Objective(s) specified in the department’s strategic plan
WHY is your program relevant to the strategic objective?	These questions relate to the long-term, sustainable development result(s) that the program can reasonably be expected to produce in support of a strategic objective.
WHO do you want to reach?	These questions relate to the "reach" of program activities and outputs in terms of target groups, clients, co-deliverers, partners, and other stakeholders.
WHAT results do you expect to achieve?	These questions relate to the short-term (or intermediate) result(s) of program activities or outputs that are believed to contribute to achieving the long-term results.
HOW are you going to achieve your objectives?	These questions relate to program inputs, processes, activities and outputs.

Work steps 1 through 5 address questions related to the WHY, WHO, WHAT AND HOW of a program initiative.

Establish Program Level Objectives That Contribute to the Strategic Objectives

STEP 1: Confirm Program Role

WHY is your program relevant to the strategic objective?

Rationale for this step: Defining the role that the program is intended to fulfill with respect to strategic objectives provides a basis for establishing program targets and performance measures.

	Have you established the links between the main activities and outputs of the program and the department’s sustainable development objectives? (e.g. activity/output "Y" contributes to, or detracts from, strategic objective/outcome "X")
	The department's sustainable development strategy should identify the department's significant sustainable development aspects as well as its strategic objectives for addressing these aspects and the measure that will be used to indicate progress.

Table 1: Linking Program Activities and Outputs to Strategic Objectives

Main Activities or Outputs of Program	Contributes to / Detracts from a strategic sustainable development objective(s)	Specify the Strategic Objectives or Outcomes That the Program Activity or Output Contributes to

STEP 2: Identify the Key Program Activities and Outputs

Rationale for this step: This step is essential to ensure that program managers and staff focus on key issues that contribute to the achievement of the department’s strategy for sustainable development.

Have you identified the key activities and outputs of the program in terms of their significance (e.g. High, Medium or Low) in contributing to the department’s strategic objectives?

Table 2: Identifying the Key Program Activities and Outputs

STEP 3: Identify Program Stakeholders and Issues

WHO do you want to reach?
Rationale for this step: In order to formulate a set of sustainable development objectives, it is essential to identify who program activities and outputs are intended to serve, influence or target, who the other principal groups affected are and how they are affected.

	RANK - High, Medium or Low Significance
Program Activities And Outputs	Strategic Objective from Table 1:	Strategic Objective from Table 1:	Strategic Objective from Table 1:
Activity 1	H	H	L
Output 1	L	H	L

For the medium and highly significant program activities and outputs identified in Table 2:

Have you identified the sustainable development issues (desired effects and undesirable effects) associated with these activities and outputs?
Have you identified the principal groups affected?

Table 3: Identifying Key Sustainable Development Issues and Affected Stakeholder Groups

Main Program
Activities and Outputs
in Order of Significance
(High, Medium)

Sustainable Development Issues

Stakeholder Groups
(Affected Parties)

Desired Program
Effects

Undesirable Program
Effects

Positively Affected

Negatively Affected

Activity 1

Output 2

STEP 4: Identify What the Program Aims to Accomplish

WHAT results do you expect to achieve?
Rationale for this step: Desired results should be defined in terms of outcomes that then become the focus for determining appropriate objectives, milestone targets and measures.

Table 4: Defining Results

Referring to the "undesirable" program effects and the "negatively affected" stakeholder groups (Table 3):

Have you established a desired long-term sustainable development outcome for each of these program activities or outputs?

Document the positive effect(s) that need to be produced

Have you established the near-term outcomes that can be expected to lead to the long-term outcomes?

Explain WHO needs to take WHAT action(s) to produce the desired effects

While this work step focusses on establishing objectives to redress identified undesirable outcomes, it is possible that positive effects can also be further reinforced or improved.
STEP 5: Identify Responses and Performance Requirements
HOW are you going to achieve your objectives?
Rationale for this step: Performance objectives must be defined in operational terms to be managed effectively.

Program Activities and Outputs
in Rank Order of Significance
(High, Medium)

Desired Sustainable Development Results
(Objectives)

Long-Term
Strategic

Near-Term
Intermediate

Activity 1
Output 2

Table 5: Performance Requirements Relative to Responses and Results

Have you defined the performance requirements necessary to achieve the desired sustainable development results?

What level of compliance, participation, reduction, behavioural effect, etc., is required by whom, to achieve the result? By when is it required?

Objective(s)
(from Table 4)

New or Modified
Activities, Outputs or Other Program Response(s) Necessary to Achieve the Objective(s):
(from Table 4)

Performance Requirements Relative to each Activity, Output or Other Response necessary to Achieve the Desired Results
(Targets)

Section 2

Establish Performance Measures

The next four work steps are intended to assist the user in establishing sound performance measures as well as accountability and resource requirements for implementation.

STEP 6: Identify Potential Performance Measures
Rationale for this step: Performance measurement is required to understand the gap between actual and expected levels of achievement and when corrective action may be warranted. The results indicated by a performance measure will generally be compared with expectations specified by a performance target (which might be based on a benchmark best practice, a technical standard or some specified progression from the baseline value). Therefore, performance measures should correspond with performance targets and indicate the extent to which the organizaton is achieving these performance expectations. Performance measures are an important source of feedback for effective management.

Have you established a list of performance measures that correspond to your performance targets?

The Set of Measures Should Address Each Aspect of the Performance Framework

Recalling the performance framework outlined above, some performance measures will reflect HOW well the program was managed. Such measurements may focus on environmental hazards and risk management, resource intensity, compliance, or conformance with relevant aspects of standard operating procedures and policies (e.g. procurement). Other measures will reflect WHAT was achieved for WHOM. These measures will generally focus on the intermediate effects of program outputs and activities. Measurement in this area can involve observation of behaviour (e.g. implementation of an environmental management system, increased "3R" behaviours); content analysis (e.g., changes to regulation, policies, or procedures); or feedback (e.g. survey responses). Finally, some measures will allow a judgment to be made on whether the long-term objectives, which provided the rationale for WHY the program was funded, were met. These measures involve monitoring long-term phenomena occurring in society, the environment, and the economy that can plausibly be linked back to the program initiatives, outputs and intermediate effects. These measures serve as the ultimate barometers of program success.

Table 6: Establishing Potential Performance Measures

Objectives
(from Table 4)

Activities, Outputs or Other
Program Responses
(from Table 5)

Performance Requirements
(Targets from Table 5)

Potential Performance Measure(s)

STEP 7: Establish Information Capabilities and a Baseline for Each Measure

Rationale for this step: Understanding what information is currently available to the organization as well as the organization’s capabilities for gathering and analyzing information is an important first step in the selection of performance measures.

The process of establishing baseline measures for each measure will shed light on the organization’s information capabilities and gaps. Baseline measures help to clarify the implications of objectives in terms of "level of effort" and resource requirements, and they facilitate assessment of the extent to which progress has been made from an initial condition. Baseline information provides further context that helps to clarify the magnitude of performance challenges and achievements.

Table 7: Establishing Baselines for Measures

Have you established (or can you readily establish) the initial value or baseline of each measure?

Potential Performance Measure
(from Table 6)

Units

Initial or Baseline Value

Measure 1

Measure ‘n’

STEP 8: Assess the Adequacy of Performance Measures

Rationale for this step: Once a list of candidate performance measures has been developed, the next step is to select a set of performance measures that are suitable for tracking performance toward specified sustainable development objectives. There are both CONTENT and QUALITY requirements to be considered when selecting performance measures. Screening the set of measures also helps to ensure that there are no costly measurement redundancies or gaps.

Content Considerations:

Sustainable development has been defined as "development that meets the needs of the present without compromising the ability of future generations to meet their own needs." This suggests that there are two main "content" considerations that sustainable development objectives and performance measures should address. These are:

present impacts of programs on human and ecosystem well-being, including positive, negative and potential unintended effects; and

known or reasonably likely future impacts of programs on human and ecosystem well-being.

Table 8 is intended to help users assess the candidate performance measures developed in previous steps. Each performance measure should address at least one of the above-noted "content" considerations. However, it is unlikely that any single measure will cover them all.

Have you screened your set of measures to assess whether the set covers the "content" considerations of sustainable development?

Table 8: A Screening Tool for "Content" Considerations

Performance Measures for Sustainable Development Strategies

LIST
Candidate Measures

Indicates Positive
Program Outcomes
(Y/N)

Indicates Negative Program Outcomes
(Y/N)

Monitors for Potential Unintended Program Outcomes
(Y/N)

Primarily Links to Human Well-being (Y/N)

Primarily
Links to Ecosystem well-being
(Y/N)

Focus on Present Outcomes
(Y/N)

Focus on Potential
Future Outcomes
(e.g. > 30 yrs.) (Y/N)

Measure 1
Y

N

N

Y

N

N

Y

Measure ‘n’
N

Y

Y

N

Y

Y

Y

If two performance measures address the same considerations, analysis may be undertaken with a view to eliminating one of the measures. Your objective should be to select the vital few measures that will minimize the burden of measurement at each level of the organization and provide the best information to decision makers on opportunities for performance improvement.

Quality Considerations

There is a high degree of consensus on the attributes of a good performance measure. Chapter 22 of the Auditor General’s December 1997 report summarized the attributes of a good performance measure as follows and concluded that good performance measures are meaningful, reliable and practical.

Exhibit 1 Quality Criteria for Performance Measures

Attributes Explanation

Meaningful

   understandable

clear (clearly and consistently defined)

context (explained)

concrete (measurable)

lack of ambiguity in direction

   relevant

relates to objectives

significant and useful to the users

attributable to activities

   comparable

allows comparison over time or with other organizations, activities or standards

Reliable

accurately represents what is being measured (valid, free from bias)

data required can be replicated (verifiable)

data and analysis are free from error

not susceptible to manipulation

balances (complements) other measures

Practical

feasible financially

feasible to get timely data

Table 9 below is intended to help users assess the quality of the performance measures selected for the strategy to determine their overall value for decision making.

Have you "screened" your measures against criteria for good performance measures?

Table 9: A Screening Tool for Quality Considerations

Performance Measures for Sustainable Development Strategies:
Screening Tool for Quality Considerations

Selected
list of performance measures that satisfy sustainable development content cirteria

Meaningful
(y/n)

Reliable
(y/n)

Practical
(y/n)

Measure satisfies content and quality criteria
(y/n)

Understandable

Relevant

Comparable

Measure 1
Y

N

Y

Y

Y

N

Measure ‘n’
Y

Y

Y

Y

Y

Y

Sustainable development performance measures should correspond to management objectives. Therefore, many of the performance measures developed at the program level in support of sustainable development strategies will be organization-specific. In the case of "cross-cutting" policy issues addressed by programs or common operations management issues that departments must confront, departments may have similar sustainable development objectives and performance measures. Co-ordination of objectives and measures under these circumstances may facilitate improved issue management, reporting and oversight of the government’s progress toward sustainable development.

Summary

You should now have:

documented the program objectives for sustainable development (table 4)

documented activities, outputs or other program responses necessary to achieve the objectives (table 5)

documented the requirements relative to those activities, outputs or other program responses that are necessary to achieve the desired results (table 5)

established performance measure(s) to track progress toward meeting the requirements (table 6)

STEP 9: Establish Accountability and Resources for Implementation

Rationale for this step: An accountability system formalizes the relationship between results, outputs, activities, and resources. It allows people to see how their work contributes to the success of the organization and clarifies expectations for performance.

Have you established accountability for implementation?

Table 10: Establishing Accountability for Implementation

Program
Objectives for Sustainable Development

Work Step 4

Responsible
Party(s) for Achieving
Objective

Activities, Ouputs or
Other Responses
Necessary to Meet
Objectives

Work Step 5

Responsible
Party(s) for
Managing
Activities or
Outputs and
Meeting the
Requirements

Performance Measure(s)

Work Step 8

Responsible Party(s) for Evaluating Measures

Objective 1 Response 1 Measure 1

Objective 2 Response 2 Measure 2

For each objective, have you estimated the resource requirements necessary to respond successfully?

Table 11: Identifying Resource Requirements for Implementation

Sustainable Development
Objectives

Activities, Ouputs or Other Responses Necessary to Meet
Objectives

Resource Requirements

Human

Financial

Other

CONCLUSION

A set of performance measures should support a broader explanation of performance results – a performance story – for managers and executives and for internal and external stakeholders. Performance information should explain how the resources committed to specific initiatives for achieving sustainable development objectives did or did not achieve the specified results. A set of performance measures will likely be required to provide a coherent performance storyline traced by Why, Who, What and How.

FEEDBACK

Language selection

Search and menus

Search

Developing Performance Measures for Sustainable Development Strategies

Getting Started

Organization of the Workbook

Section 1

Apply a Performance Framework

Establish Program Level Objectives That Contribute to the Strategic Objectives

STEP 1: Confirm Program Role

STEP 2: Identify the Key Program Activities and Outputs

STEP 3: Identify Program Stakeholders and Issues

STEP 4: Identify What the Program Aims to Accomplish

STEP 5: Identify Responses and Performance Requirements

Section 2

Establish Performance Measures

STEP 7: Establish Information Capabilities and a Baseline for Each Measure

STEP 8: Assess the Adequacy of Performance Measures

Main Program Activities and Outputs in Order of Significance (High, Medium)	Sustainable Development Issues		Stakeholder Groups (Affected Parties)
	Desired Program Effects	Undesirable Program Effects	Positively Affected	Negatively Affected
Activity 1
Output 2

	Referring to the "undesirable" program effects and the "negatively affected" stakeholder groups (Table 3): Have you established a desired long-term sustainable development outcome for each of these program activities or outputs?
	Document the positive effect(s) that need to be produced
	Have you established the near-term outcomes that can be expected to lead to the long-term outcomes?
	Explain WHO needs to take WHAT action(s) to produce the desired effects

Program Activities and Outputs in Rank Order of Significance (High, Medium)	Desired Sustainable Development Results (Objectives)
	Long-Term Strategic	Near-Term Intermediate
Activity 1
Output 2

	Have you defined the performance requirements necessary to achieve the desired sustainable development results?
	What level of compliance, participation, reduction, behavioural effect, etc., is required by whom, to achieve the result? By when is it required?

Objective(s) (from Table 4)	New or Modified Activities, Outputs or Other Program Response(s) Necessary to Achieve the Objective(s): (from Table 4)	Performance Requirements Relative to each Activity, Output or Other Response necessary to Achieve the Desired Results (Targets)

Objectives (from Table 4)	Activities, Outputs or Other Program Responses (from Table 5)	Performance Requirements (Targets from Table 5)	Potential Performance Measure(s)

Potential Performance Measure (from Table 6)	Units	Initial or Baseline Value
Measure 1
Measure ‘n’

	Performance Measures for Sustainable Development Strategies
LIST Candidate Measures	Indicates Positive Program Outcomes (Y/N)	Indicates Negative Program Outcomes (Y/N)	Monitors for Potential Unintended Program Outcomes (Y/N)	Primarily Links to Human Well-being (Y/N)	Primarily Links to Ecosystem well-being (Y/N)	Focus on Present Outcomes (Y/N)	Focus on Potential Future Outcomes (e.g. > 30 yrs.) (Y/N)
Measure 1	Y	N	N	Y	N	N	Y
Measure ‘n’	N	Y	Y	N	Y	Y	Y

Attributes	Explanation
Meaningful
understandable	clear (clearly and consistently defined) context (explained) concrete (measurable) lack of ambiguity in direction
relevant	relates to objectives significant and useful to the users attributable to activities
comparable	allows comparison over time or with other organizations, activities or standards
Reliable	accurately represents what is being measured (valid, free from bias) data required can be replicated (verifiable) data and analysis are free from error not susceptible to manipulation balances (complements) other measures
Practical	feasible financially feasible to get timely data

	Performance Measures for Sustainable Development Strategies: Screening Tool for Quality Considerations
Selected list of performance measures that satisfy sustainable development content cirteria	Meaningful (y/n)			Reliable (y/n)	Practical (y/n)	Measure satisfies content and quality criteria (y/n)
	Understandable	Relevant	Comparable
Measure 1	Y	N	Y	Y	Y	N
Measure ‘n’	Y	Y	Y	Y	Y	Y

Sustainable Development Objectives	Activities, Ouputs or Other Responses Necessary to Meet Objectives	Resource Requirements
		Human	Financial	Other