This Web page has been archived on the Web.

1997 October Report of the Auditor General of Canada

Main Points

11.2 A number of recent government initiatives support a focus on results, but concerted effort is required to bring about change. This study sought to support this change by drawing together experience from Canadian government programs and some other jurisdictions that have made significant progress. We selected examples for the study that illustrate different contexts and approaches.

11.3 We found cases where managers were measuring performance, communicating results information internally and externally and using the information to improve results. Managers of these programs were able to point to improvements in the key outcomes that they and their ministers were trying to achieve for Canadians. They also found that managing for results improved their management practices, program activities and programs' credibility.

11.4 The way managing for results is implemented will vary depending upon the situation of a particular program, but we found a number of consistent themes. Senior management leadership and commitment at all levels of government are particularly important. Effective managers create an organizational climate that encourages managing for results. Also important is agreement on the results to be expected and how to measure them. Although difficult to achieve, such agreement is necessary in order to focus activities and permit an assessment of progress.

11.5 Continued progress in managing for results will require ongoing leadership and attention across government. In particular, decision makers need to show that results count, by asking about results and visibly using them to make decisions.

Introduction

Results Focus Is Needed

11.7 Meeting the expectations of Canadians for programs that work and finding more cost-effective means of delivering them requires managing for results . When managing for results, ministers, senior officials, managers, staff, central agencies and Parliament make decisions based on what a program is achieving for Canadians - the results that citizens value - and at what cost. In particular, there is a focus on outcomes, that is, the benefits realized. Putting this approach into practice means agreeing on expected outcomes, measuring progress toward them, using the information and reporting results.

11.8 Canadians benefit from receiving better services and programs, ministers benefit from providing better value on things that matter to the electorate, and managers and employees benefit from greater satisfaction in serving the public well.

A better balance between managing inputs and results is needed

11.10 Probity and prudence are important elements of the government's management framework, but more emphasis on accountability for results would allow the government to reduce unnecessary rules and regulations under which managers operate. This would give them greater flexibility to respond to their unique and changing circumstances. Increased accountability for results would encourage managers to give more attention to producing benefits for Canadians. A sensible reduction in rules and regulations would empower them with the authority to do so. This should be done without sacrificing attention to essential aspects of public sector management, such as due process and fairness.

The Time Is Right

Government is committed to managing for results

Our government is committed to delivering programs that work for the Canadian taxpayer. To better deliver programs and services, the federal Public Service must move from a culture of rules and processes to one that also focusses on innovation and results.

A number of government initiatives demonstrate and support a results focus

11.15 Better performance information for expenditure management. As reported in Chapter 5, Reporting Performance in the Expenditure Management System, the Canadian government has recently launched several initiatives to renew the system for planning and managing expenditures and reporting on results. All these initiatives directly or indirectly demand that departments set clear performance expectations, measure and report program performance, and use that information in managing programs.

11.16 Emphasis on service quality. Service quality is a tangible result that taxpayers have demanded and can immediately see. In 1990, with its public service renewal initiatives, the government announced its movement toward service standards, and confirmed its intentions in successive budgets. The public service Declaration of Quality Service Principles, issued in late 1995, affirmed the government's commitment to setting service standards. In September 1996, our chapter on service quality reported that implementation was slow. In response, later that year, the government publicly renewed its commitment to delivering high-quality service to Canadians. The establishment of service standards and reporting of progress toward their achievement are important dimensions of managing for results.

11.17 Alternative delivery mechanisms encourage a greater focus on results. The government's efforts to rethink the need for government intervention and to identify more cost-effective ways of delivering government programs have resulted in approaches that give a greater degree of independence to those delivering programs and that can involve third parties, such as private organizations and other levels of government, in delivery. These arrangements raise the issue of accountability for results, which is at times reflected in legislation or formal agreements, such as the legislation for the new Canadian Food Inspection Agency, federal-provincial agreements that give provinces more responsibility for employment training, and federal-provincial agreements for the Canada Infrastructure Works program. In partnership delivery arrangements, it is essential that all parties understand and agree on what results are to be accomplished with the new delivery arrangement, and know the extent to which the results are being achieved. Focussing on results can help overcome interorganizational barriers and help identify innovative approaches.

11.18 Information technology makes managing for results more feasible. Greater use of information technology makes it easier to collect and report on results information. Being able to easily compare information from different sources helps managers focus their activities more effectively.

Many other governments are changing to a focus on results

There is enough experience to identify lessons learned and best practices

Focus of the Study

• to propose and explain a management framework for encouraging greater progress toward managing for results;
• to identify lessons learned and best practices in moving toward managing for results; and
• to determine incentives and disincentives faced by departments and program managers in adopting a results-based management culture.

11.23 Cases included in the study. Our study sought programs that showed some federal experience in managing for results or that were well advanced in moving in that direction. We identified four such programs. We also examined one department that had been moving in this direction for a number of years. There may be other departments and programs that could also have served as examples of progress. We did not carry out a survey across government to find these illustrative cases, but sought them out through less formal inquiry.

11.24 The four programs are:

• North American Waterfowl Management Plan (NAWMP). The Plan is intended to address the decline in population of waterfowl in North America. It is a partnership program administered jointly through the International Plan Committee (U.S./Canada/Mexico), and implemented and funded through regionally based joint ventures involving federal, state, provincial, non-governmental organizations and the private sector. From 1986 to 1996, actual contributions toward the Plan activities in Canada totalled about $360 million, with almost half coming from the U.S. (U.S. partners contributed over $172 million, Canadian provinces $80 million, Canadian federal government $59 million, private sources $49 million).
• Canada Centre for Mineral and Energy Technology (CANMET). This federal government laboratory, part of Natural Resources Canada (NRCan), has about 700 employees and a budget of about $90 million to conduct and sponsor research for the economical, safe and environmentally responsible recovery and use of Canada's mineral, metal and energy resources.
• Travellers Program. This program of Revenue Canada's Customs Border Services is concerned with the international movement of people and their accompanying baggage as they enter Canada. It has almost 2,600 employees and budgeted expenditures of approximately $113 million.
• Investigation and Control (I&C). I&C is part of Human Resources Development Canada. It employs approximately 1,100 employees and uses a budget of about $55 million to prevent, deter and detect abuse and fraud by claimants of the Employment Insurance program. This chapter focusses on the I&C function. We did not assess the management of I&C activities bearing on other programs than Employment Insurance. Overall departmental progress in managing for results is reported in Chapter 17 - Human Resources Development Canada - A Critical Transition toward Results-Based Management.

• Department-wide accountability framework in Environment Canada. Environment Canada, with approximately 4,600 employees and expenditures of over $546 million, has developed a framework that describes the results expected from its three business lines. The Department has put in place a number of processes for implementing the framework.

11.27 Definitions used. The results of government programs are the products and services provided by these programs and the benefits and other effects that they produce. Programs deliver two kinds of results: outputs , the direct products and services produced by government activities, such as an Employment Insurance cheque or some requested information; and outcomes, the intermediate and long-term effects of those outputs on individual Canadians and our society.

Findings

A Framework for Managing for Results

"the government is committed to

• defining the results it wants to achieve;
• giving managers the resources, tools, information and guidance to achieve these results; and
• measuring and demonstrating actual achievements."

11.30 This structure for managing for results is consistent with our review of relevant literature and experiences in other jurisdictions and in Canada. This review suggests that organizations that successfully manage for results:

• foster an organizational climate that encourages managing for results;
• agree on expected results;
• measure actual results to improve performance; and
• effectively report performance.

Managing for Results Has Been Beneficial

Programs report improved outcomes for Canadians

11.34 As shown in the example in Exhibit 11.2 , the North American Waterfowl Management Plan reported securing and enhancing a large area of waterfowl habitat and can demonstrate a related growth in waterfowl populations. Favourable weather and rainfall have also contributed. Case 1 provides additional detail on how a community of common interests supported managing for this result and on how that community manages for results.

11.35 As another example, shown in Exhibit 11.3 , savings to the Employment Insurance Account generated by Investigation and Control (I&C) activities at Human Resources Development Canada have increased significantly. I&C officials attribute the increase to refocussed activities resulting from implementation of a savings performance indicator, and other factors, including increased resources, technological advances, and more severe penalties. Factors contributing to the implementation of this key indicator, and the implications for the I&C function, are described in Case 2 .

11.36 As shown in Exhibit 11.4 , managers of the Travellers Program reported that the percentage of travellers complying with laws and regulations on the importation of goods and the payment of taxes and duties has increased. Program officials believe that this increase comes from the use of information on results to focus activities better and from public communication of their refocussed efforts. In their view, refocussing their activities has improved compliance by making enforcement efforts more visible to travellers and by encouraging greater traveller co-operation through reducing the frustration that accompanies long wait-times at the border. Officials recognize that other factors have also played a role in increased compliance, including increased personal exemptions and declines in cross-border shopping. Case 3 describes how traveller compliance and satisfaction became issues that supported a transition to managing for results in the Travellers Program.

Progress in improving performance in other jurisdictions

11.38 The U.S. Veterans Health Administration reported the use of performance information to target the most important opportunities to improve health services to veterans. This practice has lowered the mortality rate for cardiac procedures by an average of 13 percent over the last eight years.

Establishing credibility

11.40 At Environment Canada, the development of a corporate accountability framework provides managers with the opportunity to demonstrate to others their achievements and how they contribute to meeting the Department's long-term objectives. Case 6 describes how collaborating within a corporate framework for managing for results benefits managers at Environment Canada.

Putting the Framework into Practice

Fostering the right organizational climate

• demonstrating commitment and leadership;
• using external and internal levers;
• creating supporting incentives;
• building expertise; and
• developing a capacity to learn and adapt.

11.43 The right climate for managing for results can be fostered at all levels of government. Credible government-wide support will encourage a focus on results, and a supportive corporate culture in a department will reinforce efforts of program managers, as in the case of Environment Canada.

11.44 Our study focussed mostly on the program level, where support is essential. As we noted in the Introduction, a number of initiatives, either government-wide in scope or supported by the central agencies of government, are contributing to the creation of a government-wide climate for managing for results. Exhibit 11.5 depicts the important organizational factors that encourage and support managing for results.

11.45 Strong senior-level leadership and continuing commitment. Managing for results takes hold when senior management visibly supports the approach and is poised to take advantage of opportunities to move toward a focus on results. Our cases and the review of experience of other jurisdictions suggests that senior management commitment and leadership is the most common and perhaps most important feature of successful managing for results. We found strong leadership shown through such actions as:

• ensuring that the organization's mission (or the program's strategic objectives) focusses on results and is accepted and communicated broadly;
• ensuring that progress in implementing managing for results practices is a key part of performance appraisals of managers (or staff);
• supporting experimentation and innovation;
• leading by example;
• visibly and regularly assessing progress being made toward managing for results;
• demonstrating sustained interest and personal involvement in the results management initiative;
• including performance expectations in managers' accountability accords or performance contracts, and using these documents in the annual performance appraisals of managers; and
• entering into partnerships with other departments, governments and non-governmental organizations to pursue a common interest in results.

11.47 Senior leadership was a particularly important element in our cases. A senior manager at CANMET initiated the focus on managing for results and actively sustained it over a number of years. The focus on managing for results in the Travellers Directorate was led by the efforts of the senior managers involved. In both cases, strong and persistent leadership was needed to overcome resistance to the approach.

11.48 Using external and internal levers. External and internal factors or events that arise can be used by committed managers as levers to encourage an organization to focus more on results. In the case of the North American Waterfowl Management Plan, a common interest in waterfowl and wetland conservation has brought a variety of parties together. It was consensus on desired results and the shared vision that allowed the Plan partnership to be formalized. At CANMET, a senior manager brought in to make the program more relevant to industry turned to managing for results as the approach.

11.49 The need to rationalize activities as part of restructuring and reorganizing can also create opportunities, although this was not the case in the Canadian programs we studied. However, to make use of the opportunities that these factors create, management has to recognize and be prepared to respond to them. The organization also needs the capacity to respond, which includes appropriate supporting incentives, expertise and adaptive capacity.

11.50 Supporting incentives. Traditional budget-driven and process-oriented management practices may be resistant to change because of the existing incentive structure. As suggested earlier, incentives in the past have often run counter to efforts at measurement and a focus on results. In CANMET, for example, resistance was encountered due to the traditional scientific incentives based on publication frequency. As well, resistance to specific results approaches adopted may arise from a conflict within an organization between goals, such as service and financial goals. This was the case in the Employment Insurance (EI) program at Human Resources Development Canada, where generating savings to the EI Account was not seen by some staff as consistent with working to increase the employability of clientele. Overcoming these sources of resistance requires attention to existing incentives, both tangible and intangible, and the development of incentives that support managing for results. We found that a variety of incentives can be used, including:

• ensuring that managers (or staff) are provided at least annually with an occasion to demonstrate what their programs have accomplished. For example, CANMET, Investigation and Control, and the Travellers Program use accountability accords as incentives. These agreements between levels of management specify the results to be achieved by managers over a certain time frame and are used in reviews of the manager's performance;
• providing needed management flexibility to program managers (or staff) to allow them to move toward managing for results. In CANMET, managers are given increased authority in return for being accountable through accountability accords for achieving performance expectations;
• supporting programs that supply good performance information. The Treasury Board Secretariat has supported Investigation and Control efforts to generate savings to the Employment Insurance Account by allocating resources tied to performance targets; and
• encouraging the development of intangible incentives in the organization (or program), such as recognition or rewards for managing for results.

11.52 Innovation will be needed to implement managing for results. This implies a certain amount of trial and error. It is important that the response to mistakes incurred while innovating is measured and balanced and does not serve as a disincentive to further needed innovation. It is necessary to learn from errors and to move on.

11.53 Expertise in measurement concepts and practices. Management and staff may not immediately understand how measurement is best implemented, and may not fully understand, without advice, the implications that results-based measures have for managing operations. Unless managers and staff become comfortable with measurement, resistance to managing for results is likely. Management of the Employment Insurance program's Investigation and Control function found, for example, that savings indicators were not implemented as rapidly as they could have been because managers initially did not see the measure's implications for deciding which cases to investigate.

11.54 Consequently, managing for results requires training and communication on key concepts, as well as the exchange and sharing of experience, knowledge and best practices. In a report on progress in implementing the U.S. Government Performance and Results Act , the U.S. General Accounting Office stated: "To make the most of results-oriented management, staff at all levels of the organization must be skilled in strategic planning, performance measurement and the use of performance information in decision making. Training has proven to be an important tool for agencies that want to change their cultures."

11.55 Adaptive capacity. Maintaining the capacity to achieve program results requires a learning organization, one that stays relevant and responsive to the changing needs of clients and the general public. Managers need the capacity to prepare themselves to cope with emerging challenges. The proactive monitoring of the business environment for emerging challenges and their potential impact on programs and program management is part of this capacity. Determining the effective responses involves systematically reviewing past experiences, learning lessons from past mistakes, and taking action to avoid repeating the same mistakes. The North American Waterfowl Management Plan, for example, operates on a 15-year planning horizon and is updated every five years. Evaluations test the assumptions of the Plan and provide information on the effectiveness of specific approaches.

11.56 The changing nature of government programs constitutes a significant barrier to implementing and maintaining a focus on results. Because of this, implementing practices for managing for results is never really finished. Restructuring in Natural Resources Canada led to CANMET being split into two separate branches in 1995, at which time its functions were joined with related policy functions. Due to the splitting of CANMET, and to other reorganizations, the business plans had to be restructured and only recently reflect CANMET's orientation toward managing for results as clearly as they previously did.

Agreeing on expected results is key

• objectives, especially outcomes;
• performance indicators; and
• specific performance expectations.

11.59 As illustrated in Exhibit 11.8 , CANMET has established internal agreement on expected outcomes through management retreats, discussions with management and staff and the use of accountability accords between executives and managers. Committees of directors general provide a forum for sharing experiences and practices and the exchange of information. Agreement outside the Department is fostered through the use of advisory committees involving external stakeholders.

11.60 Program objectives are the first step. Our cases provided good examples of clear statements of the results to be accomplished. Program objectives were stated as short-term, intermediate or ultimate outcomes or results to be accomplished, rather than as activities to be carried out.

11.61 Managing outputs is a necessary part of ensuring the efficiency and effectiveness of a program. However, outcomes represent the true value resulting from the investment of tax dollars. Therefore, managing outcomes, as well as outputs and activities, provides greater assurance that the intended value, reflected in the organization's mission and overall objectives, will be achieved. To accomplish this, there needs to be agreement on what outcomes are implied by the mission and objectives.

11.62 Appropriate performance indicators are agreed to. Results can be measured in many ways, using many different kinds of information. A performance indicator is the measure actually used to assess a specific aspect of performance. No single indicator is adequate, and choosing the best set of performance indicators is central to ensuring that the right results are being measured. Without agreement on a balanced set of appropriate indicators, there is a risk that inappropriate performance will be encouraged. A balanced set of indicators will typically include indicators of both outputs and outcomes that reflect the key aspects of program performance.

11.63 For example, in developing a measure of the success of its efforts to detect and deter fraud and abuse, Investigation and Control management noted risks to focussing on "direct savings"- the total value of overpayments established and related penalties imposed. Inappropriate behaviours could be encouraged, such as allowing cases to "age" so that a larger amount could be recovered. Also, a focus on direct savings could have detracted attention from preventing fraudulent claims. To address these concerns, management added "indirect savings" - an estimation of the amount of inappropriate payments prevented through early intervention in a case. Investigation and Control also tracks a small number of secondary indicators, such as return on investment (the ratio of the total savings generated to the total I&C operating costs) to help ensure that its efforts remain cost-effective.

11.64 Agreement on clear and concrete performance expectations. Performance expectations describe the desired level of performance. Without agreement on clear and concrete performance expectations, it is difficult to know what level of performance is expected and to assess the extent to which expectations have been met.

11.65 Expectations are often expressed as targets. To be meaningful and to motivate staff to perform better, performance targets are frequently set to be challenging, but attainable. The North American Waterfowl Management Plan set targets for waterfowl populations and habitat conservation based on the partners' collective perceptions that population levels of the early 1970s were healthy and met public needs.

11.66 The targets can provide the basis for a performance contract between subordinates and their supervisor. The use of an appropriate participatory or consultative process in developing and implementing systems to measure program performance helps managers and staff to reach agreement on the system components, and to secure their commitment to use them. Such a consultative process helps to draw on the variety of skills and experiences of managers and staff and to develop sound and balanced results expectations. Customs Border Services of Revenue Canada, of which the Travellers Program is part, undertakes a joint planning process with its regions, producing accountability contracts between the assistant deputy minister (ADM), regional ADMs and regional directors. The accountability contracts specify expected performance and are the subject of discussion and input from field staff.

Performance Is Measured to Improve Results

• measuring and reporting performance;
• using performance information to refine, focus and improve activities; and
• reviewing and updating indicators and measures.

Results are measured and reported

11.69 Exhibit 11.9 illustrates that the Travellers Program uses ongoing measurement and periodic studies to assess changes in performance and progress toward outcomes. It also shows that the information is reported to all levels inside the organization for use in decision making and management, and is also reported to outside users.

11.70 A reluctance to measure and report outcomes has at times been a barrier to measuring and reporting results. Managers may be reluctant to measure and report outcomes, because these are often influenced by factors outside the control of the program and are therefore not under the control of management to the same extent as are outputs. Management will have direct control over such outputs as the number of inspections conducted, and the number of reports produced, but considerably less control over the impact these have on the behaviour of program clients and the resulting consequences for Canadians in general. The potential clients of the information may question the extent to which reported results can be attributed to program activities, which can contribute to management's reluctance to measure and report outcomes. Yet it is outcomes that are the most important indicator of the value received from public funds.

11.71 One lesson that has been learned is that a number of approaches to measurement may be needed. Outcome indicators will generally tell only how well the specific characteristic being measured is doing, not necessarily what the program's contribution has been. For this reason and other reasons, a number of approaches will be necessary.

11.72 Results can be measured in several ways - through ongoing measurement systems, periodic evaluation studies, occasional reviews, ad hoc cost/benefit analyses, client surveys, benchmarking comparisons, or long-term longitudinal studies - as appropriate to the nature of the program and the type of indicators used to measure performance. A combination of approaches is often necessary. Both quantitative and qualitative measures have a role.

11.73 Managers need to know more than simply the results. Measurement, perhaps through evaluations, should help managers to understand the extent to which the activities of their programs actually contribute to desired objectives, in relation to the role played by other factors.

11.74 In some instances, the required information may already be available. In other instances, additional or new information collection practices will have to be implemented. At the time of implementing the NAWMP, waterfowl populations were already being monitored on an annual basis. However, measurement of the area of habitat protected had to be implemented to monitor progress toward meeting targets for this important result.

11.75 Another lesson that has been learned is that it is important to limit measurement and reporting to a restricted number of indicators. Indicators or measures of performance are most useful when they reflect the key dimensions of program performance. Since outcome, impact or effectiveness indicators are relatively less common than other types of performance indicators in government departments, and because of their crucial importance to managing for results, we have focussed on such types of indicators in this study.

11.76 The number of indicators chosen should be manageable, relevant and appropriate to the needs of the users of such information. Managers in the Investigation and Control function and the Travellers Program found that having too many indicators was a barrier to their use. Managing for results took hold when fewer key indicators were used.

11.77 Credibility is important. Reliable procedures for collecting data and assessing results against expectations are essential for providing credible and understandable information to program managers. This may involve specific assessments of the reliability of information. Reliability stems from how the indicators are measured and from the quality of the information used. Assuring credible information may require formal assessment of measurement procedures and data. In discussions between the Treasury Board Secretariat and Human Resources Development Canada for additional resources to obtain additional savings through Investigation and Control activities, the credibility of estimates of direct and indirect savings was paramount. Credibility was aided by conducting an evaluation of these indicators, independent of program management.

Performance information is used to improve performance

11.79 Improved performance requires that, in addition to being reported and used externally, information be used internally to improve related management practices and program activities, as well as to support decision making. In our cases, we found a number of instances where results information was used to improve departmental practices, including the following:

• Improve planning. In 1994, the NAWMP was reviewed and updated based on results achieved, weaknesses and challenges. The updated plan implemented several recommendations regarding bird populations, habitat and supporting policies. Evaluation of the results of specific Joint Ventures is used to select the most effective habitat conservation methods. In the Prairie Habitat Joint Venture, a computerized planning model uses performance data to forecast costs and results.
• Focus activities better. Investigation and Control uses performance information to focus activities on areas likely to yield the greatest savings.
• Assess policies, practices and regulations. Investigation and Control has recently set up a unit in order to use performance information for assessing the implications of proposed changes to policies and practices. Environment Canada uses waterfowl population counts to review and modify hunting regulations.
• Assist in resource allocation or reallocation. Customs Border Services uses information on traveller compliance as one of the inputs to a computer model used to calculate staff required for border offices to meet performance targets and service standards. The Department has used traveller compliance levels and service quality indicators, along with other performance information, to support requests for and obtain additional resources. Environment Canada is working to link its results information to expenditure management, to aid in making difficult program choices.
• Demonstrate accountability between levels of management and assist in performance appraisal. In CANMET, Performance Feedback Reports compare achievements against performance expectations included in accountability accords. These are discussed by supervisors with their subordinates during mid-year and year-end reviews and annual performance appraisals.
• Monitor for problems and correct them. Customs Border Services uses information on traveller non-compliance to identify border offices where increased inspection and enforcement are necessary.

Indicators and measures are reviewed and updated

11.82 Improvements may be needed to keep pace with a changing environment and with needs of the public. New program policy and features, and changes to priorities, budget levels, program delivery mechanisms, or organizational structure can quickly outdate a performance information system, affecting its credibility among users. Consequently, the relevant components of a performance information system must be modified to make the system relevant to prevailing circumstances, and to maintain its validity and credibility.

11.83 Based on experience gained within a program, changing circumstances, and experience gained in similar programs elsewhere, the performance indicators, expectations, and measurement and communication strategies used will need periodic review and improvement. The best strategy is to begin with a reasonable set of indicators and measures, and to continue to refine and develop them with experience.

11.84 Improvements may also be needed to adapt to changing levels of performance. With experience over time, an organization learns to perform better. Under these circumstances, performance targets used previously may no longer be challenging or realistic. Consequently, performance expectations need to be revised periodically to better reflect an organization's current operational capacity.

11.85 Review of the NAWMP reflects the need to adapt both to changing needs and changing levels of performance. The Plan, originally conceived of as a "duck plan", appears to have been successful in increasing waterfowl populations. Based on its success and current conditions, the 1994 review resulted in revision of waterfowl population goals. The Plan has also been expanded to include multi-species and biodiversity objectives to support the broader goals of biodiversity conservation, sustainable development and integrated resource management. In particular, management believes that the Plan's reflection of broader benefits and values of wetlands and associated habitats will help to maintain and broaden the Plan partnership.

Conclusion

11.87 Managing for results benefits Canadians by contributing to better program performance. It also benefits public service managers by adding credibility to their efforts and assisting them in many of their management responsibilities, such as planning, decision making, responding in a timely fashion to queries, focussing activities and identifying and finding solutions to problems. And finally, it benefits ministers by focussing programs on the benefits they want to achieve for Canadians.

11.88 Past experience shows that shifting the focus from managing inputs to managing for results is not easy, but it can be done. The change takes time, usually about four to five years. Key to managing for results is persistence and the creation of a receptive climate, both across government and within individual programs and organizations.

11.89 Creating the climate presents challenges. Barriers are created by existing incentive systems that reinforce the management of inputs and activities, and by conflicting goals and public sensitivities. Key among these barriers is the desire to avoid risks. Overcoming these barriers requires strong leadership that can recognize the opportunity created by significant events in the organization or its environment and by communities of common interests in external partnerships. In organizations that manage for results, this leadership is shown by senior managers who overcome barriers by recognizing and communicating the value of information on results, consulting and seeking agreement on expected results and their measurement, providing supporting incentives, ensuring that the required expertise is developed and ensuring that risks are dealt with rather than avoided entirely.

11.90 Such change does not come without costs. Nonetheless, many of those who are managing for results see the costs of doing so as "part of doing business", entailing no additional costs beyond that of the management practices replaced.

11.91 However, where the necessary supporting systems are not in place, costs will be incurred by their implementation. Careful consideration needs to be given to ensuring that the costs of the measures chosen are reasonable in relation to their benefits. The cost of not managing for results is less efficient and less effective programs.

11.92 Experience with managing for results is not yet widespread in the federal government. Ensuring continued progress will require ongoing attention and leadership across government, as reflected in efforts to improve reporting to Parliament and to encourage the sharing of experiences among government departments. It will also require internal and external decision makers to show that results count in good management by asking for information about results and by visibly using it as a basis for their decisions.

11.93 As leadership changes and as programs and their environments change, progress that has been accomplished can come undone. For this reason, an organization that manages for results is a learning organization - one that continually monitors its environment and learns from its experiences and past performance.

Treasury Board Secretariat's response: We are pleased that the Auditor General has undertaken a study on managing and accounting for results in the federal government. We agree that establishing a climate within government that balances incentives for results and good management practices is important and that it will take some time to become fully realized. Essential steps are to define, measure and report on results.

The government initiatives to strengthen the management focus on results include: changes in the Expenditure Management System; improved reporting to Parliament; the development of government performance indicators; and modernizing comptrollership.

We look forward to working with the Auditor General to identify good practices and to communicate them to departments, Parliament and Canadians.

About the Study

Our objectives were:

• to propose and explain a management framework for encouraging greater progress toward managing for results;
• to identify lessons learned and best practices in moving toward managing for results; and
• to determine incentives and disincentives faced by departments and program managers in adopting a results-based management culture.

The study looked at progress in moving toward managing for results. We reviewed management practices that create an organizational culture that fosters managing for results, agreeing on expected results, measuring results, using information on results to improve performance and effectively reporting results. We concentrated on cases in several jurisdictions where results-based management practices are in use, or where initiatives are well under way to move toward managing for results. We reviewed their experiences and lessons learned regarding incentives, facilitators or enablers, and disincentives, inhibitors or constraints.

We looked at experiences in five Canadian federal government programs: The North American Waterfowl Management Plan jointly administered by Environment Canada, the Canadian Centre for Mineral and Energy Technology at Natural Resources Canada, the Travellers Program at Revenue Canada, Environment Canada's Department-wide accountability framework and the Investigations and Control Function of the Employment Insurance Program at Human Resources Development Canada.

We also reviewed progress in other jurisdictions, including U.S. federal government agencies complying with the Government Performance and Results Act of 1993 on a pilot basis, and progress in provincial or state governments, notably Alberta and a number of U.S. states, and in some local governments.

As this was not an audit, we did not verify the reliability of results information reported by any of the programs or organizations chosen as examples for this study.

Criteria

Because this was a study, we did not compare practices against a predetermined set of criteria. Rather, in conducting the study, we identified a set of principles that constitute a framework for managing for results, and assessed their applicability to the Canadian context.

Approach

We reviewed the relevant literature on the experiences in other jurisdictions, conducted on-site interviews with officials in Alberta; Portland, Oregon; Sunnyvale, California; and Texas, and reviewed related documentation. We conducted interviews with managers in four Canadian federal departments and reviewed related publications and documents. We also conducted interviews with the Treasury Board Secretariat on progress toward managing for results in these departments and on government-wide initiatives related to progress made.

In addition, we drew upon the evidence and findings from the April 1997 audit of Reporting Performance in the Expenditure Management System, conducted jointly with this study.

Study Team

Stan Divorski
Leslie Levita
Gerry Nera
Frances Smith
Shiv Sundaram
Mary-Louise Sutherland
Tom Wileman

For information, please contact John Mayne, the responsible auditor.