Introduction

Background

4.1 The Federal Sustainable Development Act requires the Minister of the Environment (renamed Environment and Climate Change) to develop a Federal Sustainable Development Strategy (FSDS). The Minister is required to report to Parliament at least once every three years on the federal government’s progress in implementing the FSDS. The Department, Environment and Climate Change Canada, is responsible for preparing this report. The Commissioner of the Environment and Sustainable Development must examine each FSDS progress report to assess the fairness of the information presented on the federal government’s progress in implementing the FSDS.

4.2 The Minister has issued three progress reports to date. The most recent is the 2015 Progress Report of the Federal Sustainable Development Strategy (released in February 2016), which relates to the 2013–2016 FSDS. The four themes of the 2013–2016 FSDS are

• addressing climate change and air quality,
• maintaining water quality and availability,
• protecting nature and Canadians, and
• shrinking the environmental footprint—beginning with government.

These themes had 8 goals, which were supported by 34 targets and 225 implementation strategies. The 2015 Progress Report provided performance information about the 8 goals and 34 targets.

4.3 The 2011 Progress Report (The Progress Report on the Federal Sustainable Development Strategy 2010–2013) described the systems and strategies needed to implement the FSDS and explained how results would be measured and shared in future reports. Because the 2011 Progress Report had been prepared less than one year after the 2010–2013 FSDS had been tabled, the report contained no information on the federal government’s progress in implementing the FSDS. Rather, it stated that subsequent progress reports would track the implementation of the FSDS and that a second and more substantive progress report would be tabled in the fall of 2012. Consequently, the Commissioner undertook no assessment.

4.4 The 2012 Progress Report of the Federal Sustainable Development Strategy described the progress in four areas: addressing climate change and air quality, maintaining water quality and availability, protecting nature, and shrinking the environmental footprint—beginning with government. Our assessment found that the report represented an important and useful step in communicating and accounting for progress toward the federal government’s sustainable development goals and targets. We concluded that while the report provided a sense of progress, it did not include sufficient information to fairly present the government’s progress in implementing the 2010–2013 FSDS.

Focus of the review

4.5 This review assessed the fairness of the information on the federal government’s progress in implementing the Federal Sustainable Development Strategy. As well, we considered whether recommendations from our work in the 2013 Fall Report of the Commissioner of the Environment and Sustainable Development, Chapter 8—Federal and Departmental Sustainable Development Strategies had been reflected in the 2015 Progress Report of the Federal Sustainable Development Strategy.

4.6 This document does not present the results of an audit; rather, the observations and recommendations that follow are the results from our review of the 2015 Progress Report.

4.7 We assessed the fairness of the information contained in the 2015 Progress Report using the same criteria that we used to assess the information contained in the 2012 Progress Report. We sought to determine whether the performance information presented was fair. Specifically, we reviewed whether the information was

• relevant (the information reported tangible and important accomplishments in context against objectives and costs),
• meaningful (the information described expectations and provided baselines against which performance was compared),
• attributable (the information demonstrated when and how the program made a difference), and
• balanced (the information presented a representative and clear picture of performance in a way that does not mislead the reader).

These criteria were supported by the same audit questions we used to assess the 2012 Progress Report. We did not assess the reliability and accuracy of the data used to report on the results in the 2015 Progress Report.

4.8 More details about the review’s objectives, scope, approach, and criteria are in About the Review at the end of this report.

Findings, Recommendations, and Responses

Assessing the 2015 Progress Report of the Federal Sustainable Development Strategy

4.9 Overall, we found that the information in the 2015 Progress Report of the Federal Sustainable Development Strategy provided a fair presentation of the government’s progress in implementing the 2013–2016 Federal Sustainable Development Strategy. It conveyed a general sense of progress made and also included specific information regarding progress on many of the 34 targets. However, the Progress Report did not always explain why progress had been slow in some areas. In addition, the Progress Report did not present information on how sustainable development had been considered in decision making.

4.10 This finding is important because sustainable development strategies allow the federal government to advance sustainable development and make decision making more transparent and accountable to Parliament. Periodic reports that present a fair picture of progress are fundamental to both the credibility and the impact of the strategies.

Reporting of the information in the 2015 Progress Report was fair, but information was lacking in some areas

4.11 We found that the information included in the 2015 Progress Report provided a fair presentation of the progress in implementing the 2013–2016 Federal Sustainable Development Strategy (FSDS). In particular, we found the following:

• Information provided for the 34 targets was relevant and reflected actions on initiatives designed to meet government priorities. However, the 2015 Progress Report lacked information on key risks and detailed cost information for targets.
• Information was meaningful given that, in contrast with the 2012 Progress Report, more targets included quantitative baselines, standards, or thresholds, allowing readers to assess progress. However, we found that when results differed from expected accomplishments, more information was needed to explain the differences.
• Information was attributable in that the report provided performance information for 85 percent of the targets for the 2012–2015 reporting period (the period since the last progress report in 2012). In many cases, charts, graphs, or other figures illustrated recent progress and trends.
• Overall, the information in the 2015 Progress Report was not always balanced. Although the progress report presented performance information in a neutral tone, it did not explain the causes for a lack of progress or the major challenges that exist for 33 of 34 targets.

4.12 Our analysis supporting this finding presents what we reviewed and discusses

• relevance of information,
• meaningfulness of information,
• attribution of information, and
• balance of information.

4.13 This finding matters because periodic reports that present a fair picture of progress are fundamental to both the credibility and the impact of the federal sustainable development strategies. Progress reports present a fair picture of progress when they are relevant, meaningful, attributable, and balanced.

4.14 Our recommendations in these areas of review appear at paragraphs 4.26, 4.39, and 4.56.

4.15 What we reviewed. We reviewed whether the information in the 2015 Progress Report was relevant by assessing progress information for the targets, information about risks, and information about costs. We reviewed the meaningfulness of performance information, the clarity and measurability of targets in the 2015 Progress Report, and the information provided by Environment and Climate Change Canada on extent of progress. We reviewed the attribution of performance information and progress that was reported since the last Progress Report in 2012. We also reviewed whether the performance information in the report was balanced and had been reviewed by a third party.

4.16 Relevance of information. We reviewed whether the 2015 Progress Report included

• progress information that focused on results and outcomes, initiatives, links to government priorities, the external environment, key risks (such as exposures to danger, harm, or loss, and reasons for the topic’s importance); and
• cost information to help the reader to understand the progress made.

4.17 We also looked at how the report reflected the following recommendation from our 2013 review of the 2012 Progress Report: “Environment Canada should work with other federal departments and agencies to include more consolidated financial information in future progress reports.”

4.18 We found that progress information for the targets included

• results and outcomes (unless the target was focused on process),
• initiatives to influence results,
• links to government priorities, and
• high-level discussions about the external operational environment and key risks (except for the six targets relating to greening government operations).

4.19 With the exception of the six targets relating to greening government operations that are internally focused, we found that all targets provided some basic information to help readers understand the external aspects of the topic reported. Nevertheless, we found that the report needed more detail to explain the significance of risks.

4.20 Information about the significance of risks is important, because it can influence stakeholders’ assessments and decisions and readers’ opinions. When available, the report should include information on key risks—such as those to human health—to help readers understand the importance of results.

4.21 The 2015 Progress Report contained some examples of targets that presented the significance of risks, notably on human health:

• Target 2.1: Outdoor Air Pollutants—the information included a graph illustrating the risk of death from heart or lung failure attributable to air pollutants.
• Target 2.2: Indoor Air Quality—the information indicated that radon is the second leading cause of lung cancer (smoking is number one) and the leading cause of lung cancer for non-smokers. Radon is a naturally occurring radioactive gas that cannot be seen, smelled, or tasted and that can get into homes undetected.

4.22 In one case, we found that information in the 2012 Progress Report characterizing human health risks had been omitted in the 2015 Progress Report. Specifically, target 2.3 of the 2012 Progress Report identified baseline levels of chemicals in the Canadian population in relation to the management of hazardous chemicals, which enabled tracking exposure trends relating to human health risks. The corresponding target in the 2015 Progress Report (target 4.8) omitted this information.

4.23 We found that the 2015 Progress Report provided some basic cost information on funding for programs for 53 percent of the targets (18 of 34). This percentage improved upon the 2012 Progress Report, which contained some cost information on funding for 39 percent of targets.

4.24 However, the cost information in the 2015 Progress Report did not include spending information for 30 of the 34 targets. The exception, Annex A, did break down spending in detail by department, theme, and year (2012–2013, 2013–2014) on the Clean Air Agenda.

4.25 Given its lack of cost information, the 2015 Progress Report reflected our 2013 recommendation only to a limited extent.

4.26 Recommendation. In future progress reports, Environment and Climate Change Canada should include more cost information on funding and spending, and when relevant, more specific information to explain key risks.

The Department’s response. Agreed. In collaboration with responsible departments and agencies, Environment and Climate Change Canada will include relevant financial and risk information.

4.27 Meaningfulness of information. We reviewed whether the 2015 Progress Report of the Federal Sustainable Development Strategy included

• a comparison of actual accomplishments with expected performance and whether differences were explained; and
• performance information, including baselines, that uses indicators that allow for the comparison of progress over time (trends).

4.28 We found that the performance information for 23 of 34 targets compared accomplishments with expected performance. However, the information for only 9 of these targets interpreted the differences.

4.29 As an example, information about Target 3.8: Marine Pollution—Releases of Harmful Pollutants did explain the difference between accomplishments and expected results. This target sought to reduce the number of releases of harmful pollutants by vessels by 5 percent from 2013 to 2016. The 2015 Progress Report revealed, however, that releases increased annually by an average of 20 percent from 2009–2010 to 2013–2014. The report attributed this rise to a 70-percent increase in marine patrol hours from 2009 to 2010. As a result, 97 percent more vessels were monitored in 2013–2014 than in the previous year. This higher monitoring captured releases that otherwise may have gone unreported.

4.30 We also observed that performance indicators for five targets had been updated since the last progress report in 2012. These changes can complicate comparing progress over time. Nonetheless, because the reporting framework for the Federal Sustainable Development Strategy had been in place for a relatively short period, it is reasonable that the Department would have reconsidered and updated some targets and indicators.

4.31 In our 2013 review of the 2012 Progress Report, we recommended that the FSDS targets be clear and measurable and allow for comparison with benchmarks where appropriate.

4.32 We found that Environment and Climate Change Canada modified 21 of the 34 targets between the draft 2013–2016 FSDS and the final strategy. These changes generally improved the clarity of the targets. In 12 of those 21 cases, the targets were also improved in one or more of the following ways so that they

• were more specific,
• had clear deadlines (they were time bound), or
• were measurable.

4.33 We also found that the 2013–2016 FSDS improved upon the 2010–2013 FSDS by attaching baselines to nearly all targets. This allowed readers to interpret performance information in the 2015 Progress Report in relation to a reference measure or the situation at a particular time.

4.34 For example, Target 3.1: On-reserve First Nations Water and Wastewater Systems in the draft 2013–2016 FSDS was to “increase the percentage of First Nations with water and wastewater systems with low risk rating by 2016.” This target was made clearer and more specific with added baselines in the final strategy to “increase the percentage of on-reserve First Nations water systems with low risk ratings from 27% [in 2009–11] to 50% by 2015” and to “increase the percentage of on-reserve First Nations wastewater systems with low risk ratings from 35% [in 2009–11] to 70% by 2015.” This improvement allowed a straightforward comparison of accomplishments with expected performance.

4.35 Given these observations on the clarity and measurability of the targets, we found that the 2015 Progress Report partially reflected the recommendation that we made in 2013.

4.36 We also reviewed whether the 2015 Progress Report reflected our previous recommendation on statements on the extent of progress. In our 2013 review of the 2012 Progress Report, we recommended that in “future progress reports, Environment Canada should clearly state the extent of the progress that has been made and whether targets have been achieved.”

4.37 The executive summary of the 2015 Progress Report included a table summarizing progress against the FSDS targets. This table allowed readers to understand quickly whether progress had been made. However, the report did not indicate whether progress was on track to meet targets or did not indicate whether progress had been achieved for 32 of 34 targets. The report did present, at the goal level, a general statement of some of the remaining challenges. As such, the 2015 Progress Report partially reflected the recommendation that we made in 2013.

4.38 In our view, readers would better understand the extent of progress if the report included a short statement indicating whether targets had been met or are on track to be met. This text should also describe any difficulties encountered in achieving results. For example, performance information relating to Target 3.2: Drinking Water Quality indicated that work was on track to achieve the target of developing 15 drinking water quality guidelines by 2016. The information also explained why achieving the target is possible.

4.39 Recommendation. In future progress reports, Environment and Climate Change Canada should clearly indicate whether targets have been achieved and whether progress is on track to meet targets. The reports should also specify any difficulties encountered in achieving results.

The Department’s response. Agreed. In collaboration with responsible departments and agencies, Environment and Climate Change Canada will improve how it reports on progress.

4.40 Attribution of information. We reviewed whether the 2015 Progress Report of the Federal Sustainable Development Strategy included

• the contribution made by various initiatives to achieve targets, notably presenting the accomplishments made since the last progress report (the period between 2012 and 2015);
• the contribution of key partners or other external factors; and
• data showing evidence that progress had been evaluated.

4.41 A progress report should present performance information that demonstrates how federal actions contributed to progress on the FSDS goals and targets. The report should also note the involvement and contribution of others, where applicable.

4.42 We found that the 2015 Progress Report detailed the contributions of various initiatives that related to all of the 34 targets. In addition, 32 of the 34 targets provided some information on the contributions of key partners and other external factors such as international agreements and partnerships.

4.43 We found that performance information was clearly presented for 29 of 34 targets for the 2012–2015 period. In 15 of those cases, the information included trends or progress over a longer time. In some cases, graphs or charts illustrated progress by year, allowing readers to compare progress since the last progress report in 2012 and from earlier years. For example, for Target 4.3: Terrestrial Ecosystems and Habitat Stewardship, the report provided data on how much habitat had been protected as of 2014. The information also included explanations of the degree to which certain habitats had been protected since as early as 1990.

4.44 We found that performance information for the remaining five targets did not clearly present information on progress since the last progress report. Availability of current data was a limitation for these targets; however, the reasons were not explained. For example, we found that performance information for Target 3.10: Agri-environmental Performance Metrics did not include current performance information. Rather, the progress information pertained to the period between 1981 and 2006, and there was no indication why current data was unavailable.

4.45 We found that for 29 of 34 targets, the performance information showed evidence that progress had been evaluated over time. However, in many cases the report presented this information generally at the goal level and not at the target level. The Canadian Environmental Sustainability Indicators program administers most of this performance tracking, which evaluates progress. Hyperlinks in the 2015 Progress Report allowed Canadians to access downloadable data and graphics online.

4.46 Balance of information. We reviewed the balance of performance information and whether a third party reviewed the 2015 Progress Report of the Federal Sustainable Development Strategy. We also reviewed whether the 2015 Progress Report used a neutral tone and presentation, and whether the report included

• positive and negative aspects of performance; and
• key objectives, major challenges, and significant performance information.

4.47 We also looked at how our related recommendation from our 2013 review of the 2012 Progress Report was reflected: “Environment Canada should ensure that future progress reports are fair and balanced, presenting progress to date and the remaining challenges.”

4.48 A balanced progress report presents a representative, clear, and complete picture, without misleading the reader. It uses a neutral tone and discusses both positive and negative aspects of progress as well as the challenges to meeting targets.

4.49 We found that the report generally presented performance information in a neutral tone and format for 33 of the 34 targets—an improvement over the 2012 Progress Report.

4.50 We found that 20 of 34 targets (59 percent) presented some negative aspects of performance alongside positive aspects.

4.51 We found that progress information for each of the 8 high-level goals presented major challenges pertaining to 21 of 34 targets. The major challenges to a target’s future achievement were provided only in one case, Target 3.8: Marine Pollution—Releases of Harmful Pollutants.

4.52 Explaining remaining challenges is particularly important when progress is clearly lacking. For example, Target 4.5: Marine Ecosystems stated that by 2020, 10 percent of coastal and marine areas will be conserved through a network of protected areas and other effective area-based conservation measures. Our audit in 2012 on marine protected areas found that about 1 percent of coastal and marine areas had been conserved. As of 2014, this number remained unchanged. The 2015 Progress Report did not address this lack of progress and failed to discuss why the government had not protected more marine areas.

4.53 Due to the limited information presented on major challenges remaining, the 2015 Progress Report did not fully reflect our 2013 recommendation for a fair and balanced report. Our recommendation on this issue appears in paragraph 4.56.

4.54 We also looked at how our related recommendation from our 2013 review of the 2012 Progress Report was reflected: To achieve a fair and balanced report, “Environment Canada should explore options for having the progress report verified by a third party or by an independent multi-stakeholder review panel before publication.”

4.55 We found that the Department had solicited and received third-party comments on early drafts of the Progress Report. As such, the 2015 Progress Report reflected our 2013 recommendation related to third-party review for Environment and Climate Change Canada in relation to the 2015 Progress Report.

4.56 Recommendation. Environment and Climate Change Canada should ensure that future progress reports are balanced by presenting additional information on positive and negative aspects of progress, and describing the remaining challenges to be overcome in order for the targets to be achieved.

The Department’s response. Agreed. In collaboration with responsible departments and agencies, Environment and Climate Change Canada will ensure that future progress reports are fair and balanced by presenting additional information on positive and negative aspects of progress, and describing the remaining challenges to be overcome in order for the targets to be achieved.

The 2015 Progress Report was missing important information on integrating sustainable development into government decision making

4.57 We found that, in contrast to the information provided in 2012, the 2015 Progress Report of the Federal Sustainable Development Strategy (FSDS) did not discuss either progress on integrating sustainability into government decision making or the results relating to the government’s commitment to strategic environmental assessments.

4.58 Our analysis supporting this finding presents what we reviewed and discusses

• integrating sustainable development into decision making.

4.59 This finding matters because one of the aims of the Federal Sustainable Development Act is to integrate sustainable development into government decision making. The 2013–2016 FSDS states that strategic environmental assessments seek to incorporate environmental considerations into the development of policies, plans, and programs on an equal basis with economic or social considerations, in order for decisions to be made in support of sustainable development.

4.60 In our 2013 review of the draft 2013–2016 FSDS, we stated that additional targets and implementation strategies related to integrating sustainable development considerations into government decision making would

• complement the government’s green procurement target,
• help ensure that the FSDS targets incorporate all aspects of decision making, and
• provide a basis for monitoring and reporting progress in this area.

4.61 Our recommendation in this area of review appears at paragraph 4.66.

4.62 What we reviewed. We reviewed whether the 2015 Progress Report contained information on federal government efforts to integrate sustainable development into government decision making, including the government’s commitment to strategic environmental assessment.

4.63 Integrating sustainable development into decision making. The 2013–2016 FSDS reinforced the government’s commitment to strategic environmental assessment as an important tool to integrate environmental considerations with social and economic considerations in government decision making. In our 2013 review of the draft 2013–2016 FSDS, the Commissioner of the Environment and Sustainable Development stressed the importance of including targets and implementation strategies related to integrating sustainable development considerations into decision making. These targets and strategies are important because they allow for monitoring and reporting progress in this area.

4.64 The Federal Sustainable Development Act acknowledges the need to integrate environmental, economic, and social factors into all government decisions. Given the importance of the matter, we considered whether the 2015 Progress Report reported on this topic as had been done in the 2012 Progress Report.

4.65 We found that the 2015 Progress Report contained no discussion of progress in integrating sustainable development into government decision making. Likewise, the 2015 Progress Report did not include performance information relating to the government’s commitment to strategic environmental assessment. In contrast, the 2012 Progress Report included a section on transparency and accountability, which contained information on progress toward the government’s commitment to strategic environmental assessments. The 2012 Progress Report also included examples of progress to update the strategic environmental assessment practices in 5 departments and agencies of the 26 required to contribute to the FSDS.

4.66 Recommendation. Environment and Climate Change Canada should ensure that future progress reports contain information on progress related to integrating sustainable development into decision making, using tools such as strategic environmental assessments.

The Department’s response. Agreed. Environment and Climate Change Canada will explore ways to report on how sustainable development is being integrated into decision making.

Conclusion

4.67 Based on our review of the 2015 Progress Report, we concluded that in general, the information provided a fair presentation of the progress made by the government in implementing the 2013–2016 Federal Sustainable Development Strategy. We noted instances where improvements could be made. Nothing has come to our attention that causes us to believe that the information is unfair.

4.68 The 2015 Progress Report provided information on progress, including a table summarizing key information. However, in most cases, the report did not indicate whether targets had been met or were on track to being met, nor did it always explain the lack of progress or the challenges in meeting the 34 targets. The 2015 Progress Report also lacked contextual and cost information for many of the 34 targets. In addition, the report did not present performance information related to integrating sustainable development into decision making.

4.69 Although the 2015 Progress Report reflected aspects of the recommendations from our 2013 Fall Report of the Commissioner of the Environment and Sustainable Development, Chapter 8—Federal and Departmental Sustainable Development Strategies, we concluded that the 2015 Progress Report did not fully reflect these recommendations.