1994 Report of the Auditor General of Canada
Chapter 9—Science and Technology—Overall Management of Federal Science and Technology Activities
The government has committed itself to managing its science and technology portfolio more strategically
Parliamentarians have no basis to hold the government accountable for its spending on science and technology
Assistant Auditor General: Robert R. Lalonde
Responsible Auditor: Richard Flageole
9.1 The government recently committed itself to managing its science and technology portfolio more strategically, announcing its intention to "put in place a true strategy, one with real priorities, real direction and a real review of results." Previous efforts to do so in the last 30 years have failed.
9.2 There has been much activity but few results. Previous government-wide reviews have not produced results-oriented action plans that lay out priorities to meet Canada's economic and social needs and opportunities. Some of this lack of progress can be attributed to a lack of overall government-wide leadership, direction and accountability for implementing desired changes.
9.3 While establishing real priorities will be difficult, it is essential. There are not sufficient resources to meet all needs. Government spending on science and technology must be focussed on areas with the greatest need, opportunity and potential pay-back. The challenge is to determine what these areas are and to then actually implement the necessary changes.
9.4 Successful development and implementation of a true strategy will require the joint effort of federal departments, provinces, industry and universities, and persistence and leadership by Cabinet. This may require a revised structure for science and technology in the federal government.
9.5 The government needs a framework and indicators to monitor Canada's overall performance in science and technology and to determine the success of its own efforts to support science and technology. The indicators must reflect both the federal government's objectives for its own agenda and larger national objectives that consider all public and private sector stakeholders in science and technology.
9.6 Parliamentarians have no basis on which to assess whether the government's expenditures on science and technology reflect Canadian needs and opportunities, and to hold the government accountable for results.
9.7 Parliamentarians can play an important role in ensuring that an appropriate, balanced and workable science and technology strategy is developed and implemented.
9.9 Technology plays an important role in achieving economic goals in socially acceptable ways. People are now more aware of sustainable development. Companies soon will be expected to meet environmental standards in order to be competitive in the global economy, just as they are already expected to meet quality standards. Science and technology can help to meet environmental standards related to such things as mine tailings, forestry practices and industrial waste.
9.10 Economic growth is important to maintain and improve our standard of living. But science and technology go beyond contributing to economic growth; they contribute to our quality of life. For example, methods to test the food that we consume, means of improving the safety of our transportation modes and the technology available to improve health care quality are all supported by various activities in science and technology, in both the public and private sectors.
9.11 In many cases, it takes years before the discoveries from research are developed to the point where they are commercialized or incorporated in regulations. A nugget of new knowledge may lay dormant, perhaps for decades, until other advances in technology require that knowledge. Then it may become a critical factor in technological advancement.
9.12 The federal government allocates a significant level of resources to science and technology activities. Its policies and the deployment of its resources influence Canada's ability to prosper through innovation. Effective interaction between the research community (including universities) and the private sector will contribute to Canada's economic well-being. Given the potential benefits and the extensive costs involved, the successful management of federal science and technology activities is of profound significance.
Significant resources are allocated to science and technology activities9.13 In 1993-94, the federal government spent approximately $6 billion on science and technology activities. In addition, companies in the private sector claim approximately $1 billion in tax credits each year for research and development. The expenditures of $6 billion represented 11 percent of total government spending, excluding expenditures for such purposes as public debt charges and payments to persons and other levels of government.
Who spends the federal money and where is it going?9.14 Federal science and technology funds are allocated primarily to federal organizations, industry and universities. Exhibit 9.1 provides a breakdown of science and technology expenditures for 1993-94, by performing sector. Federal organizations accounted for approximately 59 percent of total expenditures, employing approximately 35,000 people.
9.15 Federal science and technology activities support a variety of missions and mandates, including health and safety, protection of the environment, communications and social and economic development. These activities are divided into two principal components: research and development, and related scientific activities.
9.16 Research and development refers to creative work undertaken in a systematic way to increase the stock of scientific and technical knowledge and to use that knowledge in new applications. Related scientific activities are those that complement or expand research and development by generating, disseminating and applying new scientific and technological knowledge. They include, for example, the gathering of scientific data through such means as geological and fisheries surveys, and the maintenance of meteorological records.
9.17 Of the $6 billion budget for science and technology in 1993-94, the federal government spent approximately $3.5 billion or 59 percent on research and development. Of that amount, $1.7 billion was spent in federal organizations and the remainder was paid to outside parties. The federal government is consequently a major player in Canadian research and development activities. Exhibit 9.2 shows who funds and who performs research and development in Canada.
9.18 Although almost 60 distinct federal organizations are involved in science and technology activities, 17 of them spend over 90 percent of the resources. The federal organizations spending the most on science and technology include the three granting councils ($860 million), Environment Canada ($666 million), the National Research Council ($512 million), Natural Resources Canada ($476 million) and the Canadian Space Agency ($422 million). Exhibit 9.3 provides a breakdown of federal spending on research and development and related scientific activities for the 17 most important organizations, for 1993-94.
How much research and development is Canada doing compared with other countries?9.19 The level of national expenditures on research and development as a percentage of gross domestic product is widely recognized as a significant indicator of the relative creative effort of a country. Exhibit 9.4 provides a comparison by the Organization for Economic Co-operation and Development (OECD) of research and development spending among its member countries for 1991. In percentage of gross domestic product, Canada ranked 14th among the 24 OECD countries; its effort in research and development was lower than that of all G-7 countries except Italy.
Audit Objective and Scope9.20 The overall objective of the audit was to assess whether the federal government has established clearly what it wants to achieve with its science and technology activities and whether the activities are managed in a way to yield maximum return.
9.21 In our 1981 audit of research and development activities, we focussed on the management of activities in specific departments and agencies. For this audit, we adopted a broader approach, in which we also examined the central machinery in place for the overall direction and co-ordination of federal efforts, the management of scientific personnel and the provision of tax incentives to industry for research and development. We did not examine how grants and contributions for science and technology are administered by the three granting councils and by government departments and agencies.
9.22 The results of our audit are presented in four interrelated chapters. This chapter outlines our conclusions on the federal government's overall management of science and technology activities and addresses some government-wide issues. Chapter 10 describes our audit of the management of those activities in a sample of federal departments and agencies. Chapter 11 deals with the management of scientific personnel. Finally, our observations and recommendations on the program of tax incentives for scientific research and experimental development are presented in Chapter 32.
9.23 In this chapter we report on the mechanisms used to establish priorities and direction, to co-ordinate activities among the various stakeholders, to measure results and to inform Parliament on the overall performance of the government in managing its $7 billion annual investment in science and technology. Overall, we expected to find strategic direction, priorities and co-ordination of federal science and technology activities. We also expected to find that results from these activities were assessed and communicated to Parliament in a timely manner. The audit observations and recommendations in this chapter are consequently addressed to the federal government as a whole and not to any of its specific departments or agencies. Since no specific department or agency is responsible for the overall policy direction and co-ordination of science and technology activities, we were not provided with responses to our recommendations. We expect the government to address our recommendations as part of the federal science and technology review currently being carried out, and described in paragraphs 9.24 to 9.27 of this chapter.
Observations and Recommendations
Deciding What to Do: A Long-standing Challenge
The government has committed itself to managing its science and technology portfolio more strategically9.24 The need to improve our economic performance and to create jobs was a major issue in the last federal election. Canada's lagging performance in using science and technology to these ends, as well as the lack of strategic direction and focus for the government's efforts, were particularly noted.
9.25 In the February 1994 Budget speech, the government committed itself to managing its science and technology portfolio more strategically, announcing its intention to " put in place a true strategy, one with real priorities, real direction and a real review of results." The Minister of Industry was asked to put forward a paper on science and technology, clearly stating the government's priorities, to set the stage for an intense national dialogue leading to a new national strategy.
9.26 On 28 June 1994, the Minister of Industry and the Secretary of State for Science, Research and Development launched a major review of federal science and technology activities. The purpose of the review is to "help determine how federal spending in science and technology can best be applied to creating economic growth and jobs within the context of sustainable development, enhancing the quality of life and advancing knowledge."
9.27 The review has three main components: an internal review of activities of federal departments and agencies; an independent assessment by the National Advisory Board on Science and Technology; and consultation with interested Canadians. An Interdepartmental Secretariat has been established in Industry Canada to co-ordinate the process. Three task groups are responsible for the internal review, focussing respectively on wealth and job creation in the context of sustainable development, the enhancement of quality of life and the advancement of knowledge. The task groups comprise representatives from federal departments and agencies conducting science and technology activities in these fields. The review is planned for completion in early 1995.
Previous efforts to implement a science and technology strategy have failed9.28 The issues of strategy, priorities, direction and results of science and technology activities are not new and this is not the first time the government has attempted to resolve them. In fact, they have been subjects of discussion and controversy for more than three decades. Exhibit 9.5 provides a brief summary of some of the numerous past studies and initiatives that dealt with the matters currently being addressed by the government. Considering its track record, we believe that the government faces a tremendous challenge in developing and implementing a new science and technology strategy.
9.29 Policy direction, priorities and co-ordination of federal science and technology activities were addressed extensively in the 1963 report of the Royal Commission on Government Organization (Glassco). The Commission noted the government's lack of vision for those activities and criticized the central machinery then in place for policy determination. It raised serious concerns about the lack of co-ordination and the piecemeal expansion of government scientific activity after the end of World War II, and concluded that Canada's scientific policy at the time was the result, not the cause, of growth in the many scientific activities undertaken by government.
9.30 Following the Glassco Commission, a Science Secretariat was established as part of the Privy Council Office in 1964. Then, in 1966, the Science Council of Canada was established. In 1968, the Council issued a discussion document entitled Toward a National Science Policy for Canada , proposing a series of national objectives and a structure for organizing new undertakings in Canadian science. The debate on science policy continued.
9.31 In 1967, under the chairmanship of Senator Maurice Lamontagne, the Senate Special Committee on Science Policy launched a thorough examination and assessment of science and technology in Canada. The Committee addressed many of the questions being reviewed today. It made 45 recommendations regarding targets and strategies and 27 additional recommendations for the organization of federal science and technology activities.
9.32 In brief, the Committee called for both sectoral science policies and an overall science policy that would encourage industrial innovation and contribute solutions to social problems. It also stressed the need for evaluating federal scientific research to enable the government to maximize the return on its investment and shift resources to new needs and opportunities as they present themselves. It concluded that a coherent organizational system must replace the conflicting pattern of responsibilities then in place.
9.33 A number of the recommendations of the Lamontagne Committee regarding the organization of government were acted on throughout the 1970s. In 1971, the government appointed a Minister of State for Science and Technology, who was made responsible for the overall formulation of policy and co-ordination of activity. During that period, the government also established the Natural Sciences and Engineering Research Council and the Social Sciences and Humanities Research Council as separate entities.
9.34 Between 1971 and 1983, the Ministry of State for Science and Technology made various unsuccessful attempts to elaborate some kind of science and technology policy. However, no clear science policy was put in place and the government's efforts were still piecemeal.
9.35 In the years following the 1984 election, there was a concerted effort to develop a science and technology policy and strategy. Numerous reviews and consultations took place from 1984 to 1991. A key advancement was the signing of the National Science and Technology Policy in March 1987 by the federal, provincial and territorial governments. A Council of Science and Technology Ministers was also established to serve as a consultation and co-ordination mechanism between the two levels of government.
9.36 In the same year, the federal government issued a document called InnovAction - The Canadian S&T Strategy , which identified five key sectors for intervention. It also issued a document known as A Decision Framework for Science and Technology , which outlined principles, objectives and guidelines for the management of federal activities, the roles of federal players and a process for annual reporting to Cabinet on science and technology activities.
9.37 It was also in 1987 that the National Advisory Board on Science and Technology (NABST) was created to advise the Prime Minister on the government's overall policies and priorities in science and technology. Members are leaders in the business, education and labour communities. The Board was, and still is, chaired by the Prime Minister. In 1988, a new department, Industry, Science and Technology Canada, was formed by merging the Department of Regional Industrial Expansion with the Ministry of State for Science and Technology. Another organizational move in the late 1980s was the formation of a House of Commons Standing Committee on Research, Science and Technology.
9.38 Another important development came in August 1989 with the release, by the National Forum of Science and Technology Advisory Councils, of a document entitled The Halifax Declaration: A Call To Action . That document laid out five broad issues where common action was needed. The Council of Science and Technology Ministers subsequently issued, in 1991, a National Science and Technology Framework for Action . It was aimed at providing direction for meeting science and technology imperatives and proposed a long-term action plan that identified specific goals and fundamental principles to guide government efforts. It also noted that a range of indicators must be in place to assess those efforts and to measure progress, and it proposed specific indicators of inputs and outputs.
9.39 Parallel to those initiatives, the National Advisory Board on Science and Technology issued a series of reports addressing various strategic and organizational issues. One of the reports, Revitalizing Science and Technology in the Government of Canada (the Report of the Committee on Federal Science and Technology Expenditures, known as the Lortie report), made 40 recommendations regarding the organization, financing and evaluation of federal laboratories.
9.40 With all the efforts in the last 30 years to provide some overall direction and to streamline the management of its science and technology activities, it would be reasonable to expect the government to be able to enunciate clearly what it is trying to achieve through those activities and to direct its efforts in a way to yield maximum return. Unfortunately, this is not the case.
9.41 In a report this year on federal science and technology priorities, the National Advisory Board on Science and Technology noted that there was no evidence of an explicit rationale for the allocation of science and technology spending among departments and agencies. The Board was unable to find a rational explanation for why individual budgets were at particular levels or why one organization's budget was larger than another's. It appeared to the Board that certain departments and agencies had benefited from historical incrementation in their budgets. Our audit findings clearly support the Board's conclusion. The present allocation of funds among various fields of science and technology is more incidental than the result of a well-formulated strategy.
9.42 There has been much activity but few results, although the 1987 strategy and decision framework have contributed to the achievement of some desirable results. For example, federal departments and agencies have worked more closely with industry and have increased their revenue levels. Many industry-based advisory boards have also been created.
9.43 However, the framework is far from being implemented in departments and agencies. For example, one of the thrusts in the 1987 framework was to devolve the in-house performance of science and technology to the private sector and universities. Yet we observe that the proportion of total federal science and technology resources spent in-house decreased only from 60 percent in 1987-88 to 59 percent in 1993-94; in our view, this does not reflect the spirit of the desired shift expressed in the 1987 decision framework.
9.44 Some of this lack of progress can be attributed to a lack of overall government-wide leadership, direction, focus on results and accountability for implementing desired changes. As we point out later in this chapter, Industry Canada is not well positioned to provide effective leadership among departments with science and technology responsibilities.
Learning from the Past: Some Conditions for Future Success9.45 We fully support the government's announcement in the February 1994 Budget speech that it will put in place a strategy for research and development with real priorities, real direction and a real review of results. Governments in many industrialized countries are faced with this challenge. All international and national economic indicators show that time is of the essence. If Canada is to maintain its economic well-being, the government must, after 30 years of trying, succeed this time. Past efforts have revealed several lessons for the current science and technology review and the future implementation of a strategy ' lessons that we believe are important and that are presented in this section of the chapter.
There is an urgent need to set real priorities9.46 As noted in paragraph 9.19, Canada has only modest resources to invest in science and technology, compared with many other developed countries. With the emergence of new players in the Pacific Rim and elsewhere, Canada will probably be doing even less of the world's research in the future. Under these conditions, it is essential that our resources be invested in a way that will yield maximum return. As stated previously, this is not the present case.
9.47 We believe that it is time to address specific questions and issues. For example, if the government wants to use science and technology as a means of generating wealth and creating jobs, it should consider focussing its investment in those sectors of the economy that are the most promising in terms of potential value added by federal science and technology activities. The government should clearly understand the functioning of our innovation system, our challenges to remain competitive, and the needs and opportunities it can address given its strengths and particular capabilities. The most cost-effective means of making the desired difference must then be selected and their economic impact demonstrated clearly.
9.48 Although the need has long been recognized, the government has not been able to do these things in the past. However, we believe that with the right leadership, discipline and framework, it would be possible for the government to institute methods and processes for making the necessary tough choices across and within federal departments and agencies.
9.49 To be able to make valid choices, the government must be clear about what it wants to achieve. A lack of focus at the government-wide level contributes to a lack of focus in departments and agencies. As we report in Chapter 10, our audit of selected science and technology organizations identified a need for improved understanding of needs and sectoral opportunities. This understanding requires the joint effort of scientific and non-scientific groups in departments and agencies with science and technology programs, and collective effort by departments, including the Department of Finance and the industry sector units in Industry Canada. Science and technology are only a means to achieving broader social and economic objectives. Chapter 10 also reports that science and technology organizations need to have clearer goals and priorities. We believe that establishing priorities at the government-wide level would contribute to their achievement within departments and agencies.
9.50 In summary, determining real science and technology priorities will require addressing fundamental questions such as: What are the greatest needs and opportunities? Where must the government be involved and why? Where should and could the government be involved and why? What should and could the government's involvement be? Without answers to these questions, the current review is unlikely to provide the government with a true strategy. Furthermore, we would expect to see the strategy identify priorities at the government-wide level as well as the sectoral level.
Priorities are not enough - direction is also needed9.51 A true strategy will also provide direction with regard to who will do what, how the desired results will be achieved and when. While this touches on the mandates of departments, there is a need for top-down, government-wide direction. For example, if environmental technology is selected as a priority for generating wealth and for government involvement, direction is needed on how the government intends to harness the necessary expertise and resources in departments and agencies such as Agriculture and Agri-Food, Environment, Fisheries and Oceans, National Research Council and Natural Resources. Collective effort cannot be assumed without central direction from the government.
9.52 In the past, science and technology policy directives have been issued without adequate direction for implementing them. For example, in Chapter 10 we comment on departments' pursuit of collaborative arrangements and revenue. The absence of government-wide guidelines and criteria has contributed to departments being involved in activities that we believe they should question.
9.53 The lack of adequate direction in the past has also slowed the implementation of change. In Chapter 10 we report that departments are making efforts to improve their relevance and impact. However, there is still much progress to be made. A strategy with real direction is needed to accelerate the desired changes and improvements. The situation needs continued study, such as the National Advisory Board on Science and Technology has carried out, but follow-up action and direction are necessary to produce results.
The present science and technology review needs to provide a results-oriented action plan9.54 The 1987 strategy and decision framework established several policy objectives, such as the devolution of in-house science and technology to industry and universities. However, it did not establish explicit targets. As reported in paragraph 9.43, the actual results achieved to date do not reflect the spirit of the framework; we believe the absence of targets has been a contributing factor.
9.55 As we report in Chapter 10, we came across numerous situations where the expected results of science and technology programs are defined in such general terms that they do not create a sense of urgency in the performers and do not provide a basis for accountability. We also report in Chapter 32 that the Department of Finance has not yet evaluated the impact of its $1 billion annual tax incentives to industry for scientific research and experimental development. As for priorities, focussing on results at the government-wide level will contribute to the achievement of results at the departmental level.
9.56 We conclude that if the government intends to achieve real results from its investment in science and technology, then it needs to establish results-oriented targets at the government-wide level. This needs to be one of the outcomes of the current science and technology review. For example, if the government wishes to emphasize the development and export of environmental technology-related products and services, then it should establish realistic targets and measure progress against these targets. Past efforts have focussed on inputs (expenditures), processes and activities.
Stakeholders must work together9.57 The need for stakeholders to work together is not new. In the early 1970s, the Lamontagne Committee recommended that the government establish a central mechanism based on a "concerted action" model whereby central agencies work in partnership with line departments and agencies.
9.58 We believe that one of the reasons why the 1987 effort did not work was that departments and agencies did not support it because they were not an integral part of the exercise. It did not reflect their ideas and concerns but was a central agency product. This time, departments and agencies have been asked to be part of the review process. For the exercise to succeed, departments and agencies need to work together as a team to address overall requirements rather than their own particular interests.
9.59 The 1987 policy clearly recognized the need to also take into account the priorities of the federal, provincial and territorial governments in science and technology. It recognized the need for governments to work in greater harmony and to co-ordinate their efforts more closely. An important mechanism to ensure such co-ordination was the creation of the Council of Science and Technology Ministers; however, it has not met for almost two years. We have been informed that the current phase of the federal review will be extended to the national level and will provide for formal federal-provincial dialogue. We believe that such policy consultation and co-ordination with provinces is an important condition to avoid disharmony and duplication of effort.
9.60 Collaboration among federal departments and with the provincial governments is not enough for science and technology efforts to succeed. Industry and universities must be involved to a great extent. Consultations must include discussion of sectoral needs and opportunities if the government expects to put in place a strategy that has real priorities and real direction.
9.61 Everyone agrees that team effort is essential. In our view, the government cannot assume that stakeholders will work together without concerted and persistent effort from the very top.
9.62 The federal government should ensure that:
- the new federal science and technology strategy is the result of a concerted effort among all stakeholders; and
- the strategy provides for a mechanism to encourage congruence of policies, goals and strategies among all stakeholders and to minimize duplication of effort.
A revised structure may be needed to oversee the government's science and technology portfolio9.63 It is important that there be an effective mechanism for overseeing the government's science and technology portfolio. The mechanism is required to perform such roles as leading the development of government policies; leading government-wide reviews and priority-setting exercises; preparing an annual report to Parliament; and monitoring overall trends, performance and results.
9.64 The required mechanism needs to be credible to central agencies and line departments and agencies, and to be linked into the decision-making and resource-allocation processes. It should also take into account the line authority that ministers have over the activities in their departments. There may be some merit in the Lamontagne Committee's conclusion that the delivery of science and technology programs should remain diffused among departments and agencies, rather than centralized into a "super Science and Technology department".
9.65 The Canadian government has made several attempts to establish co-ordinating mechanisms for government-wide science and technology policies and priorities. The inherent drawback of these mechanisms is that they either are not part of the government decision-making and resource-allocation processes or are the responsibility of a department such as Industry Canada, which itself is involved in the delivery of science and technology programs. The result has been that the central science and technology agencies have not been held in high regard.
9.66 Other countries have adopted various models. It is not the role of this Office to tell the government how it should organize itself to manage. However, the matter is becoming more and more important as the government makes science and technology a priority while its fiscal pressures continue to increase.
9.67 The government should put in place an effective framework to co-ordinate the activities of departments and agencies in achieving the intended results of its new science and technology strategy.
Sustained political will and leadership are needed to bring about change9.68 We found that one of the key factors that contribute to making government laboratories and science and technology programs successful is having executives who are persistent and provide leadership. Persistence on the part of management ensures that there is continuous improvement and repeated effort to make research and development activities relevant and focussed, and that accountability is not only established but practised and enforced. Leadership by management provides vision and direction. In addition, we believe that politicians who oversee the government's involvement in science and technology must also be persistent and provide leadership.
9.69 The Canadian government has certainly demonstrated a strong will to make its involvement in science and technology more effective and relevant. However, that will has not been sustained over the years. Reviews and public consultations are announced with great publicity. The same applies when official reports are released. Unfortunately, this has been only at the start of the process. The results are yet to come.
9.70 As mentioned previously, government-wide science and technology reviews have not produced results-oriented action plans that lay out priorities to meet Canada's economic and social needs and opportunities. The fact that there has not been concerted follow-through on the reviews has exacerbated the situation.
9.71 In 1994, the government committed itself to specific goals related to science and technology. Cabinet will need to be persistent if the outcome of the current review is to reflect those goals. Leadership that transcends governing departmental mandates is required.
9.72 We support the view that a government-wide science and technology review and priority-setting exercise should be carried out regularly. Again, however, this will require leadership that transcends governing mandates. Changes need to be made in the context of a long-term view. Research and development cannot be turned off and on. It can take years to develop capable teams and to transfer discoveries for application by industry and regulators.
The Government Must Know What Is Being Achieved9.73 The government's commitment to set real priorities, provide real direction and focus on results will require significantly better information on planned and actual results than is currently available.
9.74 On a government-wide basis, the currently available information comprises the expenditure and activity-oriented information reported in Part III of the Estimates presented to Parliament, and statistics that focus on important overview information such as expenditures, number of scientists and number of alliances. Such information does not provide a basis for assessing performance and setting real priorities.
Evaluating results will be a challenge9.75 Establishing meaningful performance targets and evaluating results of science and technology will be a challenge. Many of the desired results might become evident only after a number of years. It is often difficult to make the connection between the outcome of a research project and, for example, the creation of jobs. Other countries face the same challenge.
9.76 Our audit revealed that the significant amount of analytical work conducted over the years by various federal organizations has provided a basis for evaluating results of science and technology. At the departmental level, organizations such as CANMET at Natural Resources Canada and the Communications Research Centre at Industry Canada have undertaken work on developing performance indicators, even though complete processes are not yet in place to collect and compile the information.
9.77 At the government-wide level, Treasury Board Secretariat has moved with a series of departments and agencies toward developing frameworks for monitoring and evaluating results. A Statistics Canada advisory group on the development of science and technology statistics has proposed developing a nation-wide framework to be used in evaluating the results of the federal investment. The success of these initiatives, however, is ultimately dependent upon departments and agencies providing information that, as we report in Chapter 10, is currently unavailable.
9.78 There is a need to start developing and reporting some key measures and indicators. Evaluating research and development is a complex subject. It can never be truly comprehensive. Even if truly comprehensive evaluations were technically feasible, they would not be affordable. What needs to be done is to evaluate the most important components.
What a framework to assess results should include9.79 Based on our audit, we offer the following suggestions for a framework to assess the government's involvement in science and technology. The government-wide framework will need to include results-oriented information on such topics as:
- the impact of government-wide policies, for example, the policy to transfer, where prudent and feasible, government in-house science and technology activity to the private sector and universities;
- progress and results achieved by the government's involvement at the sectoral level, that is, in government laboratories performing research and development aimed at sectors such as aquaculture, mining, forestry, biotechnology, environment, pharmaceuticals and health; and
- the impact of initiatives such as the scientific research and experimental development tax credit program.
9.81 Given the nature of science and technology, the assessing of results will need to encompass both progress and actual results. Companies base their decisions to allocate additional resources, or to terminate research and development projects, on progress to date and continued fit of expected results with market conditions. So, too, the government needs information on the progress of its science and technology initiatives in order to support decision making.
9.82 Assessing results will need to include outputs and the contribution of those outputs to end results. For example, objective assessment of results will take into account the scope and significance of the contribution of technology developed in a government laboratory to such end results as jobs, exports and tax revenue. Performance measures will comprise a mix of quantitative and qualitative indicators. The information sources will include documented success stories, periodic inventories of the movement of technology and expertise, client feedback, review and priority-setting exercises, sector scans, measurement systems and independent evaluations.
9.83 The government needs to make decisions on science and technology. However, it will be limited in its ability to make sound decisions and set real science and technology priorities, provide real direction and focus on results until it has better information on results in its science and technology portfolio.
9.84 The government should define, design and implement a framework for assessing its science and technology policy and program initiatives. The information should link planned results, progress, actual achievements, expenditures and foregone revenues.
Parliamentarians and the Public Must Be Better Informed on Activities and Performance9.85 In our 1992 chapter on departmental reporting, we stated that the business of government must be transparent, irrespective of Parliament's use of the information it is given. We recommended that information to Parliament include reporting on global stewardship. Chapter 7 of this Report discusses these notions in the context of sectoral activities like science and technology. The following paragraphs deal with our observations on science and technology reporting.
9.86 Because of the importance of the federal investment in science and technology, and the scattering of activities among numerous organizations, we believe that the availability of information on the overall activities and results of these organizations is an important condition to ensure a better accountability.
There is no comprehensive reporting on federal science and technology activities9.87 Information to Parliament on government science and technology activities is confined to Part III of the Main Estimates for individual departments. With respect to science and technology expenditures, the Part IIIs tend to report the inputs of programs and activities and do not provide a basis of accountability for results.
9.88 At the overall level, statistics are published by Industry Canada and Statistics Canada on expenditures and human resources involved in science and technology activities, but they do not provide any information on the nature of such activities and their results.
9.89 An attempt was made in the 1987 document called A Decision Framework for Science and Technology to gather and report on federal science and technology activities overall. That document, approved by Cabinet in January 1987, called for an annual overview of all federal activities in science and technology to be prepared by the then Minister of State for Science and Technology. The overview was to address the following issues:
- progress toward the alignment of those activities with the guidelines and priorities established by Cabinet;
- key issues, gaps, or duplication in the overall federal effort, taking into account the regional, national and international context;
- periodic in-depth reviews of the programs or issues;
- special reports of advisory groups established by departments, for example, in accordance with the Technology Centres Policy;
- potential or actual impact on science and technology of policy initiatives in other areas (taxation, investment, patents, procurement, labour adjustment, free trade); and
- major institutional changes or financing requirements for the management and support of science and technology.
Parliamentarians have no basis to hold the government accountable for its spending on science and technology9.91 In conclusion, current reports do not provide parliamentarians with the information they need to assess whether the government's investment in science and technology reflects Canadians' needs and opportunities. Available information does not help parliamentarians direct questions to the government about its strategies and priorities, the way funds are allocated among departments and agencies, the results of the intervention mechanisms used by the government and the research activities it carries out. Parliamentarians are not provided with information on co-ordination with other stakeholders or on current problems facing science managers.
9.92 We believe that parliamentarians need to be provided with information on government science and technology programs and activities that would answer four basic questions with respect to these programs:
- What are the government's mission and lines of business?
- How does the government carry out its lines of business to achieve its mission?
- What are the government's strategic objectives for realizing its mission, and its plans for managing the significant public resources under its control ?
- How did the government do at meeting its objectives, and how much did it cost ?
9.94 The government, in consultation with Parliament, should establish a cost-effective reporting framework linked to its accountability structure to provide Parliament, on a regular basis, with information about its performance in managing its science and technology activities overall.
Future Directions for Parliament
Parliamentarians could play an important role in the current science and technology review9.95 As described previously, the current government initiative follows 30 years of unsuccessful attempts to establish a national science and technology strategy. Canada must decide what it wants to do. Our audit has shown that an effective, highly focussed, national science and technology strategy is critical to survival and growth in today's high-technology economic environment. Ensuring that such a Canadian strategy emerges from the proposals to be put forward by the Minister of Industry could be one of the most important roles for parliamentarians in the near future.
9.96 Several challenges lie ahead. A first challenge for parliamentarians and for the appropriate parliamentary committee will be to ensure that there is a real results-oriented science and technology strategy with real priorities and real direction. A second challenge will be to ensure that there is a concerted effort to implement the strategy and achieve the intended results. Finally, a third challenge will be to ensure that a proper accountability infrastructure is put in place.