2004 March Report of the Auditor General of Canada
Chapter 1—National Research Council Canada—Management of Leading-Edge Research
1.1 The National Research Council's (NRC) Governing Council is not fulfilling its duties to direct and control the work of the corporation through the President as required under the National Research Council Act. The Governing Council's responsibilities are not clearly defined and therefore are not well understood by the Council members. Thus, the Governing Council lacks important governance and accountability structures. As a result, there is no effective challenge to management plans and decisions. The National Research Council Act has not been updated to incorporate best practices for corporate governance, such as those in more recent legislation establishing a governing council.
1.2 The NRC needs to review its corporate mechanisms for setting priorities to avoid an imbalance between its research activities and available funding. Although it has had no core budget increase, it is currently launching several long-term initiatives that include major increases in infrastructure and staff. Meanwhile, the NRC is facing important challenges in funding most of its current activities.
Background and other observations
1.3 The NRC has earned international recognition in a broad diversity of scientific disciplines. It has a long history of making valuable scientific discoveries that contribute to the well-being of Canadians and others worldwide. While most of the research at the NRC is well managed, the leading-edge research funded predominantly by parliamentary appropriations is not subjected to a rigorous priority-setting framework at all institutes. If this situation continues, core competencies needed by the NRC to maintain Canada's competitive edge in scientific research worldwide could be eroded. In our view, the weakness in the priority-setting framework results from a failure to implement strategic plans and guidelines.
1.4 With an anticipated shortage of researchers looming, the NRC needs to be able to continue recruiting and retaining highly qualified researchers to be a leading research and development organization. To do so, the NRC must further develop its human resources management strategic direction, based on an assessment of the risks and opportunities to address key challenges in institutes. Further, it needs to have a coherent and comprehensive action plan to implement its strategic direction.
1.5 After seven years of collecting and reporting corporate performance data, the NRC is continuing to improve its performance measurement and reporting practices through a new corporate performance management framework. It now needs to strengthen and implement that framework and to address weaknesses in its Performance Report.
The National Research Council has responded. The National Research Council has accepted and agreed with all our recommendations and has provided individual responses. The responses include statements of actions already underway or planned to address the recommendations.
1.6 Federal science and technology (S&T) activities play an important role in fulfilling government responsibilities in areas such as health and safety, protection of the environment, communications, and social and economic development. Statistics Canada estimates that the federal government spent $8 billion on S&T activities in 2002–03, of which $5 billion was for research and development (R&D).
1.7 A number of federal government departments and agencies are involved in R&D. Of these, the National Research Council (NRC) is the government's most comprehensive research arm. The NRC had expenditures of about $800 million in 2002–03, funded through parliamentary appropriations and government transfers (80 percent) and self-generated income (20 percent). For the same period, the NRC had just under 4,000 employees, as well as 1,200 guest workers from Canadian and foreign universities and public and private sector organizations. The NRC is larger than most Crown and departmental corporations in parliamentary appropriations received, number of employees, and number of buildings it owns and manages.
1.8 The NRC's mandate under the National Research Council Act is to ". . . undertake, assist or promote scientific and industrial research . . ." that is in the national interest of Canada. The Act also sets out a specific mandate for the NRC related to ". . . the investigation and determination of standards and methods of measurements . . . ." The Weights and Measures Act sets out an NRC mandate for the calibration and certification of measurement standards. The NRC's vision, as stated in its Vision 2006 document published in April 2002, is "Recognized globally for research and innovation, [the] NRC is a leader in the development of an innovative, knowledge-based economy for Canada through science and technology."
1.9 The NRC views its role as unique in the transfer of technology and dissemination of knowledge to industry and in enhancing innovation in Canada. It sees itself as a "public champion" for enabling Canadian companies and communities to develop sustainable knowledge-based growth while reinforcing connections to universities and industries. Unlike most other research organizations, the NRC institutes span the research spectrum, from exploratory to applied. They involve a wide range of sectors including aerospace, biotechnology, construction, information and communications, manufacturing, nanotechnology, and ocean engineering. Much of this research is done in collaboration with Canadian industry, universities, other government entities, and international organizations.
1.10 The NRC has 18 research institutes and 2 technology centres across Canada, as well as the Canada Institute for Scientific and Technical Information and the Industrial Research Assistance Program (Exhibit 1.1). NRC institutes are located in nine provinces, each focussing on different areas of research, comprising anywhere from 60 to 320 staff, and reporting to the NRC corporate office through a director general and vice-president. The technology centres are smaller in size and operate on a cost-recovery basis. In 2002–03, the NRC had expenditures of about $450 million for its institutes and technology centres, $100 million for its corporate services, and $245 million for other activities (see paragraphs 1.12–1.14).
Contributions of the National Research Council
1.11 The NRC has a long history of making valuable scientific discoveries that contribute to the well being of Canadians and others worldwide. It has earned international recognition in a broad diversity of scientific disciplines. It is a key player in maintaining Canada's competitive edge in the knowledge-based economy.
Some National Research Council discoveries and successes
1.12 The Canada Institute for Scientific and Technical Information (CISTI) is Canada's largest publisher and disseminator of scientific, technical, and medical information. In 2002–03, CISTI spent about $53 million providing just under a million documents to Canadian academics, industries, governments, and medical professionals and others around the world.
1.13 The Industrial Research Assistance Program (IRAP) is intended to stimulate wealth creation for Canada through technological innovation in Canadian small and medium-sized enterprises. It does this through technical and business advice, financial assistance, access to business information, contacts, and national and international networks. In 2002–03, IRAP spent $151 million helping 12,000 private sector firms across Canada. These costs included $95 million in IRAP contributions.
1.14 Other NRC 2002–03 expenditures included $41 million in contributions for the Tri-University Meson Facility.
1.15 The NRC actively encourages the creation of new research-based enterprises from its laboratories. It reports that it has created 55 new companies in a variety of sectors since 1995 and that this has generated employment for about 500 Canadians and cumulative private investments of $250 million. These companies were formed either by NRC employees (after having resigned or taken a leave of absence) or by non-NRC principals using NRC technologies. As well, a number of specialized agencies and services have grown out of the NRC over the years, including Atomic Energy of Canada Limited, the Medical Research Council (now known as Canadian Institutes of Health Research), the Natural Sciences and Engineering Research Council, and the Canadian Space Agency. (See photograph)
Focus of the audit
1.16 Our audit objectives were
- to assess the National Research Council's systems and practices for setting strategic direction for its scientific research activities, including its corporate governance and accountability structure; and
- to determine whether it managed these activities in a way to maximize results and whether it measured and appropriately reported the results and impacts of its efforts.
1.17 We assessed the significance, risks, auditability, and interest to Parliament of the NRC's activities and programs. We identified four lines of enquiry for examination: corporate governance and setting of corporate strategic direction, research management at the institute level, human resources management, and performance measurement and reporting.
1.18 Some quantitative information in this chapter is based on data provided by the NRC. We have assessed these data through a process of analysis, comparison, and discussion. Unless otherwise indicated in this chapter, these data should be treated as unaudited.
1.19 Our audit did not include a review of the quality of the NRC's research. Program evaluations to assess whether program objectives are being met are the responsibility of the NRC. Since 2001, the NRC has conducted five evaluations—the Biotechnology Program (delivered by five NRC institutes), the Institute for Chemical Process and Environmental Technology, the Industrial Research Assistance Program, and two initiatives supported by the NRC (the Tri-University Meson Facility and the Canadian Technology Network). Further details on the audit are found at the end of the chapter in About the Audit.
Observations and Recommendations
Governance and accountability regime does not allow effective challenge of management decisions
1.20 Corporate governance refers to the process and structure for overseeing the direction and management of a corporation so that it carries out its mandate and objectives effectively. Good corporate governance is important to maximize performance, prevent financial losses, and help to achieve the corporate mandate.
1.21 The National Research Council (NRC) is a departmental corporation, listed under Schedule II of the Financial Administration Act. The President of the NRC is the Chief Executive Officer of the corporation and Chair of the Governing Council.
1.22 Section 3.(1) of the National Research Council Act establishes ". . . a Council, to be called the National Research Council of Canada, consisting of a president and not more than twenty-one other members . . . ." However, the legal title of the corporation, including all of the researchers, other staff, and managers, is also the National Research Council of Canada. In order to distinguish between these two groups, the NRC uses the term "Governing Council" when referring to the President and the 21 external members appointed by the Governor in Council. For the purposes of this section, we will distinguish between the Governing Council and the corporation.
1.23 The Act provides significant duties and powers to the Governing Council. Section 4 specifies its duties as having "charge of such matters affecting scientific and industrial research in Canada as may be assigned to it by the Governor in Council." Section 5.(1)(b) grants the Governing Council the power to "control and direct the work of the Council [corporation] through the President." The key responsibilities we normally associate with controlling and directing the work of an organization include approving corporate strategic plans, monitoring progress against those plans to achieve stated objectives, approving budgets, and approving accountability reports or annual reports.
1.24 The responsibilities of the Governing Council are very broadly defined by the Act. We expected to find a resolution setting out the terms of reference by which the Governing Council would fulfil its mandated responsibilities, including any important distinctions between the role of the President as Chair of the Governing Council and as Chief Executive Officer of the corporation. Management has advised us that no such terms of reference exist and that the Governing Council conducts business in accordance with the National Research Council of Canada General By-law, approved by the Governor in Council in December 1990. The by-law is administrative in nature and deals essentially with the scheduling and conduct of Governing Council meetings; it is silent on the responsibilities of the Governing Council in fulfilling its mandated role under the Act.
1.25 We expected that the Governing Council would have established a structure and defined its requirements for management information, including planning, financial, and operational information, to make informed decisions about the strategic direction of the corporation and to provide an effective challenge of management decisions. We found no evidence of specific requirements for management information going to the Governing Council. However, management does provide institute strategic plans and other relevant corporate proposals to the Governing Council for approval. We noted that the Governing Council lacked important structural tools to assist it in providing its oversight function, such as the use of an audit committee or a human resources management committee.
1.26 Role of Governing Council is unclear. We attended two meetings of the NRC Governing Council, in June and October 2003. In addition, we interviewed six members of the Executive Committee of the Governing Council to obtain their views on the role that they and their colleagues on the Council are mandated to perform under the National Research Council Act. We also interviewed the President of the NRC to obtain his perspectives.
1.27 Our discussions revealed a lack of clarity among Governing Council members about the role of the Governing Council—that is, whether it is simply advisory in nature to the President or whether it includes the obligation to direct and control the work of the corporation through the President. The Governing Council is currently operating primarily as an advisory body. The NRC has advised us that, according to its research, the role of the Governing Council as an advisory body has not changed over the past three decades. We are concerned about this apparent inconsistency with the enabling legislation, and believe that it has led to the absence of the key Governing Council governance initiatives and structure noted earlier. A review is needed to reconcile the gaps between the statutory role of the Governing Council, which provides the expectation of oversight of the corporation's operations, and its current practices.
1.28 Enabling legislation has not been updated. The NRC's enabling legislation was first enacted in 1917; the most recent amendments related to the Governing Council's powers were enacted in June 1979. We reviewed the Canadian Institutes of Health Research Act, enacted in April 2000, as an example of the government's current practices when establishing a Governing Council. Like the NRC, the Canadian Institutes of Health Research (CIHR) is a departmental corporation listed under Schedule II of the Financial Administration Act.
1.29 The Canadian Institutes of Health Research Act is similar to the National Research Council Act in that it confers on the CIHR's Governing Council the mandate to manage the affairs of the corporation. However, it also differs from the National Research Council Act in important areas, such as providing greater clarity in the intent of its provisions, as follows:
- It establishes both a corporation and a governing council and makes a clear distinction between the two entities.
- It clearly stipulates the powers and functions of the corporation and the responsibilities of the Governing Council.
- It makes the Governing Council explicitly responsible for managing the corporation and stipulates the following key responsibilities: developing the CIHR's strategic directions, goals and policies; evaluating its overall performance, including achievement of its objective; approving budgets and funding for research and other expenditures; and establishing policies for dealing with collaborative partnerships.
- It makes the Governing Council responsible for establishing, maintaining, and terminating Institutes of Health Research, including a periodic review of the mandate and performance of each institute.
1.30 The NRC must ensure that its Governing Council has the necessary governance mechanisms in place to fulfil its mandated role. In order for the Governing Council to fulfil the role to "control and direct the work of the Council," it will be essential to ensure that members have the appropriate skills and experience. In reviewing the role of the Governing Council, the NRC could consider the best practices for corporate governance as reflected in more recent enabling legislation for departmental corporations.
1.31 Review of corporate senior management structure is needed. The NRC's review of the Governing Council's structure and governance mechanisms would also provide a timely opportunity to review its corporate senior management structure. Crown corporations with the complexity, staff, and budget comparable to those of the NRC normally have vice-presidents of finance and human resources management. While the NRC's research activities are headed by three vice-presidents, the functions of corporate services, finance, and human resources management are headed by directors general reporting to the President. It would be appropriate for the NRC to review its corporate senior management and accountability structures.
1.32 Recommendation. The National Research Council (Governing Council) should define its role to meet its assigned responsibilities under the National Research Council Act and should put in place the necessary governance mechanisms to implement that role.
National Research Council's response. While recognizing the historical perspective that the Governing Council has functioned essentially as an advisory body for decades, the National Research Council accepts this recommendation and will work with the Governing Council to more clearly define the Council's role. The NRC will also ensure that it has the systems and structures in place to provide the necessary organizational support to fulfil that role.
1.33 Recommendation. The National Research Council should include a review of its corporate senior management structure to ensure appropriate accountability within the corporation and to a restructured Governing Council.
National Research Council's response. The National Research Council accepts this recommendation and has already taken steps to add the Director General of Finance and the Director General of Human Resources to the Senior Executive Committee. A further review of the senior management structure will be undertaken in 2004, after the arrival of the new President of the NRC.
1.34 Recommendation. The National Research Council should seek advice on what remedies are available to it under the National Research Council Act to facilitate the realignment of the Governing Council's role, including ensuring that it has the right profile of Council members and, if appropriate, proposing amendments to the legislation to better reflect best practices for governance of departmental corporations.
National Research Council's response. The National Research Council accepts this recommendation and will strike a task force, involving Governing Council members, to follow up on this recommendation and will consult the appropriate individuals in the Privy Council Office and the Department of Justice Canada to develop a proposed course of action.
Setting corporate strategic direction
1.35 Governance within the NRC involves both internal staff and external representatives (Exhibit 1.1). The NRC Governing Council includes the President and up to 21 other members. NRC management's Senior Executive Committee is composed of the President (Chair), the three vice-presidents, and the Secretary General; it acts as the main internal decision-making body in all matters pertaining to the NRC's operations. Each NRC institute, as well as the Canada Institute for Scientific and Technical Information and the Industrial Research Assistance Program, is headed by a Director General and has an external Advisory Board that provides strategic advice on program and policy matters relevant to the institute or program. Both the Governing Council and advisory boards are composed of external representatives, mostly from private industry and academia.
Good practices in developing vision
1.36 Setting corporate strategic direction is one of the key activities of any organization; it allows the organization to transform its vision into a corporate strategy, including the identification of future priorities based on an assessment of opportunities, risks, and potential resources.
1.37 We expected the NRC to ensure that its vision and priorities were linked to its mandate and the government's agenda, and established after consultation with key stakeholders.
1.38 Except for the human resources management component, the NRC followed good practices in preparing its current vision document, Vision 2006: Science at work for Canada. A February 2002 internal audit concluded that Vision 2006 reflects government direction and that significant environmental scanning and consultation allowed the NRC to identify many opportunities for it to contribute its expertise to improve technological solutions for Canadians. The NRC's vision document outlines five strategic pillars and related corporate goals, strategies, and desired outcomes for a five-year period (2001 to 2006). Our review indicates that the vision is closely aligned with the federal government's March 1996 science and technology strategy and February 2002 innovation strategy.
Need to review mechanisms for setting priorities
1.39 We expected the NRC to ensure that its strategic decisions and priorities were based on a thorough analysis of its operating environment, options, risk, and future impacts, and reflected in the allocation of internal funds. We also expected the NRC to periodically reassess the value and continuing relevance of its research areas.
1.40 We are concerned that the NRC's current and future expansion could negatively affect its overall effectiveness, and we question how long the NRC can continue to expand with no increase to its core budget. The combined effect of the following factors makes selecting priorities and assigning scarce resources an increasing challenge for the NRC:
- The NRC is facing important resourcing challenges.
- There has been a significant increase in long-term research activity, including major expansion of facilities, with no core budget increase.
- The NRC does not have a regular and rigorous process to synthesize its priorities and make strategic decisions based on a realistic assessment of capacity.
- The NRC's last comprehensive review of all its research areas that resulted in a refocussing of research efforts was in 1995; the NRC has limited flexibility in reallocating internal funds. (See photograph)
1.41 The NRC faces important resourcing challenges. The NRC is involved in a wide variety of research areas and plans to add more. Although some specific research-related initiatives have recently been funded through appropriations (for five to six years), the related increase in staff and infrastructure puts additional pressure on the NRC's corporate services. Its vision includes desired strategic outcomes that will involve further increases in research activity. Meanwhile, the NRC is facing important challenges in funding ongoing operations, including maintaining its infrastructure and state-of-the-art equipment, paying increasing utility costs and taxes, and hiring staff.
1.42 Unlike federal government departments, the NRC must bear the cost of maintaining its buildings, including utility costs and payments in lieu of taxes. In 2001, it submitted a long-term capital plan to the Treasury Board Secretariat that identified a cumulative shortfall of $308 million by 2005. The plan identified total requirements, net of yearly capital appropriations, for maintenance and repairs to the NRC's 181 buildings (60 percent of which were constructed over 30 years ago), upgrading of research equipment, and new buildings and equipment. The NRC has since received funding for specific research-related initiatives, which included coverage of about half of the capital plan shortfall. We estimate the shortfall to be about $150 million now.
1.43 Although the NRC has implemented a number of energy-saving measures over the years, increasing utility costs and payments in lieu of taxes are having a negative impact on funds available for research. In the last four years, these annual costs have increased substantially—from $19 million in 1998–99 to $31 million in 2002–03.
1.44 The NRC was facing a tight financial situation during 2003–04, due to higher than expected expenditures and lower revenue. Senior management had to take a number of measures, including a 1.5 percent budget cut across the organization. In addition, some institutes may be at a crucial point in being able to ensure continuing operations. We found evidence that some institutes had previously been unable to fully accomplish their stated priorities because of a lack of funding to hire additional qualified staff. (See photograph)
1.45 Increased long-term research activity, including major expansion of facilities, launched with no core budget increase. Increased research activities at the NRC have resulted from additional appropriations of $430 million up to 2006–07 for targeted projects and from investments from other federal government agencies, provincial governments, universities, and industry. A large number of major capital projects have recently been completed, are ongoing, or are planned. Since 2000–01, the number of staff has increased by 470 (14 percent), and more staff will be needed once the new institutes and research centres are fully operational.
1.46 The new projects and ongoing or planned initiatives include two new research institutes—the National Institute for Nanotechnology in Edmonton, Alberta, and the Institute for Nutrisciences and Health in Charlottetown, P.E.I., and 10 new Industry Partnership facilities (attached to an NRC institute and used to house start-up companies). There are also several new research centres or locations, including the following:
- Aluminium Technology Centre in Ville Saguenay, Quebec
- Aerospace Manufacturing Technology Centre in Montréal, Quebec
- Canadian Photonics Fabrication Centre and Gas Turbine Environmental Research Centre in Ottawa, Ontario
- E-business research centres in Fredericton, Moncton, and Saint John, New Brunswick and Wireless Systems in Sydney, Nova Scotia
- Centre for Sustainable Urban Infrastructure in Regina, Saskatchewan
- Language Technology Research Centre in Gatineau, Quebec
1.47 We reviewed four of the recent strategic initiatives to assess the appropriateness of the decision-making process and the elements considered (see case study Four strategic initiatives by the National Research Council). We found that all four initiatives are linked to the NRC's mandate and the government's agenda, that there were good consultations with key stakeholders, and good analyses of options, risks, and partnership opportunities. However, there was no assessment of the potential financial implications of these initiatives on the NRC's current operations.
1.48 These initiatives are all expected to become key elements of the NRC's future research activities and will likely require ongoing operational funding for decades to come. However, the funding received for these initiatives was for five to six years, with very little commitment from the government for future operational funding.
1.49 In recent years, the NRC has received additional parliamentary funding for specific initiatives but has had no increase in its base funding for ongoing operations since at least 1997–98 (Exhibit 1.2). Although it has been able to benefit from increased self-generated revenues through increased fee-for-service activities and collaborative partnerships, it expects future increases of these revenues to be limited. The funding uncertainty puts the long-term sustainability of current and new initiatives at risk.
1.50 The NRC lacks a regular and rigorous process to synthesize its priorities. The NRC has chosen not to have a corporate business plan. Instead, it relies on Vision 2006 to document key corporate goals and strategies and on five-year institute strategic plans to provide more detailed information on how each institute intends to accomplish the corporate vision. Institutes, programs, and branches also prepare yearly planning outlooks. However, the NRC has no overall process for synthesizing priorities and making related decisions based on a realistic assessment of capacity.
Need for a comprehensive review of research areas
1.51 We expected the NRC to periodically review the value and continuing relevance of all its research areas, including its current institutes. This would allow it to set priorities for its activities at the highest level and identify if trade-offs were necessary based on available funding. We also expected the NRC to periodically review its allocation of funding to institutes, programs, and administrative branches. This would allow it to ensure that its internal funding of activities reflected its current priorities.
1.52 The NRC conducted a comprehensive review of the value and continuing relevance of all its research areas as part of the federal government's Program Review in 1995. At that time, it realigned its programs and services. In 1998, it also converted the research focus of one of its institutes to redirect efforts and facilities to the emerging fuel cells initiative.
1.53 The NRC has practices that help fine-tune the research focus of its institutes. It conducts program evaluations and independent peer reviews of its institutes and programs. All institutes also prepare strategic plans once every five years, which are vetted by the NRC's senior executives and approved by the Governing Council. A number of research activities have been reoriented as part of this process and other processes. However, as discussed in the section "Research management at the institute level," we found that lower-level reviews of certain institute activities are inadequate. Two of eight institutes audited had poor systems and practices for periodically reviewing the value of ongoing research projects.
1.54 In 1999–2000, the NRC conducted what it referred to as the Performance Review and Reallocations Exercise. It is our understanding that the objective of this exercise was to review institutes' past performance and future directions in order to ensure, through budget reallocations, an appropriate balance of funds across the organization. The NRC has indicated that this was a very comprehensive review, but we were unable to confirm this based on the limited documentation provided. The review identified differences among institutes in terms of past performance results (ratings varied from adequate to excellent) and their potential to contribute to science, technology, and wealth creation for Canada. We would have expected such a review to result in appropriate refocussing of research efforts, with priority given to those areas with the most potential. However, because the NRC felt that the 1999 budgets represented a threshold to sustaining research activities and scientific integrity in each institute (as established during Program Review), decreases beyond two percent could not be justified.
1.55 We noted instances of fine-tuning of the research focus of some institutes, including small budget adjustments. However, given the important resourcing challenges facing the NRC, it would benefit from conducting a high-level systematic and transparent review of the value and continuing relevance of all its research activities. Such a review would identify the need for any major trade-off decisions based on available funding, including the need to terminate some research activities if warranted.
1.56 Recommendation. The National Research Council should develop a corporate business plan that sets clear priorities, objectives, and strategies based on its financial capacity. As part of this process, it should develop a rigorous priority-setting mechanism and conduct a comprehensive review of the value and continuing relevance of all of its activities, including refocussing of research efforts as appropriate.
National Research Council's response. Through its Vision 2006, the National Research Council has a strong strategic-level vision for the organization that is linked through strategic and operational plans at the institute level to the business of the NRC. The NRC will build on its existing strategic strengths at the corporate level, and it will develop and implement a more formal corporate business planning system, including an annual rolling three-year business plan. The NRC also accepts the need for a periodic comprehensive review of its programs, and it will undertake to define a process that will achieve the desired result, taking into consideration the unique characteristics of a research organization.
Research management at the institute level
Research funded primarily by appropriations provides significant contributions
1.57 Research is the NRC's primary activity. The NRC does much of its research in partnership with industry, other governments, and educational institutions in Canada and abroad. An NRC internal audit of partnership practices for collaborative research completed in 2002 concluded that, in general, NRC institutes managed partnerships well. The audit included a review of institute awareness of partnership opportunities; the links with the NRC's mandate and vision; the documentation of project objectives, risks, success factors, and resourcing; and project management practices. We therefore focussed on other research projects that were funded predominantly by parliamentary appropriations and managed solely by the NRC. Those projects mostly involve leading-edge research of no immediate commercial interest and innovative new approaches that allow NRC researchers to maintain their core capabilities and leadership in a highly competitive knowledge-based economy. (See photograph)
1.58 For our review, we selected 43 medium- to high-dollar value research projects and/or programs in eight NRC institutes. We selected institutes with a view to ensuring a good representation of geographic locations, disciplines, and size.
1.59 The NRC was unable to provide information on the total number of research projects it was conducting or on the number and costs of research projects totally funded by parliamentary appropriations. Management explained that its research programs are a complex mix of basic research, collaborative research with universities and industry, and fee-for-service work, and it is difficult to identify projects funded solely by appropriations. In the absence of such information, we were unable to assess the relative importance of these research projects compared with those involving partnership funding. However, while the NRC has moved toward more partnership programs in recent years, projects funded primarily by appropriations continue to provide a significant contribution to the NRC's research activities. Indeed, the new scientific discoveries of today will lead to the new products and industries of tomorrow.
Some institutes need a better framework for setting priorities
1.60 We expected the NRC to have an appropriate priority-setting framework (Exhibit 1.3) for selecting, reviewing, and terminating research projects so that only those offering the best value for Canada were retained. At the same time, we acknowledge the need for flexibility so that the framework is applied with a level of rigour commensurate with the estimated cost and risk of research activities.
1.61 We found evidence of some very good practices. All institutes in our sample have mechanisms in place to ensure that research activities are focussed on pragmatic problems. Researchers we spoke to were passionate about their research, knowledgeable about their science, and linked to the scientific community. Many are adjunct professors at universities and active members of the scientific community nationally and internationally. Most researchers are involved in industrial partnership research activities and this has resulted in an enhanced awareness of the needs of the users. Virtually all researchers we interviewed had a clear vision of the benefits of their research for Canadians.
1.62 Further, two institutes in our sample review all their research activities (new initiatives, ongoing projects, and related activities) on a cyclical basis, allowing all projects to be scrutinized collectively in a transparent and informed context.
1.63 The NRC's systems and practices do not provide assurance that only research projects offering the best value for Canada are retained. Of the eight institutes that we visited, two had good to excellent practices in almost all areas related to selecting, reviewing, and terminating research projects; four had a mix of good and weak practices; and two had major weaknesses in their priority-setting frameworks. We also found that there was a good priority-setting framework for the Genomics and Health Initiative research projects that are conducted in a number of NRC institutes.
1.64 The NRC has mechanisms in place to ensure that institute strategic plans are in line with the corporate vision. All institutes complete five-year strategic plans that are reviewed by the NRC's senior executives and approved by the Governing Council. These documents outline the institute's roles and research priorities for attaining the NRC's vision and corporate goals. However, most of the institutes visited lack documentation such as records of decision and rationale for approving specific projects or programs. Thus, we were unable to determine if activities at the project/program level were consistent with the NRC's vision and goals.
1.65 We found that certain research management practices varied considerably among institutes; for the most part, they are a function of institute preference rather than a specific rationale. While some met our expectations, many fell short. Weaknesses and possible consequences of research management practices are identified in Exhibit 1.4.
1.66 We recognize that one of the challenges of research activities is to find an appropriate balance between oversight and scientific creativity. However, with research projects at the core of the NRC's vision and strategic objectives, a good priority-setting framework in institutes is important. This framework should include consideration of the NRC's vision and goals, potential uses and users of the research, risks, and views from peers. It should also include a rigorous project approval process and a periodic review of the value of projects. We found that two of the eight NRC institutes visited and the GHI research projects had such frameworks.
1.67 Recommendation. The National Research Council should improve the priority-setting framework in its institutes so an appropriate process is in place to ensure that only research projects offering the best value for Canada are retained.
National Research Council's response. The National Research Council strives to ensure that research projects are selected on the basis of their scientific merit, their contribution to national science programs, and their value for Canada. The NRC is therefore committed to developing a priority-setting framework for its long-term research based on institute best practices identified in the audit report. It will focus on the life-cycle practices for selecting, reviewing, and terminating research projects. The NRC will also identify best practices in the NRC and elsewhere in documenting decisions related to project selection, approval, and termination and will reflect these in the proposed priority-setting framework.
Institutes lack relevant documentation of key decisions
1.68 Our findings on research management are based in large part on interviews with 63 NRC institute staff members of various hierarchical levels, including directors general, directors, project leaders, and researchers. Although we also reviewed NRC documents related to the research activities selected, documentation of project management processes and decisions was lacking. Key decisions such as project selection, approval, termination, and major shifts in project direction were generally not well documented.
1.69 Many institutes rely heavily on the collective experience and knowledge of staff for project selection and management. As the NRC expands into various new areas and as senior managers and researchers retire, there is a strong risk that key corporate knowledge and the capacity to share information and processes will be lost.
1.70 Further, the NRC's mechanisms for handling project information do not allow it to analyze and reconfigure data to provide fundamental information, such as compilation of all NRC projects, cost of a project within a given program, and linkages between file systems. We also noted that the information was scattered and that mechanisms to cross-reference and integrate information were weak.
1.71 Recommendation. The National Research Council should clearly document key decisions related to research project selection, approval, termination, and major shifts in project direction to ensure that important corporate knowledge is not lost.
National Research Council's response. The National Research Council strives to ensure that research projects are selected on the basis of their scientific merit, their contribution to national science programs, and their value for Canada. The NRC is therefore committed to developing a priority-setting framework for its long-term research based on institute best practices identified in the audit report. It will focus on the life-cycle practices for selecting, reviewing, and terminating research projects. The NRC will also identify best practices in the NRC and elsewhere in documenting decisions related to project selection, approval, and termination and will reflect these in the proposed priority-setting framework.
Human resources management
1.72 The NRC aims to be a leading-edge organization in research and development. Its objective is to be a major innovator in human resources management. Over the last three years, the NRC has invested considerable effort to develop and implement a strategic approach to its human resources management (HRM).
1.73 The NRC operates in an environment with fierce competition for new resources. It competes with industry and universities at the national and international levels while operating in a government context. Tailoring a strategic HRM approach that can support the strategic plans of more than 25 institutes, programs, and branches is a huge challenge. (See photograph)
1.74 The NRC cannot simply apply strategic management practices from the private sector. Generally, some of the core requirements to manage human resources include an established strategic planning process, the development of an HRM plan supporting the strategic business plans, and the involvement of the Human Resources Branch as a strategic partner. In our opinion, the NRC is moving in the right direction in implementing this emerging trend.
1.75 We looked at how the NRC developed and put in place its expected HRM strategic outcome, goals, plan, and priorities to achieve Vision 2006.
Strategic direction for Vision 2006 needs to be developed further
1.76 More extensive environmental analysis is needed. We expected to see a formal HRM environmental analysis that captures future-oriented and critical information on the NRC's potential HRM challenges for the next five years. The analysis is intended to ensure the relevance of the long-term HRM strategic outcome, goals, plan, and priorities.
1.77 In finalizing its Vision 2006 document in 2001, the NRC added a human resources management strategic outcome, "Outstanding People—Outstanding Employer." However, this was included without a rigorous consultation with institutes, programs, and branches, and without an HRM environmental analysis of the constituencies of the NRC to identify risks and opportunities.
1.78 In support of its Vision 2001, the NRC did a comprehensive job of developing competencies, which are used for staffing and in some cases for learning and performance management. However, with the new Vision 2006, we found very little future-oriented information available from the NRC's institutes, programs, and branches. For example, the following information was lacking:
- the types of competencies that are now obsolete and the types of new competencies required to implement the five-year institute strategic plans (based on the NRC competency model);
- the availability of and demand for needed competencies in the labour market;
- clear information, at the institute level, on compensation competitiveness for researchers in comparable organizations; and
- the impact of the financial situation in institutes, programs, and branches on attracting and keeping highly qualified people.
1.79 In January 2001, the NRC put in place a Human Resources Management Steering Committee to oversee HRM initiatives. The committee is composed of a vice-president, some directors general, some directors, and the Director General, Human Resources Branch. Although this committee provides input on behalf of stakeholder representatives, the Human Resources Branch needs to capture comprehensive strategic thinking and insights within institutes, programs, and branches—to determine what should be addressed and in what order to support the strategic plans. At the time of our audit, the Human Resources Branch had not collected this information.
1.80 We noted that institute managers we interviewed possess a good knowledge of their external and internal human resource environment, but their knowledge is not documented formally and captured at the corporate level.
1.81 In the fall of 2003, the NRC started an HRM environmental analysis. This comprises a review of challenges from the perspectives of worldwide and federal government science and technology, and an assessment of the HRM practices in institutes, programs, and branches. The analysis does not yet include an adequate assessment of risks and opportunities.
1.82 HRM strategic goals, plan, and priorities are too broad to provide direction. The NRC's strategic planning guidelines indicate that strategic goals and objectives are to provide the framework for more detailed levels of strategic planning. The framework should clarify the vision and address strategic opportunities and organizational issues.
1.83 In its 2003–04 Report on Plans and Priorities, the NRC defined its HRM strategic goals in generic terms, such as "recruit and retain highly qualified people, reward professional development and productivity, provide an outstanding place to work . . . ." However, these goals, and the related plan and priorities, are so broad that they can include almost any HRM initiative; thus they do not provide direction or establish key priorities. One of the NRC's priorities is to implement a new employment philosophy by 2006. This philosophy is an ideal vision of human resources management and sets forth best practices in all HRM areas. This was not intended to suggest priorities. By the end of our audit, the NRC was debating its future HRM priorities to make them more strategically focussed.
1.84 Need to develop an effective partnership. To successfully develop and implement an HRM strategic plan, the Human Resources Branch needs to be actively involved in the organization's overall strategic planning and management processes. This would ensure that strategic initiatives are evaluated on their human resources implications. It would also allow the Branch to obtain a better understanding of the missions of institutes, programs, and branches and the issues confronting institute managers.
1.85 In May 2003, the NRC modified the role of the Human Resources Management Steering Committee to ensure that it focusses on high-priority human resource issues instead of directing the HRM agenda. This provides the NRC with an opportunity to review the outdated 1993 policy on the Human Resources Branch roles and responsibilities and to build an effective partnership between the Branch and senior management to set the appropriate strategic direction.
1.86 Recommendation. The National Research Council should continue its environmental analysis based on credible, factual, and future-oriented information at institutes, programs, and branches. It should focus on the most relevant human resources management challenges as a basis for setting priorities. The National Research Council should also clarify its strategic goals and develop measurable objectives linked to those priorities within a defined period of time. The Human Resources Branch should play a partnership role with senior management in setting the strategic direction.
National Research Council's response. The National Research Council agrees with these recommendations and will continue its efforts to ensure that the current and future human resources (HR) challenges and needs of the institutes, branches, and programs are identified. This environmental analysis will form the basis for establishing human resources management priorities for the organization. This priority-setting process is well underway, with the first comprehensive HR plan scheduled for completion in Spring 2004. The NRC has already taken steps to ensure that HR planning is integrated with and supportive of strategic business planning, including the development of measurable objectives. To emphasize the essential strategic partnership, the Director General of the Human Resources Branch will become a member of the Senior Executive Committee.
The critical challenges identified in the recommendation are consistent with the human resources management priorities outlined in the NRC Report on Plans and Priorities for 2004–05. In this context, the NRC is in the process of building capacity to address recruitment, hiring, succession planning, and compensation issues.
An action plan is needed to implement the strategic direction
1.87 Given the comprehensiveness of the new employment philosophy, the number and diversity of NRC institutes, and the limited resources available, we expected to see an HRM action plan focussed on priorities.
1.88 In the absence of HRM priorities derived from an environmental analysis, the NRC used the Human Resources Management Steering Committee to determine the overall agenda for HRM. The Committee dealt with issues that mainly represented the pressing immediate needs rather than changes needed to meet the NRC's future challenges.
1.89 In our review, we focussed on the HRM issues as identified by the institutes we visited and by some Human Resources Branch representatives. These issues included the NRC's ability to continue recruiting highly qualified researchers, including the impact of compensation for researchers and the need for succession planning.
1.90 Recruiting and hiring practices need to be more tailored to the NRC's needs. Managers and researchers expressed concern about the time it takes to hire a new employee through an external competition and the lack of flexibility in attracting highly qualified researchers. The NRC supports the use of networking to identify excellent potential candidates in research areas. Its 2002–03 records show that over one third of its researchers were recruited by "word of mouth." However, the NRC has not yet put in place a specific hiring procedure that responds expeditiously to the operational requirements of its institutes and is consistent with its general staffing objective and policy. It risks losing excellent candidates unless the current hiring process is tailored to the needs of institutes.
1.91 The NRC recently reviewed its recruitment practices and activities to find innovative ways of recruiting staff and linking all activities in a comprehensive manner. It developed a global recruitment strategy that focusses on targeted recruitment activities that emphasize reputation- and image-building. It has also been conducting an extensive review of its hiring practices, with the objective of streamlining the process to be more efficient and effective.
1.92 The NRC has proposed many changes to its current recruiting and hiring practices. In our opinion, these changes should be pilot-tested in some institutes, with a focus on highly qualified researchers, before full implementation.
1.93 The NRC's Vision 2006 goals emphasize recruiting highly qualified people, while its employment philosophy emphasizes looking for the "best of the best." In light of the funding available and the anticipated shortage of researchers, the NRC needs to clarify what type of people it needs to be a leading-edge organization in research. It also needs to align its recruitment, staffing, performance management, and pay policies accordingly.
1.94 Managers view compensation for researchers as an important challenge. Many managers and senior researchers we interviewed emphasized the issue of compensation competitiveness with universities and industry in their communities as a road block in attracting highly qualified researchers. Furthermore, the available funding is not always sufficient to cover a full term of employment or limits the capacity of institutes to staff researchers in continuing positions, where appropriate. We therefore expected that the NRC would have taken steps to closely examine the issue of salary competitiveness for researchers.
1.95 The NRC has no compensation policy. Such a policy is necessary to specify how the NRC will pay its employees and how it positions itself in relation to competitors. The policy should consider internal and external factors that affect the NRC's capacity to deliver its mandate.
1.96 The NRC has not carried out a thorough study to determine whether there are institute- and program-specific issues it should be dealing with related to compensation of researchers. It has no quantitative or qualitative data from managers and researchers to determine the nature and extent of the problem related to the competitiveness of salaries. As well, it has no data on the number of potential candidates for researcher positions who refused to apply in 2001–02 and 2002–03 due to compensation or other considerations. Since networking is one of the key methods for recruiting, project leaders in institutes have information on researchers who were not interested in jobs at the NRC because of compensation or other considerations. In the fall of 2003, the NRC planned to build its capacity to deal with compensation issues.
1.97 Urgent need for succession planning. Our review of demographic data compiled annually at the institutes we visited indicates that an average of 28 percent of researchers will be eligible for retirement in five years. We share the concern of the NRC's managers and senior researchers about the researchers' current demographic profile and the fact that universities, an important source of recruitment for the NRC, have the same demographic profile. An academic study estimated that over the next eight years, universities are going to need 10,000 PhDs to replace retiring faculty members, and Canada is going to produce only 4,000 of them. In addition, the NRC has a unique situation with employees, particularly researchers, who stay long beyond normal retirement age at very high wages. The NRC will need to address this issue in conjunction with the need for new expertise in some institutes. These challenges need to be taken into account in the near future.
1.98 Recommendation. The National Research Council should develop a comprehensive human resources management (HRM) action plan in line with an HRM strategic plan. The action plan needs to address the critical challenges of recruitment, hiring and compensation practices, and succession planning at both the corporate and institute levels.
National Research Council's response. The National Research Council agrees with these recommendations and will continue its efforts to ensure that the current and future HR challenges and needs of the institutes, branches, and programs are identified. This environmental analysis will form the basis for establishing human resources management (HRM) priorities for the organization. This priority-setting process is well underway, with the first comprehensive HR plan scheduled for completion in Spring 2004. The NRC has already taken steps to ensure that HR planning is integrated with and supportive of strategic business planning, including the development of measurable objectives. To emphasize the essential strategic partnership, the Director General of the Human Resources Branch will become a member of the Senior Executive Committee.
The critical challenges identified in the recommendation are consistent with the HRM priorities outlined in the NRC's Report on Plans and Priorities for 2004–05. In this context, the NRC is in the process of building capacity to address recruitment, hiring, succession planning, and compensation issues.
Performance measurement and reporting
Measuring the performance of research and development activities is complex and difficult
1.99 Research results come in many forms and become evident at different times. For example, sometimes the only identifiable result in the near term is an increase in the body of knowledge. That new knowledge may turn out to be of pivotal importance at some unknown time—possibly years in the future or in another discipline. The complexities of measuring performance at the NRC are magnified by the diversity of its research and development activities, carried out in some 20 institutes and technology centres across Canada as well as in a variety of programs and initiatives.
1.100 The NRC has several years of experience in results measurement, including experience with ongoing measurement and periodic program evaluation. In 1996 it established its first corporate performance framework and, as part of that framework, developed definitions of common performance indicators and performance reporting outlines for its institutes, programs, and branches. In 2003 it launched a new corporate performance management framework for Vision 2006 and was implementing it as we carried out this audit; the framework is scheduled to be completed by 2004–05. At the time of our audit, the NRC was also leading a government-wide project intended to address many of the common challenges to performance management in research organizations. The project, involving about 20 other federal science-based departments and agencies, is expected to develop into an ongoing network for those organizations.
1.101 Although the NRC was in a period of transition for measuring and reporting on its performance, we focussed our audit work in this area on its new performance management framework and its most recent Performance Report.
The new performance measurement framework poses challenges
1.102 In our view, the new performance measurement framework incorporates important and encouraging advances over the one it replaces. The more important of these are the following:
- The NRC plans to use the framework to move from performance measurement to performance management.
- The new framework is more focussed and manageable in its approach to results measurement, with 28 key indicators instead of the more than 80 used previously.
- The information for the indicators will come from a variety of sources, including ongoing data collection, program evaluation studies, bibliometric analyses, and peer reviews.
1.103 However, the NRC has developed few meaningful standards or targets against which to compare results achieved. Measuring results is a necessary but not sufficient step in measuring performance. Measuring performance requires some assessment or analysis of achievements against realistic and meaningful expectations; these are set out, for example, as targets or standards. Only two of the strategic outcomes associated with the five strategic pillars of the NRC's Vision 2006 have concrete targets. However, these are targets only for 2006 and provide no basis for measuring performance in the intervening period.
1.104 Achieving desired results is important, but achieving value for money is also important. One requirement for assessing performance, therefore, is to link results with the costs incurred in achieving them. We found that the NRC's financial information systems compile cost information along three business lines, rather than in relation to the key performance indicators or the five strategic pillars, and the NRC has no plans to change this. We believe that the failure to link results with costs limits the usefulness of the NRC's corporate performance management framework for purposes of management as well as accountability to Parliament.
1.105 In February 2002, an internal audit was conducted to assess the NRC's progress in implementing the recommendations in our 1994 Report, Chapter 10, Science and Technology: Management of Departmental Science and Technology Activities. The audit found there was no uniform automated system or process for institutes to gather performance information for reporting to the corporate level. An October 2002 internal audit to assess the NRC's management of partnerships found errors in the performance information related to collaborative partnerships in four of the five institutes audited. The NRC informed us that while there is little verification of the reliability of performance information, efforts are being made to simplify and automate data gathering, thus easing the burden and reducing the possibility of error. In addition, risk-based verification of performance information is a potential area of future internal work.
1.106 Implementing the performance management framework throughout the NRC is a complex undertaking, extending over a considerable period of time. Although the NRC has identified the planned start and end dates for several major tasks, it does not have a comprehensive, structured implementation plan that sets out such details as responsibilities for completing tasks, the nature of the tasks, the resources that will be required, and the costs that are likely to be incurred.
1.107 Moreover, we noted that there are already signs of some slippage in achieving planned milestones. For example, definitions for the new key performance indicators were to have been developed and communicated across the organization in the summer of 2003. This milestone, which could be significant in assuring the quality of the 2003–04 performance information, was not achieved.
1.108 Recommendation. The National Research Council should establish clear and concrete targets for the results measured by its key performance indicators. It should also move urgently to establish a comprehensive and adequately resourced plan for addressing gaps and implementing its new corporate performance management framework. This planning process should include steps to link results to the costs incurred in achieving them and continue to explore ways to strengthen systems and practices for assuring the reliability of performance information.
National Research Council's response. The National Research Council is committed to and will continue to implement and improve its new performance management framework as part of the Modern Comptrollership initiative. Actions are underway to establish a comprehensive and adequately resourced plan to address the gaps in the corporate performance management framework, including setting performance targets for key performance indicators, developing a system to link resources to strategic outcomes, and building an automated performance data gathering system along with procedures to improve the reliability of performance information.
Performance Report needs to be improved
1.109 We compared the NRC's Performance Report for the period ended 31 March 2003 with the corresponding report tabled one year earlier, and we found it to be a better document. The most recent report has a greater focus on performance and results, including a summary table of key results related to commitments made in its 2002–03 Report on Plans and Priorities and a summary of the findings of recently completed evaluations and internal audits. Further, the overall layout of the report is clearer, and it makes better use of charts and tables.
1.110 We also assessed the most recent Performance Report using the model developed by our Office and outlined in our 2002 Report, Chapter 6, A Model for Rating Departmental Performance Reports. The model is generally consistent with the Treasury Board Secretariat's guidelines for performance reporting, released in 2001. It is important to note that we did not audit the information in the Performance Report.
1.111 Although information for the new key performance indicators is not yet available to the NRC, its most recent Performance Report is presented within the context of Vision 2006. We noted some strengths and several ways that it can be improved and provide better accountability to Parliament.
1.112 The report includes a good discussion of the contribution of the NRC's planned results to relevant government priorities. In addition, it demonstrates clearly that planned results are consistent with the NRC's mission and mandate and its operating environment. However, the report does not include a discussion of risks to the achievement of planned results.
1.113 A summary table in the report sets out key results against performance commitments made in the 2002–03 Report on Plans and Priorities. Most of the commitments are to carry out activities or to produce outputs. Only a few are expressed as outcomes (for example, "stimulate foreign investment by creating new technology-based companies and providing highly qualified jobs for Canadians"). For the most part, the commitments do not set out the direction of planned change or concrete targets for the amount of change over a specified time.
1.114 Since most of the performance expectations are not concrete and few of the expected results are expressed as outcomes, it is difficult, if not impossible, for a reader to form any judgment about the NRC's performance or to determine whether the results reported amount to performance successes or shortcomings. Further, the report does not link the resources used by the NRC to the strategic outcomes.
1.115 With few exceptions, information in charts and tables is clearly labelled and easy to understand. However, there are no discussions or representations about the quality of the data.
1.116 Finally, the report does not provide examples of the use of performance information in decision making within the NRC; nor does it provide any discussion of lessons learned or of actions taken or planned to address performance issues.
1.117 Recommendation. The National Research Council should present results for its 28 performance indicators against clear and concrete expectations in its future performance reports, and it should link costs to results—at least at the level of strategic outcomes.
National Research Council's response. The National Research Council is committed to continuing to improve its performance report and, by addressing the gaps that exist within its corporate performance management framework, will improve on the comprehensiveness of its reporting to Parliament.
1.118 We found that the National Research Council's corporate governance and accountability structure does not provide an effective challenge of management decisions. We are concerned that the role of the Governing Council is not clear among members of the Council. We are also concerned that the Council is not carrying out the duties we would expect to fulfil the responsibilities assigned to it by the National Research Council Act. We noted that the Act has not been updated to reflect best practices in corporate governance as reflected in more recent legislation.
1.119 The National Research Council generally followed good practices for developing its vision, which is closely aligned with the federal government's 1996 science and technology strategy and its 2002 innovation strategy. However, the NRC continues to expand the scope of its research activities and infrastructure, despite a lack of core budget increase. We concluded from our case study of four recent strategic initiatives that there were good consultations with key stakeholders and good analyses of options, risks, and partnership opportunities. However, there was no assessment of the potential financial implications of these initiatives on the NRC's current operations. We also noted the need for a comprehensive assessment of the NRC's research activities.
1.120 We found some examples of good to excellent practices for selecting and terminating leading-edge research projects that are of no immediate commercial interest. However, we also found a number of instances where the NRC did not have the appropriate systems and practices. We identified the absence of priority-setting frameworks and a lack of appropriate project documentation as key areas for improvement to enhance corporate oversight for all projects. This would ensure that all research being performed is providing the best value for Canadians. It is important to note, however, that our findings relate to a sample of projects funded primarily through parliamentary appropriation. They cannot be extrapolated to include projects involving collaborative partnerships, which we did not review.
1.121 The NRC has recognized the importance of human resources management (HRM) to achieving its Vision 2006 by making it one of the five pillars and strategic outcomes. However, the environmental analysis supporting the HRM strategic priorities needs to be stronger, and the strategic direction needs to be developed further. In addition, the NRC lacks a comprehensive human resources management action plan in line with an HRM strategic plan that will enable it to address future critical challenges in the institutes, programs, and branches. We are concerned that the lack of a clearly supported way forward and the NRC's challenges with succession planning could limit its ability to conduct the future research activities of its institutes and programs as planned.
1.122 In developing a new performance management framework, the NRC has taken an important step toward managing for results and improving performance reporting in its Performance Report. However, gaps remain in the framework and the NRC faces challenges in implementing it. Key weaknesses we noted included few meaningful standards or targets against which to compare results achieved, no means to link costs with results, no uniform automated systems or processes for gathering institute performance information, and the absence of a comprehensive and structured plan for implementing the framework.
About the Audit
Our audit objectives were
- to assess the National Research Council's (NRC) systems and practices for setting strategic direction for its scientific research activities, including its corporate governance and accountability structure; and
- to determine whether the NRC managed these activities in a way to maximize results and whether it measured and appropriately reported the results and impacts of its efforts.
Our more specific objectives were to determine whether the NRC
- periodically reassesses the appropriateness of its research activities and whether its strategic decisions are made with adequate consideration of risks, options, stakeholders, and potential impacts;
- has appropriate systems and practices for selecting and terminating research projects, and ensuring adequate corporate oversight;
- is able to ensure that it has sufficient qualified staff available for its current and future needs; and
- is able to use performance information on its key activities to effectively monitor and manage its operations and report to Parliament on the achievement of its objectives.
Scope and approach
Our planning work included a review of all the NRC's key activities and programs. This was done from February to June 2003 through site visits to 15 NRC institutes; a review of key documents; numerous interviews of NRC staff (President, vice-presidents, all corporate branch directors general, Director General of the Industrial Research Assistance Program, Director General of the Canada Institute for Scientific and Technical Information, and directors general of institutes visited) and of some key stakeholders (partners in industry, universities, and government).
Based on our knowledge of the NRC and its key challenges, we assessed the significance, risks, auditability, and interest to Parliament of the NRC's activities and programs. We identified four lines of enquiry: setting of corporate strategic direction, research management at the institute level, human resources management (HRM), and performance measurement and reporting.
We selected a sample of recent strategic decisions as a case study to assess the quality and adequacy of information available for decision making and assess the overall decision-making process. We selected eight institutes to assess research project management practices and six institutes to assess HRM practices.
We conducted numerous interviews with NRC Governing Council members, senior executives, and staff. We also examined various NRC documents, including its most recent Performance Report (2002–03) to assess the adequacy of its accountability reporting.
Our audit did not include a review of the quality of the NRC's research. Program evaluations to assess whether program objectives are being met are the responsibility of the NRC. Since 2001, the NRC has conducted five evaluations—the Biotechnology Program (delivered by five NRC institutes), the Institute for Chemical Process and Environmental Technology, the Industrial Research Assistance Program, and two initiatives supported by the NRC (the Tri-University Meson Facility and the Canadian Technology Network).
We drew our audit criteria from the NRC's enabling legislation, guidelines, and key documents such as Vision 2006, Report on Plans and Priorities (2003–04), Performance Report (2001–02), and internal audit reports. We also consulted the Treasury Board Secretariat's modern comptrollership practices and the expectations developed by our Office based on our experience in value-for-money auditing and on generally accepted good management practices.
- We had the following expectations:
- The NRC's corporate systems and practices should provide for periodic reassessment of the value and continuing relevance of existing research areas. They should ensure that its strategic decisions and priorities are linked to its mandate and the government's agenda; established after consultation with key stakeholders; based on a thorough analysis of its operating environment, options, risks, and future impacts; and reflected in the allocation of internal funds.
- The NRC should have appropriate systems and practices for selecting, monitoring, and terminating research projects to ensure the best value for Canada. We would expect this to include ensuring risk analysis, peer reviews, identification of potential uses and users, periodic review of projects' value, corporate management oversight, and assurance of a link to the NRC's vision and goals.
- The NRC should have a competent workforce with the appropriate mix of employment relationships (permanent, temporary, contract personnel, or partnerships) to achieve its short- and long-term strategic and operational objectives, in a timely and cost-effective way. We would expect the NRC's systems and practices to include the definition of desired key competencies; strategies to meet the desired workforce profile, and adequate strategies for recruitment, selection, hiring, employment, termination/retirement, and training. These would be aimed at ensuring that staff have the desired attributes and that the NRC has the flexibility to meet operational requirements, has adequate turnover of staff, and has the ability to maintain the overall competency of its workforce.
- The NRC's performance management framework should provide clear and concrete performance expectations and credible and balanced performance results to allow for effective management of its vision and promote good accountability to Parliament and Canadian taxpayers.
Assistant Auditors General: Richard Flageole and Nancy Cheng
Principal: Reno Cyr
Director: Denis Scott
For information, please contact Communications at (613) 995-3708 or
Nanotechnology— The building of objects on a scale that is measured in billionths of a metre. (Back)