This Web page has been archived on the Web.

2007 February Status Report of the Auditor General of Canada

Main Points

What we examined

The National Research Council Canada (NRC) is the federal government's most comprehensive scientific and industrial research organization. It reports to Parliament through the Minister of Industry and is governed by a Council of 22 federal appointees, including the President.

We examined the progress made by the NRC in addressing the recommendations from our 2004 audit, when we reported problems related to the NRC's corporate governance, mechanisms for setting strategic direction, management of research projects and human resources, and performance measurement and reporting.

Why it's important

Through its science and technology activities in areas such as aerospace, biotechnology, and information and communication technologies, the National Research Council makes an important contribution to the fulfillment of the government's responsibilities in areas such as health and safety, protection of the environment, communications, and economic development. The NRC also supports the efforts of Canadian small- and medium-sized enterprises to improve their technological competitiveness and innovative capability.

The NRC employs about 4,000 scientists and support staff in 18 research institutes and 2 technology centres across Canada. Its annual spending is about $800 million. It is essential that the NRC be able to recruit and retain the best scientists and researchers and manage its research activities strategically to ensure that its work remains relevant for Canada's future research needs in science and technology.

What we found

• The National Research Council has made satisfactory progress overall since 2004 in responding to our recommendations. It has addressed the recommendations in the area of corporate governance, corporate strategic direction, and human resources management. However, progress is unsatisfactory in the documenting of key decisions at the institute level and in performance measurement and reporting.
• The NRC's Council has approved a new role for itself, one that is better aligned with the NRC's mandate under the National Research Council Act. Nevertheless, some members said their role was still not clear to them. The Council has begun to exercise its revised role by approving or recommending for approval several key policy and budgetary proposals since June 2005. It has also established two new standing committees—the Audit, Evaluation and Risk Management Committee and the Human Resources Committee. However, at the time of our audit, about half of the positions on the Council were vacant, despite the NRC's efforts to have the government appoint additional Council members. The number of vacancies hampers the ability of the Council and its standing committees to function effectively.
• The NRC has put in place new management systems and processes to implement a new corporate strategy, the NRC Strategy, released in May 2006. Development of its first corporate business plan, currently planned by March 2007, awaits the completion of critical milestones scheduled for the first year of the NRC Strategy.
• The NRC follows several good practices in selecting and monitoring research projects, but essential management information on projects is not always easily or uniformly retrievable. Although most institutes have priorities for selecting, reviewing, and terminating research projects, documentation such as rating sheets or rankings is often unavailable or incomplete. Peer review and outside input are sometimes only implicit.
• While the NRC has created a plan and established key positions for human resources management, it has not set strategic priorities for human resources management and linked them to key functions such as recruitment, staffing, and succession planning. This is a task scheduled for completion as part of implementing the NRC Strategy.
• The NRC has not improved its performance reports to Parliament.

The National Research Council Canada has responded. The National Research Council Canada has accepted all of the Auditor General's recommendations and has committed to take action. Its responses follow the recommendations throughout the chapter.

Introduction

3.1 The National Research Council Canada (NRC) is the government's most comprehensive research organization. The Parliament of Canada established the organization in 1916 under the National Research Council Act to ". . . undertake, assist or promote scientific and industrial research . . ." that is in Canada's national interest.

3.2 The majority of the NRC's annual expenditures of about $800 million—80 percent—comes from parliamentary appropriations. The NRC generates the remainder of the revenue itself. It employs about 4,000 scientists and support staff who work in the NRC's 18 research institutes and 2 technology centres across Canada. The NRC also attracts approximately 1,200 guest workers from Canadian and foreign universities and research and development organizations. This reflects the organization's philosophy of collaboration and partnership.

3.3 Through its Canada Institute for Scientific and Technical Information, the NRC disseminates information about science, technology, and medicine to researchers and innovators in fields of strategic importance; such as biotechnology, biosciences, molecular sciences, fuel-cell technology, and e-business. Through the Industrial Research Assistance Program, the NRC helps small- and medium-sized Canadian enterprises to improve their technological competitiveness and innovative capability. In the process, the NRC helps to build local and national economic systems and technology clusters.

What we found in 2004

3.4 In our March 2004 Report, we examined several major components of the NRC's management systems and practices, including the role of its Council. The Council, which consists of 22 members including the President, is discussed in greater detail at the beginning of the section on corporate governance. We noted that the Council was not fulfilling its duties to control and direct its work through the President, as the National Research Council Act requires. The Council's responsibilities were not clearly defined, and therefore members did not clearly understand them. The Council also lacked important governance and accountability structures. As a result, there was no effective oversight to management plans and decisions.

3.5 We noted that the NRC needed to review its corporate mechanisms for setting priorities, to avoid an imbalance between its research activities and available funding. Without any core budget increases, the NRC continued to launch several long-term initiatives that included major increases in infrastructure and staff. The NRC faced important challenges in financing most of its activities.

3.6 While most of the research at the NRC was well managed, the NRC's institutes did not subject the research, predominantly financed by parliamentary appropriations, to a rigorous priority-setting framework. We were concerned that if this situation continued, it could erode the core competencies the NRC needs to achieve its aim of being a world class institution.

3.7 To be a leading research and development organization, the NRC needed to be able to continue recruiting and retaining highly qualified researchers. To do so, the NRC needed to further develop its human resources management strategy, after assessing the risks and opportunities involved in addressing critical challenges at all its institutes. Further, the NRC needed to develop a coherent and comprehensive action plan to implement its strategy.

3.8 After seven years of collecting and reporting corporate performance data, the NRC was continuing to improve its performance measurement and reporting practices, through a new corporate performance management framework. It needed to strengthen and implement that framework and to address weaknesses in its annual performance report to Parliament.

3.9 The NRC accepted all our recommendations and indicated that it was already acting on them, or planned to do so.

Important changes since 2004

3.10 In February 2005, the government appointed a new president to the NRC. In May 2006, the NRC published its new strategy document, Science at Work for Canada: A Strategy for the National Research Council 2006–2011 (the NRC Strategy). Since the arrival of the new president, a new senior management team has been put in place for the NRC's organization (Exhibit 3.1).

3.11 Management achieved an important milestone in better governance in June 2006, when it presented the first NRC audited financial statements to the members of the Council. These audited financial statements provide reliable information that can assist strategic decision-making. Presenting financial information in accordance with generally accepted Canadian accounting standards improves transparency and helps stakeholders hold the NRC to account.

Focus of the audit

3.12 This follow-up audit examined the same five lines of enquiry we identified in our 2004 audit: corporate governance, setting of corporate strategic direction, research management at the institute level, human resources management, and performance measurement and reporting.

3.13 We interviewed selected members of the Council, senior NRC officials, and staff; and we examined various NRC documents to determine the progress the organization had made in implementing each of our 2004 recommendations.

3.14 More details on the audit objectives, scope, approach, and criteria are in About the Audit at the end of this chapter.

Observations and Recommendations

3.15 The NRC has made satisfactory progress in implementing seven of the ten recommendations we made in 2004. Senior management has informed us that they have delayed implementing some recommendations or elements of other recommendations by approximately two years because of their concern with putting in place a new strategic vision for the organization, the NRC Strategy, and certain accountability measures.

3.16 Implementing the NRC Strategy is a complex, challenging, and time-consuming undertaking for the NRC. At the time of our audit, most implementation activities were just under way or in the planning stages. Given the importance of implementing the NRC Strategy, the associated risks, and the fact that implementing some of our recommendations will depend on implementing the NRC Strategy, we may re-examine the NRC's progress at a future date.

Corporate governance

The Council plays a new more effective role

3.17 The National Research Council (NRC) Act does not distinguish between the "Council" referring to the 21 government-appointed members plus the President, and the "Council" referring to the officials and staff comprising the corporate body. The NRC practice at the time of our 2004 audit was to refer to the 22-member Council as the Governing Council. Since the NRC no longer uses the term "Governing Council," in this report we will use the Council to refer to the appointed members and the National Research Council Canada or NRC when referring to the corporate body.

3.18 The NRC Act defines the responsibilities of the Council very broadly. Section 5(1) (b) of the Act grants the Council the power to "control and direct the work of the Council through the President." In 2004, we observed that the NRC's governance and accountability regime did not allow the Council to challenge management decisions effectively.

3.19 In our opinion, the main responsibilities associated with controlling and directing the work of an organization include: approving corporate strategic plans, monitoring progress against those plans to achieve stated objectives, approving budgets, and approving accountability reports or annual reports. In order for the Council to fulfill this role, it is essential to ensure that the members have the appropriate skills and experience.

3.20 Soon after the tabling of our 2004 Report, the Council established the NRC Council Task Force on Governance, with a mandate to review the governance issues we had identified. The task force presented its report for discussion to members of the Council in its meetings in October 2004 and February 2005. In June 2005, the Council approved the task force's recommendations, including the new role for the Council (Exhibit 3.2).

3.21 The Council has begun to carry out a more effective role. For example, since June 2005, it has reviewed and recommended to the President for approval the financial statements for the 2004–05 and 2005–06 fiscal years, and endorsed the NRC Strategy. However, it has not begun approving financial statements as intended in the statement of its new role. Our interviews with selected members of Council identified that a lack of clarity about the role of the Council continues, despite the approval of its new role.

The Council established new standing committees

3.22 We noted in 2004 that the Council lacked important structural tools, such as the use of an audit committee or a human resources management committee. At the time, the Council only had the Executive Committee, the only committee specifically referred to in the NRC Act.

3.23 In October 2005, the Council established two additional standing committees and approved their memberships—the Audit, Evaluation and Risk Management Committee and the Human Resources Committee. It also approved the terms of reference of the two committees.

Vacancies in Council membership hamper its effective functioning

3.24 In order for the Council to exercise its mandated responsibilities, it is essential to ensure that its members have the appropriate skills and experience. In September 2005, the Council's Executive Committee approved a general profile of its membership. In addition to regional, sectoral, academic, and gender balance, the profile requires that, as a whole, the Council

• is comprised of senior executives and generalists with an overall view of issues related to research and development policy, corporate governance, and research management; and
• consists of members who can provide informed advice about financial administration, audit, program evaluation, strategic planning, and human resources management.

3.25 The NRC does not appoint members to its Council; they are appointed through orders-in-council. The NRC makes recommendations to the government on who should be considered for appointment. We would expect that the NRC's recommendations to the government reflect the organization's assessment of the competencies of its current membership, relative to the desired profile of the Council as a whole. This would assist the government in identifying the critical skills required for an effective Council and the best candidates to fill the positions.

3.26 During our audit, the Council never had more than 11 of the 21-member positions (plus the President) filled, which the NRC Act authorizes. These vacancies make it difficult for the Council to function properly as a whole, especially with its two new committees. The NRC had raised its concerns about the vacancies with the Minister of Industry and had recommended several candidates.

3.27 Recommendation. The National Research Council Canada should advise the government that the Council has adopted the general profile for its ideal membership. To assist the government in appointing new Council members in a timely fashion, the National Research Council Canada should identify suitable candidates based on the criteria in the general profile.

The National Research Council Canada's response. The National Research Council Canada agrees with this recommendation and has taken or is in the process of taking a number of steps to address it. NRC will continue to advise the government of its view of the general profile for its ideal membership on Council to assist it in appointing new Council members in a timely fashion. NRC will also continue to develop and communicate lists of candidates based on this profile, to the Minister of Industry, the Industry Portfolio, as well as to the Privy Council Office.

3.28 Recommendation. The National Research Council Canada should clarify the role of the Council, at the corporate level and for members of the Council, in a manner that is consistent with the Council's approved new role and its mandate under the National Research Council Act.

The National Research Council Canada's response. The National Research Council Canada is in the process of clarifying Council's role as mandated by the National Research Council Act. This clarification, expected to be completed by the end of 2007, will be embedded thereafter in the orientation of new members.

Corporate senior management structure has been strengthened

3.29 After the government appointed a new president in February 2005, the NRC announced a new management structure, which took effect in October 2005. The previous management structure consisted of three vice-presidents—two in charge of all research and development activities and the other one responsible for technology and industry support. There are now five vice-presidents: three for the research and development program (one for each of the three portfolios—life sciences, physical sciences, and engineering), a fourth vice-president responsible for technology and industry support, and a new position of Vice-President, Corporate Services, created in February 2006.

3.30 The Chief Financial Officer and the Director General, Human Resources Management Branch were made members of the NRC's Senior Executive Committee (SEC) in February 2004. Other SEC members include the President; the vice-presidents; and the Secretary General, a corporate secretary function supporting the President, the Council, and the SEC.

3.31 Management has informed us that the NRC is adopting a portfolio management approach at the corporate level. The approach will involve initiatives that cut across the three research portfolios and are designed to allocate limited resources more effectively, in line with the NRC priorities in the NRC Strategy. At the time of our audit, the NRC had yet to formally implement this approach.

3.32 We found that the overall progress for corporate governance issues was satisfactory (Exhibit 3.3).

Setting of corporate strategic direction

The National Research Council Strategy follows Vision 2006 and emphasizes collaboration and commercialization

3.33 In 2004, we concluded that the National Research Council Canada followed good practices in preparing a strategic document called Vision 2006: Science at Work for Canada. However, we found that the NRC did not have an overall process for synthesizing priorities and related decisions, based on a realistic assessment of capacity. The NRC had chosen to rely on Vision 2006 and its institutes' five-year strategic plans, without having one corporate business plan that would document major corporate goals and strategies to accomplish the corporate vision.

3.34 We recommended that the NRC develop a corporate business plan that set clear priorities, objectives, and strategies, based on the organization's financial capacity. We also recommended that as part of this planning process, the NRC develop a rigorous priority-setting mechanism and conduct a comprehensive review of the value and continuing relevance of all its activities.

3.35 In February 2005, the NRC's new President launched the renewal strategy to review the NRC's vision and to develop a new corporate strategy. The strategy consists of four phases: Environmental Scanning, Strategic Direction, Strategy Development, and Strategy Implementation. The first three phases of the project were completed by May 2006, with the release of the NRC Strategy.

3.36 The NRC Strategy states the NRC's role is "to be a critical instrument of the federal government, translating science and technology into social and economic well-being for Canada." It spells out three goals for the NRC:

• contribute to the global competitiveness of Canadian industry in key sectors and to the economic viability of communities;
• strengthen Canada's innovation system; and
• make significant contributions to Canada's priorities in health and wellness, sustainable energy, and the environment—areas critical to Canada's future.

3.37 The NRC Strategy emphasizes improved collaboration with the NRC's partners to achieve greater commercialization of its research results.

First corporate business plan awaits completion of critical milestones

3.38 We reported in 2004 that, although the NRC had a strategic planning process in place to support Vision 2006, not all of its institutes followed the process and submitted five-year strategic plans. Some institutes prepared annual planning outlooks instead. At the time of our follow-up, some institutes continued to prepare five-year strategic plans or annual planning outlooks or both. At the same time, they began to prepare business plans, as the new planning process requires. The introduction of the NRC Strategy has made it even more important and urgent for the NRC to develop a corporate business plan.

3.39 The NRC Strategy articulates 4 strategies, 17 important actions, and 86 milestones (over five years) that will guide the NRC in implementing the strategy and achieving its goals. The NRC established project teams to carry out these major actions and interdependent milestones, organized into five projects:

• planning, performance, and resource management (PPRM);
• research programs;
• organizational change under a project called "One NRC;"
• business review; and
• sustainable organization.

3.40 While the PPRM project team will establish new management systems and processes, the other four project teams are carrying out critical decisions and crucial analyses, which the NRC needs to put the NRC Strategy to work.

3.41 The NRC has developed detailed terms of reference and plans for the PPRM project, and the team is well on its way to developing the business planning process and corporate performance management framework. Yet, the contents of a corporate business plan will require the NRC to complete 29 milestones identified in the NRC Strategy, including the following crucial activities:

• complete its evaluation of competencies, capacity, and opportunities;
• assess and augment its capacity for commercialization;
• establish priorities within each research portfolio;
• identify key players in the Canadian innovation system;
• complete the Council's assessment of the competency portfolio;
• rationalize existing programs; and
• evaluate potential areas of collaboration.

3.42 The NRC must carefully carry out these activities if it is to address the two critical weaknesses that we identified in 2004—its failures to determine the financial implications of proposed new strategies and to assess the value and continuing relevance of all its programs. The NRC has assigned the responsibility for carrying out these activities to the other four implementation project teams. At the time of our audit, these interrelated projects were in the early stage of implementation.

3.43 The PPRM project has already designed and piloted the business planning process. The process will integrate and align business plans that the institutes and programs submit, with the NRC's corporate business plan. The NRC is planning to produce its first corporate business plan by March 2007. It will be a challenge for the NRC to complete all the critical milestones indicated above.

3.44 We noted that the NRC had reviewed certain aspects of Vision 2006, as it developed the NRC Strategy. In our view, the NRC would benefit from a more formal and systematic assessment of lessons learned from Vision 2006.

Significant risks are associated with implementing the National Research Council Strategy

3.45 Implementing the NRC Strategy is a complex and challenging undertaking. Given the NRC's experience with Vision 2006 and the risks that document identified, a number of conditions must prevail if the organization is to implement the NRC Strategy successfully. Through its own risk assessment and core competency assessment, the NRC has identified significant risks, including

• the capacity of the organization and its institutes to implement new systems and processes;
• its capacity to absorb change;
• workload issues;
• its competency in corporate governance, planning, and performance management; and
• reports indicating evidence of difficulty in implementing major reform.

3.46 We are satisfied with the progress that the NRC has made in putting in place the new systems and process needed to produce the corporate business plan (Exhibit 3.4). We also believe that, although the NRC is keenly aware of the complexity and risks involved in implementing the NRC Strategy, it needs to devote attention to monitoring this endeavour.

3.47 Recommendation. The National Research Council Canada should take action to ensure all critical milestones are met to develop the corporate business plan on a timely basis. It should also monitor closely the implementation of the NRC Strategy to ensure that the complexity and risks are recognized and properly addressed.

The National Research Council Canada's response. The National Research Council Canada agrees with this recommendation. As noted in the Report, the implementation of the NRC Strategy is a complex and challenging undertaking involving reviews of programs, processes, and resources. NRC is committed to implementing the NRC Strategy.

NRC's corporate business plan, which operationalizes the NRC Strategy, will be completed by 31 March 2007. NRC will continue to monitor its implementation for important milestones and risks.

Research management at the institute level

3.48 In 2004, we noted that the National Research Council Canada's mechanisms for handling project information did not allow it to analyze and reconfigure data to provide fundamental information, such as a compilation of all NRC projects, the cost of a project within a given program, and linkages between file systems. Information was scattered, and the NRC had weak mechanisms to cross-reference and integrate information. The organization also lacked documentation of project management processes and important decisions, such as the reasons projects were selected, approved, and terminated. We recommended that the NRC clearly document major decisions on projects to make sure it does not lose important corporate knowledge.

3.49 We also recommended that the NRC improve its priority-setting framework for projects in institutes, to include a rigorous project-approval process and a periodic review of the value of projects. The NRC committed to developing a priority-setting framework for its long-term research, based on institute best-practices that the audit report identified. It further committed to identifying best practices in the NRC and elsewhere, for documenting decisions related to project selection, approval, and termination; and to reflecting these best practices in the proposed priority-setting framework.

Clear criteria for project selection used, but documentation remains uneven

3.50 Our 2004 audit focused on research projects funded predominantly by parliamentary appropriations and managed solely by the NRC. Given the NRC's increasing focus under the NRC Strategy on collaboration and commercialization, during this follow-up audit, we examined projects that represent the full range of the types of funding at NRC institutes.

3.51 We asked all institutes for a list of their current projects, including basic information that would allow us to identify some projects for further analysis. We also asked each institute to indicate how long it took to respond to this request.

3.52 The staff time that each institute reported that it had expended in responding to our request varied significantly, ranging from under one hour to one or more days, and in one case—two weeks. This indicates, as we observed in 2004, that basic information is not always easily retrievable.

3.53 We selected twenty-six projects from nine institutes to examine further how institutes manage projects:

• five projects had budgets or annual expenditures between $1 million and $9.5 million,
• sixteen projects were between $100,000 and $1 million, and
• five projects were less than $100,000.

For each project we requested documentation that demonstrated what criteria the institutes used to select the project, any rating system involved, and the formal record of decision. To assess the quality of ongoing monitoring of projects, we requested documentation on the process, timing, and outcomes of project reviews, including any peer review processes that provided outside input into the evaluation of projects.

3.54 We discuss the results of our review of research projects in the following paragraphs and summarize them in Exhibit 3.5. The results are summarized by institute and not by project.

3.55 Although we found considerable variation in how institutes select and monitor projects, we found the following good practices:

• All institutes have a well-defined project planning cycle with clear expectations and criteria for project selection, related to the NRC objectives, as well as their strategic plans. All institutes' project selection criteria included the availability of human and other resources, as well as scientific or industrial merit. However, in some cases a significant criterion was existing researchers' capacities, which could potentially conflict with the NRC's corporate aim to be flexible and responsive to changing needs.
• Three institutes documented their process for using the criteria by rating and/or ranking different projects.
• Three institutes used a risk analysis framework to assist in selecting at least some of their projects.
• Eight institutes had an external peer, international, or industry review of proposed and ongoing projects. Some of these institutes view commercial or external funding as an implicit substitute for peer review, particularly when the funder itself conducted a review process. All institutes used internal reviews to assess project proposals.
• Seven institutes conducted a formal review of progress on a periodic basis, in addition to budget monitoring.
• We assessed the progress in addressing our recommendation on priority-setting at institutes as satisfactory (Exhibit 3.6).

3.56 Among the practices that require attention and improvement, we observed the following:

• Six institutes did not have documentation for systematically using the framework in the form of rating sheets, rankings, and records of decision. Indeed, several of the nine institutes reported to us in 2006 that their practices for project selection, monitoring, and termination were essentially the same as in 2004.
• Six of the institutes did not systematically assess financial, safety, or other risks; or conduct cost-benefit analyses of projects.
• One institute submitted projects to an external reviewer, but took no action to ensure that the review was ever received or considered.
• Two institutes did not document that the projects were formally monitored beyond budget reports.

3.57 We expected to observe institutes and their partners sharing best practices. The NRC informed us it had intended to share best practices among institutes, but we did not observe this happening. Most institutes said they used the NRC's Project Management Guide, which is also based on best practices. It was not clear how they used it.

3.58 In 2005, the NRC piloted a Research Management Self Assessment Tool, based on best practices across many facets of research management. This tool is designed to assist institutes in assessing how effective their research management was and in identifying opportunities for improvement. However, the NRC is still piloting this tool, which is not yet in general use. Of the nine institutes we examined, only two had been involved in the pilot.

3.59 Recommendation. The National Research Council Canada should further develop and implement decision-making tools and templates for project selection, as part of the NRC Strategy. It should also improve the project documentation and information retrieval systems required to support these decisions.

The National Research Council Canada's response. The audit report identifies best practices that exist in the institutes the auditors examined. These best practices and others in the remaining institutes will continue to be developed and strengthened as part of the Planning, Performance and Resource Management (PPRM) project for implementing the NRC Strategy. Mechanisms will be developed as part of the next project planning cycle for 2007–08 to ensure their transfer and application throughout the National Research Council Canada. It should be noted, however, that peer review and external consultations are not always appropriate and would not always be performed. This is the case for matters of national security and client confidentiality.

Human resources management

3.60 With 48 percent of the National Research Council Canada's budget devoted to people, many of them highly trained scientists and technicians, and with the NRC Strategy under way, human resources management (HRM) remains vital to the NRC's success.

Implementation of Human Resources Management reform awaited the National Research Council Strategy

3.61 The NRC's guiding principles for human resources management are described in its 2002 Employment Philosophy. As we observed in 2004, this philosophy is based on a strategy that is ambitious and high level, but that does not indicate priorities. The guiding principles include

• recruiting and retaining outstanding people;
• giving employees the opportunity and support to grow professionally, to utilize their strengths, and to produce to the maximum of their capabilities;
• rewarding employees based on their level of professional development and productivity in their job; and
• creating a respectful partnership with employees, based on trust and understanding.

3.62 The HRM Steering Committee concluded in 2005 that it needed to revise the Employment Philosophy to include more operational priorities and more realistic goals. This did not happen, because the NRC believes now that the Employment Philosophy should be revised only if required, in conjunction with other HRM practices, as part of implementing the NRC Strategy.

3.63 In 2004, in response to our audit, the NRC completed a human resources management plan and, in the 2005–06 fiscal year, created two new positions—Director, Total Compensation, and Performance Measurement Consultant. The NRC expects that the new positions will support and enhance the implementation phase of the NRC renewal strategy, and, at the same time, address our 2004 concerns about HRM capacity and planning. Because the positions have only recently been staffed, we were not able to observe what they have accomplished.

3.64 A similar situation exists for the environmental analysis, which is a required foundation for HRM planning. We recommended in 2004 that this analysis needed to be more extensive and should include an adequate assessment of risks and opportunities, and credible, factual, and future-oriented information from institutes, programs, and branches. In 2006, the NRC informed us that such a scan had begun, building on the NRC Strategy environmental scan. At the time of our audit, the HRM environmental scan was still under way.

Human Resources Management policies and practices are not yet aligned with the National Research Council Strategy

3.65 In 2004, we noted that the NRC needed to clarify the qualifications of the people it needs to employ to achieve its objective of being a leading-edge research organization. The NRC also needed to align its recruitment, staffing, performance management, and pay policies accordingly. We noted that the NRC had been reviewing its hiring practices, with the intention to streamline the process.

3.66 We reviewed the most recent progress report (July 2006) produced by the NRC's Modernization of Hiring Initiative. The progress report described several operational improvements and noted that future initiatives need to include updating the hiring process to delegate more to institutes, programs, and branches.

3.67 The NRC instituted a Leadership Enrichment and Development Program (LEAD) in 2005, to identify and develop NRC employees with management potential and to provide them with opportunities to develop management and leadership skills to enhance their own job performance. The Senior Executive Committee viewed this as potentially helping to address succession planning issues. As well, two of the pilot business plans prepared by institutes, (see paragraph 3.71), identified succession planning as a key HRM issue.

3.68 Compensation remains an important area. This concerns not only salary, but all forms of rewards. Although the description of the newly established position of Director, Total Compensation does not formally include strategic planning for compensation, we were informed that planning and policy will constitute part of this job. This is essential, because the NRC will continue to try to offer rewarding careers and an environment that will attract top scientists and engineers, benefiting them and Canada. To do so, the NRC will need to consider carefully how recruitment, compensation, promotion, performance management and appraisal, awards, rewards, opportunities for professional contacts, and training can all contribute to this goal.

3.69 The NRC needs and intends to conduct a comprehensive review of the way HRM policies and practices can reinforce its revised strategic priorities. It also needs to develop an action plan to make any required changes. Monetary compensation and promotion opportunities are obvious and important motivators, but they are not the only potential incentives or disincentives to help the NRC and its staff align their goals.

A promising business planning process has started

3.70 As institutes write the business plans that the NRC Strategy expects from them soon, they will have to include HRM planning. To do this, they will need information about the future and analyses they often lack now.

3.71 We reviewed the pilot business plans of three institutes. Two of the three contained an HRM component. The more complete of the two identified a number of HRM areas that the institutes will need to address. These include, among others,

• leadership roles and responsibilities,
• performance management,
• reward systems,
• strategic long-term recruitment needs,
• retention,
• succession-planning gap analysis, and
• learning and development.

Since many institutes lack sufficient planning capacity to address these issues, the NRC will need to consider how to improve that capacity.

3.72 In our view, the HRM component of the institute business plans is a good step in the right direction (Exhibit 3.7). It underscores the NRC's need to update a corporate HRM plan linking institute business plans with corporate HRM priorities.

3.73 Recommendation. The National Research Council Canada should, as soon as possible, clearly align and integrate its Human Resources Management regime with its strategic priorities as expressed in the NRC Strategy. The regime should link compensation, rewards, promotions, awards and other forms of recognition, recruitment, succession planning, and performance management.

The National Research Council Canada's response. The National Research Council Canada agrees with this recommendation and has already taken steps to address these issues as part of the "One NRC" implementation project of the NRC Strategy. NRC will integrate compensation, rewards, promotions, awards and other forms of recognition, recruitment, succession planning, and performance planning with the strategic priorities of the NRC Strategy. This will be done via NRC's business planning process. The NRC corporate business plan, which will be released in March 2007, will reflect NRC's priorities for the next three years and will capture key human resources requirements to deliver on those priorities.

It is anticipated that the strategies to address recruitment, succession planning, performance management, and compensation (as it applies to position-based classification) will be developed by mid-2007, while strategies to address rewards, promotions, compensation, awards and other forms of recognition will be completed by mid-2008. The priorities for addressing these human resources issues will be confirmed with the development of the NRC corporate business plan.

Performance measurement and reporting

The National Research Council Strategy delays improvements to performance management

3.74 In 2004, we recommended that the National Research Council Canada do the following:

• establish clear and concrete targets for the results its main indicators measured,
• urgently establish a comprehensive plan with adequate resources to address gaps and implement its 2004 corporate performance management framework,
• link results to the costs incurred in achieving them, and
• continue to explore ways to strengthen systems and practices that ensure the reliability of performance information.

3.75 We also recommended that, in future performance reports, the NRC present results for its 28 indicators against clear and concrete expectations. In its response, the NRC stated that it would continue to improve its performance management framework and performance report.

3.76 The Management, Resources and Results Structure (MRRS) is a Treasury Board Secretariat policy that took effect on 1 April 2005, to support a government-wide approach to reporting performance information. In June 2005, the NRC informed us that it would align its performance management framework with the MRRS.

3.77 The NRC Strategy envisions that the performance management framework will be finalized in the early implementation stage of the Strategy and will be the driver for the NRC's planning process. NRC management informed us that the task of developing the MRRS has been delayed due to the need to align it to the NRC Strategy. Therefore, no such "driver" existed during the pilot of the business planning process. We found that there was little consistency in the performance information included in the pilot business plans submitted by institutes (Exhibit 3.8).

Performance reports have not improved

3.78 Our 2004 examination of the NRC's past performance reports up to the 2002–03 fiscal year found that, although there had been steady improvements, there were still several ways to improve the reports and provide Parliament with better accountability.

3.79 Our review of the NRC's most recent performance reports (for the 2004–05 and 2005–06 fiscal years) indicates that there is no significant improvement from the reports that we reviewed in 2004. The NRC has not developed clear and concrete targets for its major indicators.

3.80 Until the NRC develops and uses a corporate performance management framework, as envisioned in the NRC Strategy, its performance report is not likely to improve.

3.81 Recommendation. The National Research Council Canada should expedite the completion of its Management, Resources and Results Structure, based on the existing corporate performance framework, and align it to the NRC Strategy. This would provide for meaningful performance information to be included in its 2007–08 Corporate Business Plan and future performance reports.

The National Research Council Canada's response. The National Research Council Canada is committed to developing an approved corporate performance framework based on the balanced scorecard approach and directly aligned to NRC's Strategy by 31 March 2007. The integration of the balanced scorecard with NRC's corporate business plan will provide performance measures and costs linked to the NRC Strategy.

The completion of the Management, Resources and Results Structure is scheduled for March 2008 and will be greatly expedited through the newly approved institute, program, and branch business planning process, which collects all of the information necessary to comply with the Treasury Board Secretariat Management, Resource and Results Structure policy.

Conclusion

3.82 The Council has strengthened its role. It has demonstrated that by getting involved in major policy and budgetary decisions since June 2005. Yet, we noted differing views among some Council members about their specific role. The National Research Council Canada should clarify the role of the Council at the corporate level and for members of the Council, in a manner that is consistent with the Council's approved new role and its mandate under the National Research Council Act (NRC Act).

3.83 The Council adopted a new committee structure. However, at the time of our audit, nearly half of the seats in the Council were empty. The vacancies inhibit the capacity of the Council and its committees from performing their roles properly. Since the Council approved a general profile of the competencies, skills, and characteristics that its members need before the Council can function effectively, we recommend that the NRC identify suitable candidates based on the criteria in the profile to assist the government in appointing new members.

3.84 The NRC's Vision 2006, which existed when we audited in 2004, is succeeded by the NRC Strategy, which emphasizes collaboration and commercialization. The NRC has put in place new management systems and processes to implement the NRC Strategy, and it plans to produce its first corporate business plan by March 2007. Yet, the NRC faces many critical milestones in completing this on a timely basis. Because of the importance and risks involved, we recommend that NRC management closely monitor and take action as needed to address the risks in implementing the NRC Strategy.

3.85 We observed that the use of clear criteria to determine priorities and guide project selection is widespread in the NRC institutes. We noted several good practices for selecting and reviewing research projects, but opportunities exist for greater use of priority-setting frameworks and documentation in the form of rating sheets or rankings.

3.86 Essential management information about projects is not always easily or uniformly retrievable. We recommend that the NRC improve its decision-making tools and templates for project selection, including documentation of key project decisions and retrieval systems necessary to support these decisions.

3.87 The NRC has made satisfactory progress in implementing our 2004 recommendations on human resources management by implementing several initiatives and making the Director General, Human Resources Branch part of the Senior Executive Committee. We noted, however, that full implementation of the recommendations must now await further development of the NRC Strategy. A Human Resources Management regime that links compensation, succession planning, performance management, and other HRM functions to the NRC Strategy is essential if the Strategy is to succeed.

3.88 The development of the NRC Strategy has delayed the preparation of the corporate performance management framework. The NRC should quickly respond to the need to have meaningful performance information by completing this framework.

About the Audit

Objectives

The overall audit objective was to assess the progress that the National Research Council Canada has made in addressing the concerns we raised in Chapter 1 of our March 2004 Report. This follow-up audit assessed the extent to which measures the NRC put in place since 2004 have improved the management of its scientific research activities.

Scope, approach, and criteria

In 2004, we made 10 recommendations in five areas: corporate governance, corporate strategic direction, research management at the institute level, human resources management, and performance measurement and reporting. In this follow-up audit we interviewed selected Council members, NRC senior executives, and staff; and examined various documents to assess the NRC's progress in implementing each of our 10 recommendations.

We expected that the measures the NRC took would lead to the following conditions:

• The NRC's corporate systems and practices should stipulate periodic reassessment of the value and continuing relevance of existing research programs. They should link the organization's strategic decisions and priorities to its mandate and to the government's agenda. The NRC should establish its priorities through consultation with major stakeholders, and they should be based on a thorough analysis of the NRC's operating environment, options, risks, and future impacts. The allocation of internal funds should reflect the priorities.
• The NRC should establish appropriate systems and practices for selecting, monitoring, and terminating research projects, to ensure the best value for Canada. This should include risk analysis, peer reviews, identification of potential uses and users, periodic review of the value of projects, corporate management oversight, and assurance of a link to the NRC's vision and goals.
• The NRC should employ a competent workforce, providing the appropriate mix of employment relationships to achieve its short- and long-term strategic and operational objectives, in a timely and cost-effective way. The NRC's systems and practices should include a definition of desired major competencies; strategies to hire the right people to complete the desired workforce profile; and adequate strategies for recruitment, selection, hiring, employment, termination/retirement, and training.
• The NRC's performance management framework should provide clear and concrete performance expectations and credible and balanced performance results to manage its vision effectively and promote good accountability to Parliament and Canadian taxpayers.

Audit work completed

Audit work for this chapter was substantially completed on 30 October 2006.

Audit team

Assistant Auditor General: Nancy Cheng
Principal: Reno Cyr

Auditor: Chantal Turgeon

For information, please contact Communications at 613-995-3708 or 
1-888-761-5953 (toll-free).