This Web page has been archived on the Web.

1981 Report of the Auditor General of Canada

Introduction

12.1 The National Research Council of Canada (NRC) is governed by the National Research Council Act ( R.S.C. 1870, c.N-14 ). NRC is a departmental Crown corporation and reports to Parliament through a designated Minister, currently the Minister of State for Science and Technology.

12.2 NRC has two operational programs: the Scientific and Industrial Research Program (Research Program) and the Scientific and Technical Information Program (Information Program).

Objectives

- To promote, assist and perform research for the creation of new knowledge derived from the natural sciences and engineering.

- To apply knowledge derived from the natural sciences and engineering to the solution of problems of national concern.

- The application and use of engineering and the natural sciences to assist industry in Canada with the development of new and improved processes, methods, products, systems, techniques and services.

- To develop and maintain standards and criteria based on the natural sciences and engineering.

- To provide and maintain services associated with scientific and technical information to meet the needs of the government and people of Canada.

- To conduct research into the need for and methods of accomplishing the transfer of scientific and technical information in response to user needs.

- To participate in the overall provision of library and information services for the government and people of Canada.

Resources

(Exhibit not available)

Organization

(Exhibit not available)

12.7 Programs. In its Research Program, NRC operates eight Ottawa-based laboratory divisions and three regional laboratories in addition to a number of national research and test facilities. Research staff is normally organized in divisions based on scientific discipline or by area of application. For example, the Division of Chemistry and the Hertzberg Institute of Astrophysics are discipline divisions; building research is an area of application. Divisions are headed by Division Directors who delegate project management responsibilities to Section Heads within their Divisions. Division Directors report to the Senior Vice-President, Finance and Operations. The Divisions are arranged in three groups, and a Director in each group is designated as the Group Director for purposes of planning and co-ordination.

12.8 As part of NRC's responsibilities for technology transfer and industrial assistance, the Program for Industry/Laboratory Projects (PILP), the Science and Engineering Student Program (SESP) and the Industrial Research Assistance Program (IRAP) are carried out under the direction of the Vice-President, Industry.

12.9 NRC also operates three major interlaboratory programs, each directed by a Program Manager who manages research activities or projects that cut across disciplinary and application areas. NRC's energy program is an example of this kind of activity. Program Managers report to the Executive Director, Interlaboratory Programs.

12.10 The Scientific and Technical Information Program was established in 1974, and its Director reports to the Vice-President, External Relations. The Canada Institute for Scientific and Technical Information (CISTI) is the major activity of this program. CISTI is involved in collecting, storing and disseminating scientific and technical information to universities, industry and other users. Another activity of this program is the publication of the Canadian Journals of Research.

12.11 The Council and Committees. The governing Council, composed of a President and up to 21 members, is appointed by Governor in Council. Council members are chosen from industry, universities and government departments and agencies. Members serve on a part-time basis. At 31 March 1981, the Council had 19 members, 10 from industry, 6 from universities and 3 from the public service. The President of NRC serves a term not to exceed five years; however, he is eligible for reappointment. The Council, chaired by the President, is responsible for general policy direction only. Day-to-day control is provided by management.

12.12 The governing Council is assisted in its work by a number of Committees. Membership on three of the Committees is drawn from NRC's senior management and from representatives of various laboratories:

- The Management Committee is NRC's senior management committee, carrying out or arranging for strategic planning and setting priorities.

- The Program Selection Committee is responsible for examining and selecting proposals for programs or projects requiring new resource commitments. The selected program or project then becomes part of NRC's operational plan and is included in resource requests to central agencies.

- The Tactical Studies Committee provides a management and communication link between the laboratories and senior management regarding priorities and resource allocation.

- Ad Hoc Review Committees review and evaluate specific parts of NRC research and service activities. They are chaired by either current or former Council Members.

- Advisory Committees advise on specific NRC research programs and projects and report either to Council or to particular division heads.

- Associate Committees are formed to address national needs and priorities, consider specific problems, provide a neutral forum where there is a shared federal/provincial interest or to act as a Canadian liaison for membership in international scientific organizations.

Environment

12.15 Historically, management of research and development activities has been characterized by a degree of informality and individual and group research autonomy. This is based on the concept that, in areas of pure and applied research, bureaucratic processes should not interfere with scientific creativity. Undue interference, some research organizations have maintained, can hamper the motivation and morale of staff, thus jeopardizing creative activity.

12.16 As with other areas of high technology and science, knowledge, creativity and skill are essential to progress at NRC. All research organizations must be capable of adapting to the availability of special technical resources, and often flexibility is limited by a lack of personnel with required research skills and expertise. In addition, the specialized nature of the human resources required means that it is important to plan carefully for future initiatives; the disciplinary skills of scientists and engineers are not easily transferred to other areas.

12.17 Over 30 different disciplinary and application areas and laboratories operate within NRC. These range from theoretical and basic research in areas like spectroscopy to applied research concerning applications of standards in fields such as building research. Thus, the types of work vary considerably, with management needs in areas such as information and control varying as well. Further, research and development activities are often characterized by long time delays between the initial investment of resources and the achievement of expected returns. This time lag can be for months or decades, depending on the nature of the work and the resources devoted to it. Therefore, it is not possible to manage all research work in the same way.

Audit Scope

12.19 To determine whether NRC has demonstrated due regard for economy and efficiency in the acquisition and use of its resources and has satisfactory procedures in place to measure the effectiveness of its programs, we assessed the systems and processes for strategic planning, operational planning and project management, and review and evaluation. In addition, we examined relevant support functions including financial management, management of materiel and capital projects, and internal audit.

12.20 Because the major initiatives of NRC are directed toward industrial support and development, most of our audit work was devoted to programs and projects related to this area.

Summary of Audit Observations

12.22 Our review of the Information Program and capital projects, which comprise approximately 19 per cent of the resources allocated to NRC, indicated that, in these areas, satisfactory systems and procedures were generally in place.

12.23 Through its Ad Hoc Review Committee process, NRC has taken reasonable steps to determine the scientific quality and relevance to objectives of its research efforts. The focus of our audit in the research program was not on the quality of the research effort at NRC but rather on the management systems and procedures.

12.24 Basic research. Our audit disclosed that NRC's management systems as applied to areas devoted exclusively to pure research, which makes up about 26 per cent of NRC's expenditures, were generally adequate for ensuring that due regard was paid to economy and efficiency of operations. As well, in many cases, program effectiveness in these areas was being measured, based on peer review of individual research efforts.

12.25 Applied research. NRC's role and the emphasis of its programs have shifted in recent years. Since 1974, NRC's Council has given priority to policies and programs for advancing and assisting industrial research and development in Canada. In 1979, NRC began a long-range planning exercise in which objectives were revised and themes or priority areas were identified. This plan, called The Urgent Investment , has received approval in principle from the Government.

12.26 Given this reorientation toward industrial development, we concluded that, even though NRC has been initiating a number of changes in its management systems, there were still several significant deficiencies in these systems as they relate to applied research, which accounts for approximately 37 per cent of NRC's resources.

12.27 First, long-range plans were not sufficiently specific in support of NRC's objectives to provide adequate guidance to its operational planning efforts so as to ensure that appropriate programs would be developed, projects selected, and achievement measured.

12.28 Second, there was a lack of a formal planning process to link NRC's objectives and the actual laboratory programs and projects being carried out, except with respect to projects requiring new resources. In addition, the information and data from outside sources, such as industry, were generally inadequate for ensuring that research programs and projects, which have as their objectives assistance to industry, were likely the most effective.

12.29 Third, the laboratory projects proposal, planning and approval processes were not adequately documented, and there was a general lack of documented, systematic monitoring of continuing programs and projects. Those procedures in program planning, selection and approval that did exist were directed primarily toward initiatives requiring new or additional resources.

12.30 Fourth, satisfactory procedures were not in place across NRC to measure the effectiveness of its applied research programs.

12.31 Financial function. Staff of the Financial Services Branch were not appropriately involved in the financial aspects of program planning or in the development of information systems that had financial implications. In addition, there was a need to improve the financial information available to managers for planning, controlling and evaluating the activities for which they were responsible, including revenue-earning operations.

Observations and Recommendations

Strategic Planning

12.33 Because a decision to embark on a new program or project can have important resource implications for years to come, an important point in the planning process is the selection of research programs or projects. It is essential that the selection of programs or projects is consistent with objectives and that expenditures are justified. This usually means that the process must be reasonably formal, rigorous and comprehensive for large organizations such as NRC.

12.34 We examined the strategic planning process at both the corporate and divisional levels of NRC. We looked at five aspects of strategic planning -- stating objectives, the planning process, technology transfer, identification of long-term trends, and identification of industry needs.

12.35 Stating objectives. We found that NRC's objectives had not been translated into operational terms, and priorities had not been defined. For example, one of NRC's major objectives stated in the Estimates is "the application and use of engineering and the natural sciences to assist industry in Canada with the development of new and improved processes, methods, products, systems, techniques and services". Because of the limited research resources within NRC and given the broad spectrum of Canadian industry, we would expect to find a documented set of specific industrial priorities. Our premise is that the benefits to be derived from assisting Canadian industry vary with the choice of these priorities.

12.36 Support for the need for such focusing can be found in the Electrical Engineering Division. The 1980 Ad Hoc Review Committee for one section of this Division commented that there was a need for stronger, central direction, that is, a unified policy and defined goals; program development was in response to specific requests and available expertise; objectives could have been made clearer and more specific. This was done in the Solar Energy Program, where a planning framework had been developed for use in project and program selection.

12.37 If objectives are not adequately clarified, either through stating them more specifically or through a process of translating them into operational terms, relevant goals or targets cannot be set, and thus it is difficult to monitor research activities and evaluate programs and projects. Because NRC management had not defined its objectives in operational and specific terms, it could not hold research groups accountable for achieving objectives. This also made it more difficult for Parliament to hold NRC accountable for achieving objectives.

12.38 NRC should translate its objectives into operational terms to guide its planning efforts and to provide for subsequent assessment of performance.

12.39 Planning process. There was no mechanism for co-ordinating and reviewing strategic planning efforts across Divisions, except for initiatives requiring new or additional resources. There were few procedures to assist Divisions to improve their planning processes and methodologies, to undertake broad strategic studies on behalf of NRC, or to monitor progress against the long-range plan and inform senior management of progress.

12.40 NRC should ensure that procedures are put in place to co-ordinate strategic planning efforts and to assist Divisions to improve methodology for planning.

12.41 Technology transfer. Although NRC had a number of mechanisms for transferring technology to industry, such as direct negotiation with industry by laboratories, internal or external contacts, joint research and the Program for Industry/Laboratory Projects, we found no comprehensive, documented approach for achieving this technology transfer objective.

12.42 Several of NRC's Ad Hoc Review Committees found that attempts at technology transfer had been poorly co-ordinated and based on misconceptions concerning how such transfers should be achieved. Committee reports cited instances where companies had been selected as recipients for this type of transfer, even though the companies were not commercially viable and therefore were not likely to be able to use the technology.

12.43 NRC should develop a comprehensive approach for technology transfer as part of the strategic planning process.

12.44 Identification of long-term trend. We observed that there were deficiencies in the amount, quality and completeness of information gathered from both inside and outside NRC to be used as input to the strategic planning process.

12.45 It would be reasonable to expect that, as part of the strategic planning process, a formal assessment of the external environment would be done. This assessment would include long-term technological and scientific trends, changes in the economic and political environment, and the long-range prospects and needs of industry. Although formal long-range needs assessment and market studies had been completed in some Divisions, others based their long-term plans on informal, internal discussions. There was in general, a lack of data supporting formal long-range analyses of future requirements for research and little attempt to assess long-term potential economic or social benefits of undertaking research programs. Research managers were generally aware of changing trends in technology; however, there was no one at NRC responsible for co-ordinating "external" information or for analyzing and projecting trends that could have a significant effect on ongoing research efforts.

12.46 The implication of this situation is that planning could be based primarily on short-term considerations without proper recognition of long-term needs and trends. This, in turn, could mean that resources would not flow to those areas that would be of the greatest benefit to Canada.

12.47 NRC should ensure that analyses of environmental trends and related research needs are conducted by each Division, as appropriate, or by NRC as a whole, and that the results of these analyses are integrated to provide a basis for developing its operating strategies.

12.48 Identification of industry needs. A specific major problem area is the identification of industry needs. The Industrial Materials Research Institute in Montreal has undertaken a number of technical-economic studies for materials research. The Laser Chemistry Group and the Solar Energy and Computer Technology Research Programs have initiated long-range needs assessment studies. Generally, however, NRC had no formal procedures for assessing long-range industrial research support requirements.

12.49 Many Divisions informed us that they kept in touch with research requirements in industry through informal contacts, seminars, publications, and by visits to industrial research laboratories. While this is a necessary part of maintaining an awareness of industrial needs and technological trends, there is also a need for a systematic assessment of industrial research support requirements. The lack of such an assessment meant that little guidance had been provided to managers who are responsible for establishing criteria for selecting projects to assist industry at the laboratory level.

12.50 Nine out of 17 Ad Hoc Reviews indicated that the assessment of long-term industrial needs for research was inadequate. This weakness should be a matter of major concern to NRC, since one of the critical elements of a strategic planning process is the identification of long-range client or market needs. There must be a means of ensuring that this takes place and is subject to continuing review and re-assessment. Without such a process to assess client needs, there is a risk that substantial sums of money could be spent with insufficient justification in terms of industrial support, economic benefit or prospects of transfer to industry.

12.51 NRC's Divisions should undertake further analyses of the needs of industry for research support.

Operational Planning and Project Management

12.53 Project management is a process concerned with setting specific objectives, targets and milestones, monitoring progress and making decisions about the allocation or reallocation of resources for individual projects. It focuses specifically on control and direction of research projects by individual units within an organization.

12.54 At NRC, we would have expected to find that:

- divisions and other appropriate organizational units had completed an operational plan annually, including details of goals and anticipated resource allocation for major programs and projects;

- each unit had appropriate procedures and guidelines in place for preparing project proposals, screening them, and documenting their progress and review;

- each division had a project management system in place, identifying for each program and project the resources required, the objectives and a documented plan for achieving these;

- specific clientele and their requirements were clearly identified; and

- provision had been made for monitoring and periodic review of project progress.

12.56 Our audit indicated that NRC's systems and procedures were generally adequate for effectively planning and controlling the Industrial Research Assistance Program (IRAP) and the Science and Engineering Student Program (SESP). We found that the goals and objectives of each Program had been stated in terms specific enough to identify the companies toward which each Program was directed and to develop criteria for selecting the most appropriate proposals for funding. In general, we found the process for reviewing applications and selecting proposals and for assistance was adequate for ensuring that approved applications met the selection guidelines and were consistent with the objectives of these two Programs. However, we found some instances of deficiencies in their administration.

12.57 The newer interlaboratory programs such as Computer Technology Research, Solar Energy Research and Development, and the Industrial Materials Research Institute (IMRI) were in various stages of implementing project or task management systems. For example, IMRI sets up a project dossier for each project undertaken. The dossier includes a project initiation sheet, a project justification report, quarterly progress reports, and a report when the project is completed. Many of the older Divisions, however, used their annual reports to describe ongoing programs and the resources devoted to each of them. Objectives and accomplishments were usually incorporated into the reports; however, our review of the annual reports indicated that they lacked detailed information on individual projects in terms of plans, objectives, and accomplishments with respect to plans.

12.58 Initial allocation of new resources to a program is screened, ranked in priority and co-ordinated by a Program Selection Committee and approved by the Management Committee and Council. The Program Selection Committee is a recent innovation at NRC; approval for new resources was formerly given by the Management Committee on the advice of the Tactical Studies Committee. Once the request for new funding is approved and the funds obtained through the Estimates, the program area and attendant resources become part of a Division's continuing program. It is then up to the Division, usually the Director, to approve new projects or tasks within each program area or to reallocate resources to other projects within his Division. Depending on the size of the project and the size of the Division, this process may be delegated to the Section Head. For example, in Divisions such as the National Aeronautical Establishment, Electrical Engineering, and Chemistry, the Section Head can usually decide to embark on small new projects that fall into the program area he supervises. Usually, however, the Director decides on major resource commitments, external contracts or shifts in resources between projects. Generally, ideas for projects originate with the scientists and are approved at the laboratory level.

12.59 In most of the Divisions, there was little documented identification of proposed projects, the rationale for their approval, the resources spent on the project, or what was being achieved. Divisional managers informed us that projects were managed through informal management systems and formal, but not always recorded, review meetings. Any dissemination of results was usually through research publications. Throughout this process, there was little systematic review of programs against goals or objectives set for them.

12.60 Although a project identification system was being implemented, at the time of our audit few divisional or senior managers had put in place procedures that would give them a comprehensive picture of all projects and studies under way in a particular year, the resources to be expended on each, and the expected achievements or targets for each project. There was no mechanism to document progress reviews, report activity to senior management or to hold managers to account.

12.61 NRC should develop procedures for the use of Divisions in identifying and reviewing both new initiatives and continuing programs. Procedures should provide for documentation of review results, including the continuing relevance of project objectives.

12.62 NRC should implement project management systems, including as a minimum:

- formal project justification and selection and review procedures;

- identification of project goals and milestones related to operational plans; and

- periodic progress reporting and review against milestones and goals.

Review and Evaluation

12.64 In examining the review and evaluation procedures of NRC, we expected to find that:

- there was an evaluating plan in place, with procedures for evaluating periodically all projects and programs for which evaluation was reasonable in the circumstances;

- information about the results of program or project evaluation was being communicated to senior management; and

- recommendations arising from evaluations were being acted on appropriately and follow-up actions were being monitored.

12.66 We found that a second system of project review was based on elements of personnel appraisal systems. Scientists generally rely on peer group recognition in their chosen field; therefore, appraisal procedures in the basic research areas are heavily slanted toward assessing numbers of publications, citations, and other measures of stature in the scientific community. The orientation of this type of appraisal is toward the technical and scientific performance of the individual. This approach is consistent with procedures followed by other basic research institutes.

12.67 A third level of evaluation occurs through the Ad Hoc Review Committees. Reviews are conducted periodically by a selected group of outside experts in particular fields. The mandate of these reviews is broad and includes a review of activities of selected sections in relation to NRC objectives and the quality, effectiveness and future direction of research activities.

12.68 A fourth level consists of the program effectiveness evaluations of the Industrial Research Assistance Program, the Science and Engineering Student Program, the Program for Industry/Laboratory Projects and the Technical Information Service.

12.69 Basic research. Our findings indicated that, although the National Research Council used the second method -- evaluating the scientific efforts of personnel involved in basic research -- evaluations were a by-product of the Council's promotion system. Under this system, formal appraisals were only produced for personnel being considered for advancement.

12.70 Although NRC used measurement criteria accepted by the international scientific community in evaluating scientific personnel and their activities, there were no explicit procedures for evaluating the activities of research teams involved in basic research, except in cases of above average performance.

12.71 NRC should ensure that, where feasible, all basic research team activities are evaluated on a formal and regular basis.

12.72 Applied research. NRC has attempted to assess the economic benefits resulting from the Industrial Research Assistance Program and the Science and Engineering Student Program. The evaluation results were reported in The Urgent Investment . Our review of the methodology used in these evaluations revealed a number of deficiencies.

12.73 NRC had developed certain indicators of economic benefits for use in its IRAP evaluation; however, in our opinion, these indicators were deficient as measures of benefits. One of the indicators was the ratio of the dollar value of companies' sales, derived from projects partially funded by IRAP, to IRAP costs. The use of this indicator was not appropriate because it took into account only IRAP costs and not costs to companies involved in IRAP projects that might have exceeded IRAP costs. The indicator did not take into account certain factors in addition to IRAP that might influence sales. That is, sales could be affected by the company's own marketing efforts, competitors' activities, customer reaction and general economic conditions. Given the deficiencies in this indicator, the benefits flowing from IRAP may have been overstated.

12.74 Our audit also revealed deficiencies in the information included in the annual report on SESP. The statement of financial benefits resulting from SESP projects of three to four-month duration was often based on companies' unverified projections of increased sales, gross profit and reduced operating costs over a five-year period. In addition, we found inconsistencies in the methods used to calculate financial benefits resulting from SESP projects.

12.75 Except for the specific program evaluations mentioned previously, we found that none of the Divisions had developed formal procedures for evaluating the impact of the results of individual applied research projects or programs. There was no evidence to indicate that post-project or program reviews were carried out. These reviews are particularly important for projects or programs of an applied nature in order to assess whether short and long-term objectives set out for the project or program have been met.

12.76 NRC should ensure that, where it is feasible and cost-effective, applied research projects and programs are periodically evaluated in terms of continuing relevance and results achieved. In carrying out evaluations, appropriate and cost-effective methodologies should be used.

12.77 Reporting. There was little evidence to indicate that information about Division activities, progress toward objectives, and results achieved was summarized and communicated to senior levels of management to provide an indication of progress or objectives achievement. We were told that some information is communicated informally to senior management, but that there was no means to ensure that this was done regularly. Where information was formally communicated through annual reports produced by selected Divisions, it provided only a project-by-project review of technical progress, lists of publications, patents and meetings attended. Except for Ad Hoc Reviews, there were few plans for reviewing projects and programs periodically or for communicating results to management.

12.78 At the time of our audit, there was no designated group responsible for evaluating NRC's programs. Without designated responsibility of this kind within the organization, NRC can have no assurance that cyclical evaluations related to achieving objectives will take place. NRC has proposed the establishment of a program services secretariat that would include this evaluation function.

12.79 NRC should establish procedures to co-ordinate, plan, and monitor evaluation efforts and ensure that all results are reviewed by senior management.

12.80 Ad Hoc Committee Reviews. External Ad Hoc Committee Reviews are a positive initiative; however, we observed inconsistencies in the approaches used by various Committees. For example, some focused on projects; others on general management procedures. NRC has recognized that the Committees do not always provide a rigorous review of achievement of objectives.

12.81 NRC should examine the procedures for Ad Hoc Committee Reviews and establish criteria to ensure consistency and adequacy in coverage and approach.

Information Program

12.83 Our review indicated that the management systems and procedures in place were generally satisfactory. Management had, at the time of our audit, initiated a study to identify user needs and their relation to CISTI.

Support Services

12.85 The Financial Services Branch provides a central service to the decentralized NRC organization. Divisional administrative officers provide for financial needs in Divisions and are thus relied on to identify problems and opportunities for improvement. These officers take functional and program direction from the Senior Vice-President and functional direction from Financial Services.

12.86 The responsibilities of senior financial staff were limited to operational systems and controls and preparation of budgetary reports for central agencies. The Financial Services Branch did not participate directly in the financial aspects of program planning and was not always consulted in the development of information systems that had financial implications.

12.87 We found that functional guidance in operational processing controls and in matters related to financial policies was satisfactory. NRC has recently strengthened these areas, and financial staff plan further improvements.

12.88 NRC should clarify the mandate of the financial function to include involvement in the relevant aspects of management decision making, including planning and evaluation, and responsibility for reviewing the financial control aspects of all management information systems.

12.89 Financial information. Managers require information on progress against plans, both for resources applied and results or outputs generated, on which to base decisions on corrective action needed. Information for controlling annual appropriations is also required. At NRC, the FINCON system (an on-line commitment control and financial reporting system) provided adequate information for appropriation control and some additional statistical information.

12.90 Managers also need to have information, in a form related to their operations, that shows costs and revenues compared to budget. Analysis of variances would also normally be expected to show the reasons for deviations from budget and identification of possible savings or overspending. A few Divisions have developed their own systems to give project-oriented information of this nature; however, there was little systematic accumulation and reporting of financial information related to project results and expenditures. NRC recognizes the need for such information and, to address it, has been developing a project information system over the past two years.

12.91 Although it is accepted that the information systems of individual Divisions may be different, it is necessary that the results can be aggregated for senior management and others. Except for appropriation control, there was no overall plan to ensure that management received adequate information on which to base decisions.

12.92 NRC should develop a plan to meet managers' needs for financial information for planning, monitoring, control and evaluation. The plan should ensure that:

- financial information for planning and evaluating activities is related to the information used for monitoring and controlling operations and is reported to senior management; and

- information on the results of activities identifies costs and revenues compared to budget and includes analyses of variances, where appropriate.

12.94 We found that internal control over the billing and collection functions was adequate. However, rate-setting policies did not provide sufficient guidance to laboratory and program managers who make most of the decisions with respect to revenue. Regular reports were not provided to management setting out the effect of rates which do not recover the cost of the services concerned or of discounts given to certain customers.

12.95 Central management reviewed rate revisions for some cost-based services; however, rates were inconsistently calculated. For example, in the National Aeronautical Establishment wind tunnels, the rates did not recover capital costs from Canadian users although they did from foreign users. Based on current levels of usage, the recovery of these costs in the 30 foot wind tunnel alone would raise about $170,000 more revenue annually.

12.96 NRC procedures do not require managers to advise central management routinely when they provide new types of services without charge to outsiders even in situations where a charge might reasonably be made and the amount involved is sufficient to warrant making a charge. In the changing environment of NRC, it is possible that potential sources of revenue may be overlooked.

12.97 Financial information regarding revenue-producing services was limited to recording annual budgets and actual charges. Information to indicate maximum potential revenue, the extent to which the capacity of revenue-producing facilities was being used, the effect of discounts, and other information on the management of revenue was not systematically gathered and communicated. As a result, management did not know whether maximum allowable revenue was being collected.

12.98 NRC should make guidelines for rate setting more specific to ensure that line managers consider all relevant costs in setting rates.

12.99 Financial information systems should display to management the results of revenue-earning operations. In particular, the extent to which the capacity of revenue-producing facilities is being used and the effect of discounted or no-charge services should be displayed.

12.100 NRC should review periodically all activities and facilities providing services to parties outside the government to ensure that its revenue policies are being uniformly applied.

12.101 Materiel management. Since 1978, NRC has had a Central Inventory Control Office and now has a computerized inventory record system. This system produces an inventory of all NRC capital equipment. Although we noted some errors in information provided, generally it appeared to meet management's needs for inventory information and control.

12.102 Capital projects. As indicated in Exhibit 12.1, approved capital expenditures for 1980-81 amounted to approximately $31 million, or 13 per cent of NRC's estimates. We reviewed the planning and acquisition phases of two capital projects -- the Remote Manipulator System and the Fire Research Station. Both of these projects were nearing completion at the time of our audit.

(Exhibit not available)

12.103 Although there were no documented internal project management guidelines, our findings indicated that the management systems and procedures for those two projects were satisfactory.

12.104 Internal audit. Since its inception in June 1979, the internal audit group has completed five projects and has established its mandate and plans for audit coverage. Most of these projects were carried out in late 1980 and early 1981 and annual audit plans were reviewed and found satisfactory by the Office of the Comptroller General in 1980 and 1981.

12.105 The Chief of Internal Audit reports to the President of NRC; however, the internal audit group has had only general contact with the President and the Audit Committee. The group expects that the President will review and clear important audit reports in future.

12.106 Recent internal audit reports have addressed financial compliance and procedures to measure effectiveness; however, it is too early for us to assess whether internal audit can be used as a comprehensive management control by senior management and the Council. Because internal audit results were not available at the time of planning our audit, we were unable to rely on them in determining our audit scope.

Information for Parliament

12.108 Planning priorities for the 1980s. NRC's long-range plan, The Urgent Investment , describes six major "themes" and strategies for achieving them. One theme, for example, is to stimulate the gross domestic product of the technology intensive industrial sector of the economy from $20 billion to $40 billion by 1990, and to achieve specific employment and balance of payments benefits. Disclosure of specific objectives and their desired long-term results would give users of the Estimates a basis for assessing the direction of the program and for questioning specific milestones and resources requested. More work needs to be done in specifying objectives and milestones before these can be included in the Estimates.

12.109 Results and outputs of applied research. Given NRC's primary emphasis on research in direct support of industrial innovation and development, the Estimates should identify, where appropriate and reasonable, the expected results of applied research. This would help parliamentarians assess the value of these results in relation to the funds requested.

12.110 Results and outputs of basic and exploratory research. Although it is extremely difficult to identify results and outputs for the area of application that NRC calls "advancement of knowledge", this type of research should, nevertheless, be dealt with in more detail in the Estimates because it consumes a sizeable share of program expenditures. Where intended results and outputs cannot be stated precisely, the Estimates should describe the work in progress and classify basic and exploratory research projects or programs by expected life span.

12.111 Personnel. NRC research staff are a highly specialized and valuable resource as well as the most critical ingredient in achieving the organization's objectives. In fact, the skills and interests of personnel partly determine the choice of basic research projects many years into the future.

12.112 Personnel and associated costs are NRC's largest single expenditure item. Information from the current "Blue Book" concerning the number of scientific, professional and technical personnel should be expanded into a profile of the types of staff, their levels and average salaries, and the degree of disciplinary specialization throughout NRC. An explanation of the professional staff development and training expenditures might also be related to the staff profile.

12.113 Over the past few years, expenditures on research and development contracts and other special and professional services have increased until they now amount to almost two-fifths of budgeted payroll costs. An explanation should be provided in the Estimates of the rationale for and the types of work done by special and professional service contracts.

12.114 Grants and contributions. The Public Accounts Committee has requested more details in the Estimates on grants and contributions generally. This is especially appropriate for NRC because, after salaries and wages, contributions are the second largest item in its Estimates. Contributions toward research in industry should be classified by industry, types of business and location. If this information is not known when the Estimates are prepared, it should be provided in the Public Accounts. The objectives of major projects like the Tri-University Meson Facility (TRIUMF) should be explained, in addition to the expected effects and how these relate to the program objectives. The Estimates should also show the eligibility criteria for funding and the response time for processing applications for grants and contributions.

12.115 Major capital project. NRC currently has several large capital projects. In addition to the information now shown, the Estimates should state the purpose of projects, distinguish new projects from those begun in prior years, show the estimated completion date and future requirements year by year, and explain changes from the estimated total cost for a given project in the previous year's Estimates.

12.116 Revenue. Although revenue is not a particularly large item in relation to total expenditures, there is a need to demonstrate to Parliament that a logical, consistent revenue policy is being followed. An explanation should, therefore, be provided of the services for which NRC does and does not charge and the basis of their rates or fees if a charge is made. Where practical, a comparison should also be shown of the costs and revenues of particular services.

Summary of Recommendations and NRC's Comments

Strategic Planning

Recommendation

NRC's Comment

Recommendation

NRC's Comment

Recommendation

NRC's Comment

Recommendation

NRC's Comment

Recommendation

NRC's Comment

Operational Planning and Project Management

Recommendation

NRC's Comment

Recommendation

- formal project justification and selection and review procedures;

- identification of project goals and milestones related to operational plans; and

- periodic progress reporting and review against milestones and goals.

NRC's Comment

Such a project management system, however, used in its entirety, is inappropriate for fundamental research. Research whose results are foreseeable is very often poor research.

Review and Evaluation

Recommendation

NRC's Comment

Recommendation

NRC's Comment

Recommendation

NRC's Comment

Recommendation

NRC's Comment

Support Services

Recommendation

NRC's Comment

Recommendation

- financial information for planning and evaluating activities is related to the information used for monitoring and controlling operations and is reported to senior management; and

- information on the results of activities identifies costs and revenues compared to budget and includes analyses of variances, where appropriate.

NRC's Comment

Recommendation

NRC's Comment

Recommendation

NRC's Comment

Recommendation

NRC's Comment