1988 Report of the Auditor General of Canada

Main Points

The Canadian International Development Agency (CIDA) is responsible for the largest portion of Canada's expenditures dedicated to helping less-developed countries sustain greater social and economic progress. This year's report focusses on CIDA's contracting, aid information, and Special Programs Branch contributions to non-governmental organizations. (paragraphs 9.1 to 9.4.)

Contracts are an essential element in CIDA's delivery of international aid. Work performed under these contracts, however, does not consistently respond to project needs approved by CIDA and the recipient country. (9.16 to 9.34.)

CIDA does not have adequate procedures to ensure that goods are purchased at prevailing market prices or that they have the required Canadian content. (9.35 to 9.42.)

Monitoring of contract implementation is a required part of contract administration. CIDA has procedures to review contractor performance but these need improvement to ensure monitors are in place at the right time and produce the appropriate reports. (9.43 to 9.48.)

CIDA has an Aid Information System to provide financial data and track project implementation. This system which took six years to develop, costing about $40 million, produces adequate financial information but CIDA is not achieving value for money from its project management capability. (9.49 to 9.59.)

CIDA provides funds through the use of contribution agreements to Canadian and international non-governmental organizations involved in international development. Most of these organizations are implementing projects in accordance with their agreements with CIDA. (9.60 to 9.67.)

CIDA's monitoring procedures are not well matched to the nature and types of non-governmental organizations receiving funds from the Special Programs Branch within CIDA. (9.68 to 9.79.)

Background

9.1 The Canadian International Development Agency (CIDA) is responsible for the largest portion of Canada's expenditures dedicated to helping less-developed countries sustain greater social and economic progress. This responsibility is carried out through the International Development Assistance Program (IDA), which provides goods and services, and sometimes funds, to recipient countries in a co-operative effort to assist in their development.

9.2 International development requires CIDA to co-operate with a wide range of partners in Canada and overseas. Consequently, CIDA draws on the expertise of hundreds of Canadian private firms, voluntary organizations and other federal and provincial government agencies, as well as the governments, institutions and voluntary organizations of the developing countries themselves. Through these organizations, CIDA is able to deliver assistance that would be beyond the capacity of the Agency using its human resources alone.

9.3 CIDA's budget of approximately $2.1 billion (1987-88) is divided into Bilateral programs ($957 million); Food Aid ($400 million); Voluntary Organizations ($256 million); Multilateral programs ($152 million); International Development Banks ($146 million); Humanitarian Assistance ($53 million); Industrial Cooperation ($47 million); and Administration ($92 million).

Scope

9.4 In our previous examinations of CIDA's main programs, conducted in 1979 and 1984, we observed a number of system weaknesses in the preparation of contracts to procure goods and services, as well as in the management of information. Monitoring of contributions to voluntary organizations was another area of concern. This report focusses on these issues in greater detail, in addition to reviewing the information provided to Parliament in Part III of the Estimates.

9.5 Contracting. CIDA spends approximately $1.1 billion annually to procure goods and services for countries receiving aid. These goods and services include a wide variety of commodities, manufactured products and technical expertise to support specific development objectives. Our audit work entailed a review of the overall contract process, focussing on definition of requirements, statements of work, pricing, Canadian content and contract administration. This review included the examination of 39 contracts totalling $242 million, signed in the period from April 1986 to June 1987, that were randomly selected from CIDA files.

9.6 The 39 items in our sample represent all major types of contracts in which CIDA is involved, with the exception of contribution agreements, intergovernmental agreements and contracts with CIDA co-operants in the field. (Contribution agreements are discussed in our section on Special Programs.) In addition to reviewing CIDA contract files at headquarters, we examined activities for 27 of the contracts at project sites in 10 countries. This field audit work formed the basis for observations that could not have been obtained through file reviews at CIDA headquarters alone. Details gathered from project site audit work, in conjunction with file data, are presented in several case studies illustrating some of the consequences of weaknesses we observed generally throughout the contracting process.

9.7 Aid Information. The CIDA Aid Information System (AIDIS) is the electronic record of the Agency's financial and non-financial activities. Our first review of this system was described in the section of our 1984 annual Report dealing with the Comprehensive Agency Management Plan. At that time, CIDA planned that the Aid Information System would be fully operational in October 1984, with enhancements planned for completion in 1987. With these changes now complete, we audited AIDIS to determine how the cost of the system compares with original estimates, the extent to which the cost of development has been controlled and how well the system addresses the financial and non-financial information requirements of the Agency.

9.8 Special Programs. CIDA's Special Programs Branch is responsible for encouraging and supporting partnership relations with Canadian-based and international non-governmental organizations. Non-governmental organizations are an increasingly important source of international development expertise. The Financial Administration Act governs CIDA contributions to these organizations and entails certain accountability requirements. We assessed adherence to these accountability requirements in our examination of Special Programs.

9.9 Information to Parliament. The purpose of Part III of the Estimates is to provide more and better information to Members of Parliament, thereby permitting them to carry out their review of expenditures more effectively. This information should enable Parliament to gain a basic understanding of each program and assess planned and actual performance in terms of results and resources. In this context, our audit examined the information presented in CIDA's Part III to see whether the document accurately reflected CIDA's program structure and program objectives. We also looked at the style of presentation and the extent to which performance information was provided.

Contracting

9.10 The management of contracting is vital in the delivery of bilateral aid, subsequent to the initiation and approval of specific development projects. Contracting activity is governed by the Financial Administration Act and by government contract regulations; CIDA policy manuals contain detailed information on the process.

9.11 Broadly described, bilateral aid begins with the preparation of a Country Program Review document outlining CIDA's long-term development strategy for the recipient country. From this general strategy, CIDA and the recipient country select a number of projects and Plans of Operation are developed. These projects are implemented through contracts for goods and services that meet the development needs of the recipient country. A project's Plan of Operation may require a number of separate contracts.

9.12 CIDA's Project Team Leaders are responsible for all phases of the contract process and must ensure compliance with CIDA policy and with governmental contract regulations. The contracting process also requires CIDA to respect the role of other government departments involved in Canadian government procurement, in particular the Department of Supply and Services (DSS). Rules governing Canadian government procurement require that DSS purchase goods on behalf of CIDA, but DSS may delegate this authority to CIDA. CIDA has authority to contract for services within approved limits.

9.13 Although CIDA and DSS normally conduct Canadian government international development procurement, authority for procurement is sometimes given to the recipient country. The CIDA Project Team Leader, however, is still responsible for ensuring that contract terms and conditions match Canadian contracting regulations as closely as possible. A CIDA Principal Resource Officer provides technical expertise and advice to the Project Team Leader. Although CIDA sometimes provides in-house resources for contract monitoring, Project Team Leaders often have contracts with Canadian firms to do this work in the field.

9.14 Our audit of the contracting process indicated that CIDA's bilateral manuals and contracting handbooks conformed to the government contracting regulations and the Financial Administration Act. The contracting process is separate from that of bilateral project management, where the needs of the recipient country are identified and agreed on in a Memorandum of Understanding. Our review did not include an examination of CIDA's procedures to identify project needs or to evaluate competing contract proposals, because CIDA keeps only successful proposals on file. CIDA is currently reviewing its policy on records management.

9.15 Our audit focussed on active or completed contracts. Of the 39 contracts we examined, two-thirds contained no significant departure from CIDA contract policies. The remaining contracts revealed weaknesses in requirements definition, statements of work, pricing, Canadian content and contract administration. Concerns similar to ours were raised in recent internal audit work on procurement.

Requirements Definition

9.16 Contracts are an essential element in CIDA's delivery of international aid. Work performed under these contracts, however, does not always respond to project needs approved by CIDA and the recipient country.

9.17 Requirements definition is the first step in the contracting process. The Project Team Leader must interpret the information contained in the project approval documentation and Plan of Operation in order to clearly specify the requirements of the contract. The Project Team Leader must also ensure the contract requirements are consistent with items, elements or components identified in the project approval documentation and Plan of Operation.

9.18 Our review indicated that CIDA practices need improvement to ensure that contracts for the procurement of goods and services consistently match the requirements identified in the project documentation. These requirements are stated in the project Plan of Operation or Memorandum of Understanding signed by CIDA and the recipient country. CIDA program manuals provide a formal process for reviewing contracts in relation to program priorities and to project approval documentation; however, the process is not consistently followed. Approximately half of the contracts in which we observed deficiencies contained inadequate references to the key project requirements identified in the Memorandum of Understanding and Plan of Operation.

9.19 The omission of key contract provisions pertaining to project requirements has a significant impact on CIDA's ability to hold contractors to account. A contractor is legally required to perform only the work identified in the signed contract. Where all significant project requirements are not included in the contract, CIDA cannot ensure that the work will meet the requirements identified in the project approval documents agreed on by CIDA and the recipient country.

CASE STUDY

Hydroelectric Dam - China

Issue: Requirements Definition

9.20 In August 1986, CIDA signed a $13 million contract with a consortium of Canadian engineering firms and provincial power utilities to conduct a feasibility study on the technical and economic viability of the Three Gorges Hydroelectric Dam project in China. The consortium completed the work in late 1987 and submitted a draft report for approval to the international steering committee's panel of experts in January 1988. At the time of our audit, most of the report had been approved and the Chinese authorities were satisfied with the quality of the work.

9.21 This project, as approved by Treasury Board, had two principal objectives. The first objective was to develop and implement a program of transferring hydro-electric power project technology to the Chinese. The second objective was to produce a feasibility study of the Three Gorges project acceptable to international lending institutions.

9.22 Our audit disclosed that the contract statement of services developed jointly by CIDA and the Chinese authorities addressed only the second objective. Although a significant transfer of technology took place during the feasibility study, the contractor was not charged with the responsibility of developing and carrying out a formal technology transfer program as stated in the project documentation.

9.23 The absence of this objective in the statement of services led to a gap between what the Chinese had expected in terms of technology transfer and what they actually received. Our audit revealed that, while they were satisfied with the feasibility study's results, they did not obtain the understanding they had anticipated with respect to the methodology, principles and procedures used to arrive at the study's conclusions. The contract contained no details on how transfer of technology was to be achieved; for example, in methodology for sedimentation calculations; construction design; environmental impact analysis and resettlement; economic and financial analysis of dam construction, including cost-benefit analysis; design programs for power planning and transmission; and project planning and management.

9.24 Our review also indicated that CIDA does not challenge the need for goods when the Agency is not the prime contributor of funds to a development project. In the project approval process, estimates of the funds required for a project are made at an early stage without full knowledge of all contract or project implications. In some cases, the amounts eventually needed to fulfill project requirements are different than were estimated and approved. This sometimes leads to a re-profiling of the project components. Often, CIDA or the recipient country will identify a new contract requirement within the current project framework. Such newly identified contract requirements should undergo appropriate challenge, but CIDA does not carry out this challenge consistently, particularly when it is not the main contributor of funds to a project. Instead, CIDA relies on the judgement of the other contributors. And in cases where the contract is reviewed by CIDA, there is no assurance it will be done by an officer with sufficient sectoral expertise to understand the contract in relation to overall project objectives.

9.25 It is essential that CIDA review contract requirements when the recipient country is the designated contracting authority. In these cases, CIDA's responsibility is to ensure the contract process does not contravene Canadian government policy or fail to address the contract requirements of the recipient country. The primary purpose in designating the recipient country as the contracting authority is to enable the country to develop expertise in dealing with Canadian suppliers. However, there are no criteria to indicate when this authority is to be granted. Without direct review of the contract requirements, CIDA is unable to know how closely they match the project requirements. This can lead to a situation where funds are used for purposes that match neither CIDA's project authorization nor the recipient country's priorities, resulting in procurement that may be of little benefit to the recipient country.

CASE STUDY

South Sumatra Coal Transport

Issue: Review of Contract Requirements

9.26 The objective of the Indonesian South Sumatra Coal Transport project is to improve the transportation of coal from the Bukit Asam mine to the sea terminal on the coast of Sumatra. Indonesia is the contracting authority. Using parallel financing, 75 percent of the cost of the project is funded by a commercial loan from the Export Development Corporation; CIDA is responsible for providing the remaining 25 percent. CIDA's responsibility for the project, as approved by Treasury Board in 1981, was to provide funds for the purchase of locomotives and a railway communications system. These goods were only one component of a larger Indonesian project which included the refurbishment of rail lines. In 1986, the Indonesian government identified a need for additional steel rails that were to be paid for using CIDA and EDC financing. CIDA subsequently approved the re-profiling of the project to include the purchase of rails. However, because the project is 75 percent financed by EDC through a commercial loan, CIDA did not challenge the need for the quantity of rails to be purchased; nor did it seek the advice of the project monitor in Indonesia. Indonesia then purchased from a Canadian supplier 197 kilometers of steel rails costing $15 million, of which CIDA paid $3.75 million.

9.27 Our audit found that 116 kilometers of this rail, representing $2.2 million of CIDA funds, were not needed for the purposes described in the project documentation. Of these, 66 kilometers will be used to replace rails that do not need to be replaced until 1994. The remaining 50 kilometers are for a maintenance line needing only used rails that were readily available at the site. At the time of our audit, 14 months after delivery and one month before expiry of the supplier's guarantee, approximately 50 kilometers of rail were still unused.

Statement of Work

9.28 A statement of work or description of services in a contract is mandatory when CIDA retains a contractor in the field. It is the responsibility of the Principal Resources Officer to provide a clear statement of work specifying the objectives to be attained and the type and scope of the work to be done. The statement of work must clearly specify the responsibilities of the contractor and other parties to the contract so that questions of interpretation can be avoided. It should provide a description of measurable outputs to which progress and payments can be related, including a schedule for completion of the work. The statement of work should also identify the requirements for approval and acceptance of the work at various stages and at completion.

9.29 Our review indicated that CIDA procedures do not ensure sufficiently precise contract statements of work to provide all parties with a common understanding of what service is to be performed. Because statements of work frequently do not contain sufficient detail on standards of contractor performance, it is difficult for CIDA to assess and record performance information.

CASE STUDY

Canada-Tanzania Wheat Program, Phase III

Issue: Statement of Work

9.30 The purpose of Phase I of the Canada-Tanzania wheat program, which was begun in 1970, was to develop varieties of wheat that would grow well in Tanzania's dry areas, and to supply equipment to two farms. Phase II installed the infrastructure and necessary equipment for five more farms and continued to assist in research. This was described in our 1984 annual Report. Phase III of the project is designed to train the Tanzanians to manage the farms and research programs with less outside assistance and to replace worn equipment as necessary. In April 1987 CIDA signed a contract, termed an "arrangement", with Agriculture Canada, to act as the Canadian executing agent in a $13 million technical assistance program for Phase III.

9.31 In our review of the statement of work in the arrangement, we were unable to find specific details on outputs required of the Canadian executing agent. Instead, the arrangement identifies a group of activities required to be conducted as part of the administrative responsibilities of the executing agent. Although the arrangement indicates that the Canadian executing agent is to recruit and send a specific number of Canadians with various skills to the farms in Tanzania, there were no details on what results were to be achieved by these people. We had expected the statement of work to identify outputs such as the establishment of applied research programs and training courses in crop management, land use, financial management, equipment and building maintenance, inventory control and plant management.

9.32 A key activity of the arrangement calls for the preparation of an inception plan which would give "more concrete expression to the guidelines in the Plan of Operations", indicating what work would be accomplished to complete the contract. The inception plan was to be prepared mainly by the Project Field Director in Tanzania. Our review indicated that, after one year, the executing agent had not yet sent a field director to the project. Consequently, the inception plan was prepared in the absence of a Phase III Project Field Director as required. Although CIDA personnel visited the project, no technical monitor was hired until ten months after the arrangement was signed with the executing agent. Our view is that the dollar value of this contract is of a magnitude to warrant a clear statement of work in advance of signing the contract, particularly since the arrangement calls for 75 percent of the annual budget of the arrangement to be paid in advance. Without a clear statement of work, it is not possible for CIDA to determine if the work to be done is consistent with the needs of the recipient country that provided the justification for the contract.

9.33 Contract formulation is a highly specialized activity, and lack of clarity has implications for all parties. Without a sufficiently clear statement of work, CIDA cannot enforce the terms of the contract or ensure that the objectives of the project are being fulfilled. For the contractor, lack of precision can lead to work activity that does not match CIDA's expectations or meet the needs of the recipient country. For the recipient country, this can lead to disappointment in the results achieved. Understanding among all parties as to what is to be accomplished is further clouded by CIDA's practice of not disclosing contract terms of reference to the recipient country. Given the number of participants and the complexity of international development work, failure to establish a precise statement of work, or to ensure full disclosure to the recipient, creates the opportunity for misunderstanding, thereby jeopardizing the benefits of CIDA's initiatives and expenditures.

9.34 CIDA should ensure that all contracts contain clear statements of work that reflect project plans of operation and should obtain agreement with the recipient country on contract obligations and outputs.

Management response: Agreed. CIDA will ensure that in the future all contract obligations on work to be performed will be disclosed to the recipient country prior to contract signing. However, CIDA would like to indicate that between the plan of operations and the contract drafting stage, projects undergo progressive refinement as the result of a variety of processes including the request for proposals or call for tenders, modifications identified in the selected proposal, and ongoing consultations with the recipient country. As a result of this process, contract requirements and statements of work may not match the plan of operation approved months earlier, but may well reflect recipient needs more effectively. While recipient country agreement on all contract obligations and outputs might prevent misunderstandings this may increase the amount of time required to initiate project implementation.

Pricing and Canadian Content

9.35 CIDA does not have adequate procedures to ensure that goods are purchased at prevailing market prices or that they have the required Canadian content.

9.36 Pricing. To obtain the best price on goods and services, government contract policy states that, where it is cost effective and reasonably efficient, all qualified Canadian firms should have an opportunity to compete for contracts. When competitive bids cannot be obtained, contract regulations require CIDA to document that goods or services were obtained at a fair price in the circumstances. However, CIDA cannot demonstrate that it always receives a fair price because in these non-competitive situations it does not always compare quoted costs to current prices charged by the same suppliers to other purchasers. CIDA requires suppliers to provide a Fair Price Declaration, but it does not regularly monitor the prices of commodities frequently purchased in its aid program. It also does not review the prices obtained through the use of Fair Price Declarations before the contracts are signed, and when it does discover, after delivery, that pricing was not fair, it does not always attempt to recover the difference.

CASE STUDY

Procurement of Potash

Issue: Pricing

9.37 Following CIDA's approval of a procurement request, a recipient country sent a request for bids to a number of Canadian potash producers. On August 25, 1986, three Canadian potash companies informed CIDA of their decision to submit a joint bid. CIDA subsequently requested and received ministerial approval that allowed the recipient country to enter into non-competitive contracts with the supplier consortium. The total value of these contracts amounted to $31.9 million. For each purchase order approved, CIDA obtained a fair price declaration stating:

The prices quoted are not in excess of the prices currently charged by ourselves to our most favoured customers for similar goods in like quantities under comparable conditions and are considered fair and reasonable.

9.38 During 1987, after completion of these contracts, CIDA performed a price analysis comparing them with the suppliers' other commercial contracts. The analysis showed that a weighted average cost paid by CIDA was $68.54 compared with a weighted average cost of $63.36 (U.S. Funds - FOB Vancouver) for commercial contracts with the same firms for potash sold in the same months as the CIDA purchases. This difference in cost amounts to approximately $1.2 million.

9.39 CIDA conducts audits on firms with whom it has service contracts; these audits are directed primarily toward the analysis of profit margins and overhead rates. No similar audit procedure is applied to contracts for goods where authority to purchase the goods has been delegated to CIDA or the recipient country.

9.40 CIDA should establish audit procedures to ensure the accuracy of fair price declarations.

Management response: Agreed. Although CIDA maintains some commercial pricing information on a few key commodities, it has not conducted formal price reviews on a regular basis. CIDA will establish appropriate audit procedures to ensure the accuracy of fair price declarations.

9.41 Canadian content. The provisions of the Canadian aid program require CIDA to ensure that goods purchased with funds tied to procurement within Canada contain at least two-thirds Canadian content. This provision is particularly important in contracts involving manufactured goods. CIDA's policy is to obtain a written certificate from suppliers attesting to the Canadian material, labour and overhead in goods to be procured. However, CIDA has no procedure to ensure that these certificates are accurate and thus has no assurance it is adhering to this provision of the Canadian aid program.

9.42 CIDA should establish procedures to ensure the accuracy of Canadian content certificates.

Management response: Agreed. CIDA's current policy is that Canadian content certificates are audited at the request of the Project Team Leader only if there is evidence to suggest that further investigation is required or a complaint is received. CIDA will review this policy and implement appropriate procedures to ensure the accuracy of these certificates.

Contract Administration

9.43 Monitoring of contract implementation is a required part of contract administration. CIDA has procedures to review contract performance but these procedures need improvement to ensure monitors are in place at the right time and produce the appropriate reports.

9.44 CIDA staff's responsibilities in the contracting process focus mainly on front-end activities such as identifying projects, selecting suppliers, negotiating rates and obtaining contract approvals. The Project Team Leader is accountable for the satisfactory completion of the contract, but implementation and monitoring are usually delegated to executing agents or to a CIDA Principal Resource Officer. Because it is not possible for CIDA officers in headquarters to be at all locations where projects are in progress, contract monitors are the expert "eyes and ears" of the Project Team Leader. However, our audit found that contract monitors are sometimes not hired or assigned until after a substantial portion of the contract work has been finished. In other cases, monitors have not been hired until months after the delivery of goods, which the monitor was to have overseen.

9.45 CIDA contracting policy requires periodic reports on the progress of work. These are required to provide salient facts for the reporting period and to signal when 65 percent of the financial limitation of the contract has been reached. However, our audit identified contracts where the executing agents and contract monitors failed to provide the required progress reports. As a result, the Project Team Leader did not have timely assurance that the contractor was meeting all contract obligations.

9.46 CIDA Project Team Leaders are accountable for contractor performance, but they are not always able to use their authority to enforce the terms of the contract. One reason is that contract provisions may not be sufficiently clear on what is to be delivered. Another reason is that the reassignment of personnel within CIDA sometimes creates difficulty in continuity of contract administration and unfamiliarity with contract objectives.

9.47 A decision by the Project Team Leader to enforce contract terms by withholding funds, or by discontinuing the contract, can have a significant impact on diplomatic relations between Canada and the recipient country. Consequently, in the absence of clear contract obligations, the tendency of CIDA officers is to allow the contract to proceed in the hope that problems will disappear without direct intervention. When the Project Team Leader is dissatisfied with contractor performance, CIDA has no internal mechanism whereby contract concerns can be formally discussed to improve contractor performance. CIDA also does not have any means of knowing the degree to which lack of contract clarity impinges on contractor performance.

9.48 CIDA should ensure that contract monitors are in place at the right time to provide adequate support for the ongoing assessment of contractor performance. CIDA should also establish a method to review contract disputes.

Management response: In consultation with Treasury Board and the private sector, CIDA is currently undertaking improvements in its contracting practices to streamline the process so that contractors and monitors reach the field earlier. However, given that contract disputes have arisen in the past which have been successfully resolved by the line management hierarchy with the assistance of legal, professional and technical experts, CIDA sees no requirement to devise additional processes.

Aid Information

9.49 CIDA has an Aid information System to provide financial data and track project implementation. This system, which took six years to develop and cost about $40 million, produces adequate financial information but CIDA is not achieving value for money from its project management capability.

9.50 The purpose of examining CIDA's Aid Information System (AIDIS) was to review the cost of the system in relation to original estimates, to assess the extent to which the cost of the development had been controlled and to determine if the system is providing CIDA management with essential financial and non-financial information. Our review covered the period from 1979 to 1988.

9.51 The development of CIDA's information system began in 1979 when Phase I - Terms of Reference for a Long Range Systems Plan was drafted. The cornerstone of this plan was AIDIS, which had an estimated development cost of $9.1 million. In March 1980, a Phase II Integrated Systems Overview and Plan for Systems Development was prepared, which included all elements of the previous plan and proposed 49 sub-systems. In November 1982, CIDA awarded a tender for the detailed design of the system. By 1985, AIDIS II was fully operational and has continued to be modified and enhanced.

9.52 Our audit indicated that no cost-benefit analysis had been conducted as part of the original decision to implement AIDIS. The first cost-benefit study of the system was not done until July 1983, after $4 million had been spent. That study projected the benefits of the system to be $94.2 million over a five-year period, against a total development cost of $13.8 million (see Exhibit 9.1). Most of these savings were to be derived from improved procurement, but the study gave little indication of how this would be achieved.

Exhibit not available

9.53 An up-dated cost-benefit report issued in September 1985 revised the total quantified benefits to $36.0 million against total costs of $41.0 million (see Exhibit 9.1). This time, much of the savings derived from AIDIS was attributed to the reduction in cost overruns in bilateral development projects. The 1985 cost overruns on all bilateral projects were around $41.9 million; in 1982-83 it had been estimated at as high as $102.7 million. The report attributed about 10 percent of this reduction to AIDIS, with future savings projected at about $19 million, but there was no substantiation of this projection.

Exhibit not available

9.54 CIDA estimates that development costs for the AIDIS II portion of its information technology systems were $21.8 million as of November 1985. This estimate does not include costs related to user involvement, overhead, training, supplementary development and supporting systems, or costs incurred prior to 1980. We estimate the development costs of the financial and operating management system to be closer to $40 million. Our estimate included overheads such as fringe benefits, accommodation and support; user training time and related costs; the costs of other financial systems developed since 1979; and development costs subsequent to initial implementation in 1985.

9.55 Users of AIDIS view the system as producing accurate, timely and reliable financial information. However, the system is difficult to access, has poor data presentation and in some branches is rarely used by project officers directly. In addition, the system has a limited capacity to aggregate data for reporting and is not very flexible. This lack of flexibility is creating increased costs for developing additional mainframe software and for the extensive use of supplementary microcomputer-based applications.

9.56 Our review of the AIDIS database identified a significant number of project files where non-financial information on project milestones, notes, monthly budgets, field visits, and payment and report schedules was unreliable and consequently unused. These shortcomings, however, do not apply to all branches within CIDA. Some branches have begun to place more emphasis on the integrity and accuracy of their data and are beginning to see the benefits of these efforts. However, given that CIDA is embarking on a decentralization of its operations to the field, it is imperative that the problems encountered in developing AIDIS not be repeated in its field applications.

9.57 Our view is that AIDIS provides accurate and timely financial information similar to that provided by other government financial systems. Its cost, however, is substantially higher than the cost of other government department financial systems. Much of this higher cost is due to the non-financial, project-oriented reporting features built into the system. But because this non-financial information is not extensively used, CIDA is not gaining full value for the money it spent to develop the system. It is senior management's responsibility to ensure the organization's commitment to AIDIS and its ability to deliver quality information.

9.58 In summary, the development of AIDIS has been long and costly. It was undertaken during the formative years of CIDA's data processing organization and, in the early stages of the project, relied heavily on outside contract services without the degree of management control appropriate to its cost. With forecasted benefits imprecise and unsubstantiated, final costs are significantly higher than originally expected. An appropriate costing system was not in place throughout the life of the project, with the result that the true total cost of AIDIS is unknown.

9.59 CIDA should ensure that the non-financial data entered into the Aid Information System is both accurate and timely.

Management Response: Agreed. In 1979, the Auditor General noted that CIDA should "... ensure that top priority is given to the completion and implementation of the long range systems plan and the action plans". As a result it was decided to proceed with the design of AIDIS which was the subject of intense discussion and monitoring by the Treasury Board Secretariat and the Office of the Comptroller General. As the report notes, AIDIS provides accurate and timely financial information and there is adequate control over CIDA's program. The AIDIS was designed using hardware and software technology available at the time and therefore is not as flexible as the software now available. Management is aware of the lack of flexibility of AIDIS and is pursuing a strategy leading to a new generation of systems that will build on the investments made in AIDIS, and that will be cheaper and more flexible to operate.

Management has noted the Auditor General's observation concerning data entry, and the new generation of systems will ensure that non-financial data are entered into the system on time and accurately.

Special Programs

9.60 The primary objective of CIDA's Special Programs Branch is to encourage and to support Canadian non-governmental organizations and non-profit institutions involved in Third World development. These organizations carry out their own project planning, delivery, management and liaison activities with development countries, independently of CIDA. In 1987-88, Special Programs spent about 11 percent of CIDA's total international development assistance budget, or approximately $240 million, to support 464 organizations involved in the delivery of more than 4,600 projects in over 100 countries.

9.61 There are five special programs managed by CIDA's Special Programs Branch. Its Non-Governmental Organizations program supports the work of organizations that do much of their own fund-raising. These organizations may be denominational groups, development agencies such as CARE and HOPE International, and associations such as the Rotary Clubs. Most of their work is in small, community-based projects in the areas of education, food production, drinking water supply, public health and income production.

9.62 The Institutional Cooperation and Development Services Program supports organizations that send volunteers abroad (CUSO, for example), co-operatives, universities, colleges, unions and professional associations involved in international aid. These organizations do not have a significant fund-raising capability and receive a large portion of their funds from CIDA. This program is designed to assist the development of human resources and strengthen institutions through the direct participation of Canadian volunteers.

9.63 CIDA also channels international development assistance through its International Non-Governmental Organizations Program. This program gives priority to strengthening organizations that are active largely in assisting rural disadvantaged populations and that encourage the participation of women in the decision-making process. The two other Special Programs are the Management for Change Program and the Public Participation Program.

9.64 Our audit focussed on Non-Governmental Organizations, Institutional Cooperation and Development Services, and International Non-Governmental Organizations. These three programs account for about 93 percent of the Special Programs Branch expenditures.

9.65 Our audit included a review of CIDA's files on 25 organizations selected at random, as well as site visits to 21 projects located in six countries. We did not audit the management of the organizations selected. These organizations were visited to verify information in CIDA's files. We also examined Special Programs management procedures, with particular attention to the monitoring of contributions to voluntary organizations.

9.66 CIDA provides funds through the use of contribution agreements to Canadian and international non-governmental organizations involved in international development. Most of these organizations are implementing projects in accordance with their agreement with CIDA.

9.67 Our review of Special Programs Branch activities indicated that program funding criteria were clearly stated in accordance with government guidelines on financial administration. Furthermore, all organizations in our sample met CIDA's program criteria. In our review of 21 projects in the field, we identified only one that was not performing in accordance with the detailed project description contained in CIDA's files.

Monitoring

9.68 CIDA's monitoring procedures are not well matched to the nature and types of non-governmental organizations receiving funds from the Special Programs Branch within CIDA.

9.69 Government policies on the administration of contributions programs require that program managers put in place appropriate monitoring and control procedures to account for funds expended in relation to the achievement of program objectives. Special Programs' chief methods of monitoring and controlling its contributions are through receipt of periodic program reports from the organizations, project visits by CIDA personnel, analysis of financial statements, organizational assessments and audits.

9.70 Progress Reports. CIDA requires recipient organizations to submit periodic reports on their field activities, generally according to an agreed schedule. This enables CIDA, after review, to release funds for payment to the organization. These reports contain information on achievement toward meeting project objectives and project impacts. CIDA usually receives these progress reports as required.

9.71 Project Visits. On-site review by CIDA personnel of activities funded by Special Programs is difficult because of the large number of projects, their small size and the number of countries involved. Furthermore, the cost of visiting all project sites would be prohibitive. As a result, CIDA officers cannot visit projects frequently enough or examine them in enough depth to reliably monitor or assess either the organizations or their projects. When officers from CIDA do visit projects, they usually record their key observations, but the Branch does not aggregate key information in these reports for later use in evaluating the Branch programs. We also noted that CIDA officers at Posts have no direct responsibility for monitoring contribution projects operating in their areas. None of the projects we visited had been visited by Post staff.

9.72 Financial Statements. Special Programs Branch personnel do not consistently review recipient organizations' financial statements. Although it is Special Programs' policy to conduct this type of analysis and most officers have received some training in financial analysis, we found most files contained no evidence that analyses of financial statements were actually being carried out.

9.73 Organizational Assessments. Organizational Assessments are conducted as part of the decision-making process in funding recipient organizations. Because project site visits are not cost-effective, Special Programs uses periodic organizational assessments to review recipient organizations' methods of planning, delivering and monitoring their activities and projects. Organizational assessments focus primarily on the assessment of risk associated with funding specific organizations. These assessments do not always include on-site visits or testing of the organizations' management systems. Twelve organizational assessments were conducted in 1987-88. Our review of a sample of them revealed little information on the organizations' past performance, or tests of their financial, evaluation and audit systems.

9.74 Effectiveness. An evaluation of Special Programs Non-Governmental Organizations was carried out in 1984 based on project data gathered between 1979 and 1982. Since then, organizational assessments have been the major outputs of Special Programs evaluation activities. These assessments, however, are not adequate to determine or report on the effectiveness of the Branch programs because they do not contain performance information. Furthermore, evaluation activities are not managed and co-ordinated adequately. The Special Programs Branch evaluation plan, for example, states that:

The planning for the new evaluations to be undertaken during 1987-88 has not followed any overall SPB strategy but has been left entirely to each individual SPB division as being the best qualified to determine the reasons for which its evaluations will be required.

Evaluation plans are never met in their entirety. There is no evidence that the Branch compares actual performance against the evaluation plans or that it takes action when plans are not met. When evaluations are conducted, their purpose is sometimes unclear to the organizations under review.

9.75 Audit. Special Programs Branch policy requires that a sample of organizations be audited each year, but implementation of this policy varies from division to division. The Institutional Co-operation and Development Services division schedules regular audits and conducted six audits in 1987-88, a year in which it provided funds to 123 organizations. In the same year, the Non-Governmental Organizations division provided funds to 152 organizations, three of which it audited.

Project and Program Funding

9.76 Our observations indicate that Special Programs' procedures in the monitoring and control of contributions are inadequate, particularly in the context of recent funding trends. Special Programs Branch currently assists organizations by two major types of funding - project funding and program funding. Project funding entails assistance on a project by project basis. Program funding allows the recipient organization to independently select the projects it wishes to undertake and to manage cash flows among projects on an aggregate basis. Special Programs presently funds 46 organizations on a program basis. In 1986-87, these organizations received about $135 million, or 65 percent of total Special Programs disbursements, and the use of this type of funding is increasing.

9.77 The advantages to the organizations of program funding are security of funds over a period of up to three years, streamlined planning and project implementation, and less paper burden. Organizations funded on a program basis are not required to provide a detailed report on every project; only financial reports and narrative summaries of their field activities are required. From a Special Programs' perspective, program funding is less cumbersome administratively. However, the level of control that Special Programs maintains over the use of program funds is lower than that associated with project funding.

9.78 Contribution programs contain clear obligations to account for funds expended. Grant programs, however, place major emphasis on the decision to allocate funds but not on monitoring their use. Without application of satisfactory accountability procedures, the effect of current funding trends is to change Special Programs from a contribution program to something closer to a grant program. Such a change would require Parliamentary approval of the nature of the program, its objectives, target groups and expected results.

9.79 CIDA should develop and implement a more systematic approach to account for Special Programs Branch expenditures. This approach should include the application of monitoring and control procedures matched to the nature and types of organizations involved in contribution agreements with CIDA.

Management response: Agreed. Although we feel there is an adequate system to account for Special Program Branch expenditures we would agree there is room for a more systematic approach. In this regard, CIDA will take steps to improve the aggregation of information from field visits for analytical and evaluative purposes; and integrate audit and evaluation plans in the overall organizational assessment process and ensure that these plans are fulfilled. CIDA takes special note of the observation on current funding trends and its accountability implications.

Information to Parliament

9.80 The intention of Part III of the Estimates is to provide information about government programs, details of planned and actual results, related expenditures and other performance data useful in justifying resource requirements. We reviewed Part III of the Estimates for each of the last three fiscal years, for structure, content, style and indicators of performance. The 1987-88 Part III accurately reflects CIDA's program structure and provides more information than in previous years. It would be possible, however, to achieve greater clarity in the graphic illustrations. For example, in the introduction to the 1987-88 Part III, CIDA's aid and development activities are placed within the context of total Canadian overseas development assistance. This provides a helpful illustration of total Canadian assistance, but it is difficult to identify CIDA's direct responsibilities since the graphic display gives the impression that CIDA is responsible for all Canadian official development assistance.

9.81 The use of technical jargon has been reduced, but the Part III is still being written in an overly complex style, making the information difficult to use. This problem is reflected in the views of the Standing Committee on External Affairs and International Trade in its recent report on development assistance policies and programs:

The estimates that are tabled annually in Parliament are ... virtually impenetrable even to those who compile and use them. There are enough minor discrepancies between the estimates and other versions of planned expenditures as to raise doubts about the whole process. The supplementary explanations of the Main Estimates, known as "Part IIIs", were introduced a number of years ago to shed light on these matters, but they seem to have succumbed to the darkness too. Each year they become thicker and more complex, defeating the very purpose for which they were created.

9.82 Such views are generated by overly complex descriptions of programming and budgeting, as illustrated by the following statement on page 41 of the 1988-89 Part III:

The deterioration of the economic situation on most countries in the Americas led to the establishment of polysectorial lines of credit in support of structural adjustment programs.

And on page 15 details of the forecast budget are cryptically described:

In terms of reportable ODA, this amount has been offset by transfers to other ministries and Crown corporations, to be utilized for ODA purposes.

9.83 The Part III also contains information that is difficult to reconcile with information in other sections in the document. Different dollar values for the same budget items are often explained in notes that are unclear and that are placed at locations distant from the text. The text also contains a number of examples of poor editing, the effect of which makes the Part III difficult to follow.

9.84 Part III performance information is not sufficiently related to intentions, previous plans or trends in performance. Sections on resource justification merely contain more detailed descriptions of activities outlined elsewhere in the document. In other cases, incomplete data lead to inaccurate conclusions. For example, Figure 11 in the 1988-89 Part III compares the volume of aid to various regions of the world with the number of person-years allocated to the programs. In most regions, the number of person-years has remained stable or has decreased slightly, while the volume of aid is shown to have increased. This implies that CIDA has been able to accomplish more with the same number of person years.

9.85 This implication is reinforced in Figure 23, which illustrates the growth in International Development Assistance using person-years, current and constant dollars, and number of funded projects and contracts. The explanatory note says, "the Agency's workload has substantially increased in comparison to the person-year base which has remained relatively constant". The dollar amounts accurately portray an increase in spending over the number of person-years, but the number of contracts is more significant. The Figure shows that more assistance work is being carried out by contractors, rather than by CIDA staff. This represents a major change in the organization and delivery of Canadian aid. However, the text contains no discussion on the advantages or disadvantages of the change to contracting, in terms of the effectiveness or efficiency of CIDA's programs.

9.86 The CIDA Part III provides little interpretation of data on program activities. Given the complexity of information, it is essential that this data be placed in the context of expected levels of performance, to enable Parliament to obtain a more complete understanding of future resource allocations, which was the original intent behind the Part IIIs.

9.87 CIDA should continue to simplify the presentation of information in its Part III of the Estimates by reducing the use of jargon and complex explanations. CIDA also should ensure that its performance data is related to program plans and activities.

Management Response: The field of international development is vast and complex and it is not always possible entirely to avoid widely used and accepted terminology, on the one hand, and complex explanations on the other. The Agency has improved the presentation of the Part III of the Estimates over the last three years and will continue to improve with the collaboration of the Office of the Comptroller General and the Treasury Board Secretariat. A special effort will be made to present better performance information on our program.