This Web page has been archived on the Web.

1991 Report of the Auditor General of Canada

Main Points

18.2 The government strategy gives priority to competition, fairness and accessibility as the cornerstones of government procurement policy. Of the approximately $8 billion worth of goods and services procured annually through DSS, some 60 percent is procured on a competitive basis. Government policy requires that client departments' needs be clearly defined in a generic and unbiased way to ensure accessibility and fairness to all qualified suppliers. In addition, DSS is required to advise and encourage client departments to amend non-competitive procurement requests to permit competition, where an alternative product or source of supply exists. DSS' dual role of providing prompt and adequate service to client departments while ensuring compliance with government procurement policies may affect its challenge of client departments' requests for non-competitive procurement (18.15 to 18.49).

18.3 When DSS cannot establish a contract price by competition, key price components such as suppliers' cost and profit must be estimated and negotiated. DSS awards $3 billion worth of non-competitive contracts annually. It is therefore important that DSS procurement officers be given assistance in deciding when detailed price substantiation should be carried out, the extent of such substantiation and the amount of documentation required. They should also be provided with sufficient technological tools and advanced methods to enable them to substantiate price proposals effectively. The current profit policy should be revised to ensure that the profit calculated is related to the level of risk assumed by the suppliers. The price certification currently provided by suppliers in support of non-competitive, non-commercial price proposals should also include a certificate of accuracy, currency and completeness of the price support information as of the time the non-competitive contract is awarded (18.50 to 18.77).

18.4 Generally, DSS has procedures in place to evaluate the technical and financial capabilities of suppliers. However, there is no corporate system to record and report on supplier and product past performance (18.90 to 18.99).

18.5 DSS, one of the largest purchasers of goods and services in Canada, is undergoing significant changes. At the same time, there are continuing changes in the complexity and technical aspects of the products procured. The training and skills of procurement officers are the key to their effectiveness in providing centralized procurement services to the Government of Canada. Therefore, DSS training programs should be comprehensive and up to date, to provide procurement officers with the appropriate skills to discharge their responsibilities (18.110 to 18.119).

General Background

18.7 DSS is a common service organization internal to government and is responsible for two major government-wide functions: banking and accounting, and centralized procurement. Centralized procurement services include procurement planning, market research, selection of methods of supply, contract negotiation, issuance and administration of procurement contracts, including those for major Crown projects, and other procurement services. DSS summarizes its mission as follows: "to deliver valued services that enable our clients to achieve their objectives".

18.8 As the procurement arm of government, DSS supplies defence and civilian requirements for goods and certain services, taking into consideration national objectives such as promoting competition, fairness and accessibility, and industrial and regional development. DSS procures goods and services for the Government of Canada worth approximately $8 billion annually and participates in the administration of major Crown projects valued at $23 billion. All DSS procurement services are currently provided on a fee-for-service basis.

The Environment

18.10 Important government initiatives, such as Increased Ministerial Authority and Accountability and Public Service 2000, are proposing increased delegation of procurement authority to client departments. This would challenge the central procurement role of DSS. In addition, the Procurement Review Board of Canada was established by the Canada-U.S. Free Trade Agreement to hear supplier complaints regarding procurement falling under the Free Trade Agreement. Through its Procurement Service Line Reviews, including Acquisition 2000, DSS is re-examining its strategic directions and embarking on an ambitious plan to automate its procurement practices in response to this changing environment.

18.11 DSS is currently negotiating with Treasury Board and client departments for the virtual elimination of revenue dependency as a method of financing the central procurement service of the Government of Canada.

Audit Objective and Scope

18.13 The objective of this year's audit was to review and assess DSS' central procurement practices and management controls, with particular focus on issues and concerns identified in our 1989 opinion survey.

18.14 In this chapter, we report on our assessment of DSS' management of the procurement service for goods. In Chapter 19 we discuss issues related to DSS procurement and industrial development.

Extent of Competition in Procurement

18.16 The Government Contracting Regulations contain four exceptions to the requirement to solicit bids: in cases of pressing emergency, which is an actual or imminent life-threatening situation requiring immediate action; for amounts less than $30,000; in cases where there is only one capable supplier; and cases where it is not in the public interest to solicit bids. Treasury Board policy provides guidance in interpreting these exceptions. Low cost procurements are expected to be competed, where cost effective. Technical or legal requirements would warrant the use of the only capable supplier exemption, not simply the fact that management knows of only one potential source. Security considerations or the need to alleviate some significant socio-economic disparities in Canada are described as cases where competition would not be in the public interest.

18.17 The General Agreement on Tariffs and Trade and the Free Trade Agreement have opened up the procurement of some goods to international competition. In general, there is to be worldwide competition for goods procurement over $210,000, and competition within Canada and the United States for goods requirements over $31,000. Exemptions to these agreements, which simply state that open competition need not apply, include procurement related to national defence and security and the requirements of the departments of Transport, Fisheries and Oceans, and Communications. These account for over 50 percent of government procurement. Service contracts are also exempt from the provisions of the agreements.

18.18 These agreements permit non-competitive procurement in a number of specific circumstances, such as when there are no acceptable tenders in a competitive process, when there are legal restrictions (like patents or proprietary rights) and no reasonable substitute exists, or because of extreme urgency. Additional deliveries by the original supplier for interchangeability reasons and the development of a prototype product are other specific circumstances. For requirements not governed by the international agreements, DSS applies procurement policies which give preference to domestic suppliers. Domestic preference policies are discussed in Chapter 19.

18.19 Government contracting policy encourages the competitive process by requiring that client needs be clearly defined in a generic and unbiased way to ensure accessibility and fairness to all qualified suppliers. DSS procurement officers are required to advise and encourage client departments to amend non-competitive procurement requests to permit competition, where an alternative product or source exists. Justification for deviation from policies must be fully documented in the contract file. In addition, non-competitive contracts are scrutinized by more levels of authority than competitive ones.

Audit Objective and Scope

18.21 We reviewed procedures for procurement planning which determine whether the procurement should be competitive or non-competitive, the justification for non-competitive procurement was adequately documented, different methods of supply were assessed and the chosen method did not unduly restrict competition.

Observations and Recommendation

Because the level of non-competitive contracting is significant, overall analysis of the underlying reasons is needed

18.23 DSS systems capture basic contract information, including the contract award method (competitive or non-competitive), which allows the tabulation and reporting of statistics on the number of competitive and non-competitive procurements. However, these systems do not accurately capture the reasons for awarding non-competitive contracts, and the reliability of the internal information is questionable.

18.24 Procurement officers advised us that they examine the justification for non-competitive procurement on a case-by-case basis to ensure that it adheres to government policy. While such a review is necessary, an overall analysis of the extent of, and reasons for, non-competitive procurement would allow DSS to identify trends and problem areas where action could be taken to improve competition.

18.25 In the absence of reliable information within DSS at the time of our audit, we analyzed the justification given for non-competitive procurement in the sample of non-competitive contracts we examined. Some of the reasons given for not competing these contracts are discussed in the following paragraphs. The results are also summarized in Exhibit 18.1 .

18.26 Proprietary rights were quoted in 45 percent of the contracts examined. Where major systems are built to client department specifications, additional systems or components are often bought from the same supplier because of some proprietary rights such as patents, licences or hardware/software rights. We discussed with the Department the possibility of obtaining the rights to the design to permit future requirements to be tendered competitively. DSS officials indicated that they had found that the costs of doing so were prohibitive. Nevertheless, opportunities are still being sought to acquire design rights, particularly with respect to non-commercial goods and requirements under major Crown projects.

18.27 The only known or capable supplier was the justification used in 32 percent of the contracts examined. Prior to DSS' implementation of the Open Bidding Opportunities Policy initiative in late 1989, its search for alternative sources of supply relied largely on the information in the DSS source listings, the procurement officer's knowledge of the industry, information obtained from client departments and informal contact with procurement officers of other jurisdictions. Case 1 illustrates this practice. In April 1991, DSS also implemented the Item Information Service, a computerized data base which provides information on technically capable sources and recent procurement history, principally for spare parts. The introduction of these two initiatives for identifying other potential sources of supply should, in the future, increase the extent of competition by attracting previously unknown suppliers.

18.28 Upgrades and follow-on purchases accounted for 23 percent of the contracts examined. While the original purchase may have been won competitively, subsequent procurements are often awarded on a non-competitive basis as and when required. There is no formal requirement for client departments to provide long-range plans to DSS to help it develop the best long-term procurement option. The Office Automation, Services and Information Systems Directorate at DSS is currently experimenting with a long-range procurement approach for mainframe computers. The client department's computer needs over a five-year period are identified, and its requirements sent out for competitive tender. This results in locked-in competitive percentage discounts for future optional upgrades. We support this practice and encourage expanding its application to other directorates of DSS where feasible.

18.29 Compatibility was the justification used in 20 percent of the contracts examined. Compatibility with existing equipment could, in some cases where cost justified, be incorporated into a generic, unbiased description of the requirement in a way that could permit competition.

18.30 Operational urgency was quoted in 14 percent of the contracts examined. Case 2 is an illustration of urgent requirements from client departments. DSS must promptly address urgent requests for non-competitive procurement. However, such requests may impede the ability of DSS to adequately search for alternative sources or methods of supply.

18.31 National objectives were quoted in 8 percent of the contracts examined. Issues related to procurement and national objectives are discussed in Chapter 19.

18.32 In June 1991, in response to a ministerial request and subsequent to our audit, DSS completed a study, which had begun in March 1991, of the reasons for non-competitive procurement based on a sample of contracts from most commodity groups. While the reasons identified in the DSS study were the same as ours, the percentages differed because the DSS analysis covered most commodity groups while our analysis focussed on certain commodities. The next step would be for DSS to analyze the results of the study to identify areas where action could be taken to improve competition.

DSS' dual role of providing prompt and adequate service to client departments while, at the same time, ensuring compliance with government procurement policies may affect its challenge of departmental requests for non-competitive procurement

18.34 Although, in determining the specifications for any procurement, client departments must comply with, and are jointly responsible for, the application of Treasury Board policy on competition, in practice DSS alone is generally seen to be accountable for allowing non-competitive contracts. The overlapping roles of DSS and its client departments in this area may create tension in their relationship and make it difficult to fix responsibility and determine accountability for some of the non-competitive procurement decisions. There is a need for guidelines to ensure that the respective roles of DSS and client departments are clear and that the responsibility and accountability for ensuring competition in government contracting are shared.

18.35 Procurement Review Board of Canada. The Procurement Review Board of Canada was established in 1989 to review suppliers' complaints under the Canada-U.S. Free Trade Agreement. The current mandate of the Board covers only contracts between $31,000 and $210,000 that are subject to the Free Trade Agreement. Of the 11 decisions published by the Board from its inception to early 1991, 4 made reference to justifications for non-competitive procurement. In three of these decisions the Board concluded that DSS had not adequately challenged client departments' justification for non-competitive procurement. In its fourth decision, the Board noted that DSS had challenged the non-competitive request, insisted on competition, and obtained from the client department a generic description of the requirement, which included a need for compatibility with existing equipment. Case 3 is an example of one of these decisions.

18.36 As a result of one of its decisions on the adequacy of DSS' challenge of a non-competitive request, the Board recommended that "DSS should review the extent to which all levels of contracting officers understand their responsibilities with respect to reviewing and accepting justification for non-competitive tendering". DSS responded to this recommendation by adding relevant sections to training courses.

18.37 DSS initiative. DSS initiated the Open Bidding Opportunities Policy in November 1989 to further the objectives of competition, fairness and accessibility. The policy requires the advertising of government procurement requirements in the Government Business Opportunities publication and on its electronic bulletin board, thus giving businesses across Canada better access to government procurement opportunities.

18.38 In 1990 the policy's coverage was expanded to include the majority of goods contracts valued over $25,000. Service contracts have been included since April 1991.

18.39 Starting in July 1991, DSS plans to advertise, two weeks prior to contract award, all proposed non-competitive procurement covered by the General Agreement on Tariffs and Trade and the Free Trade Agreement. If this approach is successful in identifying other sources of supply, DSS proposes to extend it to all proposed non-competitive procurement over $25,000 starting April 1992.

18.40 DSS expects the Open Bidding Opportunities Policy to result in a more visible procurement service and one that is open to challenge. However, this policy only applies to procurement carried out by DSS on behalf of the government. It does not cover procurement carried out independently by other government departments or agencies.

Case 1. Savings Resulting from Competition

18.42 The DSS procurement officer made inquiries concerning recent prices paid by the United States Air Force and concluded that the prices quoted for seven of the parts were high. A non-competitive contract was awarded for the five remaining items, which the procurement officer considered to be fairly priced. A competitive tender was issued by DSS for the seven remaining items and bids were received from several suppliers, including the original supplier which again quoted its earlier prices.

18.43 Contracts were awarded to other suppliers for the seven tendered items. The total value of all the contracts awarded for the 12 items was for approximately $219,000. In this case, the efforts of the DSS procurement officer to compete the seven items saved the taxpayer approximately $250,000 or 50 percent of the value of the procurement.

Case 2. Urgent Requirements to Upgrade Computer Equipment

18.45 In August 1988, the client department requested a further upgrade of the same computer mainframe, asking for delivery of the required equipment by 1 November 1988, to implement another urgently needed system. A discount from list price was obtained. DSS processed the non-competitive request while obtaining the client department's agreement to the competitive tendering of any further requirements, including those projected over a five-year period. DSS informed us later that there was competition for subsequent requirements and that large discounts from list prices were obtained. Subsequent to our audit, the supplier indicated that, in his opinion, the difference in the discounts offered was due to factors of urgency, time and competition. We did not verify this opinion.

Case 3. Decision of the Procurement Review Board of Canada

18.47 The Board found no evidence that competitive solicitation had been considered. Also, DSS had not questioned the request, and agreed with the client department's justification. The Board noted that, in another case involving the proposed procurement of similar products for another department, DSS had challenged the non-competitive request, insisted on competition and obtained from the client department a generic description of the requirement.

18.48 In its response to the Board, the contractor said that its competitors' products would not be able to connect to the computer equipment without extensive development work. However, the Board noted that this was something which any competitor might have been willing to do, and which the client department might not have had to pay for or even wait long for. It stated that the competition policy should not be thwarted by internal government judgment about what the private sector can or cannot do. The Board found in favour of the plaintiff.

18.49 DSS should establish and communicate guidelines to clarify roles and responsibilities and to remind client departments that accountability for ensuring maximum competition in government contracting is shared.

Department's response: DSS recognizes that competition is one of the cornerstones of the government procurement policy and is attempting to achieve the objectives of competition, fairness and accessibility in support of the objectives of its client departments. The Department has put in place a number of initiatives designed to foster competition such as open bidding and the Advanced Contract Award Notices advertised through the Procurement Opportunities Board and the Government Business Opportunities publication.

The DSS Customer Manual clearly indicates to clients that requirements should be defined in terms that allow the use of the competitive process and result in the procurement of goods or services that will provide the best value in meeting the needs. The division of responsibilities between DSS and certain client departments are delineated in Memoranda of Understanding. DSS will ensure that any Memoranda of Understanding with client departments specify roles for ensuring maximum competition. Where no such Memoranda exist, DSS will ensure that roles and responsibilities are appropriately communicated through other sources.

Pricing Non-competitive Contracts

18.51 In its Contract Costing Memorandum, DSS sets out policies on allowable costs. Broadly, these include estimated material, labour and overhead costs incurred in performing the contract. In the case of agents, material costs would include the cost of imported goods. For determining profits, the Department has a number of guidelines, depending on the dollar value of the contract and whether the supplier is an agent or a manufacturer.

18.52 DSS policies also require that, in non-competitive procurements, the contractor sign a price certification and the contract contain a discretionary post-award audit clause. The form of the certification varies, depending on whether the goods being acquired are commercial or non-commercial products. For commercial products the contractor certifies that the price quoted is not in excess of the lowest price charged any other customer. For non-commercial products the contractor certifies that the price has been determined in accordance with the Department's cost and profit policies. Discretionary post-award audit clauses afford DSS the right, in the case of commercial goods and after the completion of the procurement, to conduct an audit to satisfy itself that the price it has paid is as low as the lowest price charged any other customer. In the case of non-commercial products DSS is entitled, after completion of the procurement, to conduct an audit of profits paid to the contractor to ensure that these were reasonable.

18.53 DSS annually awards $3 billion worth of non-competitive contracts. It is therefore important that procurement officers have the necessary skills and tools to ensure, prior to awarding the contracts, that the prices proposed by the suppliers are fair and reasonable.

Audit Objective and Scope

18.55 We reviewed a random sample of non-competitive contracts issued during the past three years valued at $267 million. Our sample was selected from a contract population valued at $2.3 billion, covering spare parts, communications and electronic data processing equipment. The sample items selected were firm price contracts for the acquisition of both commercial and non-commercial goods.

Observations and Recommendations

Price proposal substantiation is generally better for higher dollar value contracts than for lower ones

18.57 We found that the level of price substantiation varies greatly, depending on the dollar amount of the contract and the judgment, skills and ability of individual procurement officers. In general, we noted inconsistencies in both the substantiation of the proposed price and the documentation of the extent of such substantiation within DSS files. Higher dollar value proposals, which relate to contracts in excess of $2 million, are generally reviewed more thoroughly than relatively lower dollar value proposals.

18.58 We agree that the costs of substantiating low dollar value proposals must be weighed against the value of obtaining assurance that the price is fair and reasonable. However, we noted that procurement officers are provided with few guidelines as to when detailed price substantiation should be completed and the extent of such substantiation.

18.59 Generally, in lower dollar value contracts - that is, contracts with a value of less than $2 million - we made the following observations.

18.60 Price support information is insufficient. For the acquisition of non-commercial goods, contract files generally contain a summary of the supplier's estimated costs with totals for materials, labour, overheads and a profit calculation prepared by the supplier. This high level summary is generally insufficient to allow proper analysis prior to substantiation and negotiation. While procurement officers may have seen information beyond the summaries provided by suppliers in the price proposals that we found in the files, there was generally no evidence of it. As well, price proposals for commercial goods contain a certification by the supplier that the price quoted is not in excess of the lowest price charged any other customer. We noted that, while these contracts contain discretionary post-award audit clauses, no such audits of supplier certificates were conducted by DSS until the fall of 1990, when the Department introduced new audit coverage procedures.

18.61 Our audit noted that, while the Department has policies which govern the determination of allowable costs and profits, there are few guidelines that set out the level of pricing information to be provided by suppliers in responding to a non-competitive request for proposal. More detailed information in support of price proposals is necessary if procurement officers are to provide a reasonable challenge during price substantiation and if they are to negotiate with the same level of information as suppliers with respect to the proposed prices.

18.62 We noted that, while suppliers must certify adherence to departmental cost and profit policies or certify the offering of their lowest price, there is no requirement that information submitted by suppliers be certified as to its currency, completeness and accuracy at the time of the contract award. This certification is necessary to impose a responsibility on the supplier to provide the information needed for negotiation of non-competitive contracts on an equal basis by the government.

18.63 As part of the contract approval process, procurement officers express an opinion that the price being proposed is fair and reasonable. Such opinions also generally contain a statement that they believe costs proposed by suppliers for materials and labour are commensurate with the work to be performed under the contract. Our review indicated that generally there was insufficient documentation of the substantiation work undertaken by the procurement officers in the procurement files to support their judgments.

18.64 While procurement officers may have substantiated price support information, we noted that there was generally little documentation in the contract files to show that detailed substantiation of such things as estimated labour or material usage, foreign content and material quotations had been completed. Procurement officers consistently documented such things as the details of telephone conversations but not key components of price substantiation such as the basis for their judgment of the reasonableness of estimated labour hours or material use in the price proposals. Such documentation is important to establish the accountability of the procurement officer for the judgments made during price proposal substantiation and negotiation. Such documentation also provides background knowledge on products and pricing for future procurements. Our audit also noted that in several instances there was no evidence in the contract files that foreign suppliers who had been requested to provide price support information had done so before the contracts were awarded.

18.65 The DSS contract approval process is based on the dollar value of the contract and the approval authority level of the procurement officer. Approval authority is delegated to procurement officers based on their positions within the organization. In general, the higher the dollar value of the contract the more senior will be the position of the procurement officer responsible for the contract and the more senior will be the managerial review. Managerial review is an important internal control mechanism. However, the lack of documentation supporting price proposal substantiation reduces its effectiveness.

18.66 Profit policies. For determining profits, the Department has a number of guidelines, depending on the dollar value of the contract and whether the supplier is an agent or a manufacturer. Factors considered include:

• the cost of capital to the supplier;
• the level of general business risk;
• the contractual risk assumed by the supplier; and
• the supplier's Canadian value added contribution.

18.68 We also noted that a departmental study completed by DSS in 1987, which we verified, identified as a serious conceptual flaw the lack of a mechanism to ensure that the profit calculated is related to the level of return for investments of similar risk. In July 1991, subsequent to our audit, DSS officials indicated to us that the profit policy was being reviewed and that this review started in 1988.

Technology and advanced methods are needed to ensure that pricing of spare parts procurement is fair

18.70 The U.S. government responded to this concern with the Buy Our Spares Smarter program. This program uses cost reduction methods such as "break-out" analysis to search for alternative sources of supply, which are cheaper than prime contractors, agents or middlemen. The U.S. program also uses value analysis, where technical and cost analysts determine the "should cost" value of an item. Extensive use of computerized data bases is central to the program, providing information on technically capable sources of supply, procurement history and prices paid for spare parts and other equipment.

18.71 DSS initiated a pilot test in 1989 of several information data bases similar to those being used in the U.S. to assess their effectiveness in achieving reduced prices in the acquisition of spare parts. These systems have proven to be an effective tool in detecting overpricing in spare parts procurement.

18.72 After the success of the pilot test, DSS obtained Treasury Board approval to fund a Canadian Item Information Service that incorporates spare parts pricing and sourcing information for both U.S. and Canadian requirements and anticipates annual savings to the government of $12 million. The system was implemented 1 April 1991.

18.73 One area where we believe that the application of modern technological tools such as the Item Information Service could provide significant savings to the Crown is in the acquisition of initial provisioning of spare parts. Initial provisioning is the acquisition of the initial set of spare parts and equipment needed to support ships, aircraft or other large systems in major Crown projects during their early years of operation. DSS procures these spare parts, representing thousands of parts valued at millions of dollars, from the prime contractors for the projects, to ensure that the total system responsibility of prime contractors is not diminished and to maintain the contractor's warranty against defects. Initial results of work currently under way by DSS on one major Crown project indicate that significant savings could be achieved by using these technological tools to review and challenge the price of initial provisioning spares which the prime contractor proposes to acquire on behalf of the government. We suggest the extension of the application of these technological tools to cover all initial provisioning of spare parts procurement.

18.74 Through training programs or other means, procurement officers should be given assistance in deciding when detailed price substantiation should be carried out, the extent of such substantiation, and the amount of documentation required.

18.75 The price certification currently provided by suppliers in support of a non-competitive, non-commercial price proposal should also include a certification of the accuracy, currency and completeness of the price support information, as of the time a non-competitive contract is awarded.

18.76 The profit policy should be revised to ensure that the profit calculated is related to the level of risk assumed by the supplier.

18.77 DSS should ensure that procurement officers are provided with sufficient technological tools and advanced methods to enable them to substantiate price proposals effectively.

Department's response: DSS has undertaken a review of the profit policy, and a proposed policy has been drafted which is designed to achieve the objective of better ensuring that profit is related to the level of risk assumed by the supplier. Consultations with the industry are currently under way. A revised policy will be put in place when the consultations are completed.

Prior to the award of non-competitive, non-commercial contracts valued over $50,000, suppliers are presently required by DSS policy to submit a certification indicating their adherence to departmental costs and profit policies. Such a certification is normally provided by the supplier with its price proposal. Since there is no indication that suppliers have deliberately withheld supporting information, the Department sees no need to replace the current process with one requiring a certification of the accuracy, currency and completeness of the price support information as of the time a non-competitive contract is awarded.

The Supply Operations Sector has embarked on a program to streamline and modernize its operations, including the tools to verify price proposals. Item level information is the most cost-effective tool available at the present time. Improvements will continue to be made to the item information data base.

As part of its ongoing review of departmental policy and procedures, DSS is planning a review of its practices on pricing of non-competitive contracts. This includes the policy on price verification, the extent of information required of suppliers, and the roles of procurement staff, cost analysts and auditors. DSS is also revamping its training program for procurement officers and price substantiation will continue to be addressed.

Government Post-contract Financial Audit

18.79 To ensure that the prices paid for goods and services procured are fair and reasonable in the circumstances, the Department must verify the costs and profits of contractors. An independent audit of contracts, carried out by the Audit Services Group on behalf of DSS, is one of the principal tools available to the Department to discharge its responsibilities.

18.80 Chapter 16 of our 1988 Report, on DSS post-contract financial audit practices, included a review of the type and extent of contract audit carried out by DSS, the management information available, the nature and disposition of contract audit findings and the practices of other countries.

18.81 We concluded that certain aspects of government post-contract financial audit coverage at DSS were deficient and did not provide adequate protection to the Crown. As a point of comparison, we compared the Canadian practice to that of the United States government.

18.82 We recommended that DSS determine and implement the optimum mix and coverage of contract audit, develop a contract audit management information system and promptly resolve contract audit findings. Further, we recommended that the disposition of audit findings be reported to senior management and the Audit Services Group and that DSS develop a more explicit cost accounting policy, particularly for the more complex contracts, and that it has contractors disclose their cost accounting practices.

18.83 In responding to our 1988 Report, DSS stated: "The present contract audit coverage carried out by the Department continues to provide protection to the Crown in ensuring that contractor costs and profits are reasonable." The Department's response also indicated that it was conducting a review to assess the scope of contract audit coverage and its adequacy for the future and that changes would be made if necessary.

18.84 After our 1988 Report was presented to Parliament, the Treasury Board requested a review by the Office of the Comptroller General of the post-contract financial audit activity in DSS. The review, completed in April 1989, came to the same conclusion as ours in 1988. In summary the Office of the Comptroller General stated that DSS had not carried out an overall assessment of risk, nor had it developed a means of determining whether the Crown was protected through the contract audit process.

Audit Objective and Scope

Observation

DSS is currently taking steps to address our 1988 Report observations and those of the Office of the Comptroller General's 1989 review on its post-contract financial audit practices

18.87 An audit sampling and risk analysis plan is currently being tested for use by DSS contract auditors. This is expected to result in more audits than were carried out in the past for non-competitive non-commercial fixed price contracts. In addition, since the fall of 1990, price certifications provided by suppliers of non-competitive commercial goods have been subject to DSS audit.

18.88 DSS cost accounting interpretations are to be publicly distributed to provide further guidance and to address the issue of possible misinterpretation. In addition, DSS is implementing a Statement of Cost Accounting Principles questionnaire which will be completed by selected suppliers.

18.89 Because these changes are still in progress, we cannot determine their impact on the overall contract audit process. However, we will continue to monitor this function, which is an important internal control over government contracting.

Supplier Evaluation

18.91 Treasury Board and DSS policies require that the competence of suppliers be verified before awarding government contracts. Competent suppliers are those who have the technical and financial capability to discharge the contract. This capability can be measured by means such as analyzing financial statements, assessing prior experience, obtaining opinions from independent technical organizations, and carrying out evaluations of suppliers' facilities.

18.92 Suppliers invited to bid on government contracts are selected from source lists maintained by DSS or through public tendering procedures. Capability evaluations may be performed at the source listing or bid evaluation stage. Their extent varies with the circumstances and is left to the judgment of the responsible procurement officer. Treasury Board policies on bid evaluation require that in certain circumstances, such as where identical proposals are received, a supplier's track record be considered.

18.93 Treasury Board recognizes that it is not always appropriate to focus simply on price. The products offered by different suppliers are not always identical, and different suppliers' past performance is not necessarily uniform. Assessments and trade-offs must then be made between different elements of performance such as price, dates of delivery, service and logistic support, to enable DSS to achieve the overall government objective, which is obtaining the best value in procurement.

Audit Objective and Scope

18.95 We expected to find that DSS conducted reasonable evaluations of suppliers' technical and financial capabilities before awarding contracts, to ensure that the suppliers are capable of providing the required goods in the right amount, at the right time and of the required quality. We reviewed the adequacy of the management information system used by DSS to monitor and report on the post-contract evaluation of supplier performance and the use of such information.

Observations and Recommendation

Generally, DSS has procedures in place to evaluate the technical and financial capabilities of suppliers

There is no corporate system to record and report on supplier and product past performance

18.98 With a formal system, where contract performance information is shared by all parties involved, responsibility for difficulties encountered could be determined and problems addressed on a timely basis. In addition, DSS would be able to provide client departments with a record of past performance of suppliers to facilitate their procurement planning process.

18.99 DSS should investigate the feasibility and cost effectiveness of revising its computer system to include information related to suppliers' past performance.

Department's response: Since there is less reliance on the use of source lists, as a result of the General Agreement on Tariffs and Trade, the Free Trade Agreement and the Open Bidding Opportunities Policy, supplier evaluation is undergoing a significant change. The use of supplier past performance is being reviewed by the Department as part of establishing a comprehensive supplier history and capability data base.

Ship Design Contracts

18.101 Our discussions with client departments and with DSS officers indicated that various factors contribute to the problems encountered in the procurement of ship designs and that the types of difficulties illustrated in Case 4 are not uncommon in this area. They also indicated to us that DSS and its client departments are currently examining this area to determine if there are practical ways of overcoming such difficulties.

Case 4. Difficulties in Ship Design

18.103 In 1989, DSS requested construction tenders based on the revised design. A contract for $5.6 million was awarded to the lowest priced technically complying bidder. As required by the contract, the successful shipbuilder had to conduct a design check to "provide the Minister with a written statement in which the Contractor guaranteed that the design was sufficient to allow the vessel, when complete, to perform fully in accordance with the contract including the specifications". The architect engaged by the shipbuilder to carry out the design check noted in his report that he found deficiencies in the design. The firm that had designed the vessel was paid $13,000 to prepare a rebuttal in which it responded to the criticism in the above report. However, it did not convince the client department that a major design change was unnecessary. The client department directed a contract to the shipbuilder for the redesigning of the ship for an agreed amount of $300,000.

18.104 By February 1991, 10 months after the design check, construction of the ship had not started because of the need to correct the design and negotiate an increase in the price. DSS informed us that, as a result of the negotiations, the estimated construction cost of the ship increased by approximately $1 million. This amount consisted of $227,000 in modifications necessary to correct the design, $575,000 in escalation costs due to the delay, and $282,000 of additional requirements requested by the client department. On 24 May 1991, it was announced that the work had been cancelled because of a lack of resources.

18.105 Subsequent to our audit, the firm that made the original design acknowledged to this Office that there were a number of deficiencies in that design. However, in their professional opinion, the deficiencies were not severe enough to warrant a complete redesign of the vessel. We did not verify this opinion.

Quality Assurance

18.107 We recommended then that DSS consider, in consultation with central agencies, the need to clarify government policy on requirements for suppliers' quality assurance systems, including the adoption of Canadian or international quality assurance requirements. We also recommended that DSS, in consultation with its customer departments, clarify and reach agreement on an acceptable division of responsibilities for quality assurance.

18.108 We reviewed actions taken by DSS in response to these recommendations and noted that DSS has adopted various methods for promoting the use of international quality standards. The initiatives taken include intradepartmental discussions with the Product Quality Management Committee. The Canadian General Standards Board has replaced its own quality assurance standard with the International Standards Organization series as the basis for its quality listing programs. DSS reported to us that these programs are now an integral element of the sourcing process for such areas as computers, office furniture and office supplies. These programs assist client departments in establishing the appropriate level of quality required from suppliers. DSS officers indicated to us that the application of these international standards will be expanded to other products.

18.109 In support of its responsibility to plan and organize for quality assurance, DSS also developed a matrix of quality elements and related tasks. This delineates the relationship between DSS and its clients for product quality management issues. This matrix has recently been incorporated in the DSS customer manual.

Skills Development and Training

18.111 The Department is undergoing significant changes in automation and the use of technology in procurement. At the same time there are continuing changes in the products procured and their technical aspects. The training and skills of procurement officers are the key to their effectiveness in providing centralized procurement services to the Government of Canada.

18.112 DSS currently provides a variety of training courses and programs for procurement officers. These are designed to provide knowledge of the procurement function, an understanding of the procurement and supplier environment, planning and problem solving skills, and specific procurement-related skills in areas such as cost and price analysis, and accounting. The courses are supplemented by on-the-job training and courses conducted by some, but not all, operating directorates.

Audit Objective and Scope

18.114 The objective of this segment of the audit was to assess the adequacy of the current DSS training program in providing procurement officers with the skills to carry out their responsibilities. Our review was based on two recent internal studies, conducted by DSS, whose findings we verified.

Observations and Recommendation

There are insufficient mechanisms to ensure that training courses offered are up to date and meet their objectives

18.116 Course validations and quality control reviews of the training program are not regularly performed. Thus, there is no assurance that the courses are meeting their objectives. Because the procurement service has undergone such rapid changes in the technologies used and policies and procedures applied, a review of the course is needed to ensure that the material covered is appropriate and up to date.

There is insufficient training to prepare procurement officers for work on major Crown projects

18.118 The departmental studies, which we reviewed, identified gaps between the performance level of staff working on major Crown projects and the performance expectations of management for key skill areas for this type of procurement. The studies indicated that procurement officers need more training and development before starting work on major Crown projects. At the time of our audit, actions were being taken by DSS in response to these studies.

18.119 DSS training programs should be comprehensive and up to date, to provide procurement officers with the appropriate skills to discharge their responsibilities.

Department's response: DSS recognizes the importance of training for procurement officers and has recently established a formal structure, known as the Supply Portfolio Training Council, which will provide direction on procurement training needs and establish a formal link between the Supply Operations Sector and the Corporate Training Group.

In response to the internal Major Crown Project Organization Study, a new project support branch was created in April 1991. One priority that this branch will be focussing on during its first two to three years of operation will be the identification of needs and the development of a training framework for both current and future major Crown project resources. Within this framework, the branch will develop and deliver a range of courses.