2010 Fall Report of the Auditor General of Canada

Chapter 6—Acquisition of Military Helicopters

Main Points

Introduction

The acquisition process: Project management and contracting
Focus of the audit

Observations and Recommendations

Maritime Helicopter Project—Cyclone

Changes in contract provisions are not consistent with the procurement strategy
The developmental nature of the helicopter was underestimated
Full life-cycle costs, including in-service support, have not yet been determined
There were gaps in oversight and approvals by senior boards

Medium- to Heavy-Lift Helicopter Project—Chinook

Precise needs and helicopter specifications were not defined early enough
The full extent of modifications were not initially presented to decision makers
Detailed life-cycle planning was begun late
The advance contract award notice did not comply with the contracting regulations and policy
Management oversight and approval by senior boards were missing at key decision points in the project

Challenge Function of the Treasury Board of Canada Secretariat

We could not reach a conclusion about analysis and challenge provided by the Secretariat

Conclusion

About the Audit

Appendix—List of recommendations

Exhibits:

6.1—Federal organizations with a role in acquiring equipment for National Defence

6.2—Key phases, steps, approvals, and documents in government project and contract management

6.3—Timeline of key announcements and approvals for the maritime helicopter project—Cyclone

6.4—Estimated cost information for the maritime helicopter presented in the Treasury Board submissions

6.5—Timeline of key announcements and approvals for the medium- to heavy-lift helicopter project

6.6—Estimated cost information for the medium- to heavy-lift helicopters presented in the Treasury Board submissions

Main Points

What we examined

National Defence is replacing and upgrading its helicopter fleet and will spend nearly $11 billion to acquire two new types of helicopters along with long-term in-service support. The CH-148 Cyclone is a maritime helicopter that will replace the Sea King helicopters; and the CH-147 Chinook is a medium- to heavy-lift helicopter. These modern and technologically sophisticated helicopters are expected to bring new and enhanced capabilities to the Canadian Forces.

Although there are significant differences between the two acquisition projects, both have experienced cost increases and significant schedule delays. For both projects, National Defence has adopted a relatively new approach to providing for long-term in-service support.

We examined how National Defence managed the acquisitions, with emphasis on whether the work to be carried out was described clearly and consistently in key information and decision documents, risks were appropriately assessed and managed, life-cycle costs and plans were complete and timely, and senior departmental boards provided appropriate oversight and approvals. We also examined the contract award process for the projects by National Defence and Public Works and Government Services Canada.

Our conclusions relate only to the management practices and actions of public servants. We did not audit the records of the private sector contractors and, consequently, our conclusions cannot and do not pertain to the contractor’s practices or to their performance.

Our access to Cabinet confidences created prior to 2006 is governed by a 1985 order-in-council that permits access only to certain types of Cabinet confidences. Accordingly, we were not provided with information regarding decisions that were made prior to 2006.

Audit work for this chapter was substantially completed on 30 April 2010.

Why it’s important

The capabilities that National Defence is seeking to obtain in acquiring the Cyclone and the Chinook helicopters are considered by the Department as essential to the support of Canada’s military operations internationally and at home.

The total project cost of 28 Cyclone helicopters, together with initial set-up, training, provision of spare parts and long-term maintenance, is now estimated at $5.7 billion. Delivery of the first fully capable Cyclone, initially expected in 2005, was delayed to 2008 and is now expected to occur in 2012. The total project cost of 15 Chinook helicopters, together with initial set-up, training, and long-term maintenance, is now estimated at more than $4.9 billion. The first fully capable helicopter is scheduled for delivery in 2013, five years later than planned.

Given the cost and complexity of military acquisitions, how they are managed is subject to a number of regulations, policies, and controls designed to ensure that the equipment and services acquired meet the identified needs and are delivered on time and within budget in a way that enhances access, competition, and fairness. Careful planning and full costing of these projects are needed to ensure that all project elements come together in a timely and predictable way and that adequate funds are available over the long term. The demanding acquisition process requires effective leadership, oversight, and due diligence by senior decision makers across several departments. We recognize the significant efforts of many individuals involved in these projects over many years.

What we found

  • National Defence underestimated and understated the complexity and developmental nature of the helicopters that it intended to buy. Both helicopters were described to internal decision makers and the Treasury Board as non-developmental, using “off the shelf” technologies. On that basis, overall project risks were assessed as low to medium. In each case, however, significant modifications were made to the basic models. For the maritime helicopter, this will result in an aircraft that never existed before. For the medium- to heavy-lift helicopter, this will result in a new variant of the Chinook. Ultimately, these modifications led to schedule delays and cost increases beyond original plans.
  • The medium- to heavy-lift helicopter acquisition was a directed procurement using an advance contract award notice (ACAN). National Defence had initially planned to proceed rapidly to contract award by spring 2007; however, its needs and priorities were not precisely defined at the outset, evolved over the course of the acquisition, and were not finalized until 2009. The manner in which Public Works and Government Services Canada used the ACAN did not comply with the letter or intent of the applicable regulations and policies and, consequently, the contract award process was not fair, open, and transparent.
  • National Defence did not develop full life-cycle plans and costs for these helicopters in a complete or timely way. In addition, total estimated costs were not disclosed to decision makers at key decision points. Some costs have yet to be completely estimated and some elements needed for the capability are not in place. Without adequate cost information, National Defence cannot plan to have sufficient funds available for long-term operation and support of the helicopters. Moreover, without sufficient funds, National Defence may have to curtail planned training and operations.
  • National Defence did not fully comply with the oversight and approval framework established in its Project Approval Guide. For the maritime helicopter project, boards provided appropriate oversight at the preliminary project and effective project approval stages. However, neither the Senior Review Board nor the Program Management Board met to challenge and approve the information in the 2008 revised effective project approval that was related to the contract amendment approval of $262 million. For the medium- to heavy-lift helicopter, there was an absence of timely meetings, challenge, and approvals by senior boards at all key decision points in the acquisition process and before seeking Treasury Board approvals.

The entities have responded. The entities agree with all of our recommendations. Their detailed responses follow the recommendations throughout the chapter.

Introduction

6.1 National Defence is replacing and upgrading its helicopter fleet and plans to spend nearly $11 billion to acquire and maintain two new types of helicopters (all figures in this chapter are rounded, in Canadian dollars, and net of GST, unless otherwise stated). These modern and technologically advanced helicopters are expected to bring new and enhanced capabilities to the Canadian Forces.

6.2 The new maritime helicopter will replace the existing CH-124 Sea King. The Sea King entered service between 1963 and 1969 to perform anti-submarine warfare duties in support of Canadian naval surface vessels; the need to replace it was first identified in 1975. A maritime helicopter’s capabilities are a vital part of a frigate’s ability to protect itself and to exercise surveillance and control well beyond the range of the ship’s own sensors and weapons. In 2004, following a competitive procurement, two separate but related fixed-price contracts were awarded to Sikorsky International Operations Inc.: one for $1.8 billion to deliver 28 maritime helicopters and to modify one frigate from which they operate; and one for $3.2 billion to provide initial set-up and long-term in-service support. It is planned that Canadian Forces personnel will continue to perform routine maintenance and repair at base and while deployed. The new helicopter was named the CH-148 “Cyclone” in 2004. The new fleet will be located on the east coast at 12 Wing Shearwater and on the west coast at Patricia Bay.

6.3 The new medium- to-heavy-lift helicopter (capable of lifting medium to heavy payloads) is intended to provide the Canadian Forces with the ability to move personnel and equipment by air quickly and efficiently in a variety of risk environments at home and overseas. The government’s intention to buy these new helicopters was announced in Budget 2005, and the formal acquisition process commenced shortly thereafter. In 2009, a directed contract with a $1.4-billion value was awarded to The Boeing Company for 15 “Canadianized” Chinook helicopters. An amendment to this contract is planned in 2013, to provide for long-term in-service support. The new fleet will be located at CFB Petawawa.

6.4 Buying helicopters is only one part of providing the Canadian Forces with the capability to perform expected missions. Other elements must also be in place to provide the capability and sustain the equipment over its entire life, including qualified and trained personnel to operate and maintain the aircraft, infrastructure to house the helicopters, and the spare parts needed over the life of the aircraft. The maritime helicopter also requires modifications to the frigates from which they operate. In this chapter a distinction is made, where appropriate, between the costs directly attributable to the acquisition project (including but not limited to third-party contracts) and the total costs associated with the capability.

6.5 For both helicopters, National Defence has adopted a relatively new approach for long-term, in-service support. Although Canadian Forces personnel will provide routine maintenance and minor repairs while at base and while deployed, this new approach involves contracting with the original equipment manufacturer to provide initial set-up and training, spare parts, and major repairs. The approach aims to provide a single point of accountability for maintenance and availability of the fleet.

The acquisition process: Project management and contracting

6.6 Major projects to acquire defence equipment involve several federal departments (Exhibit 6.1).

Exhibit 6.1—Federal organizations with a role in acquiring equipment for National Defence

Organization Roles and responsibilities
Cabinet Serves as the ministers’ forum for discussion and decision making.
Treasury Board Approves expenditure authority for projects that exceed ministerial project approval limits.

Approves a department’s request to enter into contracts.

May allow exemptions from Treasury Board policies.

Treasury Board of Canada Secretariat As the Treasury Board’s administrative arm, develops policies and guidance for the federal government in such areas as project management and contracting.

Provides advice and assistance to departments in preparing submissions to the Treasury Board and challenges submission content.

National Defence Initiates and manages acquisition projects.

Defines project requirements.

Analyzes and recommends options.

Proposes a procurement strategy.

Prepares statements of work and plans for technical evaluation of bids.

Performs quality assurance, receives goods, and authorizes payments.

Public Works and Government Services Canada Manages the contracting process.

Conducts market analysis of industry availability and capacity.

Develops final procurement strategies.

Prepares bid documents and conducts tendering processes.

Awards and administers contracts.

6.7 Such major equipment acquisition projects are subject to legislation, Treasury Board policies, and various departmental guides and manuals. Together they aim to ensure that the projects are managed effectively and economically, that the equipment and related services are acquired in a way that enhances access, competition, and fairness, and that they result in best value. They specify key steps to be followed, types of information and mandatory documents required, and approvals that must be obtained as an acquisition project progresses through the project management and contracting phases (Exhibit 6.2). Key project information tracked through the phases of the project includes operational requirements, cost, risks, and delivery schedule.

Exhibit 6.2—Key phases, steps, approvals, and documents in government project and contract management

Flowchart

[text version]

Preliminary project approval—Within National Defence, internal approval in principle for the preferred option at the estimated cost and to proceed to the project’s definition phase. Within the Treasury Board, authorization to expend resources for the project’s definition phase. Submissions at this phase are expected to include the indicative estimate of total project cost, including life-cycle costs.

Effective project approval—Within National Defence, internal approval for the selected option and to proceed to the project’s implementation phase. Within the Treasury Board, expenditure authority to implement the project. A substantive total project cost estimate based on higher quality and reliability is expected to be included in supporting documentation.

Note: This exhibit presents an adapted version of the project management process used within National Defence and the contract management process used within Public Works and Government Services Canada in acquiring military equipment. Each process is presented separately but some activities happen concurrently.

6.8 A number of boards in National Defence are involved in providing project oversight and approvals.

  • Program Management Board—has overall responsibility for approving projects and for monitoring project performance.
  • Joint Capability Requirements Board—reviews and endorses technical and operational requirements of projects.
  • Senior Review Board—is established for each large project to provide rigorous examination of and challenge to a project and ongoing review and oversight.

6.9 For projects greater than $100 million, an interdepartmental Senior Project Advisory Committee is also established to perform the role of a procurement review committee and to serve as a forum for reviewing and discussing project objectives, requests for proposals, and other key project instruments.

6.10 The acquisition of new, technologically sophisticated and expensive capability for the Canadian Forces is a significant challenge in government procurement. The demanding acquisition process requires effective leadership, oversight, and due diligence by senior decision makers across several departments. We recognize the significant efforts of many individuals involved in these projects over many years.

Focus of the audit

6.11 This audit examined whether National Defence and Public Works and Government Services Canada (PWGSC) managed the acquisition of the maritime and the medium- to heavy-lift helicopters in compliance with selected key provisions of applicable regulations and policies. Specifically, these are: the Government Contracts Regulations; the Treasury Board’s Project Management Policy, Project Approval Policy, Policy on the Management of Major Crown Projects, and Contracting Policy; the PWGSC Supply Manual; and National Defence’s Project Approval Guide. These documents define expectations for the processes, key phases, and approvals required in managing major Crown projects and awarding and administering contracts. More specifically, we examined whether

  • technical and operational requirements were clearly defined and consistently adhered to,
  • risks were appropriately assessed and managed,
  • life-cycle plans and costs were identified,
  • key decisions were supported by complete and accurate information, and
  • appropriate and timely approvals were obtained.

We placed particular emphasis on the oversight role played by senior committees and boards.

6.12 Although the maritime helicopter project began in 1995, the activities we audited occurred between 1999 and 31 March 2010. The audit of the medium- to heavy-lift helicopter project covered activities between January 2005 and 31 March 2010.

6.13 More details about the audit objectives, scope, approach, and criteria are in About the Audit at the end of this chapter.

Observations and Recommendations

6.14 Our observations are presented separately for the two helicopter projects, the Cyclone and the Chinook. Our recommendations are provided in the section on the Chinook helicopter, but are meant to apply to both helicopter projects.

Maritime Helicopter Project—Cyclone

6.15 The process to replace the Sea King helicopters started nearly 25 years ago. A first attempt began in 1986 and resulted in a contract to purchase the EH-101. This contract was cancelled by the government in 1993 at a cost of $478 million. The acquisition process started again in 1995 as the Maritime Helicopter Project. As stated in the 1994 Defence White Paper, “Sea Kings are rapidly approaching the end of their operational life. There is an urgent need for robust and capable new shipborne helicopters.” The process is still underway (Exhibit 6.3).

Exhibit 6.3—Timeline of key announcements and approvals for the maritime helicopter project—Cyclone

6.16 Noteworthy in this acquisition are the use of a competitive procurement process leading to the award of two contracts to Sikorsky International, the announcement in 2008 of a 30-month delay in the delivery of the helicopters, and the negotiation of a major contract amendment. Subsequent to the completion of our audit work for this chapter, Public Works and Government Services Canada executed a second major amendment to the contract with Sikorsky on 30 June 2010 (see paragraph 6.34).

6.17 In 2000, total indicative costs of the 28 maritime helicopters were estimated at $2.8 billion and revised to $3.1 billion in 2003, exclusive of the cost of providing in-service support. The cost of purchasing and providing in-service support for the helicopters, and of training personnel, is currently estimated to be $5.7 billion over 20 years. This estimate does not include costs related to contracted Sea King support, new infrastructure, Canadian Forces personnel, and ongoing operating costs. In addition, the project has experienced delays. Delivery of the first fully capable Cyclone, initially expected in 2005, was delayed to 2008 and is now expected to occur in 2012.

Changes in contract provisions are not consistent with the procurement strategy

6.18 A basic principle of government procurement is that to ensure openness and fairness, project requirements should be defined consistently across the key contracting documents—statement of operational requirements, detailed specifications for the supplier selection process, the request for proposals, and contract amendments—and should not change significantly during the selection process or after a contract is awarded. This is particularly so with respect to this acquisition, given that the procurement strategy was based on awarding the contract to the lowest price bid that complies with the stated requirements. We examined whether the project requirements were described consistently in key information and decision documents and the nature and justification of the contract amendment. We also examined the basis for the chosen procurement strategy; however, we were unable to reach a conclusion on this due to limits to our access to Cabinet confidences prior to 2006 (see paragraph 6.96).

6.19 Between 1995 and 1999, National Defence conducted options analyses and recommended the procurement of new helicopters to fill the capability deficiency identified in 1994. The statement of operational requirements was approved in 1999. These were subsequently converted into detailed specifications needed for the procurement process. Only requirements deemed essential were included in the final specifications. An independent review conducted for National Defence found that the operational requirements were adequately translated into the detailed specifications used during the selection process. We performed a similar analysis of selected operational requirements and reached similar conclusions.

6.20 In 2000, two options for procurement strategies were under consideration. One was for a single competition to select a supplier of the combined airframe and mission systems; the other was for two competitions, splitting the airframe and mission systems. National Defence and Public Works and Government Services Canada (PWGSC) assessed the pros and cons of each strategy for government consideration. The government chose to have two competitions. In the fall of 2000, PWGSC issued a letter of interest that sought to identify those companies interested in bidding to become the prime contractor for the airframe or the mission systems.

6.21 In late 2002, the Minister of National Defence announced a major change in the procurement strategy: The helicopters would be purchased through a single competitive process to acquire 28 fully equipped helicopters. A second letter of interest was subsequently issued. In effect, the acquisition process was restarted, which resulted in a two-year delay.

6.22 A pre-qualification process was undertaken in March 2003, based on detailed technical specifications, and two manufacturers were deemed technically compliant. Each subsequently submitted a full bid in response to the request for proposals. The bids were evaluated based on compliance with the technical requirements, industrial and regional benefits, and lowest overall price. In July 2004, the government announced the winner as Sikorsky International Operations Inc. and two contracts (for the acquisition and the service support) were awarded shortly thereafter. Between 2004 and 2007, National Defence had regular progress meetings with Sikorsky on the status of the project.

6.23 In early 2008, Sikorsky informed PWGSC of a 30-month delay in the planned delivery of the helicopters, citing, among other things, additional work directed by or caused by the Crown (the departments) and the challenges associated with obtaining certification for a technical component in the new helicopter. This led to a contract amendment in 2008, which allowed for a delay in the delivery of the first fully capable helicopter to June 2012, 43 months after the original contract’s delivery date. PWGSC negotiated with the contractor for delivery of up to 19 interim helicopters starting in November 2010. These interim helicopters will not fully meet all of the operational requirements, in particular the mission endurance performance of two hours and 50 minutes, and will have to be retrofitted at a later date. While the interim helicopters will be suitable for testing, evaluation, and training, they will not be deployed on operations. The amendment extended the time period for providing in-service support by two years and amended the provisions contained in the original contract that set out damages associated with late delivery to coincide with the new delivery schedule.

6.24 The amendment also provided for additional “power reserve” (more powerful engines) to meet the mission endurance performance requirement and to accommodate a potential growth in performance in the future. According to PWGSC, this increased the cost of the contracts by about $84 million for the growth potential.

6.25 This amendment is significant in principle because the procurement strategy, which was communicated to industry, was based on the lowest-price bid that met the stated (essential) requirements. The process did not give any credit to bids, or portions of bids, that exceeded the stated requirements. National Defence has, in effect, acknowledged that there is value in acquiring a helicopter with this additional power reserve and is now paying for more capability than it said it needed at the time the contract was signed. It should be noted that the in-service support contract with the manufacturer specifically requires that it maintain the aircraft’s endurance performance at the desired level throughout the life of the helicopter and contract period. Consequently, in our opinion, the contract amendment is not consistent with the original lowest price compliant procurement strategy. It also raises the question as to whether a lowest price compliant strategy is compatible with the acquisition of complex military equipment requiring significant development.

The developmental nature of the helicopter was underestimated

6.26 The Treasury Board’s Project Management Policy requires that a project profile and risk assessment be prepared early in a project to document the level of risk and to develop a corresponding risk mitigation strategy. The project’s overall risk level is to be assessed as high, medium, or low, depending on the size, scope, and complexity of the project, and is to be reassessed periodically throughout the life of the project. We examined whether National Defence appropriately assessed the risks entailed in the acquisition of the Cyclone helicopter and developed a strategy to mitigate them.

6.27 The assessment of technical, cost, and schedule risk is closely tied to the complexity and developmental nature of a planned acquisition. Terminology is important in this regard, and it is helpful to think in terms of a spectrum from “off-the-shelf” to “developmental” equipment or technology. Off-the-shelf equipment is typically an existing product that is ready for delivery as is. Developmental equipment is typically a product that does not yet exist. In between these extremes are various degrees of modification to an existing product or integration of existing technologies. According to National Defence, a “simple” procurement typically involves an off-the-shelf product of low dollar value with fully defined requirements, and the absence of political sensitivity; whereas a “complex” procurement involves a complex statement of work, many systems and elements to bring together, significant resource constraints and trade-offs, and many players.

6.28 National Defence began preparing the maritime helicopter project profile and risk assessment in 2000. It was presented to its Senior Review Board, though there is no record that it was approved as required. As part of the preliminary project approvals in 2003, a revised assessment was prepared and approved within National Defence and was summarized for presentation to the Treasury Board. We could find no evidence that a formal assessment was prepared in support of the 2004 effective project approval or the 2008 revised effective project approval. The overall project risks were assessed in 2000 as low to medium, an assessment that continued through key approvals in 2003 and 2004.

6.29 As early as 2000, information provided to National Defence’s Program Management Board for preliminary project approval described the proposed acquisition project as non-developmental, using a commercial off-the-shelf airframe and mission system technologies, advising that Canada would not be the first buyer. The 2000 letter of interest issued to potential suppliers required maximum use to be made of non-developmental off-the-shelf equipment. The 2003 preliminary project approval submission to the Treasury Board informed ministers that operational requirements were written to take advantage of existing technology and that potential manufacturers have confirmed their ability to deliver a non-developmental helicopter in the time frame required. The 2004 effective project approval submission informed ministers that the airframe and mission systems were non-developmental but also acknowledged that potential technical difficulties may arise with modifying the helicopter or integrating the mission systems.

6.30 However, it is evident that this was to be a complex procurement. The project specifications included nearly 3,000 technical requirements. The winning bid by Sikorsky was to convert an existing commercial helicopter (the S-92) to military service, adapt it for marine use, and integrate numerous individual existing mission components and new technologies. According to National Defence officials, this will result in a state-of-the-art helicopter that has never existed before. The initial acquisition contract reflects this complexity in that it included $612 million for one-time engineering costs. National Defence and PWGSC sought and received approval from the Treasury Board to spend up to $659 million to cover such costs as part of the 2008 revised effective project approval submission. In our opinion, National Defence has, in effect, entered into an agreement with Sikorsky to develop a new helicopter and this should have been reflected in project risk assessments and in information provided to decision makers.

6.31 For the pre-qualification process managed by PWGSC, potential bidders were required to submit “proof of compliance” for 476 of the 3,000 technical requirements that were deemed to pose a higher risk; this involved providing reports, plans, and/or drawings describing in detail how the requirements would be met. The extent to which the contractor’s proposals met the stated requirements was assessed as part of the evaluation. No consideration was given to solutions that were already in existence (off-the-shelf) versus those requiring modification or development. In our opinion, this pre-qualification process did not adequately consider the significant technical risk associated with the developmental nature of this helicopter.

6.32 The developmental nature of the Cyclone helicopter, along with its novel features, also has implications for certifying its airworthiness. The selection process required airworthiness certification of the basic commercial helicopter at the time of the contract award and certification of the new aircraft when the first one was delivered. However, the complexity of the new aircraft has led to unanticipated certification requirements and a potential unforeseen role for National Defence in undertaking certain aspects of the certification.

6.33 In our opinion, National Defence did not adequately assess the developmental nature of this aircraft, and the risks related to cost and the complexity of the required technical modifications were underestimated. In addition, the original delivery schedule (48 months after contract award) and the identified risk related to the potential for delays were not consistent with the developmental nature of this acquisition. (See paragraphs 6.64 and 6.65 for our recommendations).

6.34 As noted in paragraph 6.16, subsequent to the completion of audit work for this chapter, the contract with Sikorsky was amended a second time on 30 June 2010, providing for delivery of “redefined” interim helicopters. These interim helicopters will not be fully compliant in areas that reflect difficulties associated with the developmental nature of the helicopter, such as the mission system software and the exchange of tactical data between the ship and the aircraft. PWGSC informed us of this amendment in July 2010.

Full life-cycle costs, including in-service support, have not yet been determined

6.35 The Treasury Board Contracting Policy states that inherent in procuring best value is the consideration of all relevant costs over the useful life of the equipment, not solely the initial acquisition or basic contract cost. Moreover, National Defence’s Project Approval Guide requires an estimation of total indicative costs when seeking preliminary project approval and total substantive costs for effective project approval. We recognize the challenges associated with estimating costs in such major acquisitions, particularly at the preliminary project approval phase, and that costing information is expected to become more precise over time. Careful planning and full costing are needed to ensure that all of the elements required to provide the needed defence capability come together in a timely and predictable way and that adequate funds are available to support the equipment over the long term. In the case of the maritime helicopter, start-up and arrival of the Cyclone must also be coordinated with modifications to the frigates and with the transition from the Sea King helicopter.

6.36 We examined whether information presented to decision makers was complete and accurate and whether approvals were given according to key provisions of legislation and National Defence and government policies and guidelines. This included project costing information contained in Treasury Board submissions and information presented to National Defence’s Program Management Board.

6.37 We found that National Defence has been slow to assess the full life-cycle costs, and some elements of these costs have still not been completely determined. In 2000, total indicative costs of the 28 maritime helicopters were estimated at $2.8 billion, and this was revised to $3.1 billion in 2003, exclusive of the cost of providing in-service support. The cost of purchasing and providing in-service support for the helicopters, and of training personnel, is currently estimated to be $5.7 billion over 20 years, an estimate that does not include costs related to contracted Sea King support, new infrastructure, Canadian Forces personnel, and ongoing operating costs. Consequently, the full life-cycle costs were not—and some still have not been—presented to decision makers at key decision points (Exhibit 6.4). Without such information, National Defence cannot adequately plan to have sufficient funds available for long-term operations and support of the helicopter. Moreover, without sufficient funds, National Defence may have to curtail planned training and operations.

Exhibit 6.4—Estimated cost information for the maritime helicopter presented in the Treasury Board submissions

  Preliminary Project Approval
June 2003
Effective Project Approval
November 2004
Revised Effective Project Approval
December 2008
(In millions of Canadian dollars)*
Capital costs  
Maritime helicopter 1,850 1,690 1,529
Ship modifications and other equipment 191 193 282
Initial set-up (includes engineering, provision of spares, training facilities) 819 838 815
Project management 158 156 210
Contingency and escalation/inflation 98 297 130
Infrastructure not included not included not included
Growth potential and other costs n/a n/a 208
Total capital costs 3,116 3,174 3,174
Personnel, operating, and maintenance costs  
Contracted in-service support not included 2,346 2,495
National Defence personnel not included not included not included
National Defence operating costs note 1 note 2 not included
Total personnel, operating, and maintenance costs 2,346 2,495
Total costs 3,116 5,520 5,669
Notes:

1. The Treasury Board submission noted that the in-service support concept was to be more clearly identified.

2. The Treasury Board submission noted that total operations and maintenance costs for the Cyclone would be $900 million more between the fiscal years 2011–12 and 2022–23 than for the Sea King.

* Figures have been rounded.

Source: Key approval documents; unaudited figures

6.38 For example, in the 2003 submission to the Treasury Board for preliminary project approval, National Defence should have presented estimated costs for infrastructure, personnel, and operations and maintenance or contracted in-service support. In fact, in reviewing the planned submission, National Defence’s Program Management Board noted the absence of this information and raised concerns about sources of funding for future operations and maintenance and the assessment of infrastructure costs. These concerns triggered subsequent discussions about the affordability of the Cyclone and of means to reduce costs, including the possibility of reducing the number of flying hours.

6.39 Estimating costs for long-term in-service support was especially challenging. Until 2003, National Defence assumed that costs for personnel, operations, and maintenance would be offset in whole or in part from the Sea King expenditures. No additional or incremental costs were identified in seeking preliminary project approval. On the basis of the bid received from Sikorsky, $2.3 billion in estimated costs for contracted in-service support for 16 years (based on flying 10,000 hours annually) was presented for effective project approval in 2004. By this time, National Defence realized that personnel, operations, and maintenance costs would exceed those associated with the Sea King by $1.1 billion over 20 years. This is significant because National Defence did not seek additional funding for the in-service support provisions, so these incremental costs will need to be covered by its existing operations and maintenance budget. Given the fixed budget for maintenance, National Defence has observed that it may have to take measures to reduce the pressure on this budget, including reducing the number of anticipated flying hours.

6.40 For land-based infrastructure, National Defence initially assumed that, despite known deficiencies, the hangars and other facilities used for the Sea King helicopters could be used for the new helicopters, thus no cost estimates were presented as part of the preliminary project, effective project, or revised effective project approvals. In 2005, however, National Defence determined that there was a need for significant investment in new infrastructure. It has since initiated approximately $340 million in projects for maintenance, spare parts warehousing, training, and squadron facilities. These projects were subject to separate Treasury Board and departmental submissions and approvals, which stressed the link to the delivery of the Cyclone helicopter.

6.41 There was also a need to extend the life of the Sea King helicopters longer than originally anticipated as a consequence of delays in the Cyclone project. This resulted in further costs. Specifically, a third-party contract was signed by National Defence in 2003 for the repair and overhaul of the Sea King until March 2008. An option to extend the period was exercised in November 2007 for an additional $168 million to cover the period up to 2014. The extension coincided with the notification that the delivery of the Cyclone helicopters would be delayed.

6.42 Taking into account all direct and related costs associated with the Cyclone capability, we estimate the total costs to be in the order of $6.2 billion exclusive of National Defence personnel and operating costs. (See paragraphs 6.74, 6.75, and 6.76 for our recommendations).

There were gaps in oversight and approvals by senior boards

6.43 In approving large capital acquisitions, National Defence is expected to follow its Project Approval Guide and the Treasury Board’s Project Approval Policy to ensure that risks are minimized and the acquisition meets the stated requirements. We focused on information that was presented internally and to the Treasury Board—as well as the role played by key boards within National Defence—for key approvals of the project, namely at the identification, preliminary project, effective project, and contract approval phases. This included approval documents, records of decisions, and minutes of meetings. We recognize that oversight within National Defence is provided by both individuals serving in senior positions and by designated boards (on which these individuals may sit).

6.44 Of the key decision points in the approval process for a major Crown project, we found that the National Defence boards provided appropriate oversight at the preliminary project and effective project approval stages. However, neither the Senior Review Board nor the Program Management Board met to challenge and approve the information in the 2008 revised effective project approval related to the contract amendment approval of $262 million. Though we were provided evidence that senior personnel signed off on the key decision documents, the absence of board involvement is important because significant changes to the cost, scope, and schedule of the project were decided without the review and challenge the board provides, as intended in the Project Approval Guide. In addition, as the project moved through the implementation phase, there were gaps in the monitoring of the project performance between May 2007 and March 2010 by the Senior Review Board.

6.45 There is also no record that the Senior Review Board met to challenge and approve other key documents such as the project charter, statement of operational requirements, and project profile risk assessments. The information contained in these documents later became part of the documents submitted for project approvals within National Defence and the Treasury Board. It is important for effective project management that internal and external decision makers have a holistic view of the project and a forum to discuss and assess project risks and performance.

6.46 Without ongoing board oversight or approvals, the maritime helicopter project is not being managed in accordance with the Project Approval Guide and the Treasury Board’s Project Approval Policy (see our recommendation at paragraph 6.92).

Medium- to Heavy-Lift Helicopter Project—Chinook

6.47 The Boeing Chinook helicopter is well known and in use by militaries around the world. Canada once owned and operated a fleet of Chinooks, but these were sold in the 1990s. With this project, National Defence is building a new capability of medium- to heavy-lift helicopters from the ground up. Among other things, there is a need for suitable facilities to house the fleet, personnel to maintain the aircraft, and pilots to fly them. Noteworthy in this acquisition are its rapid start-up and aggressive schedule following the 2005 Budget announcement, a directed contract awarded to The Boeing Company, use of an advance contract award notice (ACAN) instrument, and the as yet undetermined provisions and costs for in-service support. Exhibit 6.5 provides a timeline of key events.

Exhibit 6.5—Timeline of key announcements and approvals for the medium- to heavy-lift helicopter project

6.48 In 2006, at the preliminary project approval stage, total indicative costs for the acquisition of 16 medium- to heavy-lift helicopters were estimated at $2 billion, exclusive of long-term in-service support. The cost of purchasing and providing in-service support for 15 helicopters, and of training personnel, is currently estimated to be $4.9 billion over 20 years, which does not include ongoing operating costs and the costs of the nearly 500 Canadian Forces personnel required to operate and maintain the fleet. In addition, planned delivery of the first fully capable Canadianized Chinook has been delayed from 2008 to 2010, and then again to 2013, and the number of helicopters being purchased has been reduced from 16 to 15.

Precise needs and helicopter specifications were not defined early enough

6.49 As noted for the maritime helicopter project, project requirements should be defined consistently across the key contracting documents and should not change significantly during the procurement process. We examined whether the work to be carried out was described clearly and consistently in key information and decision documents. It should be noted that following the budget announcement, National Defence had planned to proceed rapidly through the options analysis and definition phases of this acquisition, anticipating contract award as early as the fall of 2006. We found that National Defence’s needs and priorities were not precisely defined at the outset and evolved over the course of the acquisition. They were not finalized until the contract with Boeing was signed in 2009.

6.50 The February 2005 Budget announcement highlighted the need for helicopters capable of moving personnel and equipment around in the field and in a variety of domestic and international situations. The project entered the options analysis phase in July 2005 following approval by National Defence of a broadly worded one-paragraph statement of capability deficiency. This statement highlighted a gap in helicopter support to the Land Forces in conditions like Afghanistan and also noted the need to support new task force structures then being considered. From the outset, National Defence said the helicopters were intended to serve multiple missions, which evolved from initially supporting “high, hot and heavy” operations in places like Afghanistan to eventually supporting domestic requirements in the Arctic.

6.51 Soon after the 2005 Budget, National Defence met with The Boeing Company in order to obtain detailed information about its Chinook helicopter, such as the available configurations and technological options and their associated costs. Through 2005 and 2006, additional meetings were held with Boeing and the exchange of technical and costing information continued. During this period, National Defence also conducted a market analysis by obtaining information about other helicopters, largely based on literature review.

6.52 In June 2006, National Defence approved the statement of operational requirements. It identified the following seven high-level mandatory requirements:

  • internal lift capacity: ability to carry 30 soldiers and full combat equipment weighing a total of at least 4,763 kilograms;
  • external lift capacity: ability to lift multiple loads, weighing a total of at least 5,443 kilograms;
  • minimum flying range of 100-kilometre radius with either the internal or external payload outlined above and at the temperature and altitude set out below;
  • able to provide the lift and flying range outlined above at a temperature of 35oC and altitude of 1,220 metres above sea level;
  • airworthiness certification of the aircraft to standards recognized by Canada by contract award date;
  • minimum fleet size of 16 helicopters, to be housed at two main operating bases; and
  • delivery of the first aircraft no later than 36 months after contract award and delivery of the final aircraft no later than 60 months after contract award.

These were the minimum requirements that a helicopter would need to meet and were intended for use as market discriminators in the procurement process. The statement also included 136 “rated” requirements without which, according to National Defence, operational capabilities would be “seriously diminished.” An aggressive delivery schedule was planned for the helicopters, noting the importance of an “off-the-shelf” solution to meet these needs. The new helicopters were to be housed at two main operating bases. The statement of operational requirements was not reviewed or endorsed by the Senior Review Board or by the Joint Capability Requirements Board until October 2006 and, consequently, did not benefit from the rigorous challenge these bodies are expected to provide.

6.53 By June 2006, based on meetings and discussions with Boeing and the market analysis, National Defence had formally concluded that Boeing’s Chinook was the only existing Western certified helicopter in production capable of meeting its needs. It subsequently recommended to the interdepartmental Senior Project Advisory Committee to employ an advance contract award notice (ACAN). The Committee endorsed this procurement strategy prior to preliminary project approval submission to the Treasury Board, while expressing concern about the ability to meet the planned aggressive schedules.

6.54 Also in June 2006, National Defence sought and received preliminary project approval from the Treasury Board, thereby starting the project’s definition phase. National Defence informed the Treasury Board that a contract was expected to be awarded between the fall of 2006 and the winter of 2007. The ACAN, posted in July 2006, included a revised anticipated contract award date of July 2007. After the closing of the ACAN, National Defence worked with Boeing to develop a detailed statement of work that would be the basis for the request for proposal to which Boeing would provide a bid. Achieving this aggressive schedule would have required that National Defence knew precisely the type of missions the helicopter would support, what it wanted the helicopter to do, and the technical specifications needed to achieve it.

6.55 This precise information was not in place at the time, and this affected the subsequent process of detailing specific needs and priorities. Evidence on file indicates uncertainty, before and after June 2006, regarding which types of operations were to be supported (whether land, maritime, or special operations), what mission systems would be needed, the minimum number of helicopters, and whether the helicopters would be located at one or two main operating bases.

6.56 The intended configuration of the Canadianized Chinook evolved as decisions were made about the operations it would support and about the number and location of operating bases, and as the costs associated with modifying the helicopter to meet the operational needs became clearer through discussions with Boeing. There are several models or variants of the Chinook helicopter, with different features and capabilities. According to National Defence, the seven high-level mandatory requirements could have been met by a basic Chinook model. However, in the process of detailing its specifications with Boeing, National Defence also drew from the set of rated operational requirements, effectively treating extended-range fuel tanks, an upgraded electrical system, and aircraft survivability equipment as mandatory requirements, though none had been originally identified as such. These additional modifications resulted in significant changes to a basic Chinook model and also had an impact on the timing and complexity of certification for airworthiness.

6.57 In March 2007, Boeing informed National Defence that accommodating these additional requirements would cause a delay in the delivery of the helicopter. Canadian-requested modifications also caused an increase in the cost. In an attempt to stay within the project expenditure authority approved by the Treasury Board, National Defence had to subsequently re-examine its needs in an exercise to reduce the project scope. The process to define the detailed statement of work for the request for proposal added more than two years to the contracting process than was originally planned and presented to the Treasury Board for approval in 2006. Ultimately, National Defence’s requirements were not finalized until 2009 when the contract with Boeing was signed. By the time the contract was awarded in 2009, one of the seven high-level mandatory requirements had been changed (minimum fleet size) and two would not be met (airworthiness certification and scheduled delivery of the first Canadianized Chinook, which had both been delayed).

The full extent of modifications were not initially presented to decision makers

6.58 As noted in paragraph 6.27 for the maritime helicopter project, the assessment of technical, cost, and schedule risk is closely tied to the complexity and developmental nature of a planned acquisition. In our opinion, National Defence understated the complexity of configuring this helicopter. In effect, Canada is purchasing a new variant of the Chinook helicopter that requires significant modifications to meet National Defence’s requirements and needs.

6.59 In 2006, the project was described by National Defence internally and to Cabinet and the Treasury Board as an off-the-shelf procurement, taking advantage of helicopters in production and incorporating existing technologies. Canadian-requested modifications were to be adopted only where essential. This was the basis of the information on risk, cost, and schedule presented for preliminary project approval. Risks were generally assessed as “low” to “medium,” although a formal project profile and risk assessment was not finalized or approved at this stage, or in fact at any of the three key project approval points, as required.

6.60 We disagree with the characterization of this helicopter as being off-the-shelf. It is evident that from the beginning, National Defence did not intend to procure an off-the-shelf Chinook but rather a modified one that included, among other things, extended-range fuel tanks, an upgraded electrical system, and aircraft survivability equipment. While these components existed in other Chinook models, they had never been incorporated in the Canadianized model the Department was seeking. Documentation within National Defence describes the inclusion of extended-range fuel tanks as requiring “major structural changes.” Other modifications were unique to the Canadianized Chinook. Detailed technical information and assistance were needed in order to understand the impact of the required Canadian modifications.

6.61 In March 2007, Boeing informed Public Works and Government Services Canada (PWGSC) and National Defence that the delivery schedule would need to be extended by an additional 12 months to allow for airworthiness and safety certification of the aircraft. So significant were the modifications to the basic Chinook helicopter that Boeing’s estimate included nearly US$360 million for one-time engineering costs. The extent of changes is further evidenced by the fact that the final contract awarded to Boeing includes a provision for Canada to recover some costs should another customer purchase the Canadianized version in the future.

6.62 National Defence knew, prior to seeking preliminary project approval from the Treasury Board and issuing the ACAN, that significant modifications to a basic Chinook were desired and planned. It knew also that these would increase the risks to cost and schedule. However, this was not presented to the Treasury Board when seeking preliminary project approval but should have been. We observed that the information provided to the Treasury Board when seeking effective project approval in 2009 more clearly and fully described the nature of the modifications and associated risk. Ultimately, Canadian-required modifications increased the cost of each aircraft by 70 percent more than initially quoted by Boeing in early 2006. This prolonged the negotiation of the contract by over two years and delayed the delivery of the aircraft.

6.63 The procurements of the Chinook and Cyclone helicopters raise the question of whether it is realistic to expect to be able to acquire complex pieces of military equipment that meet Canada’s unique needs using strictly off-the-shelf technologies and using a lowest price compliant strategy. It needs to be recognized that the acquisition of such complex equipment brings with it unique risks and challenges that need to be properly identified and managed using an appropriate procurement strategy.

6.64 Recommendation. National Defence should review and apply the lessons learned with these helicopter acquisitions to ensure that, for future major capital equipment acquisitions, the degree of modifications and/or development involved is fully reflected in approval documents—in the assessment of risk, project timelines, and costs—and that procurement strategies are tailored to the complexity of the equipment being acquired.

The Department’s response. Agreed. National Defence continuously strives to capture the lessons learned in undertaking complex acquisitions and, in this context, will undertake a specific review of the projects encompassed by this report. In addition, a review of the associated policies, procedures, and processes will also be conducted and they will be revised as required.

6.65 Recommendation. Public Works and Government Services Canada should review and apply the lessons learned with these helicopter acquisitions to ensure that, for future major capital equipment acquisitions, the degree of modifications and/or development involved is fully reflected in approval documents and in the assessment of risk, and that procurement strategies and contracts are tailored to the complexity of the equipment being acquired.

The Department’s response. Agreed. Public Works and Government Services Canada agrees to review the lessons learned with these helicopter acquisitions to ensure that, for future major capital acquisitions, the degree of modifications and/or development involved is fully reflected in approval documents and the assessment of risk, and that procurement strategies and contracts be tailored to the complexity of the equipment being acquired. The lessons learned will be completed by the end of March 2011.

Detailed life-cycle planning was begun late

6.66 As noted in paragraph 6.35 for the maritime helicopter project, full life-cycle costing and careful planning are needed to ensure that all elements required to provide a defence capability come together in a timely and predictable way and that adequate funds are available over the long term. This was particularly important in the case of the medium- to heavy-lift helicopter since National Defence is building a new capability from the ground up. We found it did not undertake the necessary detailed planning or costing of the medium- to heavy-lift helicopter project in a complete and timely way.

6.67 Early in the project, National Defence recognized in principle the elements needed, in addition to the helicopters, to provide full operating capability, such as personnel, training, infrastructure, operations, and long-term in-service support. In seeking Cabinet approval in principle for the acquisition in June 2006, National Defence had estimated the total long-term project costs to be $6.9 billion.

6.68 However, these total estimated costs were not presented to the Treasury Board. The costs that were presented were incomplete and underestimated, particularly at the preliminary project approval stage (Exhibit 6.6). Specifically, key items such as in-service support and personnel costs were not included. In addition, both the acquisition costs for the helicopter and infrastructure costs were initially underestimated. We observed improvements in the completeness of costing information presented to the Treasury Board for effective project approval in 2009.

Exhibit 6.6—Estimated cost information for the medium- to heavy-lift helicopters presented in the Treasury Board submissions

  Preliminary Project Approval
June 2006
Effective Project Approval
November 2009
(In millions of Canadian dollars)*
Capital costs  
Medium- to heavy-lift helicopters 1,025 1,245
Initial set-up (includes engineering, provision for spares, training facilities) 457 496
Project management 48 122
Contingency and escalation/inflation 430 232
Infrastructure 62 218
Total capital costs 2,022 2,313
Personnel, operating, and maintenance costs  
Contracted in-service support not included 2,573
National Defence personnel not included not included
National Defence operating costs not included not included
Total personnel, operating, and maintenance costs 2,573
Total costs 2,022 4,886
* Figures have been rounded.

Source: Key approval documents—unaudited figures

6.69 In 2007, National Defence and PWGSC learned that the cost of each helicopter would be significantly higher than originally estimated. At this stage, detailed negotiations with Boeing were still under way. National Defence next realized that infrastructure costs would also be significantly higher than initially assumed. This triggered a series of meetings and discussions about what helicopter requirements were essential and how to narrow the project scope in an attempt to stay within its original cost estimate.

6.70 With respect to life-cycle planning for this helicopter, given the aggressive schedule and intent to award a contract in 2007 and have delivery of the first helicopter by 2010, we expected to see equally aggressive planning with respect to all of the elements required to provide the Chinook capability. It was not until June 2009 that the Program Management Board directed the development of detailed plans for sourcing and training the personnel needed for the new squadron, infrastructure, and mitigating risks associated with the planned in-service support contract.

6.71 Details and costs concerning the provision of in-service support are still to be worked out and an amendment to the contract with Boeing is planned for 2013. Such provisions entail potentially significant financial and contracting risks. It is also planned that the in-service support provisions will be reviewed and amended at five-year intervals. According to National Defence and Boeing, this is intended to allow them to assess the costs on the basis of accumulated experience and data to the date of each review and to help reduce the risk.

6.72 In December 2006, the government decided to remove the provision for aircrew training from the scope of work planned with Boeing. Through a separate process and Treasury Board submission, a contract for this work was awarded to another firm. This is inconsistent with National Defence’s principle of providing a single point of accountability with the original equipment manufacturer and the approach taken with respect to the Cyclone helicopter. The estimated costs for this service, together with 20 years of in-service support, is $320 million (including GST).

6.73 National Defence’s full life-cycle planning for this project is still in progress and by the end of our audit had not been completed. Nor had the Department fully estimated all the life-cycle costs associated with this acquisition. Without such information, National Defence cannot adequately plan to have sufficient funds available for long-term operation and support of the helicopter. Moreover, without sufficient funds, National Defence may have to curtail planned training and operations.

6.74 Recommendation. National Defence should start estimating full life-cycle costs in the options analysis phase of its project management process and present these costs to decision makers at subsequent steps in the process as the estimates evolve. The basis of cost estimations should be included in approvals documents. National Defence should start full life-cycle planning for the preferred option in the definition phase of its project management process. Preparation of plans should be started at the time of preliminary project approval.

The Department’s response. Agreed. National Defence follows well-established processes for progressively developing and refining full life-cycle cost estimates beginning with indicative cost estimates at the identification of the requirements, followed by more substantive cost estimates confirmed through comprehensive analysis at the definition phase. National Defence continuously seeks to improve its cost estimates and, in response to this audit, will initiate a further review of the tools available and instructions in place to provide the best possible estimates to decision makers at each stage of the project approval process. Further, National Defence will ensure that relevant instructions and directives make clear the requirement to capture the basis for cost estimations in project files. Finally, while it is already a well-established requirement to initiate full life-cycle planning for the preferred option by the preliminary project approval stage, National Defence will review related instructions, tools, and compliance mechanisms to ensure that this is well-communicated and respected.

6.75 Recommendation. National Defence should undertake a review of lessons learned in the use of long-term in-service support contracts before amending the contract in 2013 with Boeing, in order to ensure that risks are appropriately identified and managed, costs are properly determined, and alternative service delivery options are considered.

The Department’s response. Agreed. National Defence will continue to collect the lessons learned in the use of long-term in-service support contracts prior to amending the contract in 2013 with Boeing. This would include reviewing the existing in-service support contracts for the airlift capability project—tactical (ACP-T), maritime helicopter project and medium- to heavy-lift helicopter project, discussing the lessons learned with project managers from the three environments (Air, Land, and Sea), validating in-service support cost models, and communicating the findings to the proper in-service support policy owners for policy amendments. The aim is to ensure that risks are appropriately identified and managed, costs are properly determined, and alternatives are considered.

6.76 Recommendation. Public Works and Government Services Canada should undertake a review of lessons learned in the use of long-term in-service support contracts before amending the contract in 2013 with Boeing, in order to ensure that risks are appropriately identified and managed, costs are properly determined, and alternative service delivery options are considered.

The Department’s response. Agreed. Public Works and Government Services Canada will review, in consultation with National Defence, the lessons learned in the use of long-term in-service support contracts before amending the contract in 2013 with Boeing, and will continue to have the risks appropriately identified in its approval documents. The Department will also request that National Defence confirm that costs are being properly determined and that alternative delivery options have been considered. The lessons learned will be completed by the end of March 2011.

These lessons learned, which will be incorporated into the amendment for in-service support for the medium- to heavy-lift helicopter project expected in 2013, will complement those already learned through the procurement of the maritime helicopter project. These include managing the procurement to produce industrial and regional benefits and reduce contract administration costs, and encouraging the manufacturer to introduce support considerations into the design process.

The advance contract award notice did not comply with the contracting regulations and policy

6.77 The Treasury Board Contracting Policy permits the contracting authority to enter into a directed contract if one of four well-defined factors exists, as set out as an exception in the Government Contracts Regulations. One of these factors, and the one used as the basis for this directed contract, is that only one supplier is capable of performing the contract. In such cases, the Contracting Policy requires that the use of this exception be fully justified. Further, it also encourages whenever possible to advertise the proposed award through an advance contract award notice (ACAN). This is intended to inform potential suppliers that a directed contract is about to be awarded and to provide an opportunity to challenge that award. We examined the basis for the chosen procurement strategy and whether it was properly approved and carried out.

6.78 The acquisition of the Canadianized Chinook helicopters was a directed procurement. It is evident from the files that National Defence concluded very early in this acquisition process that the Chinook helicopter was the only one capable of meeting its needs. As early as the fall of 2005, National Defence was considering a sole-source procurement with Boeing. As previously noted, National Defence met with The Boeing Company soon after the 2005 Budget announcement, and it held several other meetings and exchanged technical and costing information with Boeing through 2005 to mid-2006. Other helicopter manufacturers were not accorded similar treatment. In addition, National Defence and PWGSC met with Boeing to plan next steps during the ACAN posting period, raising the potential for a real or perceived bias and lack of fairness on the part of the departments. It was not until another company responded to the ACAN (see paragraph 6.82) that communications with Boeing were suspended.

6.79 The ACAN posted by PWGSC indicated the intent of National Defence to solicit a bid and negotiate a contract with The Boeing Company on the basis that “research indicates that the Boeing CH-47 Chinook is the only aircraft that meets the high level mandatory capability requirements.” This assessment was based on a market analysis prepared by National Defence. There was no justification for using the exception to the Government Contracts Regulations in the PWGSC files. Instead, PWGSC officials told us they used the market analysis prepared by National Defence as the justification. We found this justification to be incomplete: It included an assessment of selected helicopters against the four technical mandatory requirements but did not address the certification and delivery requirements. Nor did we find evidence that PWGSC evaluated Boeing’s ability to meet all of the mandatory requirements before issuing the ACAN in 2006. In our opinion, while PWGSC approved the use of the ACAN, it did so on the basis of an inadequate justification. In 2007, well after the ACAN had been posted, PWGSC requested from National Defence the required written “sole-source” justification for its files, but this was never provided.

6.80 We found the ACAN assertion that Boeing could meet the certification and delivery requirements unrealistic. The ACAN indicated the intent to award a contract to Boeing by July 2007 and to accept delivery of the first helicopter 36 months later; that is, in 2010. This assertion could only have been met by purchasing a basic in-production Chinook F model. But National Defence did not intend to purchase an existing aircraft. On the contrary, at the time the ACAN was issued, negotiations with Boeing were still under way and National Defence was still articulating its needs and technical specifications. We found no evidence that PWGSC evaluated Boeing’s ability to deliver the first Canadianized helicopter by 36 months from the anticipated contract award date. Later, a review by PWGSC of Boeing’s response to the request for proposal issued in March 2008 confirmed that Boeing was unable to meet some of the requirements. In our opinion, it is evident that National Defence and PWGSC were unprepared to solicit a bid and negotiate and award a contract within one year as anticipated in the ACAN posting.

6.81 Based on the Treasury Board Contracting Policy and the related Guide for Managers—Best Practices for Using Advance Contract Award Notices, to be considered as fair, open, and transparent, an ACAN must provide sufficient information to allow other suppliers to determine if they possess the capabilities required to satisfy the requirement and should provide a good overview of the work to be contracted. PWGSC’s Supply Manual requires that relevant information, such as a statement of work, technical requirements, and evaluation criteria, be included in a bid solicitation (which by definition includes an ACAN). In our opinion, this was not the case: The ACAN included only the seven high-level mandatory requirements from the statement of operational requirements but none of the rated ones. In addition, while the ACAN also called for Boeing to provide 20 years of in-service support, no details were provided about what this was expected to include.

6.82 Moreover, one company submitted a response to the ACAN and provided a statement of capabilities for its helicopter. Its submission was formally evaluated by PWGSC and National Defence against the seven high-level mandatory requirements and a series of related criteria developed for purposes of the evaluation. Criteria against which potential suppliers are to be assessed must be available at the time the ACAN is posted. However, in this case, the criteria were developed during the ACAN posting period in reaction to the company’s submission and, consequently, were not available for review by potential bidders when the ACAN was first posted, as is required by the Contracting Policy. The company’s submission was found to be non-compliant, in part on the basis that the documentation provided by the company did not clearly show how it satisfied the criteria. The ACAN itself did not indicate the type of documentation expected. There is no evidence that Boeing was evaluated in a similar manner using the same criteria and documentation requirements, as required by the Contracting Policy and Guide for Managers.

6.83 Overall, in our opinion, the manner in which PWGSC used the 2006 ACAN did not comply with the letter or intent of the applicable regulations and policies and, consequently, the contract award process was not fair, open, and transparent. In addition, we believe a second ACAN should have been issued in 2009 and should have included the final helicopter requirements and specifications, the revised delivery and certification schedule, an indication of willingness to pay one-time engineering development costs, and other significant changes made to the project scope. These represent a significant difference from the information included in the 2006 ACAN. While this may not have affected the outcome, this would have enhanced fairness, openness, and transparency.

6.84 PWGSC believes that the process was fair, open, and transparent because the ACAN gave potential suppliers the opportunity to challenge the assertion that only one supplier could do the work.

Management oversight and approval by senior boards were missing at key decision points in the project

6.85 In approving large capital acquisitions, National Defence is expected to follow its Project Approval Guide. We observed a lack of compliance with the Project Approval Guide and the Treasury Board’s Project Approval Policy, specifically noting the absence of timely meetings, challenge, and approvals by senior boards at key decision points in the process of acquiring the Canadianized Chinook helicopters.

6.86 At the identification phase of this acquisition, the capability deficiency document was reviewed by the Senior Review Board prior to its approval in July 2005. However, the Joint Capability Requirements Board, responsible for directing the department’s capability-based planning process and for approving strategic capital projects, did not review or approve it, as required by National Defence’s Project Approval Guide.

6.87 Internal approval documents required at the preliminary project and revised preliminary project decision points were not finalized or reviewed by the Senior Review Board or Program Management Board. Moreover, neither board met to challenge, discuss, and approve the information contained in the Treasury Board preliminary project approval submission before the Department submitted it. The Program Management Board met afterwards in March 2007 to approve the submission retroactively. The Senior Review Board did meet to discuss a draft revised preliminary project submission, but there is no record of decision indicating its approval. The Program Management Board did not meet to discuss nor did it approve this submission. These gaps in oversight are important because significant changes to the cost, scope, and schedule of the project were made during this period and decided without the senior boards’ oversight and approval. For the 2009 effective project approval, both the Senior Review Board and Program Management Board met to discuss the planned Treasury Board submission, but there is no final record of their decision or endorsement.

6.88 As the project moved through the definition phase between June 2006 and June 2009, the Senior Review Board did not meet to monitor the project’s progress. This is significant because National Defence committed to the Treasury Board in its preliminary project approval submission that this board would be convened annually to oversee the proper conduct of the project.

6.89 We also found no record of the Senior Review Board’s approval of other key documents such as the project charter, statement of operational requirements, or any project profile and risk assessment. The information contained in these documents later became part of the documents submitted for project approvals within National Defence and the Treasury Board submissions.

6.90 National Defence officials have informed us that while these boards and committees did not always meet as required, senior personnel were involved, as required, in all aspects of the management of this project. National Defence acknowledges that deviations from the Project Approval Guide and the decisions taken outside of the normal process were not well documented.

6.91 We observed that for both the maritime helicopter and medium- to heavy-lift helicopter projects, key requirements set out in the National Defence Project Approval Guide were not respected. The Guide is important because it provides a framework for managerial oversight and decision making and ensures that sound project management principles are applied and Treasury Board policies are followed.

6.92 Recommendation. National Defence should ensure that key boards fulfill their roles in a timely manner. National Defence should clarify the process and documentation required to deviate from the Project Approval Guide and the Treasury Board’s Project Approval Policy where it deems that special arrangements are necessary for complex and sensitive projects. All decisions taken during the course of an acquisition should be fully documented.

The Department’s response. Agreed. The Terms of Reference of each of the key governance boards in the Department are being refined and finalized. National Defence plans to have these terms of reference approved and in place by January 2011 with each board being chaired by the accountable decision maker (Assistant Deputy Minister or Deputy Minister Level).

As part of the planned update of the Project Approval Guide, the guidance will be adjusted to reflect the flexibility that is required for the approval of complex and sensitive projects. Specifically, in the event that it is not possible to formally convene a senior review board (such as the Senior Review Board, Program Management Board, or similar bodies), project files will be required to obtain the same authorization, approvals, and signatures even if they are pursued secretarially and/or by meeting individually with the applicable board members.

Challenge Function of the Treasury Board of Canada Secretariat

We could not reach a conclusion about analysis and challenge provided by the Secretariat

6.93 According to the Treasury Board’s document A Guide to Preparing Treasury Board Submissions, the Treasury Board of Canada Secretariat provides a challenge function to ensure that submissions have been thoroughly analyzed in light of considerations such as authorities, affordability, effectiveness, program delivery, protection of public funds, performance measurement, costs and funding, and risks.

6.94 We set out to examine the analysis and challenge carried out by the Secretariat when project and contract proposals were submitted for Treasury Board approval. We requested documentation from the Secretariat that would demonstrate the challenge it conducted in the two National Defence helicopter projects audited.

6.95 However, we were unable to conclude on this aspect of our audit. For six of the eight Treasury Board submissions requested, we received almost no documentation. For the other two, we received over 270 emails that were heavily redacted on the basis of Cabinet confidences. Moreover, even if we had received the emails unredacted, Secretariat officials informed us that much of their challenge function is undertaken verbally, and thus, there would have been no documented evidence.

6.96 Our access to Cabinet confidences created before 2006 is governed by a 1985 order-in-council that permits access only to certain types of Cabinet confidences. In 2006, a new order-in-council was intended to clarify our access to Cabinet confidences demonstrating, among other things, the challenge function carried out by the Secretariat after February 2006. Near the end of the audit work, the Office of the Auditor General of Canada (OAG), the Secretariat, and the Privy Council Office jointly developed new guidance to provide clarity on the OAG’s access to analysis and other confidential information. Despite the new agreement regarding access, too much time passed to allow new information to be considered in the course of this audit. We were therefore unable to conclude on the exercise of that challenge function before or after 2006.

Conclusion

6.97 Regarding project management, National Defence did not fully comply with the key provisions of the Treasury Board Project Management Policy, Project Approval Policy, and Policy on the Management of Major Crown Projects or with the Department’s Project Approval Guide in managing the acquisition of the two types of military helicopters selected for our audit. Specifically, we identified several gaps in the completeness of information presented to decision makers as well as approvals and oversight by senior boards at key decision points.

6.98 Regarding contract management, with respect to the maritime helicopter project, we conclude that Public Works and Government Services Canada (PWGSC) and National Defence generally complied with the provisions of the Government Contracts Regulations and the Treasury Board’s Contracting Policy. With respect to the medium- to heavy-lift helicopter project, we conclude that the manner in which PWGSC used the advance contract award notice did not comply with the letter or intent of the applicable regulations and policies and, consequently, the contract award process was not fair, open, and transparent.

6.99 We were unable to conclude whether the Treasury Board of Canada Secretariat exercised its challenge function when it received the project and contract proposals for the two helicopter projects audited.

About the Audit

All of the audit work in this chapter was conducted in accordance with the standards for assurance engagements set by The Canadian Institute of Chartered Accountants. While the Office adopts these standards as the minimum requirement for our audits, we also draw upon the standards and practices of other disciplines.

Objectives

The overall objective of the audit was to determine whether National Defence and Public Works and Government Services Canada (PWGSC) managed the acquisition of selected military helicopters in compliance with key provisions of government contracting and major capital project legislation, regulations, policies, and guidelines and to determine whether the Treasury Board of Canada Secretariat performed its related challenge function.

The audit objectives for the two lines of enquiry were as follows:

  • To determine whether National Defence managed the acquisition of selected military helicopters in compliance with key provisions of the Treasury Board Project Management Policy, Policy on the Management of Major Crown Projects, and Project Approval Policy; and
  • To determine whether PWGSC and National Defence awarded and administered contracts for the acquisition of selected military helicopters in compliance with key provisions of the Financial Administration Act, Government Contracts Regulations, and the Treasury Board Contracting Policy.

Scope and approach

Our audit focused on the acquisitions of the maritime helicopters and the medium- to heavy-lift helicopters, undertaken by National Defence for military operations.

The audit examined how National Defence managed the projects to ensure that the acquisitions met government policies related to project management and that the helicopters it was purchasing would address capability needs in a cost-efficient manner. Specifically, the audit examined whether National Defence has filled an identified capability gap with the new helicopters in meeting operational requirements. The audit also examined how the Department managed the project schedule, risks, and costs. We also examined whether adequate documentation existed to support key decisions, including whether total costs were adequately estimated and validated. Finally, we examined whether, at key project decision points, approval was made by the appropriate individual in compliance with authorities.

The audit also examined whether National Defence and Public Works and Government Services Canada (PWGSC) carried out the contracting for the acquisition and in-service support in accordance with contracting authorities. The audit assessed whether PWGSC awarded contracts in a fair, open, and transparent manner in accordance with contracting authorities.

The audit examined documents and correspondence contained in National Defence project files and PWGSC procurement files. PWGSC and National Defence officials involved in the selected projects were also interviewed.

It should be noted that our conclusions about management practices and actions apply only to those of public servants in the federal government. The rules and regulations we refer to apply to public servants; they do not apply to contractors. We did not audit the records of the private sector contractors. Consequently, our conclusions cannot and do not pertain to the contractors’ practices or to their performance.

The audit did not consider industrial and regional benefits of the projects and the associated role of Industry Canada in the procurement process.

In 2008, the government approved over $400 million to purchase and support six used Chinook D helicopters for use in Afghanistan from the United States Army through a foreign military sales contract. This acquisition was not examined in the course of this audit.

Criteria

To determine whether National Defence and Public Works and Government Services Canada managed the acquisition of the maritime and the medium- to heavy-lift helicopters in compliance with selected key provisions of applicable regulations and policies, we used the following criteria:
Criteria Sources

National Defence complies with key provisions of Treasury Board and departmental project management policies for the following phases of the military helicopter acquisition process:

  • problem identification
  • option analysis
  • definition
  • implementation

The Treasury Board of Canada Secretariat provides a challenge function to ensure that the submissions have been adequately analyzed and reviewed by its managers.

  • Project Management Policy, Treasury Board, 1994
  • Project Approval Policy, Treasury Board, 1994 and 2005
  • Policy on the Management of Major Crown Projects, Treasury Board, 1994
  • Project Approval Guide, National Defence, 1998
  • Procurement Administration Manual, National Defence, 2007

Public Works and Government Services Canada (PWGSC) awards contracts for the acquisition of selected military helicopters in compliance with key provisions of appropriate legislation, regulations, policies, and guidelines for the following phases of the contract award process:

  • procurement strategy
  • solicitation
  • evaluation/negotiation
  • contract approval

National Defence and PWGSC administer the selected military helicopter contracts in compliance with key provisions of appropriate legislation, regulations, policies, and guidelines.

The Treasury Board of Canada Secretariat provides a challenge function to ensure that the submissions have been adequately analyzed and reviewed by its managers.

  • Financial Administration Act, sections 32, 33, and 34
  • Government Contracts Regulations
  • Contracting Policy, Treasury Board, 2008
  • Supply Manual, Public Works and Government Services Canada, 1994 and 2005
  • Procurement Administration Manual, National Defence, 2007
  • Guide for Managers—Best Practices for using ACAN, Treasury Board, January 2004

Management reviewed and accepted the suitability of the criteria used in the audit.

Period covered by the audit

Although the maritime helicopter project began in 1995, the activities we audited occurred between 1999 and 31 March 2010. The audit of the medium- to heavy-lift helicopter project covered activities between January 2005 and 31 March 2010.

Audit work for this chapter was substantially completed on 30 April 2010.

Audit team

Assistant Auditor General: Jerome Berthelette
Principal: John Reed
Director: Joyce Ku

Sarah Crain
Marie-Ange Massunken
Kathryn Nelson
Jeff Stephenson
Hugues-Albert Sully

For information, please contact Communications at 613-995-3708 or 1-888-761-5953 (toll-free).

Appendix—List of recommendations

The following is a list of recommendations found in Chapter 6. The number in front of the recommendation indicates the paragraph where it appears in the chapter. The numbers in parentheses indicate the paragraphs where the topic is discussed.

Recommendation Response

6.64 National Defence should review and apply the lessons learned with these helicopter acquisitions to ensure that, for future major capital equipment acquisitions, the degree of modifications and/or development involved is fully reflected in approval documents—in the assessment of risk, project timelines, and costs—and that procurement strategies are tailored to the complexity of the equipment being acquired. (6.18–6.33 and 6.49–6.63)

National Defence continuously strives to capture the lessons learned in undertaking complex acquisitions and, in this context, will undertake a specific review of the projects encompassed by this report. In addition, a review of the associated policies, procedures, and processes will also be conducted and they will be revised as required.

6.65 Public Works and Government Services Canada should review and apply the lessons learned with these helicopter acquisitions to ensure that, for future major capital equipment acquisitions, the degree of modifications and/or development involved is fully reflected in approval documents and in the assessment of risk, and that procurement strategies and contracts are tailored to the complexity of the equipment being acquired. (6.18–6.33 and 6.49–6.63)

Agreed. Public Works and Government Services Canada agrees to review the lessons learned with these helicopter acquisitions to ensure that, for future major capital acquisitions, the degree of modifications and/or development involved is fully reflected in approval documents and the assessment of risk, and that procurement strategies and contracts be tailored to the complexity of the equipment being acquired. The lessons learned will be completed by the end of March 2011.

6.74 National Defence should start estimating full life-cycle costs in the options analysis phase of its project management process and present these costs to decision makers at subsequent steps in the process as the estimates evolve. The basis of cost estimations should be included in approvals documents. National Defence should start full life-cycle planning for the preferred option in the definition phase of its project management process. Preparation of plans should be started at the time of preliminary project approval. (6.35–6.42 and 6.66–6.73)

Agreed. National Defence follows well-established processes for progressively developing and refining full life-cycle cost estimates beginning with indicative cost estimates at the identification of the requirements, followed by more substantive cost estimates confirmed through comprehensive analysis at the definition phase. National Defence continuously seeks to improve its cost estimates and, in response to this audit, will initiate a further review of the tools available and instructions in place to provide the best possible estimates to decision makers at each stage of the project approval process. Further, National Defence will ensure that relevant instructions and directives make clear the requirement to capture the basis for cost estimations in project files. Finally, while it is already a well-established requirement to initiate full life-cycle planning for the preferred option by the preliminary project approval stage, National Defence will review related instructions, tools, and compliance mechanisms to ensure that this is well-communicated and respected.

6.75 National Defence should undertake a review of lessons learned in the use of long-term in-service support contracts before amending the contract in 2013 with Boeing, in order to ensure that risks are appropriately identified and managed, costs are properly determined, and alternative service delivery options are considered. (6.35–6.42 and 6.66–6.73)

Agreed. National Defence will continue to collect the lessons learned in the use of long-term in-service support contracts prior to amending the contract in 2013 with Boeing. This would include reviewing the existing in-service support contracts for the airlift capability project—tactical (ACP-T), maritime helicopter project and medium- to heavy-lift helicopter project, discussing the lessons learned with project managers from the three environments (Air, Land, and Sea), validating in-service support cost models, and communicating the findings to the proper in-service support policy owners for policy amendments. The aim is to ensure that risks are appropriately identified and managed, costs are properly determined, and alternatives are considered.

6.76 Public Works and Government Services Canada should undertake a review of lessons learned in the use of long-term in-service support contracts before amending the contract in 2013 with Boeing, in order to ensure that risks are appropriately identified and managed, costs are properly determined, and alternative service delivery options are considered. (6.35–6.42 and 6.66–6.73)

Public Works and Government Services Canada will review, in consultation with National Defence, the lessons learned in the use of long-term in-service support contracts before amending the contract in 2013 with Boeing, and will continue to have the risks appropriately identified in its approval documents. The Department will also request that National Defence confirm that costs are being properly determined and that alternative delivery options have been considered. The lessons learned will be completed by the end of March 2011.

These lessons learned, which will be incorporated into the amendment for in-service support for the medium- to heavy-lift helicopter project expected in 2013, will complement those already learned through the procurement of the maritime helicopter project. These include managing the procurement to produce industrial and regional benefits and reduce contract administration costs, and encouraging the manufacturer to introduce support considerations into the design process.

6.92 National Defence should ensure that key boards fulfill their roles in a timely manner. National Defence should clarify the process and documentation required to deviate from the Project Approval Guide and the Treasury Board’s Project Approval Policy where it deems that special arrangements are necessary for complex and sensitive projects. All decisions taken during the course of an acquisition should be fully documented. (6.43–6.46 and 6.85–6.91)

Agreed. The Terms of Reference of each of the key governance boards in the Department are being refined and finalized. National Defence plans to have these terms of reference approved and in place by January 2011 with each board being chaired by the accountable decision maker (Assistant Deputy Minister or Deputy Minister Level).

As part of the planned update of the Project Approval Guide, the guidance will be adjusted to reflect the flexibility that is required for the approval of complex and sensitive projects. Specifically, in the event that it is not possible to formally convene a senior review board (such as the Senior Review Board, Program Management Board, or similar bodies), project files will be required to obtain the same authorization, approvals, and signatures even if they are pursued secretarially and/or by meeting individually with the applicable board members.

 


Definitions:

In-service support—Activities required to sustain operation of the helicopter over its lifetime, including engineering, training, inspection, maintenance, and repair of equipment, and provision of spare parts. In-service support may be performed by Canadian Forces or contract personnel while deployed or at base. (Return)

Directed contract—A federal government contract awarded to a preselected contractor when the contracting authority can justify setting aside the requirement to solicit competitive bids. The contract must fall under one or more of the exceptions to competitive solicitation in the Government Contracts Regulations. Contracting authorities are strongly encouraged to provide public notification of these contracts through an advance contract award notice (ACAN). If this is done and if there are no valid challenges received to the ACAN, the directed contract is deemed to be competitive. (Return)

Total indicative costs—A rough cost projection used for budget planning purposes in the early stages of a project, based on the operational requirements, a market assessment of products, technological availability, and life-cycle costs. (Return)

Statement of operational requirements—A document stating characteristics that must be delivered for the project to satisfy the needed capability; contains the critical performance criteria necessary to evaluate technical options. (Return)

Airframe—The structural components of an aircraft, such as fuselage, empennage, wings, landing gear, and engine mounts, but excluding such items as electronics and other parts that may be replaced from time to time. (Return)

Mission systems—Suite of sophisticated electronic sensors integrated to support the operational role of the aircraft; may include surveillance and navigational control. (Return)

Project profile and risk assessment—A document that states the technical, cost, and schedule risks and outlines the management strategy to deal with them. (Return)

Airworthiness certification—A document issued by a government body certifying that the design of an aircraft meets the applicable standards for that product. A certificate of airworthiness is issued for an aircraft that conforms to its certified type design and is safe for flight in Canadian airspace. (Return)

Total substantive costs—A cost projection based on a sufficiently high degree of quality and reliability to support project approval for the specified deliverables and time frame of the project. (Return)

Advance contract award notice (ACAN)—A notice posted by departments and agencies for no less than 15 calendar days, indicating to the supplier community that it intends to award a good, service, or construction contract to a pre-identified contractor. If no other supplier submits, during the posting period, a statement of capabilities that meet the requirements set out in the ACAN, or if suppliers were not successful in demonstrating that their statement of capabilities meets the requirements set out in the ACAN, the contract may then be awarded. (Return)

 

PDF Versions

To access the Portable Document Format (PDF) version you must have a PDF reader installed. If you do not already have such a reader, there are numerous PDF readers available for free download or for purchase on the Internet: