2006 May Status Report of the Auditor General of Canada
Chapter 3—National Defence—NATO Flying Training in Canada
What we examined
We examined whether National Defence has made satisfactory progress in addressing the concerns we raised in 2002 about its management of both the NATO Flying Training in Canada program and the contract used to purchase the training. At that time, we reported that by December 2001 the Department had paid about $65 million for training that the program had been unable to provide, largely because of problems with the aircraft. This time we followed up to determine whether National Defence has properly calculated any further losses incurred since December 2001 through payments for training that did not take place and whether it has recovered all money owed. We also examined whether the Department has resolved the problems in its own management of the program, which had prevented it from taking full advantage of the training it had purchased.
Why it's important
This pilot training program involves a $3.4 billion, 20-year contract under which many payments are fixed. While demand for pilot training can fluctuate over 20 years, ongoing underuse of the available training could prove costly.
What we found
- Overall, National Defence has made satisfactory progress in resolving the problems we found in 2002, but it is still not using all the training it is paying for. It has negotiated a settlement with the contractor for the training missed up to December 2002, when the program began meeting contractual expectations. The contractor, National Defence, and Public Works and Government Services Canada have calculated that the value of the missed training totals about $89 million. The contractor and the Crown have agreed to a settlement proposal, which has been formally implemented in a contract amendment effective 19 December 2005. The agreement recognizes that the Crown has already received about $39.5 million in benefits. A further $20.5 million is held in trust for the Crown, but the timing for payment is yet to be determined. The amendment also provides National Defence with further opportunities to obtain additional training.
- National Defence is still unable to enrol enough pilots in the program to use all the training available. Since the contractor is now meeting its contractual obligations, the Department bears the cost of paying for any unused training capacity. We calculated that since December 2002, the Crown has paid about $39 million for flying training that it could not use. It appears that the Department has no recourse to recover these losses, which we believe are likely to continue for at least a few more years.
The departments have responded. Public Works and Government Services Canada and National Defence have agreed with our recommendations. Their detailed responses follow the recommendations throughout the chapter.
3.1 In May 1998, National Defence entered into a contract to purchase pilot training over 20 years as part of the NATO Flying Training in Canada (NFTC) program. In our 2002 Status Report, we reported on the implementation and management of this contract, noting concerns about management and the underuse of the program.
Key findings in 2002
3.2 Due to problems during the program's start-up, the amount of training that the contractor was able to provide was well below what the contract had stipulated. As a result, the program could not train the number of pilots National Defence said it needed to train. However, the Department was still making the fixed payments called for under the contract. By December 2001, the Department had paid about $65 million for training it missed, largely because of problems with the aircraft.
3.3 To make the program cost effective for both the Department and the contractor, National Defence had agreed to carry the program's cost with the option to sell some of its training positions to other countries. The Department marketed the program abroad and sold some of the available training positions. However, even with foreign sales, only 41 percent of the student positions National Defence had contracted for were filled by 2002. In response to our 2002 report, the Department stated it was confident that training capacity would be used more fully in the future.
Important changes since 2002
3.4 In September 2002, the House of Commons Standing Committee on Public Accounts required National Defence to present its plans for recovering the money the contractor owed. Initially, the Department responded that under the terms of the contract, the contractor did not have to provide financial compensation for training the Department did not receive. However, the Department did begin negotiating with the contractor to find ways to obtain value for the training missed. National Defence, Public Works and Government Services Canada, and the contractor have now agreed on the total amount of training missed. The contractor provided a settlement proposal, and Public Works and Government Services Canada and the contractor signed a contract amendment to implement the agreement effective 19 December 2005.
3.5 National Defence remains committed to the NFTC program and is still using it for basic flying and fighter pilot training. Other countries also use the program—mainly for training their fighter pilots, but in some cases, also for basic flying training. International participation remains critical to the program's success. Currently, six countries share the costs with National Defence, although the Canadian Forces remain the largest user.
Focus of the audit
3.6 This follow-up audit examined whether National Defence and Public Works and Government Services Canada have taken appropriate action to recover the full value for the payments the Department made for training missed. We examined how the amount of missed training was calculated and whether the settlement proposal provides the expected value.
3.7 We also looked at whether the contractor was now providing the number of training sorties paid for, under the terms of the contract, and whether National Defence was either using all those training sorties or selling the surplus to other countries. Where National Defence was not able to use training, we calculated the cost of this excess capacity and looked at whether the Department had plans for using it.
3.8 More information about the follow-up objective, scope, approach, and criteria is at the end of the chapter in About the Audit.
Observations and Recommendations
The cost of unused training
A proposal for recovering the value of missed training has been accepted
3.9 The original $2.8 billion NATO Flying Training in Canada (NFTC) contract included both fixed and variable fees. With the participation of more foreign nations, the cost of the contract increased to about $3.4 billion. The contract payment schedule stipulates the fixed fees, which in 2005 comprised about 76 percent of the total, and cover the contractor's infrastructure costs and assets such as the aircraft and simulators. The variable fees, which make up the remaining 24 percent, are based on the number of pilots trained and their use of aircraft and supplies.
3.10 The contract requires National Defence to make fixed payments that average about $120 million each year for 20 years. Prior to signing the contract, the Department determined how many pilots it needed to train each year and agreed to pay the fixed fees to secure that amount of training.
3.11 We found that, although National Defence had paid the fixed fees, the contractor was unable to meet its contractual obligations (about 81 training sorties per day) until December 2002. National Defence, Public Works and Government Services Canada, and the contractor have since determined that there was about $89 million of training missed from the start of the program until the time the contractor was able to meet its obligations. Missed training represents the training called for in the contract minus the amount National Defence actually used and the fixed costs paid, up to the end of December 2002. We examined how the three parties calculated the $89 million and found that they conducted due diligence in validating the value of training missed.
3.12 In May 2003, the contractor, Public Works and Government Services Canada, and National Defence began discussing a settlement. In February 2004, the contractor presented a written proposal to the Crown to recover the value of the missed training. In the proposal, the contractor stated it had already provided National Defence with about $27 million in training value and would provide about $12.5 million in ongoing flying instructor training and administrative services. The contractor recovered about $20.5 million by withholding payments to suppliers. This amount is being held in trust for the ultimate benefit of the Crown. The timing for when this amount will be made available is not yet decided.
3.13 The contractor also offered to provide additional basic training in 2020 and the option to sell more student positions in basic training to foreign militaries than was originally allowed in the contract. The value of this training is estimated by National Defence to be about $29 million (Exhibit 3.1).
3.14 In March 2004, National Defence asked Public Works and Government Services Canada to amend the contract to implement the February 2004 proposal from the contractor. In November 2004, National Defence and Public Works and Government Services Canada agreed that the contractor's proposal was a basis for negotiation and began reviewing the settlement.
3.15 At the time of this audit, the Crown and the contractor had agreed that the settlement proposal was acceptable and completed a contract amendment to implement the agreement on 19 December 2005. All parties now need to agree on a timeline for receiving compensation.
National Defence needs to sell additional flying training and exercise future training options in 2020 to recover losses
3.16 Currently, the contract allows National Defence to sell basic training positions to other countries up to a maximum of 11 foreign students per year. The settlement proposal would allow National Defence to increase these sales to 15 students per year. The Department must negotiate any sales itself. We found that National Defence has been less successful in obtaining foreign student commitments for basic training than for other training. Except for a handful of students trained on a special-trial basis, National Defence has not sold any of its basic training positions to other countries since 1999. The Department informed us that it has not yet done a study to examine the feasibility of selling additional training for foreign student pilots. We are concerned that the Department could find it difficult to generate extra revenue this way.
3.17 The December 2005 settlement also includes a provision for additional basic flying training for Canadian pilots after 2020. Therefore, this benefit will not be received for some time. National Defence will need to ensure that it has resolved its current underuse of the training available by 2020, and that it can enrol enough student pilots to use the training the contractor will make available. National Defence informed us that it has not yet examined the feasibility of this option because it is far in the future. However, it anticipates that it will need this training for pilots who will continue onto helicopter and multi-engine aircraft training as part of a different training contract that will still be in force at that time.
3.18 Recommendation. Public Works and Government Services Canada and National Defence should begin action to ensure that compensation is received in a timely manner.
The departments' response. Public Works and Government Services Canada and National Defence will follow up with the contractor and other relevant parties to ensure that funds due to Canada are paid promptly in accordance with the NATO Flying Training in Canada agreements. Other compensation, in the form of additional training, will be received in a timely manner based on Canada's training needs.
An additional $39 million was paid for unused pilot training
3.19 In response to our 2002 audit, National Defence had said it was confident that it would use the NATO Flying Training in Canada (NFTC) training capacity more fully as the program matured. The Department said the underuse was a reflection of the challenges inherent in starting up a new program of this complexity and magnitude. It explained that the problems caused by late deliveries of aircraft and equipment would not be repeated; thus training would not be underused. However, in 2005, we found that available pilot training was once again underused but for reasons unrelated to the NFTC start-up problems.
3.20 We found that, beginning in October 2002 the contractor had demonstrated that it could provide all the training National Defence required. It was able to maintain this performance level, and by the end of December 2002, National Defence and Public Works and Government Services Canada agreed that the contractor was now meeting its contractual obligations. Therefore, the Department's inability to utilize training it had paid for was no longer attributable to contractor problems.
3.21 Although from October 2002 onward, the program provided the number of training flights required, National Defence was, and continues to be, unable to take full advantage of this. Of about 654 student pilot training positions it paid for between the end of December 2002 and December 2005, the Department used only about 509 positions, or 78 percent. This is an improvement over the 41 percent use we found in 2002, but it still means that training positions are left empty. We estimate that the Department paid about $39 million for this unused training. Because this shortfall is a departmental responsibility, the $39 million cannot be considered as a claim against the contractor.
3.22 National Defence has two pilot training streams. All pilots take primary flying training in the Contracted Flying Training and Support program (CFTS) and basic flying training in the NFTC program. Fighter pilots, who eventually fly the CF-18 fighter aircraft, proceed onto basic, advanced, and lead-in fighter training in the NFTC program. All other pilots proceed onto advanced CFTS training. (Exhibit 3.2) Once they complete their flying training, pilots join operational training units.
3.23 The training program has remained underused because National Defence has not enrolled enough pilots into the NFTC program. In addition, however, we found that the ability of the operational training units to absorb pilots after their NFTC training is an ongoing problem—the units do not have enough room to take in all the pilots that the NFTC program would graduate if it was used at full capacity. Therefore, the Department has been limiting the number of pilots it enrolled in the program in 2003 and 2004. This trend has continued into 2005.
3.24 National Defence also decided in 2004 to lower the enrolment to the NFTC basic training level as it anticipated an interruption in the next stage of training for helicopter and multi-engine aircraft pilots. The existing contract for the helicopter and multi-engine flying training expired in August 2005, and a new CFTS contract was signed in October 2005. This new contract has a two-year transition period before training can be provided at full capacity. As a result, National Defence has made a decision to under-enrol pilots in the basic NFTC program until the next stage of training is at its capacity.
3.25 The interruption in helicopter and multi-engine aircraft flying training may create more of a backlog of Canadian Forces pilots who are waiting for training and increase their wait times. We found that, since 2002, National Defence had improved wait times and backlogs. Wait times had fallen from between 18 and 22 months to about 11 months, and the backlog of pilots waiting to start flying training had dropped from about 161 pilots to about 80. However, new delays may cause backlogs and wait times to increase again.
The Department may again have more future flying training than needed
3.26 The new CFTS flying training program for helicopter and multi-engine aircraft will also offer basic flying training similar to NFTC; this may compete with the already underused NFTC program. To prevent this situation from happening, National Defence informed us that it designed the new basic flight training to be less comprehensive than the NFTC program; it is aimed at foreign nations who do not need all the training provided by the NFTC program. National Defence plans to use the new CFTS basic flying training program for its pilots only if it has a demand for it.
3.27 Recommendation. National Defence should take immediate action to resolve pilot enrolment limitations to ensure it makes the best use of the NATO Flying Training in Canada contract and trains the number of pilots it needs. The Department should ensure it is not creating a new basic flying training program that provides added capacity it may not use.
National Defence's response. National Defence accepts these recommendations. The Department has already taken steps to resolve pilot enrolment limitations, and it will continue working to attract sufficient pilot recruits while managing the NATO Flying Training in Canada contract to produce the required number of trained pilots. National Defence remains confident that the new basic flying training option, being developed in a complementary contract, contains measures to reduce the financial impact of fluctuations in pilot enrollment and employs the lessons learned from the NFTC project that were recommended by the Auditor General.
The request for proposals for the new flying training contract addresses concerns raised in 2002
3.28 In 2002, we recommended that National Defence ensure new contracts similar to the NATO Flying Training in Canada (NFTC) program have payments tied to performance and value received. The Department said that any of these new service contracts would incorporate performance incentives and mechanisms to pay for additional value received from the contractor. We reviewed the request for proposal for the new helicopter and multi-engine aircraft flying training contract and found that payments will be based on milestones. If the contractor fails to achieve the milestones, this could result in payment holdbacks and forfeiture. Incentives are also in place for good performance.
3.29 We also recommended in 2002 that the services purchased through a long-term contract be aligned with the Department's ability to use them. The Department responded that future service contracts would incorporate strategies and structures to account for variations in demand. Unlike with the NFTC contract, we found that the request for proposal for the new flying training contract does not limit the Department's potential foreign sales of student positions; this could allow more flexibility to sell unneeded training to other countries. The new program also has mechanisms to adjust for short-term and permanent fluctuations in training demand. These mechanisms will help National Defence better align services purchased with training requirements (Exhibit 3.3).
3.30 National Defence has shown some progress in addressing the problems that we raised in 2002 concerning the NATO Flying Training in Canada program. The Department has also made progress in developing a plan to recover money owed to the Crown for training missed, which has recently resolved our main concern. Although the negotiations to reach a financial settlement took longer than the actual period in which the contractor did not meet the contract specifications, an agreement has now been reached. National Defence and Public Works and Government Services Canada need to now clarify a time frame that will ensure the Crown receives compensation in a timely manner.
3.31 The Department has not resolved pilot enrolment shortfalls and continues to incur significant losses as a result. Most of the losses are the result of under-utilization of basic flying training, which was part of the problem we found in 2002. The Department has not fully addressed problems with the pilot training process and should do so now.
3.32 We estimate that losses since the end of December 2002 amount to about $39 million and will continue to grow as the current training slowdown continues. In our 2002 audit, we raised concerns about the underuse of the NFTC contract, but National Defence assured us that the problems were temporary start-up problems and that training production would improve. In 2005, we found that training had slowed again, but for different reasons, and the financial impact has been significant. National Defence needs to take steps to make the best use of the services purchased through the NFTC contract to get the best value and use of the training it is paying for.
About the Audit
The objective of this follow-up audit was to determine how well National Defence has progressed in its management of the NATO Flying Training in Canada program since the 2002 audit. We examined whether the Department was getting back what is owed to the Crown and if the pilot training program is meeting requirements.
Scope, approach, and criteria
The audit focussed on the management framework and processes that National Defence and Public Works and Government Services Canada have in place for the NFTC program.
We assessed the NFTC contract management from the time of the last audit, April 2002 to March 2005. This represents a three-year period in the 20-year, $3.4 billion contract. While our audit focussed on this time period, certain observations had to be linked to the pre-contract planning and initial contract phases of the program.
The 2002 audit followed up on the recommendations for procurement reform made to the Treasury Board Secretariat and Public Works and Government Services Canada in the 1999 audit on alternative service delivery. This current audit did not include the Treasury Board Secretariat's and Public Works and Government Services' progress in procurement reform.
The audit scope did not include the activities of the contractor and, as a result, the follow-up audit does not include comments on them.
The audit approach included file reviews and interviews with PWGSC and National Defence. We also met the contractor.
Quantitative information in this chapter is based on data provided by the departments. These data should be treated as unaudited.
Audit work completed
Audit work for this chapter was substantially completed on 20 January 2006.
Assistant Auditor General: Hugh McRoberts
Principal: Wendy Loschiuk
For information, please contact Communications at (613) 995-3708 or
Sortie—A training flight. (Back)