2012 Spring Report of the Auditor General of Canada

Chapter 5—Oversight of Civil Aviation—Transport Canada

Main Points

Introduction

Canada’s civil aviation industry
Transport Canada’s oversight of civil aviation
Organizational structure
What we found in our 2008 audit
Current challenge
Focus of the audit

Observations and Recommendations

Regulatory framework

Transport Canada has implemented a regulatory framework that is consistent with international requirements
Transport Canada’s process for addressing some emerging safety issues is lengthy

Surveillance planning

Risk-based planning lacks rigour
Methodology for planning surveillance lacks clarity on the required minimum surveillance

Surveillance activities

Transport Canada has developed a standardized approach for consistent surveillance
Significantly fewer inspections are done than planned
Most inspections are not consistently conducted according to established methodology
Evidence of sufficient management involvement in surveillance activities is lacking

Human resources planning

Transport Canada has strengthened its human resources planning process
Progress has been made to implement human resources strategies, but a lot remains to be done

Quality assurance

The Civil Aviation Directorate still does not have an approved quality assurance plan

Conclusion

About the Audit

Appendix—List of recommendations

Exhibits:

5.1—Number of accidents between 2001 and 2010 involving Canadian-registered aircraft (travelling inside and outside Canada) and foreign air carriers in Canada

5.2—The sectors that make up Canada's civil aviation industry

5.3—Transport Canada is adopting a safety management systems (SMS) approach

5.4—Transport Canada's role has changed with the safety management systems (SMS) approach

5.5—Reporting structure for Transport Canada Civil Aviation Directorate

 

Performance audit reports

This report presents the results of a performance audit conducted by the Office of the Auditor General of Canada under the authority of the Auditor General Act.

A performance audit is an independent, objective, and systematic assessment of how well government is managing its activities, responsibilities, and resources. Audit topics are selected based on their significance. While the Office may comment on policy implementation in a performance audit, it does not comment on the merits of a policy.

Performance audits are planned, performed, and reported in accordance with professional auditing standards and Office policies. They are conducted by qualified auditors who

  • establish audit objectives and criteria for the assessment of performance;
  • gather the evidence necessary to assess performance against the criteria;
  • report both positive and negative findings;
  • conclude against the established audit objectives; and
  • make recommendations for improvement when there are significant differences between criteria and assessed performance.

Performance audits contribute to a public service that is ethical and effective and a government that is accountable to Parliament and Canadians.

Main Points

What we examined

Aviation companies that want to operate commercially in Canada must obtain an authorization. In 2011, there were more than 5,000 companies authorized to operate and more than 34,000 aircraft registered in Canada. To maintain their authorization to operate, these companies must meet the minimum safety standards required for the safe conduct of civil aviation.

The minimum safety standards for aviation companies fall under the Aeronautics Act and the Canadian Aviation Regulations. The Act and the regulations form the main part of the regulatory framework for civil aviation. The overall responsibility for the maintenance of safe, regular, and efficient civil aviation operations, including the manufacturing and maintenance of aircraft, rests with the aviation industry.

Transport Canada is responsible for developing and administering the policies, regulations, and standards required for the safe conduct of civil aviation within Canada’s borders. The Department is also responsible for overseeing whether aviation companies have complied with this safety framework, and for taking appropriate enforcement action where necessary. In 2009–10, Transport Canada spent over $148 million and dedicated about 1,400 employees to monitoring civil aviation across Canada.

Our audit examined whether the Department has managed the risks associated with overseeing its civil aviation safety program. We focused on Transport Canada’s surveillance of air carriers, aircraft maintenance organizations, and airports in the National Airports System.

Audit work for this chapter was substantially completed on 30 September 2011. Further details on the conduct of the audit are in About the Audit at the end of this chapter.

Why it’s important

Transport Canada plays a key role in helping to ensure that Canada’s civil aviation safety framework meets minimum international safety standards. While Canada compares favourably with many other countries in its aviation safety record, any deterioration would significantly impact public confidence. This makes it critical that Transport Canada maintain a robust and effective regulatory framework for civil aviation safety, especially since the International Civil Aviation Organization (ICAO) has projected a significant growth in aviation until 2025.

Identifying aviation companies that present safety risks is a highly complex process that relies heavily on the judgment and experience of Transport Canada’s inspectors across Canada and on the information that is made available to them.

The Department’s inspectors cannot be continuously present in all aviation companies to assess their compliance with aviation safety standards. Consequently, Transport Canada must use risk management techniques to decide where, when, how often, and in how much depth it inspects aviation companies in order to obtain sufficient assurance that they are complying with Canada’s aviation safety requirements. If Transport Canada does not manage these surveillance risks well, it is unlikely to focus its scarce resources on aviation companies and operations that represent the highest risks.

What we found

  • Transport Canada has developed a rigorous aviation safety regulatory framework that is consistent with standards established by the International Civil Aviation Organization, but it can take a long time to address emerging safety issues—in some cases, more than 10 years. The Department has yet to fully implement a process that will address these issues more quickly.
  • Since 2008, Transport Canada has made progress in evolving from the traditional surveillance approach—largely based on responding to regulatory requirements—to a systems-based approach designed for large and small aviation companies. This approach allows for more consistent and rigorous surveillance of aviation companies’ compliance with safety regulations.
  • While some aspects of the new surveillance program are working well, there are weaknesses in critical areas. For example, information for assessing the risk indicators that Transport Canada uses to identify the high-risk aviation companies that should be inspected is not always available or kept up to date. A minimum acceptable level of surveillance has not been clearly established to indicate how long aviation companies can operate without being inspected, and only two thirds of planned inspections have been carried out. Most inspections are not fully conducted according to established methodology and are subject to little management oversight. In addition, documentation of key decisions is weak. Finally, Transport Canada lacks a quality assurance program to continuously improve its surveillance program.
  • Transport Canada recently developed a national human resources plan for the oversight of civil aviation to help ensure that it has the resources it needs to carry out its safety regulatory program. However, the plan does not specify the number of inspectors and engineers that are needed, although the Department agreed to provide these figures in its response to our 2008 recommendation. The Department has made progress with implementing key human resources strategies, but efforts to fully implement the new surveillance approach have been hampered by the lengthy reorganization and by resistance from some inspectors.

The Department has responded. The Department agrees with all of our recommendations. Its detailed responses follow each recommendation throughout the chapter.

Introduction

Canada’s civil aviation industry

5.1 In 2010, more than 75 million passengers flew within Canada’s borders on nearly 3 million air flights. Statistics from the International Air Transport Association show that North America has one of the safest aviation industries in the world.

Facts about aviation in Canada

  • Canada’s civil aviation industry employs more than 90,000 people.
  • Canada has the second largest civil aviation aircraft fleet in the world and the second largest population of licensed pilots.

5.2 In 2009 and 2010, the total number of accidents was the lowest recorded in a 10-year span in Canada (Exhibit 5.1). Large air carriers were involved in very few accidents between 2001 and 2010, and none of these accidents resulted in fatalities. These air carriers represent more than 95 percent of revenue-generating passenger miles travelled in Canada.

Exhibit 5.1—Number of accidents between 2001 and 2010 involving Canadian-registered aircraft (travelling inside and outside Canada) and foreign air carriers in Canada

Type of air carrier 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
Large Canadian air carriers 5 6 7 3 4 7 4 6 2 6
Small Canadian air carriers 93 100 95 80 89 97 91 88 70 79
Foreign air carriers 4 2 4 4 2 1 4 2 2 1
Total 102 108 106 87 95 105 99 96 74 86
Source: Transportation Safety Board of Canada—Statistical Summary, Aviation Occurrences 2010

5.3 Canada’s civil aviation industry consists of six major sectors (Exhibit 5.2):

  • airports and aerodromes,
  • aeronautical product design and manufacturing,
  • aircraft maintenance,
  • air operations,
  • air navigation services, and
  • aviation personnel.

Exhibit 5.2—The sectors that make up Canada's civil aviation industry

Sector Components
Airports and aerodromes 26 airports in the National Airports System, 544 certified aerodromes, and 1,775 uncertified aerodromes including water airports and heliports
Aeronautical product design and manufacturing 134 manufacturers

1,025 new and modified aeronautical products each year
Aircraft maintenance 1,852 approved maintenance organizations
Air operations 2,260 air carriers

34,833 registered aircraft

383 flight training units
Air navigation services One service provider, 43 air traffic control towers, and other facilities
Aviation personnel 71,137 licences (pilots, flight engineers, and air traffic controllers)

24,000 industry delegates (to certify personnel or products on behalf of Transport Canada)

5.4 The aviation industry itself has the primary responsibility for maintaining safe, regular, and efficient air travel. Aviation companies that want to operate commercially in Canada must obtain an authorization to operate. In 2011, more than 5,000 companies were authorized to operate in Canada, and more than 34,000 aircraft were registered in Canada.

5.5 According to the International Civil Aviation Organization (ICAO), rapid expansion in the aviation industry is making it increasingly difficult to sustain a traditional approach to safety management, which primarily consists of responding to regulatory requirements. In early 2000, ICAO recommended that countries adopt a safety management systems (SMS) approach (Exhibit 5.3):

  • Background. The concept of safety management systems originated in the early 1980s in the chemical industry. The concept emphasized the need to look at an overall process or system, including the combination of human, organizational, technical, and environmental factors, rather than individual occurrences. The goal was for organizations to move from a reactive to a proactive approach by identifying hazards, analyzing associated risks, and taking appropriate measures before damage could occur. Over the years, the concept spread to other industries, including transportation.
  • Evolution. Large civil aviation companies in Canada have put in place safety management systems, and large airports are transitioning to SMS. Each company must name an accountable executive to provide leadership and foster a safety culture, and then develop safety policies, procedures, training, and ways of managing quality assurance.

Exhibit 5.3—Transport Canada is adopting a safety management systems (SMS) approach

Role Traditional approach SMS approach
Inspector Inspectors are auditors of regulatory compliance. Inspectors are system evaluators. As necessary, they may conduct traditional audits.
Company The company responds to regulatory requirements. The company proactively manages risks.

5.6 Canada was the first country in the world to regulate the implementation of SMS in the aviation industry. Since 2008, Transport Canada has required air operators whose aircraft carry 20 passengers or more and their maintenance organizations (referred to as large civil aviation companies) to use SMS in managing their safety risks. For companies developing and implementing SMS, Transport Canada has provided information and help in interpreting the regulatory requirements.

Transport Canada’s oversight of civil aviation

5.7 Transport Canada is responsible for the oversight of the civil aviation industry on behalf of the Government of Canada. Key elements of that responsibility include determining an acceptable level of aviation safety, and establishing a regulatory framework that sets out what the industry must do to achieve and maintain that level of safety. Transport Canada delivers certification services that authorize aviation companies and personnel to operate based on the requirements set out in the regulatory framework. Transport Canada also conducts surveillance activities of both aviation companies and their aviation personnel to assess whether they comply with the regulatory framework. This is done through inspections.

5.8 Transport Canada has about 1,400 employees working in civil aviation. This represents one quarter of the Department’s workforce. Human resources are located in 10 offices across the country and at headquarters in Ottawa. In 2010–11, Transport Canada spent approximately $148 million on civil aviation.

5.9 The Department’s mandate for civil aviation derives from the Aeronautics Act and the Canadian Aviation Regulations. The Act and the regulations form the main pillars of the regulatory framework for civil aviation. Under the Aeronautics Act, the Minister has broad powers to inspect, audit, and enforce regulations related to any aircraft or airport, or any premises used for the design, manufacture, distribution, maintenance, or installation of aeronautical products.

5.10 In 2005, when the Canadian Aviation Regulations introduced SMS, Transport Canada began to revise its surveillance activities to focus on measuring an organization’s ability to hold itself accountable for its own compliance. Transport Canada’s role became one of determining if a company’s safety systems complied with the regulatory framework (Exhibit 5.4). The Department also looked at how effective the company’s safety systems were. As necessary, it could conduct traditional audits.

Exhibit 5.4—Transport Canada's role has changed with the safety management systems (SMS) approach

Traditional approach SMS approach
Transport Canada inspects aircraft, records, and personnel directly. Transport Canada assesses whether aviation companies have effective processes in place to ensure safety and verifies the company's compliance with the regulatory framework. The surveillance within the SMS approach may involve traditional audits, as necessary.
Organizational structure

5.11 Transport Canada carries out its responsibilities for civil aviation through the Civil Aviation Directorate at headquarters in Ottawa and through Civil Aviation branches in five regions. The Civil Aviation Directorate is part of Transport Canada’s Safety and Security Group. The Group’s responsibilities include the development of legislation, regulations, and national standards to promote safety and security in the areas of air, marine, rail, and motor vehicle transportation. Additional responsibilities in these areas include monitoring, testing, inspecting, and providing limited contribution programs.

5.12 Headquarters is responsible for developing the content, policy, and standards for oversight activities. Regional Civil Aviation branches, along with the National Aircraft Certification Branch, the National Operations Branch, and the International Operations Branch at headquarters, carry out the oversight activities (Exhibit 5.5).

Exhibit 5.5—Reporting structure for Transport Canada Civil Aviation Directorate

What we found in our 2008 audit

5.13 In our 2008 May Report, Chapter 3, Oversight of Air Transportation Safety—Transport Canada, we examined how the Department had managed the implementation of SMS for large air carriers and associated aircraft maintenance organizations, which were the first industry groups required to implement SMS. We found that Transport Canada’s management of the transition to the new approach had several weaknesses, including the following:

  • In planning for the transition, the Department did not document risks, such as the impact of the transition process on oversight of air transportation safety, and did not identify actions to mitigate these risks.
  • The Department did not measure the impact of shifting resources to SMS activities on the frequency of traditional oversight activities, which continued for smaller, non-SMS companies.
  • The Department had not yet identified how many inspectors and engineers it needed, with what competencies, during and after the transition.
Current challenge

5.14 The ICAO has forecasted that air traffic volume will likely increase in North America by about four percent each year. This increase may more than double the current volume of air traffic in North America by 2025—the end of the forecast period. If nothing else changes, this increase in volume could lead to more accidents. The Department recognizes that it will have to do more just to keep the accident rate per revenue-generating passenger mile travelled in Canada at current levels. At the same time, Transport Canada believes that its current funding is not likely to rise significantly during that period. Consequently, maintaining the existing civil aviation accident rate in Canada will require that the Department effectively manage the risks associated with administering its air safety regulatory framework and oversight activities.

Focus of the audit

5.15 The focus of our audit was to determine whether Transport Canada has adequately managed the risks associated with overseeing its civil aviation safety program. Risks are inherent in air transportation. Eliminating accidents completely is unrealistic, and the Department will never have enough resources to ensure that every aviation company in the industry complies with all aspects of the safety regulations at all times. Therefore, it must use risk management techniques to choose what to inspect, when, and how often. It must also rely on properly trained staff to use the methodology and tools the Department provides them, as well as their knowledge, experience, and training, to identify what to inspect and to assess whether aviation companies are complying with the regulatory framework.

5.16 Our audit focused on four aspects of Transport Canada’s oversight program:

  • First, we looked at whether Canada has a robust regulatory framework and whether it is responsive to emerging safety risks.
  • Second, we looked at Transport Canada’s planning of surveillance activities and whether they are being conducted according to methods in place for that purpose. We examined Transport Canada’s surveillance activities of SMS companies, airports in the National Airports System (referred to as large airports), and small commercial air carriers and maintenance organizations. However, we did not examine Transport Canada’s surveillance of other sectors within the civil aviation industry, such as air navigation services, flight training units, general/private aviation, or personnel licensing.
  • Third, we looked at whether Transport Canada has an appropriate human resources planning process to help ensure that it has the right number of human resources with the right competencies it will need to deliver aviation services and surveillance activities.
  • Finally, we looked at whether Transport Canada has a sound plan to implement a quality assurance program.

5.17 Our audit was not designed to conclude on whether air travel is safe. That is Transport Canada’s responsibility. We also did not examine security issues, the efficiency of surveillance program activities, the Department’s activities related to certification and licensing, or educational activities that can contribute to improved air safety. More details about the audit objectives, scope, approach, and criteria are in About the Audit at the end of this chapter.

Observations and Recommendations

Regulatory framework

5.18 Transport Canada is responsible for establishing a regulatory framework to oversee civil aviation safety in Canada. The International Civil Aviation Organization (ICAO) has developed standards and recommended practices (SARPs) to help Canada and other member states meet their international obligations related to aviation safety oversight. The SARPs set minimum safety requirements for a consistent application throughout the world. In addition, Canada enters into bilateral agreements with its principal trading partners to harmonize safety standards between itself and these countries.

Transport Canada has implemented a regulatory framework that is consistent with international requirements

5.19 We looked at the regulatory framework for civil aviation safety that was in place in 2011 to determine whether Transport Canada had implemented a suitable framework to meet its international obligations and to address emerging high safety risks. We also examined whether Transport Canada had kept abreast of new amendments to ICAO’s international SARPs. We focused our work on what Transport Canada did to respond to amendments to international SARPs since 2005.

5.20 We found that Transport Canada has a comprehensive regulatory framework that is consistent with ICAO requirements. In 2005, ICAO audited Canada’s civil aviation regulatory framework and similarly concluded that it meets Canada’s international obligations. The framework consists of three key components:

  • The primary law in force is the Aeronautics Act, which establishes the Department’s authority to control civil aviation activities in Canada.
  • The Canadian Aviation Regulations set minimum legal requirements, such as those related to the safe operation of air carriers, aircraft, airports, and maintenance organizations.
  • Transport Canada’s standards set out criteria and conditions to provide the industry with a means of compliance and to help the industry understand how to fulfill Transport Canada’s expectations for meeting a regulation. The Department also develops and uses other safety measures, such as educational activities.

5.21 We also found that Transport Canada has put in place an adequate process to keep abreast of changes to international requirements. For example, it receives information regularly from ICAO about upcoming changes to SARPs and participates in several ICAO committees. ICAO senior officials said that Canada is a key participant in discussions about any proposed changes to international requirements.

5.22 ICAO accepts differences related to the international SARPs when member states have determined that they are not able to implement them, have adopted another solution, or need more time to put the new requirements in place. When Canada does not implement some SARPS, it must inform ICAO of the differences. Documenting the rationale and sharing these differences with ICAO and with the aviation industry are important because this allows the industry to make informed business and safety decisions. Information on these differences is also helpful when Transport Canada negotiates bilateral agreements to harmonize safety standards with its main trading partners.

5.23 We found that Transport Canada systematically compares Canada’s regulatory framework with amendments to the SARPs. In 2010–11, Transport Canada analyzed 11 ICAO amendments to determine the potential impacts on Canada’s regulatory framework and filed 63 differences. In all, Transport Canada has filed 178 differences to ICAO standards in the last five years. However, we found that Transport Canada has not effectively kept track of the rationale for these differences. In the absence of complete records, it is difficult for Transport Canada to ensure that these differences will be considered in the regulatory process when the Department develops new regulations.

Transport Canada’s process for addressing some emerging safety issues is lengthy

5.24 ICAO requires member states to have a process for the timely correction of safety issues. New and emerging risks can come from a variety of sources. For example, the Transportation Safety Board investigates accidents, reports publicly, issues safety recommendations, and raises high safety concerns. Nav Canada reports accidents and incidents, which are the basis of entries in the Civil Aviation Daily Occurrence Reporting System. As well, Transport Canada has a Canadian Aviation Issues Reporting System that allows the public and Transport Canada’s employees to report safety concerns. We examined whether Transport Canada had a process in place to identify, analyze, and prioritize safety issues and emerging high risks, and whether it was addressing such risks in a timely manner.

5.25 We compared emerging safety issues identified by key stakeholders with issues compiled by Transport Canada to assess whether the Department has an effective process for capturing this information. We found that Transport Canada’s process effectively identified safety issues through consultations with the civil aviation industry and stakeholders, and through participation by Transport Canada officials in many national aviation industry groups.

5.26 Transport Canada has the flexibility to determine whether safety issues will be best addressed using regulations or other safety measures, such as education, to promote aviation safety. According to the Department, the regulatory process is often not the most appropriate or cost-effective way to address or solve some emerging safety issues. We found that Transport Canada did preliminary analyses of issues and determined possible ways to resolve them. In most cases, the Department has analyzed practices in other countries. We also found that the Department has used tools such as videos, advisory circulars, and other means to alert and educate industry and the public about these safety issues and how to mitigate them.

5.27 We found that Transport Canada has recently enhanced the criteria it uses to prioritize safety issues. These criteria include factors such as the potential impact on international harmonization and key findings from the Transportation Safety Board. Transport Canada then developed a work plan to address high-priority issues. However, we found that the Department has not shared its priorities with the aviation industry and other key stakeholders, and that the time between identification of some safety issues and the date when they were finally addressed was long—in some cases, more than 10 years.

5.28 We selected and examined four safety issues raised in past years by stakeholders: the state of Canadian runways (1999); pilot fatigue (2001); the testing of aircraft wiring and flammability (1999); and aircraft collision with land or water (1995). We recognize that many steps are required in addressing safety issues, such as consultations, analysis of complex regulatory options and their impact on industry and international harmonization, and the legislative process itself. However, addressing safety issues on a timely basis is critical. Transport Canada has not analyzed its process to determine how issues might be addressed more quickly. At times, key stakeholders have expressed concern that issues they brought to the Department’s attention are not addressed quickly enough. Transport Canada’s lengthy process to determine what measure is required to address safety issues may cause uncertainties in how the industry can respond to those issues.

5.29 Recommendation. Transport Canada should analyze its process for addressing safety issues raised by stakeholders to ensure that it acts in a timely way to address significant safety issues.

The Department’s response. Agreed. By March 2013, the Department will have identified ways to accelerate follow-up on significant safety issues raised by stakeholders. The new process will continue to allow for consultation and regulatory changes.

Surveillance planning

5.30 The overall responsibility for aviation safety rests with the aviation industry. Even so, the International Civil Aviation Organization (ICAO) requires civil aviation authorities, such as Transport Canada, to adopt a systematic approach to verifying that aviation companies have effective systems for managing the safety of their operations. ICAO states that aviation companies should be subject to surveillance once every 12 months.

5.31 Given the size of the aviation industry in Canada, yearly inspections would be impractical. Consequently, Transport Canada must rely on an effective, risk-based surveillance regime to get the assurance that the industry is complying with Canada’s civil aviation regulatory framework. We focused our examination of Transport Canada’s surveillance activities on its risk-based surveillance plans for 2010–11 and on the methodology it used for surveillance planning.

Risk-based planning lacks rigour

5.32 We looked at the information Transport Canada used in developing its 2010–11 surveillance plans. Our goal was to determine whether the Department had assessed the risk that aviation companies do not comply with the civil aviation safety regulatory framework, with a focus on high risks.

5.33 Providing risk assessment guidance. We found that Transport Canada’s surveillance methodology requires inspectors and management to consider risks in determining how often surveillance will occur. In 2009, Transport Canada developed standard risk indicators based on an analysis of risk indicators used by other international transportation authorities. These risk indicators help staff to identify aviation companies that may represent a higher risk for non-compliance with the regulatory safety framework. The standard risk indicators are comprehensive, covering important risks such as changes in key personnel, changes in operations, financial and labour difficulties, management practices, safety record, and other risk factors. However, Transport Canada’s methodology does not specify what information is to be used by inspectors in assessing a company against the standard risk indicators. For example, it is not clear what type of financial information should be used to assess the risk of a company being in “financial difficulty.”

5.34 Assessing risk. In our discussions with management and inspectors, we found that they considered risk indicators when assessing each company, but the way they did so varied significantly among industry sectors, aviation companies, and regions.

  • For example, 75 large air carriers and maintenance organizations operate in Canada, including five very large companies that account for about 60 percent of revenue-generating passenger miles travelled. Transport Canada has assessed the risk indicators for the five very large air carriers and associated maintenance organizations. For the other 70 large air carriers and maintenance organizations, and about 4,000 small air carriers and maintenance organizations, the Department did not often use the standard risk indicators.
  • Except for the assessment of risks for large airports in one region, there were no formal assessments for airports using the standard risk indicators when Transport Canada planned its 2010–11 surveillance activities.

Using the same indicators is essential to ensure that risk assessments across the country are consistent and that Transport Canada is targeting the right aviation companies for inspections.

5.35 Compiling risk information. Using up-to-date risk information in developing risk profiles is essential to targeting higher-risk companies. We found that Transport Canada collects up-to-date risk information and prepares risk profiles on the five very large air carriers and associated maintenance organizations in planning for inspections.

5.36 However, for most large air carriers and maintenance organizations, for large airports, and for small air carriers and maintenance organizations, Transport Canada is missing key risk information and has no formal process in place to collect that data. Transport Canada collects a great deal of information, including information from previous inspections, companies’ operating manuals, and incidents reported in the Civil Aviation Daily Occurrence Reporting System, as well as other information on companies that transport or handle dangerous goods. However, that information is stored in different databases and is difficult to access when regional civil aviation branches are assessing the risk indicators of these companies. In addition, Transport Canada has little or no information on some important risk indicators. For example, information on the financial situation of aviation companies is often not collected. As a result, decisions on risk-based planning are often based solely on the experience of the few inspectors involved with those entities in the past. With recent organizational changes to adapt to the safety management systems (SMS) approach and turnovers of inspectors, it is even more important to ensure that the risk information used in making planning decisions is complete and reliable. The problem is particularly acute with aviation companies and large airports that were not inspected in the previous year. Without complete and reliable risk profiles to conduct risk assessments, the Department may not inspect the aviation companies that present the highest risks to aviation safety.

5.37 Recommendation. Transport Canada should clarify what information on industry and aviation companies should be used in making risk-based decisions, collect that information, assess its completeness and reliability, and develop risk profiles when preparing annual surveillance plans in the regions.

The Department’s response. Agreed. Transport Canada is committed to maintaining Canada’s very strong civil aviation safety record. It has developed and begun implementation of a comprehensive action plan to guide the modernization of its structure and improvements to its systems, processes, and tools, and to ensure that resources are targeted to areas of highest risk. The Assistant Deputy Minister of Safety and Security is accountable for the implementation of this plan, and its progress is being monitored by a senior management steering committee chaired by the Associate Deputy Minister. Successful and complete implementation of the plan is a priority for the Department.

By August 2012, the Department will complete and launch a standard risk assessment tool. The new tool is designed to collect information necessary to assess aviation companies and ensure consistency across all aviation enterprises, with regularly updated risk profiles to support a dynamic process of surveillance. These steps will improve the information base for the development of annual surveillance plans and overall surveillance activities.

Methodology for planning surveillance lacks clarity on the required minimum surveillance

5.38 For planning surveillance activities, Transport Canada must determine a minimum level of surveillance and manage its surveillance risks to an acceptable level. Setting clear minimum surveillance requirements for aviation companies is important if Transport Canada is to be sure that aviation companies are conducting safe operations, and if the Department wants to demonstrate that it is managing its surveillance risks. The safe conduct of operations is largely assured when companies comply with the terms of their authorization to operate. We looked at Transport Canada’s 2010–11 surveillance plans and interviewed those involved in their preparation to determine whether the Department had respected its own planning guidance on how often individual aviation companies are inspected.

5.39 We found that Transport Canada has developed national planning frequencies that require aviation companies to be inspected annually. This requirement is consistent with ICAO standards. According to the Department’s own planning methodology, inspectors are also required to consider risk information and availability of resources when developing their annual surveillance plans.

5.40 We also found that although the national planning frequencies state that aviation companies must be inspected every year, about 70 percent of aviation companies across the country were not inspected in 2010–11. Transport Canada’s planning frequencies also state that the decision to extend the frequency of surveillance beyond one year should be supported by a risk assessment. We found that management did not analyze and document its rationale for postponing the planned annual surveillance. This practice is important to ensure that high-risk companies are inspected each year and that the correct level of management is aware of the decision to postpone the surveillance of high-risk companies and agrees with it. We also found that the planning methodology does not provide guidance on how long a company is permitted to operate without being inspected.

5.41 Recommendation. Transport Canada should clarify the minimum level of surveillance needed to provide the necessary coverage of civil aviation companies.

The Department’s response. Agreed. By August 2012, Transport Canada will update and communicate to all inspectors and managers minimum surveillance levels for civil aviation companies.

Surveillance activities

Transport Canada has developed a standardized approach for consistent surveillance

5.42 The International Civil Aviation Organization (ICAO) requires member states to provide guidance and tools to staff who perform oversight activities. In our 2008 report on Transport Canada’s oversight of air transportation safety, we observed that Transport Canada had not explained how the day-to-day work of inspectors would change as activities related to safety management systems (SMS) were integrated with traditional oversight activities. Since 2008, Transport Canada has developed a new methodology for carrying out its surveillance activities. The methodology applies to all aviation companies authorized to operate and was designed to be flexible enough to apply to SMS companies as well as the other aviation companies. We looked at Transport Canada’s surveillance methodology to determine whether it was adequate.

5.43 We found that Transport Canada has developed a standardized surveillance methodology designed to promote consistent and rigorous inspections across Canada, and to make it possible for Transport Canada to determine a company’s compliance with regulations. The methodology provides detailed staff instructions for routine surveillance (surveillance staff instructions SUR-001), as well as guidance and audit tools to carry out the surveillance.

5.44 We examined the instructions to staff for conducting routine surveillance activities. We found that the instructions clearly define the Department’s expectations, set out the roles and responsibilities of key players involved in routine surveillance activities, and include key elements of good surveillance methodology for

  • preparing and completing inspection plans,
  • selecting sampling parameters,
  • identifying procedures on how to conduct the inspection, and
  • drafting reports and communicating inspection results to the aviation companies inspected.

5.45 However, we found that the instructions lack the necessary tools to document the results of inspection work. As well, the instructions provide little guidance on the minimum documentation needed to support the key judgments made by inspectors in preparing, conducting, and reporting inspection findings. Finally, the instructions lack guidance about how management should be involved in reviewing those decisions and how their involvement should be documented. Having sufficient and appropriate evidence to support inspection findings and having documented management reviews would help to demonstrate that inspectors are following the methodology and that the desired level of assurance is being achieved.

5.46 Recommendation. Transport Canada should review its staff instructions to clarify its expectations on the minimum documentation required and the level of management involvement expected in the oversight of surveillance activities.

The Department’s response. Agreed. By December 2012, the Department will have completed an update of, and communicated to managers and staff clear direction on, minimum requirements for surveillance documentation and management involvement.

Significantly fewer inspections are done than planned

5.47 Transport Canada’s national planning frequencies require that the higher-risk companies be included in annual surveillance plans. We looked at the actual surveillance activities conducted in 2010–11 to determine whether Transport Canada had completed its inspections as planned. We found that only 67 percent of air carriers, maintenance organizations, and large airports were inspected according to annual surveillance plans.

5.48 This result is significant because only the higher-risk companies are to be selected for inspection and included in the annual surveillance plans. If these plans constitute the minimum level of inspection that management believes is necessary to provide assurance that companies are complying with the safety regulations, then Transport Canada cannot have the assurance it says it needs. Even if more than the minimum number of inspections is planned, management still needs to assess whether, on the basis of the inspections done, Transport Canada has obtained an appropriate level of assurance.

5.49 We were told that the main reasons for cancelling or postponing inspections were to reallocate resources to other higher-risk inspections, or because planned resources were not available. However, we observed that the rationale for moving resources from planned inspections to other work or to other inspections was not documented and was not approved by senior management. We also questioned why planned resources were not available in 2010–11. We found that Transport Canada had overestimated the time inspectors had available to conduct surveillance activities. We also found that the Department did not have accurate records on the time inspectors spent on various oversight activities. In the absence of such information, it is difficult for the Department to develop realistic surveillance plans for achieving a minimum level of surveillance activities.

5.50 Recommendation. Transport Canada should determine why a number of inspections are falling short of plans and take corrective action. Deviations from original plans should be documented, and they should be approved by senior management. Senior management should also determine whether Transport Canada is still obtaining the desired level of assurance that companies are complying with aviation safety regulations.

The Department’s response. Agreed. Minimum surveillance levels will be reflected in surveillance plans from 2012–13 forward. Performance will be monitored on a regular basis, any deviations from approved levels will require respective Regional Director General and Director General Civil Aviation approval nationally, and senior management will be provided with regular updates.

Most inspections are not consistently conducted according to established methodology

5.51 Transport Canada inspectors are responsible for inspecting aviation companies against Canada’s regulatory framework. This activity requires a good understanding of the methodology they must use to prepare for, conduct, and report on surveillance activities. The surveillance process relies on the judgment of Transport Canada’s inspectors and on their training and experience to analyze and validate the safety systems and practices of aviation companies.

5.52 We randomly selected and reviewed 74 inspection files completed in 2010–11. They included inspections of large and small commercial air carriers and maintenance organizations, as well as large airports. We reviewed these files to determine whether inspection teams had conducted their surveillance according to the established methodology. We did not examine the inspectors’ judgments. Rather, we reviewed the documentation provided to us and met with Transport Canada inspectors.

5.53 In our review of files, we found examples of good surveillance practices and documentation for inspections done at the larger aviation companies that concluded on the company’s compliance to the regulatory framework. However, we found that most inspections were not consistently conducted according to the surveillance methodology. In discussion with inspectors, we often found that they did not fully understand how to apply the new methodology.

5.54 Preparation for the inspection. The instructions require inspectors to review key documentation and prepare an inspection plan, including a sampling plan, before they inspect aviation companies. This is important to ensure, among other things, that inspections focus on highest-risk areas and provide enough assurance that the company is operating within the regulatory framework.

5.55 We found that inspection plans were prepared before the inspection work began, in about 35 percent of the files that we reviewed. These plans included some of the key elements, such as the scope of the inspection, the team members, the inspection schedule, and a list of interviews. However, we found little information on the key tests that had to be done to ensure that the inspection would focus on the greatest risks. We also found some information on sampling, but sampling plans were rarely prepared. In the absence of a sampling plan, Transport Canada may not know whether enough records were tested and enough interviews were conducted to conclude on the aviation company’s compliance with the regulatory framework.

5.56 Documentation practices. Good file documentation is important to facilitate inspection reviews by management, to show that enough work has been done to support inspection conclusions on the aviation company’s compliance with the regulations and to support any enforcement action that might be needed.

5.57 In the files that we reviewed, we found that the inspectors had conducted interviews, reviewed records, analyzed documentation, identified issues, and concluded on whether the company was complying with the regulations. However, because there are no requirements for minimum documentation of work done and reporting of inspection results, the quality of the documentation varied significantly among inspectors and across regions. We found examples of documentation on interview strategies, the extent of company records reviewed, minutes of interviews, and actual company documentation reviewed and results of that review. However, this was not the case in most files.

5.58 Staff instructions require inspectors to assess whether their findings are systemic and whether they could have a significant impact on safety. However, we found that these assessments were usually not documented. In some cases, the findings in the report were different from those on the inspection worksheets. In many other cases, findings on the worksheet were not included in the inspection report. Accurate and complete reporting of significant problems is key to helping aviation companies undertake corrective action to resolve them. As well, we found that Transport Canada had trouble locating key inspection documents, such as inspection reports. In several cases, when these documents were found, they were only in draft form. In our view, Transport Canada’s documentation practices are weak and could be significantly improved.

5.59 Recommendation. Transport Canada should ensure that inspections are conducted and documented consistently according to established methodology. It should also consider whether existing documentation tools are sufficient or need to be redesigned or replaced.

The Department’s response. Agreed. A review of tools and the launch of improved tools will be completed by December 2012. Guidance and new tools to improve documentation of inspections will be provided to staff as they are approved. This documentation will be reviewed regularly as part of Quality Assurance activities.

5.60 Inspector training. Inspectors receive a delegation of authority to act on behalf of the Minister in performing certain duties, and they must be trained accordingly. In 2008, Transport Canada agreed with our recommendation that a training strategy should be developed to align with its human resources plan and address required competencies. In this audit, we looked at whether the required training had been provided to the delegated inspectors.

5.61 We found that the mandatory training for all inspectors is identified in a departmental directive. It includes courses in support of the new surveillance approach, such as an introduction to the concepts and principles of SMS, proactive interview skills, an overview of quality assurance, and surveillance procedures. We also found that the Department has implemented a process to track the training provided to all inspectors. While this training covers most of the essential aspects of the inspectors’ work, it does not include courses on working as a team—a theme that Transport Canada identified as important in the human resources plan that supports implementation of its new surveillance approach.

5.62 By the end of March 2011, a majority of inspectors had received mandatory training on the concepts and principles of SMS, on proactive interview skills, and on quality assurance. However, at that time, only 40 percent of inspectors had been trained on the new surveillance methodology. As a result, we noted that many inspections were carried out in 2010–11 by inspectors who had not received this training. Completing the training on time is important to help inspectors understand and apply the new surveillance methods. Otherwise, the Department will not have the assurance it needs that aviation companies are complying with air safety regulations. At the end of our audit, more than 65 percent of inspectors had completed the training on surveillance procedures.

5.63 Recommendation. Transport Canada should ensure that all staff involved in inspections are trained in a timely manner so they can carry out their responsibilities.

The Department’s response. Agreed. By July 2012, all current inspectors will have been given surveillance procedures training. There will be regular management reviews to ensure that mandatory training requirements continue to be met and that evolving training needs are addressed.

Evidence of sufficient management involvement in surveillance activities is lacking

5.64 Staff instructions state that inspection plans and reports should be reviewed and approved by a supervisor. Management review of inspection plans and subsequent adjustments to the plans are particularly important given the need to focus scarce resources on the highest risks. Review is also important to assure management that inspections were completed as planned and that the evidence collected supports the conclusions reached.

5.65 However, we found that Transport Canada does not have a formal management review and approval process to assess the quality of inspections or compliance with the surveillance methodology. In most cases, we found that important planning decisions, such as inspection plans and the key judgments upon which they were based, had not been approved by management. We also found that while some inspection reports for larger aviation companies had been reviewed and approved by management, the practice itself and the extent of those reviews varied significantly among regions. The fact that the documentation of inspections is weak, that many inspectors still have not received the required training as of 31 March 2011, and that aspects of the approved methodology are not being followed points to a lack of management review and approval in overseeing the surveillance program.

5.66 Transport Canada informed us that it plans to fully implement an integrated management system in 2013 to guide and support management decisions in civil aviation oversight. In developing such a system, it needs to provide for appropriate management involvement in surveillance activities.

5.67 Recommendation. Transport Canada should develop a clear expectation for appropriate management review and approval for planning, conducting, and reporting on surveillance activities to ensure that staff comply with established methodology. The nature, timing, and extent of that involvement should be documented.

The Department’s response. Agreed. By June 2012, the Department will put in place a formal process to ensure appropriate management review and sign-off of surveillance reports.

Human resources planning

5.68 In our 2008 audit, we looked at whether the Department had enough inspectors with the right skills and competencies, in the right place at the right time, to carry out its mandate. We also looked at whether the Department had an integrated human resources plan for civil aviation that was aligned with its strategic plan. We focused our audit work on how Transport Canada was managing the transition to safety management systems (SMS) for the large aviation companies. We reported that Transport Canada had not yet identified how many inspectors and engineers it needed, with what competencies, during and after the transition. We also reported that the impact of SMS was being addressed in the reorganization of the Department’s civil aviation safety program, to be completed before the end of 2009. We issued three recommendations to strengthen human resources planning: to identify future staff needs; to implement recruitment and training strategies to meet those needs; and to restrict authority if inspectors did not take the required training. The Department agreed to implement our recommendations.

5.69 In this audit, we looked at whether Transport Canada has adequately planned the human resources it will need to deliver its civil aviation safety oversight, and whether it had addressed our 2008 recommendations. We interviewed Transport Canada officials responsible for human resources planning; met with inspectors; and reviewed key planning documents, including the September 2011 transition plan for improving civil aviation safety, the 2009–12 national human resources plan for civil aviation, and the 2010 strategic plan for civil aviation. We also conducted interviews with senior management about strategies that were being implemented and their level of achievement, as outlined in the Human Resources Plan.

Transport Canada has strengthened its human resources planning process

5.70 We examined the current human resources planning process to assess whether it was aligned with the approach recommended by the Treasury Board of Canada Secretariat. We also examined the resulting plan to determine whether it addressed the recommendation we made in 2008 that the plan should specify the number of inspectors and engineers and the competencies they would need.

5.71 We found that the planning process followed by Transport Canada to prepare a human resources plan was consistent with the approach recommended by the Secretariat. It included an analysis of the civil aviation environment and priorities that are aligned with its strategic direction. The planning process involved extensive consultations among staff representatives at headquarters and in the regions and with senior management. It also took into account regional human resources needs.

5.72 As a result of the planning process, the Department identified gaps in certain competencies and proposed strategies to fill these gaps. For example, Transport Canada has acknowledged that bringing together and better integrating inspectors’ skills and knowledge are essential to achieving its objectives. Because most large aviation companies perform multiple activities like flight operations and maintenance, and have integrated systems and practices, inspectors are required to work more in multi-disciplinary teams that mirror the makeup of the aviation companies they regulate and oversee.

5.73 However, while the Department identified some gaps in competencies in its human resources plan, it did not identify how many inspectors and engineers it will need to conduct inspections under the new surveillance approach envisioned by the plan. As we noted in 2008, it is important that the Department establish the number of resources it will need to achieve its strategic objectives and to develop an appropriate strategy to obtain them.

5.74 Recommendation. Transport Canada should identify the number of resources and the competencies it will need to plan and conduct inspections under its new surveillance approach, and develop a strategy to obtain them.

The Department’s response. Agreed. The 2012–13 surveillance plan will be used to determine the number of resources and the competencies required by the Department. This information will inform updates to the national human resources plan for civil aviation, which will include strategies for recruitment and training.

Progress has been made to implement human resources strategies, but a lot remains to be done

5.75 The human resources strategies to address the gaps in competencies identified in the Human Resources Plan include the rewriting of work descriptions, the reclassification of positions (including those of inspectors) to support the new organizational structure, the development of competency profiles, and the delivery of courses to address training needs. We conducted interviews with senior management and found that the Department has made important progress toward the implementation of its human resources strategies. For example, the Department has widely consulted with staff to design the new organizational structure and validate work descriptions for inspectors. It has also put in place a new organizational structure at headquarters and in the regions. However, addressing these issues is complex, and a lot remains to be done to fully implement the human resources strategies. Transport Canada still has to finalize job descriptions for about 55 percent of positions and the classification decisions related to them. In addition, at the time of our audit, the Department still needed to deliver training on the new surveillance methodology to 35 percent of inspectors. Although Transport Canada has made it a priority to speed up its progress in achieving these strategies, the six-year period it has taken to implement the reorganization and the ongoing resistance among some inspectors have hampered the Department’s efforts to fully implement the new surveillance program.

Quality assurance

The Civil Aviation Directorate still does not have an approved quality assurance plan

5.76 Transport Canada’s 2008 Civil Aviation Directive requires that the Department plan and conduct internal quality assurance assessments within its directorate. An effective quality assurance program for evaluating Transport Canada’s surveillance activities and oversight services would set high-level direction on quality expectations, would help management determine whether established methodologies are being followed, and would facilitate continuous improvement.

5.77 In our 2008 audit, we reported that Transport Canada had no national mechanism for monitoring consistency in oversight activities or risk assessments. We recommended that Transport Canada establish a national mechanism to provide the desired level of assurance that policies, procedures, and processes for civil aviation oversight activities, including the assessment of risks, are followed consistently in all regions.

5.78 While we were scoping this audit, Transport Canada told us that it had not yet implemented a quality assurance program. Based on this information, we focused our work on the lesser expectation that Transport Canada had an approved plan to implement an adequate quality assurance program to promote continuous improvement of civil aviation safety.

5.79 We found that Transport Canada had briefly introduced some quality assurance assessments after our 2008 report, but put them on hold later that year to deploy resources elsewhere in the Department. We also found that Transport Canada conducted some quality control exercises, but these were limited to reviewing the quality of inspection reports in some regions.

5.80 At the time of our audit, Transport Canada was working on terms of reference for developing its quality assurance plan. The Department stated that it would develop a national risk-based quality assurance plan by early 2012 and put quality assurance assessments in place by spring 2013. A properly functioning quality assurance program would help to identify the weaknesses we found in how the Department prepares for, conducts, and reports on its surveillance activities.

5.81 Recommendation. Transport Canada should develop a detailed quality assurance plan and make it a priority to implement a rigorous quality assurance program that is consistent with best practices.

The Department’s response. Agreed. The Department is working on a rigorous quality assurance plan that reflects updated surveillance and inspection methodology, which will be fully completed and implemented by March 2013. Results of quality assurance work will be used to improve the Department’s work related to its oversight responsibilities.

Conclusion

5.82 Transport Canada has shown leadership in moving Canada toward a regulatory framework that includes a safety management systems (SMS) approach. Its actions were designed to promote civil aviation safety in Canada, with the objective of maintaining our good safety record into the future. As the first member of the international community to move to SMS, it faced significant challenges without the benefit of others’ experience. Transport Canada has made progress in adapting its regulatory framework to one based on SMS and in developing a surveillance methodology that is in line with an SMS approach to civil aviation safety.

5.83 We concluded that Transport Canada has implemented a suitable regulatory framework for civil aviation safety. The framework is consistent with Canada’s international obligations, and the Department has the systems and practices necessary to improve it in response to emerging safety risks.

5.84 On a program-wide basis, however, we concluded that Transport Canada is not adequately managing the risks associated with its civil aviation oversight. The significant weaknesses that need to be addressed involve how the Department plans, conducts, and reports on its surveillance activities. While we found examples where surveillance activities met our audit criteria, most inspection files that we reviewed fell short and did not follow Transport Canada’s own established methodology. We found limited evidence of management review and involvement in surveillance activities. We also found that many fewer inspections are done than planned. This is significant considering that only the companies and the operational areas considered to be of higher risk are selected for inspection in any given year.

5.85 We also concluded that Transport Canada has not adequately planned the human resources it will need to deliver its civil aviation safety program. It has developed a national human resources plan and made progress in implementing key human resource strategies, but it has not identified how many inspectors and engineers are needed to oversee civil aviation safety.

5.86 Finally, we concluded that Transport Canada does not have an approved plan to implement an adequate quality assurance program that will promote continuous improvement of its civil aviation safety program.

5.87 For decades, Transport Canada’s approach to overseeing civil aviation safety remained unchanged. The Department faced challenges as the first nation to move to an SMS-based approach in the aviation industry. It has made real progress, with large aviation companies now working in the SMS regulatory framework. Transport Canada has revised its surveillance methodology so it is consistent with this new approach, and inspections are being carried out under its instructions. Senior management now needs to concentrate its efforts on ensuring that staff apply the approved methodology consistently and rigorously, that managers provide the necessary review and supervision, and that an effective continuous improvement program is put in place. Otherwise, Transport Canada will not have the assurance it needs that the industry is operating in compliance with the regulatory framework for civil aviation safety in Canada.

About the Audit

All of the audit work in this chapter was conducted in accordance with the standards for assurance engagements set by The Canadian Institute of Chartered Accountants. While the Office adopts these standards as the minimum requirement for our audits, we also draw upon the standards and practices of other disciplines.

Objectives

The overall audit objective was to determine whether Transport Canada has adequately managed the risks associated with overseeing its civil aviation safety program.

The audit sub-objectives were

  • to determine whether Transport Canada has implemented a suitable civil aviation safety regulatory framework;
  • to determine whether Transport Canada has appropriately monitored the compliance of air carriers, maintenance organizations, and large airports against the civil aviation safety regulatory framework, with a focus on high safety risks and by using an adequate surveillance methodology;
  • to determine whether Transport Canada has adequately planned the human resources it will need to deliver its civil aviation safety program; and
  • to determine whether Transport Canada has a plan to implement an adequate quality assurance program to promote continuous improvement of civil aviation safety.

Scope and approach

Our audit focused on branches in three regions and at headquarters that are involved in the delivery of Transport Canada’s civil aviation safety program. We examined implementation of the regulatory framework and how Transport Canada was managing its risks related to its surveillance responsibilities.

We selected and reviewed significant safety issues and emerging high safety risks raised by stakeholders to assess how Transport Canada has monitored and responded to them.

We examined the information used for planning surveillance decisions and the process followed by Transport Canada for planning its annual surveillance activities in 2010–11. We also examined the methodology used by inspectors in 2010–11 for conducting planned surveillance activities. We randomly selected 74 inspection files from 2010–11 from SMS companies, airports transitioning to SMS, and small air operators and maintenance organizations. More specifically, we randomly selected the following in three regions:

  • a representative sample of 52 of 375 inspection files for small air carriers and maintenance organizations,
  • 6 large airports of the 10 airports in the National Airport System,
  • 6 of 14 inspection files for large air carriers, and
  • 6 of 13 inspection files of maintenance organizations.

At headquarters, we randomly selected 2 of 5 inspection files of very large air carriers and 2 of 5 inspection files of maintenance organizations.

Where representative sampling was used, sample sizes were sufficient to conclude on the sampled population, with a margin of error of 10 percent, 18 times out of 20. We did not examine the inspectors’ judgments or their competency. Rather, we examined whether inspectors conducted surveillance activities according to the established surveillance methodology.

Finally, we looked at Transport Canada’s plans for planning human resources and quality assurance activities.

We collected evidence through interviews with Transport Canada’s officials at headquarters in Ottawa and at several regional offices. We met with representatives from the civil aviation industry to obtain an understanding of the industry.

We did not examine the safety or efficiency of the aviation industry.

Criteria

To determine whether Transport Canada has implemented a suitable civil aviation safety regulatory framework, we used the following criteria:
Criteria Sources

Transport Canada has implemented a suitable civil aviation safety regulatory framework to address significant safety issues and emerging high risks in a timely manner.

  • Safety Oversight Manual, second edition, International Civil Aviation Organization (ICAO), 2006
  • Safety Management Manual, ICAO, 2009
To determine whether Transport Canada has appropriately monitored the compliance of air carriers, maintenance organizations, and large airports against the civil aviation safety regulatory framework, with a focus on high safety risks and by using an adequate surveillance methodology, we used the following criteria:
Criteria Sources

Transport Canada has assessed the risk of non-compliance against the civil aviation safety regulatory framework, with a focus on high safety risks.

  • Safety Management Manual, ICAO, 2009
  • Framework for the Management of Risk, Treasury Board of Canada Secretariat, 2010

Transport Canada has planned its risk-based annual surveillance according to its surveillance methodology.

  • Safety Oversight Manual, second edition, ICAO, 2006

Transport Canada has conducted its surveillance activities according to established surveillance plans and methodology.

  • Safety Oversight Manual, second edition, ICAO, 2006
To determine whether Transport Canada has adequately planned the human resources it will need to deliver its civil aviation safety program, we used the following criteria:
Criteria Sources

Transport Canada has developed a human resources planning process so that it has the appropriate number of qualified staff needed to deliver aviation services and surveillance activities.

  • Integrated Planning Guide, Treasury Board of Canada Secretariat, 2007
To determine whether Transport Canada has a plan to implement an adequate quality assurance program to promote continuous improvement of its civil aviation safety program, we used the following criteria:
Criteria Sources

Transport Canada has developed a plan to implement an adequate quality assurance program to promote continuous improvement of its civil aviation safety program.

  • International Organization for Standardization ISO 9001, 2008

Management reviewed and accepted the suitability of the criteria used in the audit.

Period covered by the audit

The audit period examined was April 2010 to March 2011. Audit work for this chapter was substantially completed on 30 September 2011.

Audit team

Assistant Auditors General: John Rossetti and Maurice Laplante
Principal: Régent Chouinard
Lead Director: Lucie Talbot
Directors: Chantal Thibaudeau and Nadine Cormier

Marie-Claude Dionne
Audrey Garneau
Michelle Gorman
Sophia Khan

For information, please contact Communications at 613-995-3708 or 1-888-761-5953 (toll-free).

Appendix—List of recommendations

The following is a list of recommendations found in Chapter 5. The number in front of the recommendation indicates the paragraph where it appears in the chapter. The numbers in parentheses indicate the paragraphs where the topic is discussed.

Recommendation Response
Regulatory framework

5.29 Transport Canada should analyze its process for addressing safety issues raised by stakeholders to ensure that it acts in a timely way to address significant safety issues. (5.24–5.28)

Agreed. By March 2013, the Department will have identified ways to accelerate follow-up on significant safety issues raised by stakeholders. The new process will continue to allow for consultation and regulatory changes.

Surveillance planning

5.37 Transport Canada should clarify what information on industry and aviation companies should be used in making risk-based decisions, collect that information, assess its completeness and reliability, and develop risk profiles when preparing annual surveillance plans in the regions. (5.35–5.36)

Agreed. Transport Canada is committed to maintaining Canada’s very strong civil aviation safety record. It has developed and begun implementation of a comprehensive action plan to guide the modernization of its structure and improvements to its systems, processes, and tools, and to ensure that resources are targeted to areas of highest risk. The Assistant Deputy Minister of Safety and Security is accountable for the implementation of this plan, and its progress is being monitored by a senior management steering committee chaired by the Associate Deputy Minister. Successful and complete implementation of the plan is a priority for the Department.

By August 2012, the Department will complete and launch a standard risk assessment tool. The new tool is designed to collect information necessary to assess aviation companies and ensure consistency across all aviation enterprises, with regularly updated risk profiles to support a dynamic process of surveillance. These steps will improve the information base for the development of annual surveillance plans and overall surveillance activities.

5.41 Transport Canada should clarify the minimum level of surveillance needed to provide the necessary coverage of civil aviation companies. (5.38–5.40)

Agreed. By August 2012, Transport Canada will update and communicate to all inspectors and managers minimum surveillance levels for civil aviation companies.

Surveillance activities

5.46 Transport Canada should review its staff instructions to clarify its expectations on the minimum documentation required and the level of management involvement expected in the oversight of surveillance activities. (5.42–5.45)

Agreed. By December 2012, the Department will have completed an update of, and communicated to managers and staff clear direction on, minimum requirements for surveillance documentation and management involvement.

5.50 Transport Canada should determine why a number of inspections are falling short of plans and take corrective action. Deviations from original plans should be documented, and they should be approved by senior management. Senior management should also determine whether Transport Canada is still obtaining the desired level of assurance that companies are complying with air safety regulations. (5.47–5.49)

Agreed. Minimum surveillance levels will be reflected in surveillance plans from 2012–13 forward. Performance will be monitored on a regular basis, any deviations from approved levels will require respective Regional Director General and Director General Civil Aviation approval nationally, and senior management will be provided with regular updates.

5.59 Transport Canada should ensure that inspections are conducted and documented consistently according to established methodology. It should also consider whether existing documentation tools are sufficient or need to be redesigned or replaced. (5.51–5.58)

Agreed. A review of tools and the launch of improved tools will be completed by December 2012. Guidance and new tools to improve documentation of inspections will be provided to staff as they are approved. This documentation will be reviewed regularly as part of Quality Assurance activities.

5.63 Transport Canada should ensure that all staff involved in inspections are trained in a timely manner so they can carry out their responsibilities. (5.60–5.62)

Agreed. By July 2012, all current inspectors will have been given surveillance procedures training. There will be regular management reviews to ensure that mandatory training requirements continue to be met and that evolving training needs are addressed.

5.67 Transport Canada should develop a clear expectation for appropriate management review and approval for planning, conducting, and reporting on surveillance activities to ensure that staff comply with established methodology. The nature, timing, and extent of that involvement should be documented. (5.64–5.66)

Agreed. By June 2012, the Department will put in place a formal process to ensure appropriate management review and sign-off of surveillance reports.

Human resources planning

5.74 Transport Canada should identify the number of resources and the competencies it will need to plan and conduct inspections under its new surveillance approach, and develop a strategy to obtain them. (5.68–5.73)

Agreed. The 2012–13 surveillance plan will be used to determine the number of resources and the competencies required by the Department. This information will inform updates to the national human resources plan for civil aviation, which will include strategies for recruitment and training.

Quality assurance

5.81 Transport Canada should develop a detailed quality assurance plan and make it a priority to implement a rigorous quality assurance program that is consistent with best practices. (5.76–5.80)

Agreed. The Department is working on a rigorous quality assurance plan that reflects updated surveillance and inspection methodology, which will be fully completed and implemented by March 2013. Results of quality assurance work will be used to improve the Department’s work related to its oversight responsibilities.

 


Definitions:

Risk profile—A risk profile identifies

  • key risk areas, risk assessments and tolerance, and an organization’s ability and capacity to mitigate risks; and
  • threats (through ongoing internal and external environmental scans, and analysis of risk information).

Source: Treasury Board of Canada Secretariat—Framework for the Management of Risk (Return)

National planning frequencies—The Civil Aviation Directorate develops national planning frequencies for surveillance, as follows:

  • SMS companies: an inspection every year—an assessment of the entire SMS every three years and a targeted inspection each of the other two years.
  • Non-SMS companies: an inspection of key functional areas once a year. (Return)

 

PDF Versions

To access the Portable Document Format (PDF) version you must have a PDF reader installed. If you do not already have such a reader, there are numerous PDF readers available for free download or for purchase on the Internet: