11.1 Our audit examined four of the grant and contribution programs included by Human Resources Development Canada (HRDC) in its 1999 internal audit. We concluded that there were widespread deficiencies in the management control frameworks of all four programs. Our findings, which covered the period to December 1999, confirmed - and extended - those of the Department's 1999 internal audit. Among other things, we found breaches of authority, payments made improperly, very limited monitoring of finances and activities, and approvals not based on established processes.

11.2 The results that projects were to achieve were often not defined in terms that could be measured. Even when they were (as in the case of Youth Internship Canada), results were not measured systematically to provide managers with feedback for making necessary improvements and to provide a basis for accountability. Some evaluation of these programs had been done. The evaluations of the Sectoral Partnerships Initiative and Transitional Jobs Fund resulted in some program changes. However, we could not support the Department`s findings on the effectiveness of the Transitional Jobs Fund.

11.3 The Department's review of 37 "problem files" identified by its internal audit for further follow-up established few cases where money could be recovered from project sponsors, because most payment errors resulted from inappropriate practices by departmental staff. Many of these practices are not acceptable - proceeding without required approvals, relying on oral contracts and paying for ineligible expenses, for example. Reporting by HRDC on the results of the review focussed on overpayments and provided little information on the problems found.

11.4 Current management is committed to addressing the serious weaknesses in the management control framework in the period up to December 1999 that our audit and the 1999 internal audit identified. HRDC has corrective action planned and being implemented in response to the problems identified in its 1999 internal audit. The actions and plans also address the deficiencies we found in our audit.

11.5 The Department has made good progress toward meeting the commitments in its Six-Point Action Plan announced in February 2000. Management is enhancing the tools and the support available to staff to improve their ability to do their jobs. Work is also proceeding on additional initiatives that expand or complement the original action plan.

11.6 HRDC management will need to sustain its efforts and attention if it is to achieve the broad-based systemic change the action plans envision. Beyond the immediate corrective steps the Department has taken, it needs to make today's extraordinary effort tomorrow's routine and fundamentally change its day-to-day approach to the delivery of grants and contributions. Management and staff of the Department need to continue the current focus on the fundamentals of control. Minimum standards of control must be in place for all projects. However, time and effort needed to review and assess proposals, monitor progress, assess performance, and evaluate results should vary with the amount of federal funding involved and the risks associated with the project.

11.7 HRDC has established an innovative system of tracking performance - one that allows for tracking improvements in the management and administration of grants and contributions. Ongoing performance tracking and internal audit are essential tools for assessing improvements in the management control framework.

Background and other observations

11.8 HRDC spent about $3 billion in 1999-2000 on grants and contributions for programs such as job creation and youth employment, as well as for employment benefits and support measures. We examined in detail four of about 40 grant and contribution programs run by HRDC: the Transitional Jobs Fund and its successor the Canada Jobs Fund, Youth Internship Canada, Social Development Partnerships and the Sectoral Partnerships Initiative.

11.9 HRDC's problems in managing grants and contributions worsened in the 1990s; audits in the late 1990s show the persistence of problems identified in the past. An internal audit in 1991 and its follow-up in 1994 led to little corrective action. Management's priorities then were to implement major policy initiatives and improve service; there was not enough emphasis on maintaining essential controls while red tape was being reduced and service improved.

11.10 More work is required to determine the resources needed to deliver grant and contribution programs. The Department's corrective actions and plans are designed to strengthen capacity by providing staff with appropriate support, training, tools and systems. HRDC is also taking steps to bring in new staff at appropriate levels. It plans to analyze workload to ensure that it has adequate resources in place.

11.11 The programs we audited had made available general information about their existence. For the most part, the promotion of these programs was passive and, in the case of some programs delivered at the local level, varied significantly among regions and local offices. Information was not always provided in a way that would promote equal access to the programs.

The chapter includes the Department's general response and its responses to specific recommendations.

HRDC has responded positively to our findings and recommendations and affirmed that it will continue to sustain the progress shown to date in meeting the commitments of the Action Plan and related initiatives.

Introduction

Grants and contributions are among the means used by government to pursue policy objectives

11.12 The government has many ways to pursue public policy objectives, including legislation and regulation, information and advice, and transfer payments to individuals, organizations and other levels of government. Transfer payments to individuals and to organizations are generally made as grants or as contributions.

11.13 Contributions differ from grants in the requirements imposed on departments and recipients. Grants are unconditional payments. If an individual or organization meets the eligibility criteria for a grant, the appropriate payment can be made without requiring the recipient to meet any other conditions. In contrast, contributions are transfer payments that are subject to performance conditions specified in a contribution agreement. The recipient must continue to show that it is meeting the performance conditions in order to be reimbursed for specific costs over the life of the agreement. The government can audit the recipients' use of contributions, whereas audit is not a requirement for grants. Because they require less accountability, grants need to be reserved for only those situations where it is demonstrably appropriate for transfers to be unconditional.

11.14 Programs that provide for grants or contributions may involve statutory or voted expenditures. Statutory expenditures are those that have been given continuing authority by Acts of the current or a previous Parliament and therefore do not require annual parliamentary approval. Voted expenditures are those for which parliamentary authority is sought through an annual Appropriation Act.

11.15 Human Resources Development Canada (HRDC) administered roughly $60 billion in 1999-2000 - the bulk of it on transfer payments. Most took the form of statutory grants such as Canada Pension Plan benefits, Old Age Security and Guaranteed Income Supplement payments under the Old Age Security Act, and Employment Insurance benefits under the Employment Insurance Act. HRDC spent about $3 billion on voted grants and contributions for programs such as job creation and youth employment, as well as for employment benefits and support measures authorized under Part II of the Employment Insurance Act. Exhibit 11.1 describes the Department's core service lines and budgeted resources for 2000-01.

Persistent shortcomings across government in managing grant and contribution programs

11.16 Our audits in various departments over more than two decades have identified persistent shortcomings in the management of discretionary grant and contribution programs. These shortcomings have ranged from problems of non-compliance with program and legislative authorities to weaknesses in program design, poor financial and management controls, and deficiencies in measuring and reporting results.

11.17 HRDC was created in 1993 from all or major parts of five departments. Included were the employment programs of the Canada Employment and Immigration Commission, providing the basis of a Canada-wide structure for delivery of the new department's programs. Many of HRDC's current grant and contribution programs evolved from programming related to employment and the labour market. In our previous audits of Employment and Immigration Canada and HRDC, we found many of the shortcomings identified recently in the management of grants and contributions. For example:

In 1983 our audit of Employment and Immigration Canada concluded that while job creation projects were selected in accordance with program objectives and criteria, inadequate monitoring posed a risk that payments would be made that were not allowable under the contribution agreements.
Our 1986 audit of two employment creation programs concluded that one of the programs had sidestepped normal administrative procedures. Further, there were few measures to ensure that project and program objectives would be achieved. Rather than public servants, it was ministers and members of the House of Commons who played the dominant role in seeking out applications and selecting projects. The other program was better designed, but the audit showed that in many cases funding for projects or enterprises had been approved based on unrealistic assessments of their continuing viability. Moreover, not enough attention had been paid to creating the permanent jobs that the program's objectives called for.
The 1992 Report of the Auditor General noted that vague project objectives in a variety of employment-related programs, coupled with imprecisely defined needs and priorities of the local labour market, prevented Employment and Immigration Canada from selecting projects on the basis of merit. Further, the Department did not respect its own minimum requirements for financial and activity monitoring.
Our 1999 report on the components of The Atlantic Groundfish Strategy (TAGS) that were managed by HRDC relied on work done by the Department's Internal Audit Bureau. Based on its work, we could provide little assurance that all contributions made under TAGS had been used for their intended purposes.

11.18 In addition to our audits, internal audits by HRDC and Employment and Immigration Canada have pointed to significant deficiencies in the management of grant and contribution programs:

A 1991 audit by Employment and Immigration Canada documented significant problems with controls, monitoring, and financial practices. The auditors noted, "There is no doubt that a persistent situation of weak controls will increase the probability of ... mismanagement resulting from negligence, abuse and even fraud."
A follow-up in 1994 found that important recommendations in the 1991 internal audit report had not been implemented, in whole or in part. Control and monitoring continued to be neglected, even in complex, high-value projects.
In 1998, HRDC's Internal Audit Bureau conducted an audit of grants and contributions under The Atlantic Groundfish Strategy. The audit identified major weaknesses in all aspects of the project life cycle. A review of project files found serious deficiencies in the management, monitoring and control of contribution agreements.
In its 1999 audit of grants and contributions, the Internal Audit Bureau examined 459 project files from a wide range of HRDC's grant and contribution programs. The audit reported systemic shortcomings in the Department's management of grants and contributions, including numerous weaknesses in control.

HRDC's problems in managing grant and contribution programs worsened in the 1990s

11.19 The management of grants and contributions requires attention to program design, appropriate tools and training for staff, and staff in adequate numbers to deliver the programs. Senior management and central agencies of the government need to ensure that there is appropriate capacity in place to deal with these programs' policy aspects, operations, and financial management and control - as well as appropriate accountability.

11.20 As early as the 1980s, our audits identified problems in the management of grants and contributions. We noted weaknesses in the design of specific programs and in the management of some stages in a project's life cycle - for example, insufficient priority placed on control and monitoring. Early audits of employment-related grant and contribution programs noted the need to ensure that trained staff were sufficient to deliver programs and the need for balance between providing service in communities and having in place proper controls and accountability.

11.21 The results of internal audits and our own work show that problems of capacity had become more serious because of many factors, including the priority given to other policy issues, reorganizations and downsizing, as well as the view that controls were of lesser importance. Audits in the late 1990s not only demonstrated the persistence of past problems but also identified systemic problems across the Department's grant and contribution programs and throughout the life cycle of projects.

The Department's emphasis was on service more than control

11.22 Following the creation of HRDC in 1993, management attention was focussed on creating the new department, managing Program Review, and reviewing many major programs under the Department's responsibility - for example, Employment Insurance and the Canada Pension Plan.

11.23 The Department went through several management changes followed by cutbacks and downsizing as part of Program Review, and some of its responsibilities were transferred to provinces through Labour Market Development Agreements. The emphasis among staff at all levels was on improving service. In April 2000 we reported on our audit of HRDC's service quality at the local level, noting that the Department had taken several initiatives to improve service to clients, including putting in place a new service delivery network.

11.24 During the mid- to late 1990s, the Department emphasized the importance of cutting red tape, empowering front-line staff, and increasing the delegation of authority, through its "Breaking the Barriers" initiatives. Other related corporate initiatives included developing a "Results-Based Accountability Framework" and programs to support staff. In contrast, the Department placed little emphasis on the importance of maintaining key financial and management controls. Appropriate controls and monitoring by management must accompany the effective empowerment of front-line staff.

11.25 As part of the major cutbacks that started in 1995, post-audit, a function to verify payments, was greatly reduced across the Department. The 1994 internal audit report had credited post-audit with the Department's only consistent monitoring of grants and contributions. Operational reviews were also cut. In addition, internal audit resources were reduced from 54 full-time-equivalent staff in 1994-95 to 33 in 1999-2000. Control points were eliminated without adequately considering the risks and the impact on the integrity of departmental controls. Downsizing and the devolution of programs and services to the provinces and to other organizations also led to a loss of corporate memory, experience and capacity - especially at the local level.

11.26 In an environment that emphasized service rather than basic controls, the Transitional Jobs Fund was introduced. Although this program required a new set of assessment skills, it was implemented without any allocation of new resources for its administration and delivery. This reinforced the relatively low priority given to administration. HRDC began to focus on its internal controls and risk management practices toward the end of the 1990s, with the introduction of initiatives related to modernizing comptrollership.

Strong public and parliamentary interest in the Department's 1999 internal audit

11.27 The release of the latest internal audit report in January 2000 generated strong interest from the public and Parliament. The findings resulted in many questions about the sampled programs, including whether funds had been spent properly and desired results had been achieved - particularly in the Transitional Jobs Fund and Canada Jobs Fund programs. In response to the audit findings and public concern, the Department released a Six-Point Action Plan. This led to further questions about whether the actions proposed in the plan could be expected to correct the deficiencies the audit had identified.

11.28 The Standing Committee on Human Resources Development and the Status of Persons with Disabilities undertook a study of the Department's management of grants and contributions. After a series of hearings and many witnesses, the Committee issued a final report in June 2000. It made 30 recommendations and requested a comprehensive response from the government by 20 September 2000.

11.29 The Committee specifically recommended that the Auditor General provide guidance in his next report "on ways to balance efficiency and flexibility in terms of program delivery and the need for sound financial management." As the Committee observed, maintaining a balance of efficiency, flexibility and sound financial management is a significant challenge to public service managers.

11.30 On 22 June 2000 the government announced that the Canada Jobs Fund program would be discontinued, although HRDC would process applications it had already received and would manage any ongoing projects to their completion. The freed-up funds are to be allocated to the regional economic development agencies.

Focus of the audit

11.31 In response to a recommendation by the Standing Committee on Public Accounts, on 8 June 1999 we advised the Chair of the Committee that we would undertake a broad-scoped audit of the management of grant and contribution programs across government. We committed at that time to auditing a number of specific grant and contribution programs, including programs administered by HRDC. When the Department released the 1999 internal audit report and announced steps it would take to correct identified shortcomings, we expanded the scope of our audit in that Department to include an assessment of its progress in implementing corrective action.

11.32 The objectives of our audit were to determine:

the extent to which reliance can be placed on HRDC's 1999 internal audit and its performance tracking system;
the Department's progress in implementing its corrective action plan;
the adequacy of the management control framework for selected grant and contribution programs;
how HRDC has measured and reported the results achieved by selected grant and contribution programs; and
the Department's compliance with financial authorities such as the Financial Administration Act and Treasury Board policies.

11.33 Our audit examined four programs from the cross-section of programs covered in the Department's 1999 internal audit. Three of the four were selected because they represented a range of expenditures, administration both by headquarters and the regions, grants as well as contributions, and varying degrees of familiarity to the public. The three programs were the Transitional Jobs Fund (budget of $300 million over three years to 31 March 1999) and its replacement program, the Canada Jobs Fund (1999-2000 budget of $110 million), the Youth Internship Canada Program (1998-99 budget of $105.7 million) and the Social Development Partnerships Program (1998-99 budget of $14.4 million). We invited HRDC to suggest a fourth program for our audit and it identified the headquarters-administered Sectoral Partnerships Initiative (1998-99 budget of $30.4 million).

11.34 Our audit of these four programs built on the Department's internal audit and extended to questions that the internal audit had not covered. We assessed not only the quality and quantity of documentation in the files but also the management of individual projects, issues of program design, and the measurement of project and program results. Moreover, we selected statistical samples of projects from each program in such a way that we would be able, from our review of project files, to project our findings to the program as a whole.

11.35 In addition, our audit assessed the adequacy of corrective actions taken by HRDC in response to its internal audit findings, including the progress it has made and reported. Given that our audit of four programs covered a broader range of issues than the internal audit had examined, we were particularly concerned to assess whether the Department's Six-Point Action Plan and other initiatives would adequately address the deficiencies we identified.

11.36 Our audit findings on the Action Plan are reported in Part I of the chapter; our detailed audit findings on the four programs we examined are in Part II. Further details on our audit criteria and approach are presented at the end of this chapter in About the Audit.

Part I - Observations and Recommendations

Internal Audit Is a Fundamental Tool for Management

11.37 In 1996 we reported that the government was going through a period of tremendous change; departments were being streamlined and restructured in an effort to reduce the administrative cost of government while maintaining or improving the delivery of programs to Canadians. We stressed the importance of effective internal audit in such an environment to help ensure that departmental programs and operations were properly managed.

11.38 Internal audit is responsible for providing senior management with sound information and advice on the adequacy of the organization's internal control systems and on how well they are performing. We expect internal audit to provide risk-based selective coverage of critical areas in a department and to report the findings for senior management's consideration. We expect senior management to direct corrective actions and subsequently to follow up on them.

Past internal audits led to little corrective action

11.39 As we would expect, in 1991 and again in a 1994 follow-up, internal audit advised senior management of internal control weaknesses in the management of grants and contributions. As part of its subsequent assessment of control risks in HRDC, internal audit identified the management of grants and contributions as an area of continuing risk; it followed up with further audit work in 1998 and again in 1999.

11.40 Employment and Immigration Canada formed working groups to address the specific recommendations arising from the 1991 internal audit. However, the efforts of the working groups led to little tangible progress at the local level. The 1994 follow-up report was particularly critical, noting specifically that the recommendations made in 1991 had not been addressed, in whole or in part. We were unable to find evidence that senior management took action to address the findings of the follow-up.

11.41 The Internal Audit Bureau's plans for 1998 defined the management of grants and contributions as a high-risk area, and the Department approved an internal audit of a range of grant and contribution programs. In early 1999, management received the report on the audit of The Atlantic Groundfish Strategy, which identified continuing problems in the management of that contribution program. However, it waited for evidence from the broader audit before taking major corrective action. The 1999 internal audit of grant and contribution programs demonstrated that there were serious and systemic problems in urgent need of corrective action.

11.42 The work of internal audit at HRDC has demonstrated its important role in providing management with assurance about whether control frameworks are effective and operating as designed. It is vitally important that management ensure the continuation of a strong internal audit group that adheres to professional standards. In addition, however, management needs to consider the results of internal audits and ensure that appropriate corrective action is taken.

The 1999 Internal Audit

Strong evidence of serious and widespread problems

11.43 The Internal Audit Bureau based its work on a sample of 459 project files drawn from some 40 grant and contribution programs in seven general areas (for example, labour market programs and youth employment programs). According to the Department, these files represented approximately $234 million in grants and contributions. The findings of the internal audit were based on a review of the documentation in the files, visits to selected recipients in all regions, and analysis of HRDC's corporate data.

11.44 The Department told us that the sample of 459 project files included 38 files erroneously drawn from outside the intended audit population. We concluded that this error in sampling was not material to the findings of the internal audit. The sampling methodology used by the Bureau was not designed (and cannot be used) to project the audit results quantitatively to the full audit universe or to any of its component programs. In other words, the quantitative results reported by the internal audit, such as dollars at risk or numbers of projects where specific controls were not working, apply only to the 459 projects it reviewed.

11.45 Although we did not assess in detail the audit work on individual files, we reviewed the methodology of the audit. In our view, the internal audit provided strong evidence of serious and widespread problems in HRDC's systems and practices for managing grants and contributions. In the files it reviewed, the internal audit found:

inadequate project selection and approval processes - for example, missing or incomplete documentation in files and failure to carry out certain consultation or verification activities;
contribution agreements that did not provide an adequate basis for both parties to exercise their rights and responsibilities or for effective financial and operational monitoring by HRDC;
inadequate monitoring of projects; and
unacceptable financial management practices - for example, ineffective controls over disbursements, inappropriate coding of transactions, lack of supporting evidence to justify expenses claimed, and lack of compliance with policies on advance payments.

11.46 Management at HRDC accepted the findings of the internal audit. Its initial response, included in the internal audit report, acknowledged the range and the seriousness of the identified deficiencies. It also noted that the lessons learned from the audit had been integrated into a comprehensive action plan for grants and contributions. As part of the plan, each of HRDC's branches, regions and program areas was to develop its own action plan. In addition, a performance tracking group was established to undertake enhanced monitoring and audits so that senior management could receive ongoing assurance about the management of grants and contributions.

11.47 When the internal audit report was released to the public and questions were raised in Parliament about the adequacy of management's response, a new, centrally developed action plan with six main areas of action was announced. The plan included withholding all payments of grants and contributions by the Department until it could be assured that key control requirements had been met. The Six-Point Action Plan was more specific on steps to be taken, timelines, and responsibilities.

Our Audit of Four Programs

We expected to find all projects managed to certain minimum standards

11.48 In our audit of four programs we selected projects that, with few exceptions, had been completed on or before 31 December 1999. This meant that the time period we covered overlapped the period covered by the 1999 internal audit. Our findings apply to approximately $570 million in grant and contribution expenditures in the four audited programs. We audited completed projects so that we could examine the full project life cycle and make an independent assessment of the nature and extent of problems that the Department's corrective action needed to address.

11.49 Our assessments took into account the significant differences in the size and complexity of projects and in the risks associated with them. We did not expect to find the same effort and attention devoted to managing small and simple projects as to managing larger and more complex projects, where financial and performance risks are inevitably greater. However, we did expect to find all projects managed to certain minimum standards, including:

requests for funding from prospective recipients that indicated how much they were seeking from HRDC, what they intended to do with the funds and what results they planned to achieve;
defensible and transparent application of project eligibility and selection criteria in recommending projects for approval;
approvals by appropriate authorities of the funding to be provided;
signed contribution agreements that set out clearly the terms and conditions of the funding and provided a basis for each party to exercise its responsibilities;
effective financial controls, including compliance with relevant authorities and financial monitoring of projects;
adherence to the terms and conditions of the agreement, with formal changes to them if required; and
monitoring of project activities and results to provide a basis for learning as well as for accountability.

11.50 In our view, observing these standards in managing projects is not optional. However, we did expect that attention to them would vary significantly, depending on the circumstances. For example, a simple letter from a prospective recipient might be fully adequate as an application for a straightforward project that involves a small contribution, whereas an application for a large, complex project would require a detailed proposal.

11.51 Our expectations for the management of grants and contributions are consistent with the Treasury Board's recently revised Policy on Transfer Payments. The revised policy consolidated a number of existing policies and provided greater clarity on issues of financial management and control and on the delivery of programs by third parties. The Treasury Board policy in effect during the period covered by our audit was less clear.

Our audit confirmed and extended the findings of the 1999 internal audit

11.52 Our audit of four programs confirmed and extended the findings of the 1999 internal audit. We found a widespread lack of due regard to probity in spending public funds and to achieving desired results. Several practices in the four programs we examined were unacceptable. These included lack of adherence to program terms and conditions, inadequate project selection processes, breaches of authority, payments made improperly, and inadequate monitoring.

11.53 We were particularly concerned about the pervasiveness of the deficiencies we found. Although the specifics varied across programs, our audit showed serious problems in all key areas of all four programs we audited.

11.54 In 81 of the files we reviewed, we found information that raised questions about the appropriateness of some payments. We provided this information to the Department. It initiated a review of all these cases to determine whether overpayments had been made. At the time we completed our audit report, the Department had completed its review of most of these cases, and had identified some overpayments. In a small number of cases, the Department had not completed its review.

11.55 In addition to auditing the management of individual grant or contribution projects, we examined other aspects of the four programs. We expected each program to have the following:

a rationale for the choice of funding instrument (grant or contribution);
clear and measurable program objectives;
clearly defined criteria for project eligibility and selection, consistent with program objectives;
capacity to deliver the program, including resources and clearly defined procedures, roles and responsibilities;
open and effective communication to make potential applicants aware of the program; and
measurement and reporting of program results.

11.56 We found several weaknesses in the design of the four programs we audited and in the measurement and reporting of program results. Part II of this chapter provides more detailed information on the four programs and what we found.

Progress in Implementing the Action Plan

The Six-Point Action Plan

11.57 In response to the findings of its internal audit, in February 2000 the Department put in place a Six-Point Action Plan, designed to strengthen the administration of grants and contributions and to rectify the problems identified by the audit. Exhibit 11.2 provides an overview of the Action Plan.

11.58 In a letter dated 7 February 2000, the Auditor General indicated that in his opinion the proposed Action Plan represented a thorough approach to corrective action on the more immediate control problems identified by the internal audit. The letter also noted that the Plan included some longer-term actions that strengthened the approach. In the current audit, we assessed the Department's progress in implementing the Six-Point Action Plan. We also looked at whether the Plan and other initiatives of the Department and the government would address our findings on the four programs we audited.

Current management is committed to taking corrective action

11.59 In an effort to respond to the issues raised by the internal audit and the questions raised in Parliament, HRDC committed significant management and staff resources to implementing the Action Plan. Specific milestones were set out, including a commitment to review all active files by 30 April 2000 for compliance with program and financial criteria, to implement regular performance tracking of grants and contributions starting in February 2000, and to report publicly on progress each quarter, starting in May 2000. The Department created a Grants and Contributions Co-ordinating Team, headed by the Senior Assistant Deputy Minister, to lead the implementation of the Action Plan. The team comprises national, regional and local representatives.

11.60 Developing and implementing the Action Plan required a significant amount of time and the commitment of HRDC management and staff. For example, project officers for the Canada Jobs Fund told us that meeting the commitments to review and revise existing project files meant putting applications on hold until the file review was over. The extended wait for approval caused enough problems for some project proponents that they withdrew their applications.

Commitments in the Action Plan are being met

11.61 The Department has generally respected the timetable it provided. It has made progress in improving and tracking the extent to which payments meet financial and program requirements. Problem files have been checked and corrected. Management has commun icated regularly with staff and has provided public reports on progress.

11.62 A number of the longer-term actions in the Plan are under way, such as providing staff with the necessary training and tools and ensuring accountability for results. An independent review by a consulting firm also concluded, "HRDC has made significant progress in implementing the Six-Point Action Plan."

Performance Tracking Directorate is established and operational

11.63 In response to a draft of the internal audit report, in November 1999 HRDC established a Performance Tracking Directorate, originally as part of the Internal Audit Bureau. In May 2000 the Directorate was transferred to the Director General, Accounting Operations. With a staff of 23 and a budget of $2.7 million, the Directorate has become an important means for continuous monitoring of the extent and impact of corrective action.

11.64 The Performance Tracking Directorate is designed to act as a quality control by measuring improvement in the administration of grants and contributions. We reviewed the sampling methodology used by the Directorate to draw a national sample of files for review, and we found that it was sound.

11.65 Given that a major component of the Action Plan's first step was the tracking of performance, regular monitoring carried out by the Performance Tracking Directorate represents good progress. If properly implemented, it should give management an ongoing measure of performance in the management and control of grants and contributions.

The Department reviewed all 17,000 active files

11.66 The Department committed itself to reviewing all active files to ensure that payments meet program and financial criteria, and it committed to taking any corrective action required. Headquarters and some regions developed checklists for use by staff in reviewing active files. Our assessment indicates that the direction the Department provided to staff on how to deal with active files complies with the requirements of the Financial Administration Act and with the Treasury Board's Policy on Transfer Payments. In addition, the checklists covered the major areas of concern our audit identified. However, we did not assess how well the checklists were applied. Based on a review of a non-representative sample of 76 files, management concluded that directives had been adequately implemented.

11.67 The Department concluded its review of 17,000 active files on 30 April 2000, as scheduled, and presented the results in a May 2000 progress report on the Action Plan. The Department reported that 16 files needed adjustments in amounts that exceeded 25 percent of the contribution agreement value, and that six overpayments totalling $3,229 had been identified.

11.68 It is not surprising that the review found so few overpayments. Only active files were reviewed and since, by definition, active files can still be adjusted, the review could not be expected to identify many overpayments. There is a higher likelihood of finding overpayments in closed files, as our audit of four programs showed.

The Department has started to report on progress

11.69 The Department's May 2000 progress report on the Action Plan indicated that the Performance Tracking Directorate had examined a sample of 76 files to confirm that the review of active files was being carried out properly. According to the report, the examination showed that all files had been reviewed in response to HRDC's national directive, and that all payments made after 20 January 2000 met the five essential financial criteria outlined in the directive (see Exhibit 11.3). These criteria are consistent with the Financial Administration Act and Treasury Board policy provisions that require certification by authorized officials of eligibility or entitlement to payment.

11.70 The report concluded that the directives on reviewing active files issued under the Six-Point Action Plan had been implemented adequately and were having a positive impact on the management and administration of grants and contributions. We note, however, that HRDC's review of 76 sampled files was not sufficient to determine whether all 17,000 active files fully met program requirements and file management standards.

11.71 An August 2000 progress report by the Performance Tracking Directorate indicated that the administration of active files has shown a substantial improvement over the findings of the internal audit. It indicated that in projects started between 1 February and 31 March 2000, there was a high level of compliance with departmental directives and guidelines relating to project applications, contracting, recommendations and approvals.

11.72 We looked at the methodology used to review the files for the August 2000 progress report and found a much wider range of issues tracked than had been covered in the 1999 internal audit - a range that corresponds more closely to our audit of the four programs reported in Part II of this chapter.

11.73 The August 2000 progress report was the first of four that the Directorate plans to prepare each year. Future reports will be able to provide a fuller assessment of a project's life cycle, including financial management.

The Department identified 37 projects for further follow-up

11.74 HRDC's review of 37 projects flagged for further analysis and correction was a major item in the action step "to check and correct problem files." The internal audit had identified 35 of the projects from the 459 projects it sampled; two others originated outside the audit sample. The purpose of the review was to complete the investigation where the initial audit work showed that financial rules might have been broken (for example, where overpayments may have occurred), as a basis for any necessary corrective action.

11.75 Work began in December 1999, although most of it took place during February 2000. The review of the 37 projects, fuelled by discussion in the media and in Parliament about the nature and extent of the problems associated with them, took on an importance well beyond what HRDC had foreseen when it first identified a need for additional work. The Department came under great pressure to conduct the follow-up review to a very high standard in a short time frame.

11.76 The Department created a task force of senior officers from its Financial and Administrative Services Branch to assess the outcome of the review. The task force signed off on the results when the follow-up work was completed. The March 2000 report, published on the Internet and provided to the Standing Committee on Human Resources Development and the Status of Persons with Disabilities, showed accounting errors across fiscal years, adjustments needed to active files, and overpayments; it also provided, for each project, a summary of the other review outcomes. Our audit confirmed that the Department made adjustments and took steps to recover the overpayments.

The review illustrated poor administrative practices and their consequences

11.77 The Department's review confirmed that poor administrative practices made it difficult to audit payments and to determine whether there had been overpayments. Program staff often made informal arrangements with sponsors that were not reflected in formal amendments to written agreements. Some of these informal arrangements were documented; others were agreed to orally or even implicitly. Some of the arrangements were contrary to the conditions established in the written agreement or to departmental and Treasury Board policies. While not acceptable practice from a government perspective, these types of arrangements have legal standing. As well, the review found that some of the formal contribution agreements were poorly constructed. For example, key clauses were missing or ambiguous and negotiated rates, rather than actual costs, were used without a clear basis. This made it difficult to determine whether the negotiated rates included ineligible costs, particularly profits.

11.78 Other departmental actions also affected the ability of reviewers to identify overpayments. For example, the Department allowed payments for expenses incurred outside the established funding period, through retroactive amendments to agreements that were already closed or through informal written or oral arrangements with the sponsor. Other examples of inappropriate practices included allowing expenses that were ineligible according to the contribution agreement or Treasury Board policy. As well, officers allowed sponsors to apply interest income earned from inappropriately large advances to increase the scope of project activities - but without a formal amendment to the agreement to increase the dollar value.

11.79 Overpayments can be established only when the Department can show that the sponsor did not fulfil its obligations as specified in the contribution agreement. The review showed that the Departments officials had ignored or not complied with internal controls, including the terms and conditions of contribution agreements. Payments made in accordance with arrangements agreed to by HRDC officials cannot subsequently be established as overpayments. Such arrangements, no matter how ad hoc, become part of the Crown's contractual obligations.

11.80 As a result, the Department's review identified and reported as overpayments only those cases it considered attributable to the actions of a sponsor. HRDC reported that it had identified $226,369.51 in overpayments, out of the $33 million in payments made to the 37 projects. One project alone accounted for $220,000 of that amount.

11.81 The Department's work on each of the 37 projects called for a full review of financial claims in relation to the terms and conditions of the contribution agreement. We concluded that there were some inconsistencies in the approach taken to the review, which limited our assurance that all issues were identified in individual projects. For example, the extent and quality of supporting documentation and working papers was thorough in some cases but limited in others. Some reviews provided no indication that the project had been assessed for interest earned on advances. HRDC's documentation indicated that in most cases all claims had been reviewed; but in a few large projects not all claims or expenses were tested.

Reporting on 37 problem projects focussed on overpayments

11.82 Parliamentarians and the media expressed a strong interest in the nature and extent of the problems in the 37 reviewed projects. Public information provided on the review in March 2000 and in the May 2000 Progress Report on the Six-Point Action Plan focussed largely on the overpayments the Department had been able to identify.

11.83 HRDC provided little public information on the nature of the administrative problems it found in the review of the 37 projects and on their consequences. The Department believed that the 1999 internal audit report had sufficiently covered the poor administrative practices found in the follow-up review of the 37 projects and that the Six-Point Action Plan had further addressed these deficiencies. It also believed that the major public concern with the 37 projects was whether or not overpayments had been made.

11.84 The administrative problems in the 37 projects identified in the Department's review confirmed and elaborated the 1999 internal audit findings. They illustrated the extent to which the system of internal controls and accountabilities had broken down, placing public funds at risk. In our view, HRDC lost a valuable opportunity to provide concrete examples of the consequences of oral agreements, retroactive amendments to agreements, inappropriately large advances at year-end, allowing claims for ineligible expenses, poorly constructed agreements and other similar practices. As a consequence, among others, the Department was unable to determine overpayments in most cases. The key deficiencies have been addressed in the Action Plan. Reporting such additional information about the problems in the 37 files would have been helpful to the public and parliamentarians as they sought to understand the issues facing HRDC and to assess the adequacy of its Action Plan and other corrective measures.

The Department is planning to review dormant files, but results are not yet available

11.85 As part of the Six-Point Action Plan, the Department will assess dormant files. These are files of projects that were active during 1998-99 or 1999-2000 but are now closed. The Department has selected a sample of dormant files, based on a set of risk factors. The results of the Department's review of dormant files were not available when we completed our audit work. However, our audit of four programs indicates that dormant files could contain a variety of problems, including overpayments that need to be recovered.

Corrective actions go beyond the Six-Point Action Plan

11.86 The Department had undertaken several initiatives before the announcement of the Six-Point Action Plan. These included establishing the Performance Tracking Directorate as well as a number of working groups to deal with specific problem areas identified in the internal audit report. The Department has since put in place other initiatives for broader corrective action needed. For example, it is examining the objectives of grant and contribution programs. It has also made organizational changes following an examination of accountability arrangements.

11.87 In addition to initiatives taken by HRDC, it is worth noting that in June 2000 the Treasury Board made changes to its Policy on Transfer Payments. The changes clarify many aspects of the previous policy. Among many new measures, the policy now requires that departments put results-based accountability frameworks in place for grant and contribution programs. It also requires departments to ensure that due diligence is exercised in making grants and contributions. Departments are expected to report on program objectives, expected results and outcomes in their report on plans and priorities. The Treasury Board will approve program terms and conditions for a specified period, usually up to five years. Departments must formally evaluate and report on the effectiveness of a program when requesting that the terms and conditions be renewed. Further, in spelling out its function as a management board the Treasury Board has indicated that it is strengthening its monitoring role.

Our audit showed problems beyond the scope of the 1999 internal audit

11.88 Our program audits identified many problems, some of which are the same as those identified by the internal audit. Others are in areas beyond the scope of the internal audit. We noted problems in:

translating the general social policy purposes into operational terms, stated as either objectives or eligibility criteria. We also noted problems in stating the expected results of programs in clear and measurable terms;
using project results and program evaluations to make necessary changes and modifications (managing for results);
providing the public with information about the availability and characteristics of a program to promote equal access;
properly managing and controlling projects - including project selection and approval as well as financial management and control; and
providing adequate support (resources, training, guidance and tools) for program delivery.

11.89 We assessed HRDC's Six-Point Action Plan and other initiatives to determine whether they address these problems.

Need to improve the way general social policy purposes are operationalized

11.90 HRDC is a social policy department that delivers a wide range of programs and services to assist Canadians of all ages, including those with distinct needs. Its mission is to enable Canadians to participate fully in the workplace and the community. General policy direction is laid out by government and approved by Parliament in the form of legislation. An essential step in improving the management of grant and contribution programs is to ensure that the design of the program, as reflected in terms and conditions approved by the Treasury Board and operationalized by the Department, reflects the policy intent.

11.91 As our audit of four programs found, this involves turning difficult concepts like sustainability, the building of community capacity, and learning systems into concrete, deliverable programs. It is particularly challenging to program designers when views differ on the best approach and when needs vary across regions. Nonetheless, the obligation remains to clarify what is to be achieved so the program can be structured to support this expectation.

11.92 Officers responsible for implementing programs and projects need to know what is to be delivered to whom, and how. In grant and contribution programs, this means clearly articulating the program's objectives or criteria, setting out measurable results to be achieved, and accompanying this with appropriate training and guidance to staff. Project officers then have clear direction on which to take concrete action.

11.93 We recognize the need for some flexibility to allow programs to respond to local realities. In our view, however, all four of the programs we audited suffered from undue lack of precision in the statement of program objectives or criteria and in the accompanying guidance. For example, in the Youth Internship Canada program the eligibility criteria for participants (unemployed or underemployed, normally aged 15 to 30, and legally entitled to work in Canada) were subject to varying interpretations. No guidance had been made available to staff to help ensure that they applied the criteria consistently. Further, the objectives and eligibility criteria for the Social Development Partnership program were stated in broad terms and were not always clearly defined.

11.94 The Department told us that it is developing additional program guidelines and additional training to help staff establish an appropriate balance between precision and flexibility. For example, the Youth Initiatives Directorate recently developed and communicated criteria for defining youth at risk and is expanding existing program guidelines.

11.95 Two of the four programs, Youth Internship Canada and Transitional Jobs Fund/Canada Jobs Fund (TJF/CJF) had defined indicators of results. In TJF/CJF, the measurement of results was built into the program design but reliable information on results was not produced as intended. Performance indicators were not defined for the other two programs. If a program's design or subsequent operational guidance is not clear about what the program is to achieve, the proper focus of the program's expenditures is subject to wide interpretation.

Plans developed to clarify program intent

11.96 The Six-Point Action Plan does not deal with the need to clarify program design and policy concepts. However, HRDC management has recognized the need and initiated a study of program management to identify the action that is needed. The study includes an examination of program mandates, objectives, design, implementation, monitoring, and areas for program and policy redesign.

11.97 The objectives of the study also include ensuring that all programs manage for results, improving learning within and across programs, and strengthening the linkages between policy development and program design. At the time of our audit, the Department was expecting to complete assessments of all its grant and contribution programs by fall 2000, with identified strategies for strengthening program management. The Department was also planning to develop appropriate performance measures as part of that exercise.

11.98 By 31 March 2001, all programs are to have results-based accountability frameworks in place. All programs are also to have learning plans that outline mechanisms for ongoing learning, dialogue and redesign, including an integrated audit and evaluation cycle. The Department has indicated that it will obtain official Treasury Board approvals by December 2002 to change program terms and conditions - well before the Treasury Board's deadline of March 2005.

11.99 As one element of the program management study, a number of programs, including the four we audited, have produced summaries of program management issues along with key elements of their action plans. The process of developing these summaries included self-assessment supplemented by peer review and discussions with the regions. In initial pilot documents we received, some managers identified a need for improvement in areas similar to those we noted in our audit. For example, managers of the Social Development Partnerships program concluded, as we did, that objectives need to be refined, measurable outcomes defined and the processes for decision making and review improved. Managers of the Sectoral Partnerships Initiative concluded that performance measures need to be validated and refined, and that program policy and guidelines need to be updated.

Improvement required in managing grants and contributions for results

11.100 Managing for results requires more than measuring results through ongoing performance measurement and periodic evaluation. It also requires using information on results to make changes to the programs and to report on their performance. As the new Treasury Board policy on transfer payments recognizes, a focus on results in grant and contribution programs provides direction on who should receive funding, whether projects are meeting expectations, and how successful the program is overall.

11.101 We expected staff in all four programs we audited to have clearly defined the results that approved projects were to achieve. We found that expected results and the related information requirements were not defined consistently. Nor did stipulating that the information was required lead to its systematic collection - even when, as in the case of TJF/CJF, the contribution agreements required the project sponsors to report on results. In the Youth Internship Canada program, expected results were defined but the information on actual project results was limited. Desired results of projects in the Social Development Partnerships and the Sectoral Partnerships Initiative programs were not defined clearly.

11.102 Three of the four programs we audited had undergone some evaluation, and an evaluation of the fourth, Social Development Partnerships, is planned. A preliminary evaluation of the TJF was carried out early in the program and several key questions were left to be answered later, in a full evaluation planned for 2000. That evaluation did not proceed. The evaluation of the Youth Internship Canada program had just been completed at the end of our audit.

11.103 We found that the work was sound in the evaluations of the Youth Internship Canada and the Sectoral Partnerships Initiative programs. We noted that some program changes were made as a result of the November 1997 evaluation of the Sectoral Partnerships Initiative and the 1998 Transitional Jobs Fund evaluation.

11.104 As we have noted, the objectives of the Department's program management study include developing appropriate performance measures and strengthening managing for results. However, as demonstrated by work we have done in other areas of government, achieving these objectives will require strong leadership from management and continuous effort.

Need to ensure equal access to programs

11.105 Public servants are expected to provide services equitably and fairly to Canadians, whether a service or program is directed to the whole population or to a specific group. Part of providing equitable access is ensuring that good information about a program is available and accessible to those who potentially qualify to participate.

11.106 Our audit of four programs found that some information on all of them was available and disseminated through a variety of media. In addition, the programs relied to varying degrees on networks and contacts maintained by project officers. In some cases, programs funded the same project sponsors year after year as part of a longer-term intervention. In our view, the methods used to promote these programs did not always ensure that potential applicants had equal access to them. Moreover, evaluations of the programs have not paid enough attention to assessing whether they have been promoted effectively and whether eligible Canadians are aware of the programs and their characteristics.

11.107 In its June 2000 report, the Standing Committee on Human Resources Development and the Status of Persons with Disabilities recommended that the Department ensure that eligibility criteria for all its grant and contribution programs were clearly identified, adequately communicated and applied equitably across the country. The Committee also recommended that to promote transparency, HRDC ensure that information available on its Web site provide the public with details about individual grant and contribution programs.

11.108 The Six-Point Action Plan itself makes no reference to equality of access to grant and contribution programs. However, the Department addressed this issue in its response to the Standing Committee. It indicated that by the end of 2000, information about all programs and their eligibility criteria would be available on its Web site and in all local offices. Local offices will provide the information to all members of Parliament before the end of the year. In addition, the Department has indicated that it will ensure that the information is kept up to date.

Need to improve the management and control of projects

11.109 Parliament sets the general purposes of expenditures and the limits on them, and the government provides public servants with further guidance on the mechanisms to be used to meet these requirements. For a grant and contribution program, Parliament establishes the program purpose and broad financial management framework, and appropriates funds or authorizes the use of funds from other sources.

11.110 Government, through the Treasury Board, approves the specific terms and conditions that programs are to observe and provides guidance on managing grants and contributions and controlling the disbursement of funds. As past internal audits and our own audit of four programs have shown, HRDC's management and control of projects and its disbursement of funds to them have been weak. The Department's response to the internal audit focussed, along with its Six-Point Action Plan, on immediate corrective action to address these weaknesses.

11.111 We found that there were problems in meeting the requirements of section 34 of the Financial Administration Act, which requires an official with the appropriate signing authority to certify eligibility and entitlement to payment. Our audit of the four programs found that staff without the appropriate delegated authority were frequently approving payments under section 34, and often did not review appropriate information to support claims.

11.112 In January 2000, HRDC stopped all payments until managers could be assured that five essential financial criteria had been met for each project (see Exhibit 11.3). In addition, the Department eliminated a problem that internal audit had identified - namely, poorly controlled codes for access to the financial system. The Department recognized that it needed to move from these immediate actions to building corrective measures into the management of all grants and contributions.

Steps taken to improve project selection and approval

11.113 Our audit of four programs found that each had problems in managing the selection and approval of projects. As already noted, some of the problems were due to a lack of clarity on what the projects were to achieve. Even allowing for these difficulties, however, we found many cases with no record of the basis for the project's selection or of who had been responsible for recommending the expenditure.

11.114 As part of improving financial management and control, the Department has taken steps to provide guidance and direction on the kinds of documentation and review of projects that it expects. If properly implemented and supported by tools and training, these steps should lead to significant improvement.

Actions in place to address problems in financial management and control

11.115 Our audit of four programs identified several problems with financial management and control. They included weaknesses in the contribution agreements, amendments made without a supporting rationale, payments made for ineligible costs and expenses, payments made without claims having been submitted, improper advance payments, inconsistent treatment of shared costs, and payments approved without the necessary authority. Details of these findings, by individual program, are in Part II of this chapter.

11.116 Part of the Department's commitment in the Six-Point Action Plan was to review all active files (see paragraph 11.66) for compliance with program and financial criteria and to take corrective action as required. The checklists used to review the active files covered the immediate areas of concern, including whether the projects met program terms and conditions.

Appropriate action taken to equip and support staff

11.117 As part of the Action Plan, the Department has undertaken a series of specific initiatives to equip and support staff in their work. Based on an assessment of workload, staff capacity and the views and concerns of staff, program staff will be given clear direction, tools, training and resources. We believe these initiatives will help address the shortcomings that our audit and the 1999 internal audit identified in the management of grants and contributions.

11.118 Among the tools HRDC is developing is a model file to replace the use of checklists. The model file contains instructions and forms to guide staff in all key aspects of financial and project management, including the aspects where our audit found deficiencies. For example, the model file offers guidance on how to identify high-risk files that require more monitoring. Staff were directed to start using the model file in September 2000.

11.119 HRDC is also developing an operational manual and policies for grants and contributions. These will provide particular guidance on financial management and control requirements. The manuals and policies will fill an important gap created in the mid-1990s, when several volumes of departmental manuals and procedural directives were replaced with only brief guidance.

11.120 To support the Action Plan, the Department is developing a new control framework with three levels of control points, a quality assurance framework, and a more consistent organizational structure at the regional and local levels. This control framework will also provide for direct oversight and will make specialist advice available to field staff handling grants and contributions. In addition, higher-level staff will handle the more complex files.

11.121 The control framework has built-in quality assurance practices. It calls for all projects to be reviewed by higher-level staff in local offices and for a small percentage of files at the regional level to be subject to post-audit. At the national level, the Performance Tracking Directorate will review between one and two percent of files. The control framework also envisions a feedback loop so that results will be used to highlight inconsistencies and procedural problems at the local level as a means to determine training needs.

11.122 The policy the Department proposes for training provides for mandatory training of managers and program staff. Program delivery training for staff addresses, for example, assessment of project proposals, contracting, payments, financial and activity monitoring, and follow-up.

11.123 We believe that when implemented in the field, the new tools and concepts being developed will go a long way toward addressing the concerns of staff, as well as those identified in our audit. However, much work on implementation remains to be done. Management will need to sustain the strong support it has shown thus far to ensure that adequate resources are obtained and that staff are trained and supported in applying the new tools, methods and approaches. As well, follow-up will be needed to ensure that staff take the training and gain the required knowledge.

More work required to determine the resources needed for program delivery

11.124 The resources required to deliver grant and contribution programs properly will vary, depending on the complexity and size of the program and the risks associated with it. Management is responsible for ensuring that the appropriate capacity is in place to deliver each program. This includes having enough staff with the necessary skills, and providing the tools, the support, the training and the information systems that they need. The Six-Point Action Plan specifically addressed the need to equip and support staff. However, the Department has identified a need for additional resources in other areas.

11.125 As indicated by our audit of the four programs and by the Department's own consultations with staff, questions remain about the adequacy of the resources (both program staff and administrative support staff) available to deliver grant and contribution programs. For example, staff told us that the requirements for monitoring (which frequently were not met) demanded too much of their time, given the other work expected of them.

11.126 Staff cuts in the 1990s had an impact on the Department's capacity to deliver programs. Nevertheless, HRDC management proceeded with the Transitional Jobs Fund without receiving any new operating funds. Staff involved in delivering the TJF faced not only additional work but also work requiring additional skills that many lacked.

11.127 The Grants and Contributions Steering Committee considered the evaluation of capacity as part of the Action Plan. It recognized that the Department's capacity to undertake grant and contribution programs had not been clearly identified. HRDC agreed with the Treasury Board Secretariat that it would undertake a workload analysis, which was under way when we completed our audit. Staff had started to identify annual workload in terms of the number and characteristics of projects administered. As an interim measure, HRDC has reallocated $20 million internally and the Treasury Board has approved a reallocation of a further $30 million from within HRDC's existing budget to strengthen its delivery of grant and contribution programs.

11.128 HRDC delivers a range of programs using grants and contributions, as do other departments. Benchmarking the resources required to deliver different types of grant and contribution programs will provide a valuable tool for government managers and central agencies to assess the adequacy of program resources.

Information systems need an overhaul

11.129 The internal audit identified deficiencies in the capacity of information systems to provide timely performance information (financial and non-financial). They included difficulty in obtaining information on sponsors' current and past projects with HRDC and the status of advances and expenses claimed. Corrective action on these deficiencies is part of the Six-Point Action Plan. A concept paper was prepared and a firm plan was to be ready by October 2000. HRDC has requested additional funding over the next three years to develop a new information system.

11.130 As a short-term solution, HRDC is upgrading an existing system to make information on grants and contributions available from one system rather than many. This interim measure was to be in place in fall 2000. It is a short-term solution, however, and will not fully address the information needs of managers at the program level. But it will create a single warehouse of data on grants and contributions and will improve the Department's capacity for external reporting.

11.131 The proposed new information system to be delivered in phases over the next three years is still at the concept stage. For maximum usefulness to management, it needs to be integrated with the financial system in a way that makes it possible for users at different levels in the Department to obtain basic financial information on contribution agreements. Managers and project officers need to be able to obtain information readily on how much has been paid to a project and on what financial commitments remain. The new system's features related to performance information were the least developed at the time of our audit.

Program Accountability

11.132 By design, and given the often general objectives and broadly stated terms and conditions, the programs we audited left public servants a great deal of discretion to determine a proposal's value and judge its eligibility. Treasury Board, a Cabinet committee of ministers, sets the eligibility requirements that departments are expected to spell out in their own guidelines and approval processes. Further, Canadians eligible for public programs expect to have equal access to them. Traditionally, public servants are responsible for administrative matters on behalf of a minister and answer to the minister.

11.133 Our work raised two issues related to management accountability and responsibility. The first concerns the role of members of Parliament and ministers; the second concerns the responsibility and accountability of staff for program decisions they make.

The role of elected officials

11.134 Elected officials outside of the executive branch of government usually do not have a direct role in program delivery. In the case of the Transitional Jobs Fund and the Canada Jobs Fund, program guidelines required that each project have a written recommendation from the member of Parliament for the area as one of the inputs for the project approval process. Such involvement in the decision-making process confuses traditional accountability relationships. The Standing Committee on Human Resources Development and the Status of Person with Disabilities noted, "Direct political involvement in HRDC's project selection process can potentially create a conflict for Parliamentarians, whose role is to hold the government accountable." The Committee concluded that the role of members of Parliament in the project selection process should remain an advisory one.

11.135 As described in Part II of this chapter, in our audit of four programs it was sometimes difficult to determine the basis on which a project had been selected. Because we reviewed only projects that had been approved, we did not review evidence that would indicate a role that members of Parliament played in decisions to reject any TJF or CJF proposals.

11.136 We found cases where project approvals were not based on the procedures established for decision making - namely, that public servants make a recommendation and ministers make a decision. Some of the projects were not eligible for the program under which they were approved; others were eligible but had not been evaluated against other projects. The credibility of established processes is weakened if they are not followed.

Steps to strengthen management accountability

11.137 HRDC is a decentralized department whose delivery of most of its programs is delegated to local and regional offices. Despite this delegation, the Deputy Minister remains responsible for the administration of the Department and, in turn, is responsible to the Minister. To deliver programs in this environment requires clear direction on what is expected as well as regular monitoring and reporting of performance. Decentralized structures require strong, clear lines of accountability to work effectively and not the type of program delivery found, for example, in the Transitional Jobs Fund. There, delivery was fully delegated and there was little central monitoring to ensure the consistent application of, and compliance with, the program's terms and conditions.

11.138 If senior management in the Department is to effectively meet its commitments to change, management structures and lines of accountability must be clear. The Department undertook an accountability review of its management structure and developed a new organizational model for the delivery of grants and contributions. Both initiatives clarify responsibilities and strengthen accountability relationships.

11.139 The aim of the Department's new organizational model is to have consistent support for project officers in all major Human Resource Centres of Canada. Three key positions have been identified. A program consultant will be responsible for technical advice, training, and quality assurance at the local level. A senior development officer will be responsible for complex cases and also for strategic development in the delivery of grants and contributions. Finally, project officers will continue to administer projects. Documentation and systems support will be provided by clerical staff. This model responds in part to concerns about the need for adequately trained and supervised staff to manage grant and contribution programs. The Department has begun staffing actions.

11.140 With appropriate staffing under the new model, project officers will be better able to meet financial and program requirements because they will be able to rely on consistent administrative support as well as advice from higher-level specialists. Segregation of duties will be improved by ensuring that enough staff are available to carry out the separate tasks.

11.141 The Department has also conducted a review of its management accountabilities, with input from 70 senior executives and managers and an External Advisory Committee (including central agency representatives and selected deputy ministers of other departments). It has also obtained assistance from external consultants. The process culminated in a memorandum to the members of the executive committee (Management Board) from the Deputy Minister, setting out a new regime to clarify and improve accountability structures.

11.142 The memorandum identified the following actions as central to increasing the clarity of decision making and accountabilities in the organization:

developing and implementing national protocols that clarify the authority, responsibility and accountability of the policy, program, corporate service and regional heads;
creating focal points of contact in the Deputy Minister's office along program development, service delivery and corporate service lines; and
establishing an executive management committee and four key departmental committees with clearly defined authorities.

Work on developing and implementing these actions has begun. These actions are in addition to the earlier reorganization of the branch administering grant and contribution programs, which established separate responsibilities for regional and national programs.

Sustained effort will be needed

11.143 As the results of previous audits demonstrate, over a number of years an unsatisfactory approach to the management of grants and contributions had developed in the Department and become widespread. Inappropriate practices had become the routine, accepted by management and supported by the absence of strong systems and controls. Practices that develop over many years frequently become entrenched.

11.144 The Department's current management is committed to implementing corrective action. It took a number of immediate steps that sent staff a clear message that change was expected. However, changes need to address the fundamental problems. The Department's internal audit, its subsequent review of 37 problem files, and our audit of four programs all revealed the extent to which the system of internal controls and accountabilities had broken down, placing public funds at risk.

11.145 The Department has begun a number of actions committed to in its Six-Point Action Plan, such as providing training and improving staff understanding. But it will need to sustain its efforts and commitment if it is to produce the desired change. Staff will need support as they change their approach to internal controls, especially as they deliver programs increasingly through third parties.

11.146 Management will need to continue monitoring the progress of corrective actions and the results achieved. The performance tracking system the Department has developed to monitor progress is an innovative approach to ongoing performance measurement. If applied systematically, it should give management a good measure of the progress being made.

11.147 As part of the incentive system for managers, their performance agreements will include commitments to improving the management of grants and contributions. The steps to implement the Action Plan form part of the basic performance agreements for 2000-01 between the Deputy Minister and all executives and managers involved in grant and contribution programs. The deadline for establishing related goals in the accountability accords was 31 August 2000.

11.148 The Department has set a clear agenda for management action in its numerous plans and initiatives. It needs to sustain the momentum that has developed. In particular, to change the way it manages grants and contributions, the Department will need to ensure that staff have a common and consistent understanding of the fundamentals of control. During the course of our work, we identified the following areas that require additional attention.

11.149 The Department should put in place quality control measures to review corrective actions, and ongoing monitoring to ensure that staff have a solid understanding of the fundamentals of financial control. Using such an approach, the Department should ensure that all overpayments from problem files and high-risk dormant files are properly identified and recovered.

Department's response: HRDC agrees with this recommendation and has taken action.

The Department had acknowledged the need for ongoing monitoring to ensure that staff have a solid understanding of the fundamentals of financial control.

This is being addressed through two measures: a new quality assurance process and the Performance Tracking Directorate. These are permanent processes and structures to detect and correct problem files, and to identify and correct deficiencies in operational policy, procedures, and employee knowledge and skills.

The new quality assurance initiative was undertaken in June 2000 to increase program expertise at the local office level. New senior operations consultants will now conduct a quality review of selected files to identify problems at the earliest possible time. In addition, financial administrators and program delivery staff have re-established a financial post-audit program at the national and regional office levels. This financial post-audit program includes the validation of expenditures in order to allow financial officers assurance that payments are in support of services provided or goods received in accordance with the prescribed policies and procedures. When either of these processes detects a problem file, it will be corrected and an adjustment or overpayment established if required.

HRDC also established a Performance Tracking Directorate, which is sampling a statistically valid selection of project files from across the country. Errors, when detected, are returned to the appropriate office for correction and establishment of an overpayment if necessary. In addition, patterns and trends are communicated to all offices delivering grant and contribution programs, with appropriate guidance and direction to reduce or eliminate such errors in the future.

The Department recognizes the importance of identifying potential overpayments and subjecting such problem cases to careful and consistent review. As part of the Six-Point Action Plan, HRDC has identified and reviewed a sample of dormant files from 1998-99 and 1999-2000. As recommended by the Auditor General, the Department has since added a quality control step to this process. Following the review by program delivery staff, Internal Audit Bureau staff have been scrutinizing the work and conducting selected on-site reviews of higher-risk projects to ensure that all potential overpayments are identified and established.

The results of the dormant file review will be published in fall 2000. In addition, HRDC will engage a private sector consulting firm to examine the Department's work and advise whether in accordance with the principle of due diligence it is cost-effective to review additional files.

11.150 The Department should ensure that key controls are strengthened as it committed to do in the Six-Point Action Plan and that they are supported by a strong internal audit function that meets the highest professional standards. The Department should also ensure that it has a well-established system for implementing and following up on corrective action resulting from internal audits.

Department's response: HRDC agrees with this recommendation and has taken action.

The Department continues to be committed to implementing the Six-Point Action Plan, including key control processes and mechanisms. As demonstrated in the progress reports issued in May and September, HRDC is reporting publicly on its achievement of the Action Plan and will continue to do so until the Plan is implemented in its entirety. In addition, as indicated in the government's response to the Third Report of the Standing Committee on Human Resources Development and the Status of Persons with Disabilities, HRDC is committed to reporting on the impact of its corrective measures in future departmental performance reports. HRDC has already contracted PricewaterhouseCoopers to conduct a second independent assessment of the implementation of the Action Plan in January 2001.

The Department has a professional Internal Audit Bureau in place and has recently acted to strengthen the role and profile of internal audit through a renewed Departmental Audit and Evaluation committee. This committee, which is one of five main executive committees, is responsible for ensuring that the Department takes corrective action on all internal audit recommendations. This action will also promote a culture of continuous learning from audits and evaluations.

Although not a part of the Department's Internal Audit Bureau, HRDC has budgeted $2.7 million annually for the national Grant and Contribution Performance Tracking Directorate. This directorate provides a consistent and continuous measurement of HRDC's compliance with file documentation requirements and fundamental financial controls. In its first report in August 2000, the Directorate confirmed that "there has been a significant, measurable improvement in overall program management and administration of grants and contributions since the 1999 internal audit." The Department has committed to publishing performance tracking results quarterly.

11.151 The Department should complete its study of program management, which is designed to ensure that desired results of grant and contribution programs are clearly defined and that results are appropriately measured and used to identify any necessary changes. In consultation with the Treasury Board Secretariat, the Department should ensure that it meets its proposed time frame for analyzing the objectives of all its grant and contribution programs and implementing appropriate changes, including measurement of results.

Department's response: The Department agrees with this recommendation and has taken action.

In May 2000, HRDC embarked on an extensive program management study that is designed to review the management and results focus of all grant and contribution programs within the Department. This study is intended to strengthen program management and performance by:

developing results-based management and accountability frameworks for all programs;
improving communication on lessons learned across programs; and
ensuring all program authorities comply with the reporting requirements set out in the Social Union Framework Agreement.

Above all, the program management study will ensure that all programs have clear objectives that shape the design and direction of the program and that they have an integrated audit and evaluation cycle. In addition, HRDC is expanding the study to incorporate a more thorough review of how program design and program information can support equality of access to programs.

This work will result in an enhanced departmental capacity to improve HRDC's grant and contribution programs. It will establish a cycle of continuous improvement in policy and program design as well as program management and administration.

This work is being carried out in co-operation with the Treasury Board Secretariat. In June 2000, the Treasury Board introduced a revised Policy on Transfer Payments to promote long-term, government-wide improvements to grant and contribution program administration. The new policy requires departments to review and resubmit their program authorities by 31 March 2005.

HRDC is fast tracking this work. As part of the program management study, all HRDC programs will have reassessed and strengthened their design, management, and accountability frameworks by 31 March 2001. The Department will then resubmit all grant and contribution program authorities to the Treasury Board by the end of 2002.

11.152 The Department should assess whether the information it provides to Canadians on grant and contribution programs adequately promotes equality of access, and it should take corrective action as required.

Department's response: The Department agrees with this recommendation and has taken action.

The Department is presently using a range of methods and techniques to provide information in a manner that promotes equality of access. In addition, HRDC is taking steps to increase the information available to Canadians on its grant and contribution programs and to make this information more accessible.

Specifically, HRDC will ensure that by December 2000, descriptions of all programs, including terms and conditions, eligibility requirements, and information on how to apply, is available on its public Web site, in every HRDC office, and to every member of Parliament.

HRDC will examine this issue further as part of the program management study referred to in the response following paragraph 11.151 and will specifically document in the program design process the accessibility and equality of access requirements. For many of these programs, HRDC believes that it will be important to consider the role of a consultative process in the community that reaches all prospective sponsors and that complements the information provided to all Canadians on its programs.

11.153 The Department should ensure that management sustains its current strong support for change to ensure that actions to equip and support staff are fully implemented and the required tools and systems provided. It should conduct ongoing monitoring of the impacts of the new tools and training and the concepts and approaches being implemented. It should also ensure that the new information system for grants and contributions is fully integrated with the Department's financial system.

Department's response: HRDC agrees with this recommendation and has taken action.

As recognized by the PricewaterhouseCoopers independent assessment of the Department's implementation of the Action Plan, "substantial senior level management involvement has been devoted to ensure progress in achieving the various activities that support the commitments under each of the six points of the Plan." Continuing management support is assured through measures taken in April establishing explicit accountability in implementing the Six-Point Action Plan for every executive with responsibility for a grant and contribution program.

In addition, HRDC has committed to reporting publicly on a quarterly basis until the Plan is implemented, and to reporting annually on the impact of its corrective measures in the departmental performance report to Parliament.

The Department recognizes the importance of well-trained and well-equipped staff in delivering grant and contribution programs. As indicated in the Department's progress reports and in the government's response to the Standing Committee, HRDC has fulfilled Action Plan commitments and taken additional initiatives to provide staff with appropriate operational policy guidance, standardized training, and improved information management systems. The Department specifically commits to monitoring and assessing the impact of operational policies and training, and to ensuring that the new information systems for grant and contribution programs are fully integrated with the departmental financial system.

11.154 In collaboration with the Treasury Board Secretariat, the Department should complete a workload analysis of the resources required to deliver different types of grant and contribution programs.

Department's response: HRDC agrees with this recommendation and has taken action.

As indicated in the response to the Standing Committee, HRDC reallocated resources in July from within its existing budget to increase the number of staff with specific responsibility for the administration of grant and contribution programs. This reallocation was based on a preliminary analysis of staffing, training and systems requirements. HRDC has since undertaken a comprehensive analysis of workload, which will be completed by December 2000.

In fall 2000, the Department is also initiating a third-party review of delivery models for grant and contribution programs. This review will assess HRDC's delivery structure in relation to those of other federal departments and will assess the appropriate balance between local, regional, and national delivery of grant and contribution programs. This review will take into account the variety of HRDC programs, the nature of partnership agreements resulting from the Employment Insurance Act and the Social Union Framework Agreement, and the existing capacity within the Department.

These reviews will be the basis for further discussions with the Treasury Board Secretariat concerning the Department's resource requirements in delivering grant and contribution programs.

Department's overall comment: The Department takes seriously all the issues raised in the chapter and in the 1999 internal audit of grant and contribution programs.

The Department is determined to build on the improvements it has already made to its administration of grant and contribution programs. HRDC will fulfil its Action Plan. It will complete the other supporting initiatives undertaken following the launch of the Plan. Furthermore, HRDC is undertaking additional steps to strengthen these actions in response to this chapter.

The challenge, as confirmed by the Auditor General, is to achieve the right balance between efficient and flexible program delivery and sound financial management. The Auditor General's report will help the Department consistently achieve program delivery that meets the needs of the Canadians it serves as well as the sound financial management that Canadians expect.

This chapter recognizes that the Department has made good progress with respect to the Six-Point Action Plan and related initiatives. It also emphasizes the need for HRDC to sustain the changes that are under way. HRDC acknowledges the need to undertake continuous work in this respect. As a result, HRDC will more explicitly incorporate the principles of modern comptrollership, or managing for excellence, as an ongoing foundation for the fundamental changes being made to its administration of grant and contribution programs.

By strengthening and integrating its work on modern comptrollership, the Department will be well positioned to fully respond to the Auditor General's statement that HRDC "needs to make today's extraordinary effort tomorrow's routine." This work will also help promote a culture of continuous learning and improvement.

In response to this chapter, HRDC will take a number of further steps. Specifically:

HRDC commits to working closely with all stakeholders, including the Treasury Board Secretariat, in the preparation of a report on the balance between effective program delivery and sound financial controls. HRDC will report by June 2001 to the Standing Committee on Human Resources Development and the Status of Persons with Disabilities on the Department's comprehensive efforts in achieving the right balance.
HRDC confirms that it will publish the results of the review of a sample of dormant files this fall. The Department will also engage an independent expert to examine its review and advise whether, in accordance with the principle of due diligence, it is cost-effective to assess additional files. This is consistent with HRDC's commitment to being open and transparent in the implementation of the Action Plan and the administration of grant and contribution programs.
HRDC will ensure that by December 2000, descriptions of all programs, including terms and conditions, eligibility requirements, and information on how to apply, are available on its public Web site, in every HRDC office, and to every member of Parliament. This action responds to the Auditor General's observations about the need to ensure that information about programs is widely available and accessible.
HRDC is expanding the program management study to incorporate a more thorough review of how program design and program information can support equality of access to programs.

In reviewing the chapter, HRDC is pleased that the Auditor General recognizes the following:

that the Department's "actions and plans also address the deficiencies we found in our audit";
that the Department "has made good progress toward meeting the commitments in its Six-Point Action Plan" and that "work is also proceeding on additional initiatives that expand or complement the original Action Plan"; and
that HRDC's "Performance Tracking Directorate represents good progress. If properly implemented, it should give management an ongoing measure of performance in the management and control of grants and contributions."

HRDC also notes that:

the Auditor General's audit was based on files that were closed before January 2000. Therefore, with the exception of a few projects that had been extended, the period covered by the Auditor General's audit precedes the corrective action by the Department;
the Department has asked for the advice of the Standing Committee on any changes that might strengthen the project selection process in relation to grant and contribution programs. This will include the role of members of Parliament, who are well placed to advise on the various interests of the communities they represent; and
with respect to efficiency, flexibility and control, the Auditor General recognized that "there is no simple answer to what the balance should be."

The Report acknowledges the broad range of initiatives HRDC has taken since the internal audit was conducted in 1999. In this respect, the Department has:

launched a Six-Point Action Plan to strengthen administration of grant and contribution programs, including the establishment of the Performance Tracking Directorate to assess adherence to policies, and the provision of extensive training and better tools to staff;
complied with the new Treasury Board Policy on Transfer Payments to further strengthen the administration of grant and contribution programs;
introduced a new quality assurance process to detect and correct errors at the earliest possible time;
initiated actions to improve accountability structures and management processes within the Department;
released two progress reports and the response to the Standing Committee report of June 2000; and
initiated a comprehensive program management study to ensure that the objectives of all grant and contribution programs are clearly defined and that performance measures are in place to demonstrate that these objectives are met.

In summary HRDC confirms its commitment to making fundamental and permanent improvements to the management and administration of its grant and contribution programs. HRDC will continue to work with all interested parties to ensure that it achieves the right balance between responsive client service and prudent management of taxpayers' money.

Part II - Four Programs Audited

Introduction

11.155 We audited four grant and contribution programs, with particular emphasis on key aspects of project management, program design and the measurement of results. Here, in Part II of the chapter, we present our audit findings and conclusions on each of the following programs:

Transitional Jobs Fund/Canada Jobs Fund (TJF/CJF)
Youth Internship Canada (YIC)
Social Development Partnerships Program (SDPP)
Sectoral Partnerships Initiative (SPI)

The period covered by our audit preceded corrective action by the Department

11.156 In order to audit the full life cycle of projects, we drew our sample from projects that had closed by January 2000. Therefore, with the exception of a few projects that had been extended by amending the contribution agreements, our audit preceded the corrective action the Department took following its 1999 internal audit.

Sampling allowed valid conclusions on each program audited

11.157 To be able to draw valid conclusions on each program as a whole, we selected all high-value projects from the populations of files we audited and stratified random samples from the remaining projects.

11.158 Transitional Jobs Fund/Canada Jobs Fund. The TJF program terminated on 31 March 1999. We included in our sample all 10 of the high-value projects that received more than $2 million each in HRDC contributions. We took a stratified random sample of 64 files from all other completed projects (1,080) over the life of the program. In CJF, we sampled 32 of 62 projects with planned end dates up to 31 December 1999 and selected all four of the projects with a high value (over $503,560). The total value of the projects selected in both programs was approximately $70 million.

11.159 Youth Internship Canada. We drew our audit sample from 8,084 projects across Canada with a total value of $185 million that were active between 1 April 1998 and 31 December 1999. We selected a stratified random sample of 77 projects - 57 with a value of up to $10,100 and 20 with a value over $10,100. Among the larger projects, we included all seven that received a contribution of over $606,000. The total value of the projects we selected was over $18.5 million.

11.160 Social Development Partnerships. This program uses grants as well as contributions. We drew our audit sample of contributions from 114 projects that were active between April 1998 and December 1999. We selected a stratified sample of projects, and included all the high-value projects (over $495,000). Our sample consisted of 47 contribution projects totalling $11.2 million. Five of these were high-value projects. We reviewed the full population of 26 grants; they ranged from about $120,000 to $3 million; two were in the range of $2 million to $3 million.

11.161 Sectoral Partnerships Initiative. From the 119 SPI projects that closed before January 2000, we drew the five highest-value projects, worth a total of $18.9 million. We also drew a stratified random sample of 44 projects worth a total of $10.8 million. Many of these projects had received funding over several years. In all, HRDC had contributed $47.1 million for the 119 closed projects. The two smallest projects we audited had each received $10,000 and the largest, $6.5 million.

Transitional Jobs Fund/Canada Jobs Fund

Program Description

11.162 The Transitional Jobs Fund (TJF) was a three-year initiative launched in July 1996. This program was implemented at the same time as the Employment Insurance (EI) reform and was designed to assist geographical areas that have historically relied more heavily on Employment Insurance and therefore would be more severely affected by changes to that program.

11.163 The objective of TJF was to create long-term, sustainable jobs for individuals. HRDC was to achieve this objective by working with the private sector, provincial and municipal governments, other federal government departments, community partners, and other organizations to generate new, permanent jobs in affected communities.

11.164 TJF received a total of $300 million in funding. It ended on 31 March 1999 and was replaced by the Canada Jobs Fund (CJF), with annual funding of $110 million. Unlike TJF, which had not been allocated an operating budget, the annual funding for CJF included almost $20 million to deliver the program. CJF was similar to TJF but had an additional objective, to strengthen communities' capacity to become self-reliant. In June 2000 the government announced that CJF would be discontinued and the associated funding redirected to federal regional economic development agencies. However, HRDC was to process any applications it had received and would manage ongoing CJF projects to their completion.

11.165 During the three years of TJF, over 1,000 projects were funded through contributions. Under CJF, 430 projects were approved in 1999-2000. The government has indicated that these programs created new jobs and leveraged millions of dollars in funding by partners. Exhibit 11.4 presents a breakdown by region of the budgets and expenditures for these programs over the last four years. Exhibit 11.5 provides examples of typical projects.

Program Design

11.166 On 26 February 1996, Cabinet agreed that a $300 million transition fund would be established for a specified period to support job creation in areas of high unemployment affected by Employment Insurance reform. In June 1996, the Treasury Board approved the terms and conditions and the funding for the TJF. The terms and conditions stipulated, among other things, eligible activities, guidelines, content of proposals, eligible categories of expenditure, authority to approve proposals and payments, and evaluation criteria. In July 1996, HRDC developed brief operational guidelines for the program.

11.167 In December 1998, Cabinet agreed that the TJF should be continued at $110 million per year and renamed the Canada Jobs Fund. The CJF was to facilitate human and social development by working with partners, particularly in the private sector, to generate the sustainable economic activity needed to create jobs in regions and communities of high unemployment (10 percent or more).

11.168 The Treasury Board approved the terms and conditions of the CJF in March 1999. They were similar to those of the TJF but in some aspects were more specific, building on the experience and knowledge acquired through the TJF. For example, the results of a 1998 TJF program evaluation led to some changes, such as better co-ordination with regional economic development agencies. Also, the Department's operational guidelines for CJF were more comprehensive and included criteria such as targeting small and medium-sized businesses and ensuring that the program was used as "last resort" funding. Exhibit 11.6 presents the key elements of TJF and CJF as set out in their terms and conditions and the guidelines.

Difficulties in targeting TJF

11.169 As a general rule, the Department decided that the program should be targeted to areas of the country and to areas within communities where unemployment rates were 12 percent or higher. The overall budget for TJF was allocated among HRDC regions (generally provinces - see Exhibit 11.4) on the basis of two criteria: the number of Employment Insurance claimants in EI regions where the unemployment rate had exceeded 12 percent in 1995 (75 percent of the budget); and the anticipated decline in Employment Insurance benefits to provinces and territories as a result of EI reform (25 percent of the budget).

11.170 The responsibility to define specific areas of eligibility within HRDC regions was left to regional and local HRDC officials so that regional and local needs could be taken into account.

11.171 We noted a wide range of approaches used by regional offices to establish areas of eligibility for TJF and to determine whether geographical areas within communities were eligible or not. Data were obtained from various sources, including Statistics Canada's Census and Labour Force Survey. As well, labour market information developed at the local level by HRDC officials was used. In addition, regional and local officials identified projects that met specific community concerns, or projects that would have positive impacts on high unemployment rates in surrounding areas.

11.172 Employment Insurance economic regions are currently the smallest geographical areas for which Labour Force Survey estimates of unemployment are available. Statistics Canada also publishes some relevant information on a limited number of cities. The information sources used by regional offices to target the program to local areas outside the EI regions that qualified (those with unemployment rates of 12 percent or higher) were not as reliable as the official measures of unemployment.

11.173 Funding projects in local areas that were outside qualifying EI regions was an important part of the program's overall design. According to the Department, allocations to these areas accounted for almost 40 percent of approved TJF projects. In our opinion, the ability to target the program equitably was compromised by the difficulty of estimating with reasonable accuracy the unemployment rates in such areas.

Targeting of CJF was more systematic

11.174 To allocate the CJF budget among its regions, the Department used an approach similar to that used for TJF. Two thirds of the funds were allocated among EI regions with unemployment rates of 10 percent or higher. An additional 8 percent was allocated among regions with unemployment rates of 15 percent or higher. The remaining 25 percent of the total budget was allocated by province and territory according to the reduction in EI benefits imposed by EI reform.

11.175 CJF terms and conditions were more precise in defining high unemployment regions and subregions. In July 1999, HRDC's Chief Actuary was asked to recommend an acceptable methodology for determining the eligibility of subregions. He recommended using Statistics Canada's Labour Force Survey, the source of data whose use for targeting government programs on the basis of unemployment rates could be fully defended. We concur with this recommendation.

Some aspects of the programs were difficult to implement

11.176 The design of TJF and CJF emphasized a partnership approach and the leveraging of funds from other sources. This emphasis required HRDC's local offices to develop and maintain collaborative partnerships with local private sector organizations, regional economic development agencies and provincial and municipal governments. Many project officers told us that this and other design elements meant that they needed new skills to deliver these programs. For example, many project officers told us that they did not have the business expertise required to assess business plans - a key part of project assessment - and did not receive sufficient training and support.

11.177 We found that the programs' terms and conditions and the operational guidelines were not sufficiently spelled out to help project officers decide who and what were eligible for funding, under what conditions, for what purposes, and in what amounts. We recognize that some local flexibility was desirable to allow the programs to be tailored to local circumstances. However, we noted inconsistencies among local offices in the interpretation of such fundamental matters as the programs' objectives and in the application of eligibility criteria, especially in the case of TJF.

11.178 For example, one local office would see the program's objective as direct job creation and would fund only projects that would create jobs directly and immediately, or in the very short term. Another office would focus more on general economic development and would fund projects that might stimulate economic activity and create jobs only indirectly and in the future. In terms of project duration, one office would recommend approval of only those projects that would last less than a year. Another would recommend approval of projects that were to last up to three years, but would arrange for three consecutive agreements of less than a year each.

11.179 Key elements of the programs were not always defined clearly, and in some cases they potentially conflicted with other aspects. For example, operational guidelines for CJF required that HRDC funding be used only as a last resort. This criterion could lead to the funding of higher-risk projects, which could conflict with the program's objective of creating sustainable jobs. Project officers told us that they found this criterion particularly difficult to apply. The Department developed guidance in April 2000, and was implementing it as we carried out our audit.

11.180 All projects had to be approved by the Minister of Human Resources Development, even though the terms and conditions of both programs provided for delegation of that authority. (One exception to this practice was the current Minister's delegation to the Deputy Minister of authority for approval of projects located in the Minister's riding.) Although the need for the Minister to approve all projects was only one contributing factor, we noted that frequently it took a long time - in some cases, months - to get projects approved. This was a source of frustration to HRDC officials as well as project sponsors.

11.181 Operational guidelines for TJF and CJF required a written recommendation for a project from the local member of Parliament. We recognize that this practice can add important input to the assessment of proposals. However, accountability is unclear when, on the one hand, Parliament approves funding and scrutinizes the government's performance in delivering programs, and on the other hand, parliamentarians have a role in the programs' decision-making processes.

Equal access to these programs was not ensured

11.182 The Minister of HRD issued a news release at the introduction of both TJF and CJF. HRDC headquarters and regional offices played only a limited role in promoting these programs.

11.183 Local offices were responsible for informing potential proponents about the programs. Some offices relied heavily for this on their partners, including economic development agencies or provincial government departments. Others developed information kits and promoted the programs through their involvement in the community. We saw evidence that members of Parliament had received information on the programs, and we were told that some had been instrumental in marketing them. Public announcements of approved projects also helped to raise awareness among potential proponents of projects. Some offices told us that they did not want to promote the programs heavily because the program budgets were limited.

11.184 In our view, the variation in the methods used to promote the programs across and within HRDC regions, including in local areas that were outside qualifying EI regions but may nevertheless have qualified by virtue of "pockets" of high unemployment, did not ensure equal access by potential applicants.

Project Selection and Approval

Key information was missing from many proposals

11.185 We found that project officers did not always have the information they needed to assess a project properly. Although proposals or business plans were submitted to HRDC for almost all projects, most of them lacked key information (74 percent in TJF and 57 percent in CJF). Information that was missing included the characteristics of jobs to be created, projected cash flows and budgets, and a clear time frame for projects. We noted also that proposals did not always provide, nor did HRDC always request, information on the nature and purpose of funding provided by other partners in a project. This prevented HRDC from determining the risk of double funding and, in CJF, determining whether HRDC funding was truly a last resort.

We found several deficiencies in project assessments

11.186 We concluded that fewer than five percent of projects were adequately assessed. In 30 percent of TJF projects and 29 percent of CJF projects, we could not find evidence that proposed activities had been analyzed to evaluate the likelihood that they would create sustainable jobs. We saw projects approved, for example, that would create only short-term jobs or that might create sustainable jobs only in the future (see Exhibit 11.7).

11.187 Many project officers told us that they did not question the incremental impact of the HRDC contribution - that is, whether the project would have gone ahead without funding by HRDC. Indeed, evidence on file indicated that almost 63 percent of TJF projects and 75 percent of CJF projects would probably have gone ahead without HRDC funding, although its contribution might have advanced the timetable or helped increase the scale of some projects. Some projects had already been completed by the time the contribution agreement was signed (see Exhibit 11.8).

11.188 In 74 percent of TJF projects it was not evident that the potential need for environmental assessments had been considered. Project officers told us, however, that HRDC often relied on its partners to ensure that where environmental assessments were needed they were done. The situation improved under CJF: environmental screening was performed in 36 percent of all projects.

11.189 Other aspects that project officers seldom considered included the possibility that other HRDC programs were funding the same project, or that a project sponsor still owed HRDC repayment of funds under previous agreements. Finally, we saw little or no evidence that when a subsequent phase of a project was assessed for funding, results of the previous phase were considered.

Approvals were not always supported by relevant information

11.190 The terms and conditions of both programs clearly stated that no financial assistance could be provided without the agreement of the appropriate provincial or territorial government. With one exception, we found evidence of this agreement in all cases. A recommendation from the local member of Parliament was on file in most cases.

11.191 We found that in some cases the Minister had not received all relevant information before approving a project. For example, we found cases where the Minister was not informed that the province, the local member of Parliament or a key partner had raised concerns about it.

11.192 Finally, we found cases where project approvals were not based on established procedures (see Exhibit 11.9). In our opinion, some of these projects did not meet the eligibility criteria.

Financial Management and Control

Many contribution agreements lacked key information

11.193 Signed agreements and schedules were on file for 95 percent of TJF projects and 92 percent of CJF projects. For the most part, the agreements or schedules reflected the Minister's approval. However, in 74 percent of TJF projects and 63 percent of CJF projects, key information was missing, or was unclear. In some cases, eligible costs were not described; in others, the activities to be completed were not indicated. We saw agreements that did not specify a funding period, or that simply stated the number of jobs to be created without indicating any characteristics of the jobs or describing how the project would ensure that the jobs were created.

11.194 We also found contradictory information in some contracts and their related schedules. For example, one section of the schedule would indicate that payments would be made on the basis of jobs created and salaries, while another section of the same agreement would indicate that the funding would be for capital purchases. These inconsistencies in agreements make it more difficult to carry out the activity and financial monitoring required to ensure that the project meets its objectives and uses the funds for agreed purposes.

11.195 About 30 percent of the agreements were formally amended in writing. Rationales for the amendments were in place for all CJF projects and all but a small number of TJF projects. Although the agreements stipulated that amendments had to be in writing, we were told that in some cases amendments were based on only oral agreements with sponsors to change the terms and conditions in contribution agreements.

Payments and advances were often mishandled

11.196 In almost all projects (92 percent in TJF and 86 percent in CJF), payments were not handled properly. The deficiencies we found included one or more of the following:

payments made for ineligible expenses ($450,000 in the 74 TJF projects that we sampled);
payments made for expenses incurred outside the funding period specified in the agreement ($9.5 million in the 74 TJF projects, and $755,000 in the 36 CJF projects that we sampled);
payments made without review of adequate supporting information;
payments made to a party other than the recipient specified in the agreement;
payments that did not respect the terms and conditions of the agreement (for example, contributions paid for salaries in the expectation that the jobs would be created after the project closed); and
payments approved without proper authority.

11.197 In some cases, the agreement indicated that HRDC would pay the lesser of a certain amount or a certain percentage of eligible costs. However, as the total cost of the project was rarely known to HRDC, in most of these cases we were unable to conclude whether the payments had been correct.

11.198 Advance payments were used often in TJF (63 percent of projects), but less commonly in CJF (36 percent). Where advance payments were used, we found that they were not managed properly in 79 percent of TJF projects and 58 percent of CJF projects. For example, some advances were not based on forecasts of cash flow, or exceeded the maximum amounts allowed under Treasury Board policy, or were not reconciled against claims and cleared before further advances were issued. Exhibit 11.10 provides an example. In the 74 TJF projects we sampled, we found almost $2 million in advances that had not been cleared at the end of the project. In many cases, advances were made at year-end and charged to that fiscal year when the activities were to take place in the following fiscal year.

11.199 We noted significant peaks in TJF expenditures before year-end, especially before the closing of the program on 31 March 1999. Indeed, we saw some cases of funding provided for projects that, at the end of March 1999, were expected to be completed in just a few days. Exhibit 11.11 shows TJF expenditures by month.

11.200 In our audit we found it difficult to determine whether agreements and payments had been approved in accordance with delegated authorities. Many local offices did not have signature cards or could not confirm the level of authority of the person who had signed the agreement or the payment. In our view, this unacceptable situation was due in part to the high turnover that occurred in the last few years and to the very generic delegation chart.

Lack of project monitoring

11.201 Activity and financial monitoring was insufficient in the vast majority of projects (98 percent of TJF projects and 79 percent of CJF projects). Project officers did not prepare risk-based plans for monitoring approved projects. We could not find evidence of any activity monitoring or financial monitoring in 46 percent of TJF projects and 7 percent of CJF projects. Final financial visits had not always been completed before the last payment was issued (78 percent of TJF and 23 percent of CJF projects).

Project Results

Information on project results inadequate

11.202 The terms and conditions of both TJF and CJF required that evaluating them against their objectives include measuring their return on investment - that is, how successfully they created jobs in areas of high unemployment and generated funds from partners outside HRDC. As well, the terms and conditions required HRDC to monitor the implementation of the program and annually report the results achieved, in terms of numbers of jobs created and numbers of dollars generated from partners. Our review of departmental performance reports for the last three years indicates that the Department has partly complied with this requirement by publishing the number of jobs expected to be created and the funding expected from partners.

11.203 We examined how information on jobs created and leverage obtained was gathered at the project level, and whether it was reliable. We concluded that information on project results was inadequate.

11.204 The standard contribution agreement includes an obligation for the sponsor to provide to HRDC, 30 days after the project's completion and again one year later, information on the numbers and types of jobs created. Only 14 percent of TJF sponsors provided the required 30-day report, and 10 percent the 12-month report. We could not reach a conclusion on the extent of compliance by CJF project sponsors because many projects were still active at the time of our audit or had only recently been completed and had not passed the deadlines.

11.205 Project officers did not properly monitor the number of jobs created, even when it was the basis for the payment of the contribution. In most cases, officers obtained information from the employer without making a site visit. To determine the number of jobs created, project officers often relied on lists of employees working at the end of the project, without recourse to information on the jobs that had existed at its beginning. In some cases, project officers relied on lists of people hired, without information on either turnover or the duration of their employment.

11.206 HRDC counted all the jobs created by a project, regardless of the extent of its contribution toward the project's total cost. We also found that HRDC and the federal regional development agency each counted all the jobs created by a project when both were involved in funding it.

11.207 Our audit found that the only information available on funds leveraged from other partners was based on estimates provided by sponsors with their project applications. Because the final costs of projects were almost never known to HRDC, it could not determine the actual contribution made by other partners.

11.208 In February 2000, following the release of the 1999 internal audit report, HRDC made an effort to account for all jobs created by TJF. Regional offices were asked to complete a template setting out the results of the program, including jobs created and leverage achieved. Telephone follow-up with sponsors and reviews of project files were used to capture the information. However, this process did not provide a constant measure of time elapsed from the end of projects. Nor did it yield any information on the actual total costs of projects and the funding provided by others.

Effectiveness Measurement and Reporting

A preliminary evaluation of TJF was completed in 1998; an evaluation planned for 2000 did not proceed

11.209 A Phase I evaluation of TJF dealt with the early stages of the program's activities and was completed in October 1998. The evaluation concluded that TJF was expected to create 29,500 new jobs by the end of March 1999. The Department stated further, "It is estimated that, without TJF, 22,000 of these jobs would not have been created."

11.210 We assessed the methodology and results of the 1998 evaluation of TJF. In part because it was a preliminary evaluation, we found limitations in it. These limitations are all the more important given that a more in-depth Phase II evaluation of the program (and CJF), which was to have been carried out in 2000, did not proceed.

Limitations in the 1998 evaluation

11.211 The 1998 evaluation did not examine the adequacy of the approach to targeting program activities in local areas of high unemployment. Nor did it consider such issues as the possible displacement effects of the program, including the possibility that publicly funded subsidies had allowed some private sector firms to hire workers away from other employers. Further, the evaluation did not base its cost-per-job analysis on all federal programs involved in funding TJF projects. Because the 1998 evaluation was completed early in the life of the program, the Department was unable to estimate its impact on future Employment Insurance claims of workers hired during the program. It was also unable to examine adequately the sustainability of jobs associated with funded projects. These issues were to have been examined in the Phase II evaluation.

11.212 The Department's estimates of TJF's employment impacts were based primarily on a telephone survey of employers, which required employers to assess the program's results at the project level. The evaluation aggregated these results to represent the program as a whole. Part of our assessment of the Department's evaluation methodology included a brief telephone follow-up of some sponsors in our TJF sample. In most cases, we found that sponsors had difficulty isolating the impact of the HRDC funding either on the progress of the project itself or on the number of jobs that were created.

11.213 We concluded that because of the difficulty of measuring job creation effects, it would have been important for the Department to corroborate employers' self-assessments with other information. The 1998 evaluation clearly identified a test to corroborate the existence of TJF's job creation effects. The report noted that information on whether or not project sponsors used TJF funds as a last resort would help in measuring the program's incremental effects - that is, the extent to which the program led to the creation of jobs that would not otherwise have been created.

11.214 This test would have required assessing whether project applicants had exhausted all other financing possibilities before obtaining TJF funding. In our judgment, the test is an important one; indeed, the Department incorporated it in the operational guidelines of the subsequent CJF program. However, the Department did not examine this issue in its telephone survey of employers. It told us that it did not do so because by the time it became aware of the test, it was too late to incorporate it in the survey.

The evaluation approach resulted in an overestimate of incremental job creation

11.215 The Department focussed strongly on assessing TJF's effects in triggering projects that would not otherwise have occurred or, alternatively, in expanding projects that would have proceeded anyway without TJF funding.

11.216 The Department's telephone survey showed that some projects would have gone ahead without TJF assistance (42 percent of the projects surveyed) but that in about half of those cases, TJF had triggered an expansion of the project and of its employment creation. However, the evaluation report indicates that in measuring the incremental employment effect, all employment associated with the expanded projects was treated as incremental - not just the jobs related to the additional activity stimulated by TJF.

11.217 The result of this approach was that the incremental job creation effect of TJF was overstated as presented in the evaluation and reported by the Department to Parliament.

11.218 The Department undertook a subsequent internal study in early 2000, again based on contacts with employers. The results of this study generally supported the initial total figure of about 30,000 jobs but did not address the estimated 22,000 incremental jobs. Yet it is this latter measure that is key to judging the program's effectiveness.

We could not confirm the program's effectiveness

11.219 We sought to confirm the program's longer-term impacts. We based our analysis on the Department's estimate, derived from its program data, that over a three-year period the program's expenditures of about $300 million had leveraged almost $3 billion in project activities in the targeted areas. We also took into account the view expressed in the 1998 evaluation that TJF had the potential to bring considerable economic relief to communities experiencing high unemployment.

11.220 We assessed the links between estimates of jobs generated by projects and estimates of total project expenditures. We noted that a small number of projects exercised a strong influence on the total leverage estimates for the program, with some estimated leverage as high as $700 per dollar of TJF expenditure. In contrast, some 50 percent of projects had leverage of less than $3 per dollar of TJF expenditure.

11.221 Much of the data on total project expenditures in the Department's program database was of poor quality. Where the data appeared more reliable (in about half the projects), we found a close relationship between the numbers of jobs at the project level and the TJF contribution, even though on average TJF accounted for only about 10 percent of the funding received by a project. This finding appears to reflect HRDC's approach to funding projects while meeting administrative guidelines related to TJF cost-per-job.

11.222 However, we found that funding provided by other partners and project sponsors was of little use in explaining the number of jobs at the project level. This finding raises questions about the reliability of the information available to the Department on the leverage of funds achieved by the program.

11.223 Finally, we analyzed the program data provided to us by the Department in combination with Statistics Canada's Labour Force Survey data. While we identified a number of problems in the quality of program data, the estimated $3 billion in total project spending over three years - if it occurred to the extent claimed by sponsors in project applications - should have been of identifiable benefit to local economies and their labour markets. However, we could find no evidence that the leveraged spending estimated was associated with longer-term improvements in general employment and in unemployment conditions in the areas targeted by the program.

Concluding Comments

11.224 We found weaknesses in many aspects of the management control framework of the Transitional Jobs Fund and, to a lesser degree, the Canada Jobs Fund. In addition, we found that the measurement of results to support learning and accountability was deficient.

11.225 Many program officers and other employees told us of problems in the Department's capacity and in their own ability to deliver TJF properly. The Department acknowledged these difficulties when it specifically asked Treasury Board to provide the CJF with operating resources to help ensure its effective delivery and monitoring.

11.226 The Department delegated full responsibility for the delivery of these programs to its regions, with little central monitoring to ensure consistency and compliance with the programs' terms and conditions. Similarly, at the regional level we found little monitoring of the quality and consistency of decisions. The regional offices played a mainly advisory role and focussed on controlling the regional program budgets.

11.227 Project officers we spoke to welcome the new tools and training being developed to support them in their work. We could also see some signs of improvement in the delivery of CJF projects approved late in 1999 and in 2000. Files were better documented, agreements were more comprehensive, financial controls were stronger and projects were monitored more closely.

11.228 The Transitional Jobs Fund ended on 31 March 1999. On 22 June 2000, the Minister of Human Resources Development Canada announced that the Canada Jobs Fund would be terminated. The Minister added that all applications with business plans received in HRDC local offices by 22 June 2000 would be processed in the normal manner, but that applications received after that time would not be considered.

Youth Internship Canada

Program Description

11.229 In 1996, the youth unemployment rate was nearly double that of other age groups in the labour force. In its June 1996 report, Take on the Future, Canadian Youth in the World of Work, the federal task force on youth indicated that due to their lack of experience, youth face their greatest challenge in getting a first job.

11.230 Following the1996 Speech from the Throne, which outlined the employment-related difficulties faced by youth, the government announced its Youth Employment Strategy to work with partners and help young people acquire work experience that would allow a transition from school to work. As part of the Strategy, HRDC launched its Youth Employment Initiatives in 1997. Through partnerships with business, labour, industry, not-for-profit organizations, communities, and other levels of government, these initiatives were aimed at helping youth prepare for, obtain and maintain employment, thus helping them make the transition into the labour market.

11.231 In a February 1997 decision of the Treasury Board, the government approved new consolidated terms and conditions for HRDC's Youth Employment Initiatives. The Youth Internship Canada (YIC) Program is one of four main activities of HRDC's Youth Employment Initiatives. YIC is a consolidation of existing programs, under the new terms and conditions approved by the Treasury Board in its 1997 decision.

11.232 As outlined in the Treasury Board decision, the overall objective of Youth Employment Initiatives is to assist youth in preparing for, obtaining and maintaining employment, and in making a successful transition into the labour market, thereby resulting in increased employment. The aim of YIC, as outlined in the program's operational guidelines, is to help youth who are unable to get a job without assistance.

11.233 Under the program, HRDC provides contribution funds to private, public and not-for-profit organizations to develop projects that help youth enhance their skills and gain work experience through internship opportunities. YIC measures its performance by two indicators: youth employed after the internship, and youth who return to school. The program is funded through annual appropriations from Parliament; expenditures from 1997-98 to 1999-2000 are shown in Exhibit 11.12.

11.234 HRDC contributions to employers or sponsor organizations subsidize the overhead and wage costs of hiring youth for internships. In some large projects, a sponsor organization is responsible for recruiting participants and placing them with employers. The participants then receive on-the-job training from employers. In a one-to-one intervention (one participant to one employer), the employer recruits the participant, receives contribution funding and provides on-the-job training. In some cases, recruitment of either sponsors or participants has been outsourced to third parties. In any case, at the end of the internship, participants are expected to either become employed or return to school. Exhibit 11.13 provides some examples of the range of projects funded by YIC.

11.235 Human Resource Centres of Canada (HRCCs) administer most YIC projects. The exceptions are some large projects administered by HRDC regional offices or headquarters.

Program Design

11.236 The key elements of the program's design are set out in its operational guidelines. Exhibit 11.14 provides a summary of those elements.

Broad application of eligibility criteria

11.237 The program's criteria for eligibility of sponsors are broad and straightforward. We found no problems in their application.

11.238 As shown in Exhibit 11.14, YIC has three criteria for participant eligibility: individuals are normally under the age of 30 and have not made the transition into the labour market, are unemployed or underemployed, and are eligible to work in Canada. YIC places greater emphasis on youth at risk (youth facing multiple barriers to employment; for example, youth who have not completed high school, single-parent youth, Aboriginal youth and street youth). However, other youth having difficulty making the transition to employment are also able to participate. Based on these criteria, a broad range of individuals could qualify.

11.239 We found that some project officers imposed less stringent requirements than others in selecting projects for funding. In some cases, new graduates from colleges and universities were assisted. In others, only those facing multiple barriers were considered. The Department told us that in part this reflects regional diversity among labour markets.

11.240 We noted that project officers were not given guidance on how to interpret the participant eligibility criteria to ensure their consistent application - for example, guidance on the definition of underemployment and the evidence needed to indicate an inability to get a job without assistance. In our view, this guidance is particularly important in a program such as YIC, where selecting youth who need no assistance may provide better apparent results (when measured in terms of retained employment) than selecting youth who are having difficulty making the transition to employment.

11.241 Exhibit 11.15 describes a case where the project did not clearly meet the program's overall aim of helping youth who are unable to get a job without assistance.

Some limitation in content of promotional information

11.242 HRDC disseminates information on YIC in several ways. At national headquarters, the Department developed documentation (fact sheets, an overview pamphlet) providing program information, primarily to prospective sponsors and employers. HRDC's Web sites also provide information on its youth initiatives, including YIC. In addition, a 1-800 information line responds to enquiries about youth programs.

11.243 We found some limitations in the content of the promotional material. Documents provide generic information about the program to prospective sponsors, employers and participants, but they do not include information such as the expected results of the program (become employed or return to school), target groups (youth at risk) and more specific eligibility requirements for participants (including a definition of "underemployment"). The documents direct youth and prospective sponsors to local HRCCs for additional information.

Project Selection and Approval

Lack of rationale to support selection and recommendation for approval

11.244 For each YIC project, we expected to find something on the project file that constituted a request to HRDC for the contribution - including information on the activities to be carried out over a specified period of time, the amount of funding requested, and the results to be achieved. For the larger projects, we expected that applicants would submit a proposal with their application. We did not expect to find as much information on small projects, but we did expect to find at least a letter from the applicant setting out the key information.

11.245 We also expected to find an assessment of the request for funding completed by the HRDC project officer, and a signed recommendation (with accompanying rationale) for approval. We expected that the rationale for recommending approval would indicate how the project would contribute to achieving the program's objectives. Again, we expected that the depth of supporting analysis would vary with the size and complexity of the proposed project.

11.246 Our review found several deficiencies:

An application, a proposal or other documentation was missing for nearly half of the small projects (up to $10,100) and 38 percent of the larger projects (over $10,100). It is not clear in those cases whether sponsors applying for YIC funding had submitted any documentation to support their requests.
Reflecting the limited documentation of the funding requests, we found a frequent lack of assessment or analysis by HRDC project officers to support their recommendations to approve project funding. In only five percent of the larger projects was there evidence that proposals had been assessed or analyzed. In 82 percent of the small projects, no rationale was provided for why the project should be funded.
We found no indication that the possible need for an environmental assessment had been considered or that any sponsors had been checked to see whether they had debts outstanding to HRDC.
Furthermore, we found that in 40 percent of the larger projects and 56 percent of the smaller projects, the project officer had not signed or dated the recommendation for approval.

11.247 A supporting rationale for recommending approval, including an explanation of how the project will support the program's objectives, would help provide assurance to those approving the project that program funds will be used for the specific purposes intended.

Deficiencies in the project approval process

11.248 We expected to find that persons with appropriate authority had approved the projects. We found that 30 percent of the larger projects and over half of the smaller projects had no approvals on file. Of the remainder, 89 percent of the larger projects had been approved by persons with delegated authority, as had 82 percent of the smaller projects.

11.249 We also expected to see a clear assignment and acceptance of responsibility for decision making. In some cases, where recruitment of potential sponsors or participants had been outsourced to third parties, we found that it was the third-party representative who had signed the formal recommendation for approval. This was the responsibility of HRDC officials.

11.250 We also found one large project whose approval was not based on the established procedures (see Exhibit 11.16). Although the Department requires applications and proposals to be submitted to the local HRCC, in this case the request for funding was made direct to the Minister's office. We found no evidence of an assessment against criteria having been completed to support the recommendation for ministerial approval.

Financial Management and Control

Lack of consistency in contribution agreements

11.251 Contribution agreements were in place for all projects, and persons with the appropriate delegated authority had signed them. However, there was a lack of consistency in the agreement templates used across the YIC program. Some projects used the agreement template of Youth Employment Initiatives, but many used agreements designed for other programs, such as Targeted Wage Subsidies, Job Opportunities for Youth, and the Canadian Jobs Strategy. This resulted in terms and conditions that differed among contribution agreements, for no apparent reason. For example, we found that in some agreements there was no reference to such standard provisions as those relating to conflict-of-interest codes and the need to repay debts owed to the Department.

11.252 Where agreements were amended, most amendments were supported by appropriate rationales.

Deficiencies in the financial management of small and large projects

11.253 Youth Internship Canada is a contribution program. The payment of federal funds is therefore dependent on the meeting of performance conditions and on compliance with the terms and conditions of the contribution agreements. This requires appropriate financial management and controls, including claims from sponsors to support the payments made, review of appropriate information to support the approval of payments, and approval of payments provided in accordance with the requirements of the Financial Administration Act.

11.254 We found that project payments were not handled properly in 60 percent of the smaller projects and 58 percent of the larger projects. These included payments that were not based on claims, expense claims that were not reviewed and certified by appropriate officers, and expenditures that were reimbursed without evidence that supporting information had been reviewed.

11.255 Advances were made to 70 percent of larger projects. In 28 percent of these projects, we found deficiencies in the handling of advances. These included advances that were not in accordance with cash flow forecasts. In addition, funds continued to be advanced to some projects even though the actual expenditures or claims were well below the amounts already advanced. Where large amounts were advanced - in a departure from the Treasury Board policy - the interest due the Crown was not accounted for. We also found cases where money was advanced inappropriately at year-end, apparently to avoid lapsing the funds. Our analysis of monthly YIC expenditures shows a spending increase in March of each fiscal year from 1997-98 to 1999-2000 (see Exhibit 11.17).

Activity and financial monitoring was not systematic or rigorous

11.256 In the larger projects, we expected to see rigorous and systematic monitoring of projects, in accordance with monitoring plans based on assessed risk. We expected monitoring to include activity and financial components - activity monitoring to ensure that the project was achieving planned results and was likely to conclude successfully, and financial monitoring to ensure that funds were being spent according to the terms and conditions of the agreement. We also expected proper documentation of on-site monitoring visits. In smaller projects, we did not expect to see anything more than off-site review of relevant documentation.

11.257 We found that:

almost none of the larger projects had monitoring plans;
on-site monitoring (consultation with sponsors, review of financial records and books) was not evident in 62 percent of the larger projects, and 29 percent of those did not show evidence of even off-site monitoring (review of progress and activity reports, correspondence, records of telephone calls); and
only half of the smaller projects had evidence of off-site monitoring.

Project Results

11.258 Information on the performance of projects is needed to draw lessons for future project selection and to provide a basis for accountability within the Department and beyond.

11.259 YIC measures its success by two indicators - namely, youth clients who are employed after the internship, and youth clients who return to school. We expected to see information on actual results against planned targets documented in each project file.

Limited information on results against targets

11.260 Planned results of projects were generally defined in measurable terms (that is, the number of participants expected to acquire employment as a result of the internship or expected to return to school). However, the information on actual results was limited. In 59 percent of larger projects and 86 percent of smaller projects, the project files did not record actual results. For 58 percent of larger projects, we also found such information was missing from the departmental information system that, among other things, records the results of projects. Where results were available in the information system, they were not consistent with those in the project files. Furthermore, project officers told us that they did not have ready access to integrated information in the Department's information systems to help them manage their projects.

11.261 Information on results needs to be input regularly to departmental information systems. Project officers also require on-line access to information to track the project's status and performance. Many told us that an integrated system is needed to collect and disseminate performance information as well as financial information for sharing at local, regional and national levels.

11.262 We were concerned to find that in three cases among the seven high-value (over $606,000) projects we reviewed, information on results achieved was not recorded in the project file. Further, in one of these cases the departmental information system had no information on results (see Exhibit 11.18).

Effectiveness Measurement and Reporting

Recently completed evaluation shows some positive results

11.263 A program evaluation of Youth Internship Canada was completed in August 2000. The evaluation reported some positive findings. Compared with a matched group that had not participated in the program, program participants recorded increased employability and earnings in the year following their participation. Employability was reported up by 11 percentage points and annual earnings up by about $3,000.

11.264 A large majority (85 percent) of participants indicated that in the month preceding their internships they had been very interested in finding full-time employment. However, 68 percent of participants said they could have found employment without participating in the internship, and 49 percent said they could have found a job closely related to their career goals. Of the 37 percent who had been employed in the week immediately before the internship, 62 percent considered themselves to have been underemployed.

11.265 The evaluation also reported that 79 percent of the participants met at least one of the following conditions for program assistance:

did not graduate from high school;
were unemployed immediately prior to the program; or
considered themselves to have been underemployed in the week prior to the internship or in the previous 12 months.

11.266 The evaluation's approach was based on analysis of a comparison group. The survey of the comparison group showed a lack of awareness about the availability of the YIC program. Only 19 percent said they were aware, or somewhat aware, of the program. This figure is consistent with our observations on limited promotional information (paragraphs 11.242 and 11.243).

11.267 We reviewed the approach used in the evaluation and concluded that it was sound. However, our audit identified two key areas that we believe deserve to be addressed more explicitly in future evaluations:

whether the program's current procedures ensure equal access to program assistance by eligible youth clients and sponsors; and
negative unintended effects when the program is used to provide employers with the financial benefits of wage subsidies rather than to provide assistance to youth having difficulties in the labour market.

Issues at the field level

11.268 As part of our audit, we interviewed project officers and others involved at the field level in delivering YIC as well as other grant and contribution programs. Several challenges faced by project officers surfaced in these discussions.

Downsizing and devolution. When the Department was downsized and programs and services were devolved to the provinces and other organizations, many project officers left the Department. There was a corresponding loss of corporate memory and experience. Experienced project officers were sometimes replaced by displaced counsellors who had little or no experience in project management.
Direction from regional offices. Many program officers at regional offices also left. In some cases, regional officers were not available to provide advice to local HRCC project officers. Where advice was given, we were told that it was not always consistent.
Lack of supervision. Also as a result of downsizing, the manager level was eliminated. This was intended to result in "empowerment" to work as teams on resolving problems. However, it left some project officers without day-to-day support, as directors were not always available to provide detailed guidance. The lack of direction and leadership only left project officers more frustrated when they were unable to resolve problems.
Training. In general, there was no mandatory training for project officers, including those without project management experience. Project officers often had to learn on the job from unofficial mentors or more experienced project officers and ad hoc training. Some of them indicated that poor practices had continued as a result.
Systems support. Information systems were not integrated to provide overall project information. Information on project status was available only through regional offices. Although local HRCC project officers input the data, they had no access to integrated information. Because project officers lacked integrated information to help manage their projects, they had to develop local systems - a duplication of effort.
Administrative support. Project officers indicated that they needed more clerical and administrative support to carry out routine tasks so they could devote more effort to managing and monitoring projects and could engage in the community networking that the program requires.

11.269 Many project officers told us that the 1999 internal audit had had some positive effects. They felt that their jobs were now perceived as more important, and they were looking forward to receiving more training. However, they were also frustrated with the workload that resulted from the Department-wide file review. In some cases, they had to put applications on hold until the file review was over. Some project officers were demoralized because they feared that they would lose some of their responsibilities, and some feared impending changes that could result in restructuring and reclassification of their jobs. More important, they felt they were being blamed for the mismanagement of grants and contributions, when in many cases it had been beyond their control.

Concluding Comments

11.270 We concluded that there were significant weaknesses in the management control framework for Youth Internship Canada in the period up to December 1999. Assessments of funding requests, which would have served as a sound basis for selecting the projects most likely to achieve the program objectives, were frequently missing. There were serious deficiencies in the financial management and control of projects, including lack of compliance with Treasury Board policy and requirements of the Financial Administration Act. In a large proportion of cases, project payments and advances were not handled properly and projects were not monitored adequately.

11.271 Planned project results were generally defined in measurable terms, but in many YIC projects, information on results was not on file or was otherwise not available. An evaluation of the program was completed in August 2000 and it reported some positive results. We concluded that the methodology used in the evaluation was sound.

Action to address the deficiencies

11.272 We audited YIC projects that had been completed by the end of December 1999. Therefore, we did not expect to see the effects of the corrective action the Department took under its Six-Point Action Plan and under other initiatives following its 1999 internal audit. However, our assessment is that the corrective action the Department has taken and plans to take addresses the deficiencies we found in our audit of YIC projects. We report the results of our detailed examination of the Department's corrective action in Part I of this chapter.

Social Development Partnerships

Program Description

11.273 The Social Development Partnerships Program (SDPP) is a research and development program that supports activities of the social not-for-profit sector. The program comprises several subprograms that provide funding to volunteer social service organizations and disability organizations assisting disadvantaged individuals.

11.274 The SDPP provides funding to projects that develop and test models designed to improve individuals' capacity for full participation in society and to strengthen related services provided in communities. The program also funds projects that develop and distribute information to client groups and key decision-makers at the policy, program development and delivery levels. Finally, SDPP provides organizational support funding for specified periods of time to selected national volunteer organizations that support activities consistent with HRDC's mandate.

11.275 The SDPP uses grants as well as contributions. Most of the support it delivers is in the form of contributions to projects that focus on developing social service models and delivery or developing capacity in the voluntary sector to represent marginalized and vulnerable communities and provide them with social services. Grants are provided to help national social service and national disability organizations build their capacity to promote their members' interests.

11.276 The total program expenditure on grants and contributions in 1998-99 was $14.4 million ($5 million in grants and $9.4 million in contributions). The budget for 1999-2000 was $15.1 million ($5.6 million in grants and $9.5 million in contributions).

Program Design

11.277 In February 1998, the Treasury Board approved the terms and conditions of the Social Development Partnerships Program. The terms and conditions identify activities and classes of recipients that are eligible for funding support. In addition, they outline provisions dealing with eligible expenses, maximum funding support and methods of payment, along with standard requirements for audit, evaluation, and accountability.

11.278 These new terms and conditions consolidated and streamlined the elements of the various program authorities that had previously existed under other federal departments. Until 1993, the National Welfare Grants program operated within the Department of National Health and Welfare; the Disabled Persons Participation Program was under the mandate of the Department of the Secretary of State. With the reorganization of government in 1993, these programs fell under the mandate of Human Resources Development Canada.

11.279 Many of the projects we examined in our audit had begun under these previous programs and continued to operate under the related authorities and guidelines, rather than the terms and conditions approved for the SDPP in 1998.

Reasons for using either a grant or a contribution are not clear

11.280 We found that the program's terms and conditions do not clearly identify situations where a grant would be more appropriate than a contribution. We noted, for example, that building a community's capacity to meet social development needs was funded through contribution agreements. At the same time, building an organization's capacity to assist the disadvantaged was funded with grants. The reasons for this difference are not clear.

11.281 Because grants are unconditional payments, they involve less stringent requirements for accountability (see Part I of this chapter, paragraph 11.13). We therefore believe that grants need to be reserved for situations where it is demonstrably appropriate for transfer payments to be unconditional. The program material provided in response to our specific questions did not contain any information on the rationale for using grants instead of contributions in any of the activities under this program.

Operational guidance to project officers is lacking

11.282 While classes of eligible recipients are clearly identified in the program's terms and conditions, eligible activities are described only in very broad terms. For example, the terms and conditions state, "Activities eligible for support under the program include a wide range of pursuits and include, but are not limited to, applied research, development, and capacity building." The statement is essentially open-ended, with no clear limits on activities that can be funded. HRDC officials told us that the eligibility criteria were intentionally formulated to be broad because of the inherent difficulty of defining concepts such as community capacity, as well as the desire to allow for negotiation with project applicants.

11.283 Except for the program's formal terms and conditions, the only guidance available to project officers is contained in guides for project applications. These tell applicants how to make project proposals and submit applications. In our view, the guides are not adequate substitutes for operational guidance aimed specifically at helping those who administer the program, partly because they do not cover the range of administrative issues involved. This was confirmed in our detailed discussions with project officers about the contents of individual project files and related program activities.

11.284 The project application guides refer to additional "funding priorities". For example, one of the funding priorities relates to "social services or social development projects that target vulnerable populations and are national in scope"; these include "research, public education, successful models, pilots, advocacy, and conferences." Key concepts are mentioned, such as social services and social development, but are not clarified. The guide for project applications by national organizations states, "Projects need to be...consistent with HRDC guidelines and limitations" - but it does not make clear what these guidelines and limitations are.

11.285 While we recognize the need to maintain a degree of flexibility, we believe that operational guidance directed specifically to project officers would help ensure that eligibility criteria are applied with discernment to achieve program objectives.

Program objectives are not defined in terms of results

11.286 The program's terms and conditions include a statement of its objectives. The SDPP is described as a research and development program that supports activities to identify, develop and promote nationally significant best practices and models of service delivery. In addition, the program aims to build community capacity to meet the social development needs and aspirations of populations that are or may be at risk.

11.287 The description of program activities is part of the objectives, and the objectives therefore suffer from the same lack of clarity as the eligibility criteria. We expected that HRDC would have defined the broad program objectives further, in terms of outcomes and measurable results. However, it has not done so.

11.288 The terms and conditions state that the objectives are to be achieved through a review and accountability framework, and in partnership with the volunteer sector. We expected that a framework would have been developed as a first step in the program's operation, providing more detailed, concrete implementation guidelines and performance indicators.

11.289 The Treasury Board asked HRDC to report, within two years after the date of approval of the terms and conditions, on the appropriateness of accountability measures developed for the grants portion of the program. At the time of our audit, the Department's response to this request was overdue. The Department told us that it intends to establish and report on accountability measures as part of developing and implementing an evaluation framework for the SDPP. Preliminary work on developing that framework was under way when we completed our audit.

Program marketing is informal

11.290 The program is promoted through the Internet and by a network that includes provincial governments, national client organizations and the academic community. As noted, application guides have been developed for each of the SDPP subprograms, describing how potential applicants can apply for financial support. For some subprograms, the Internet site includes the guides; for others, an address and telephone number are provided for further information.

11.291 The predominant characteristic of the program's marketing framework is the informality of much of the activity. For example, we heard in our discussions with project officers that some potential applicants had been invited informally to submit applications for specific projects. Sometimes this followed joint consultative seminars, particularly when projects involved research, development and testing. In our view, this informal approach to marketing does not provide adequate assurance that all eligible clients have equal access to the program.

11.292 At the time of our audit, public information about eligibility for SDPP organizational funding was inconsistent and had the potential to reduce access to that funding. This inconsistency has been rectified.

Contributions - Selecting and Approving Projects

Proposals inadequate in almost one third of projects

11.293 We concluded that just under one third of project proposals (29 percent) were inadequate; for 16 percent of the projects there was no proposal on file. Where proposals had been received, some lacked key information needed to proceed with a realistic assessment.

No record of assessments for most projects

11.294 We expected that appropriate procedures would be established to ensure that project applications and proposals recommended for approval were those that met the eligibility criteria and were the most likely to achieve the program's objectives. We expected that once established, these procedures would be documented and followed. The Department informed us that officer-level committees conducted assessments and reviews, and we found some indications of an assessment process. However, the files on 85 percent of projects showed no assessments of any kind. This is an important omission: it means that the expenditure of public funds was authorized without any record of how the projects met the eligibility requirements.

11.295 Even where assessments were on file, we had concerns about their adequacy:

some 58 percent of the assessments did not explicitly address criteria for program eligibility and project selection;
similarly, 58 percent of the assessments did not assess the reasonableness of estimated project costs; and
with few exceptions, checks of sponsors for other contracts with HRDC, including any amounts owed to the Department, were not conducted.

11.296 All project files contained recommendations for approval by the Minister. However, those recommendations were not supported by any rationale in 94 percent of the projects.

Contributions - Financial Management and Control

Contribution agreements frequently lacked key information

11.297 The contribution agreements signed by the Department and the project sponsors are important documents because they identify the details of the performance requirement and the conditions that are to be met in order for the sponsor to be reimbursed for eligible expenditures.

11.298 We found that appropriate signing authorities for contribution agreements were respected in all cases. However, we identified several concerns about the agreements themselves:

although all projects had signed agreements in place, one in five agreements did not include the necessary schedules (these schedules contain important information on performance requirements and budget details);
key information on allowable expenses was missing from all agreements. Although the agreements make reference to the program's terms and conditions (which clearly identify eligible expenses), those terms and conditions were not always available to project sponsors; and
a small proportion of agreements did not include an exact date for the project's start or end.

11.299 Forty-five percent of the agreements were amended to reflect a change in either the project's budget or its duration. Most amendments were supported by rationales and in all cases were approved by persons with the necessary authority.

Deficiencies in handling payments and advances

11.300 We expected that financial and management control of program expenditures would comply with relevant government policies and legislation. In this respect, we identified a number of serious deficiencies.

11.301 In 30 percent of projects, payments were made that were not based on claims. In addition, payments were not always supported by a review of appropriate information (proof of expenditures). In 21 percent of projects, final payments were approved before final claims or final project reports had been received. However, in all projects, staff with the appropriate delegated authority approved payments.

11.302 Advances were made to 63 percent of projects; in 93 percent of those cases, we found that the advances were not handled appropriately. Problems included advance payments that did not correspond to cash flow forecasts or that exceeded the maximum amount allowed by the agreement, and a lack of appropriate accounting for interest earned on advances.

Inadequate project monitoring

11.303 We expected that approved projects would be monitored appropriately and reported on the basis of an assessment of the risks involved. We expected to see monitoring systems in place to identify and report on a timely basis on progress and emerging problems.

11.304 We concluded that monitoring had not been adequate in 97 percent of contribution projects. We found almost no evidence that financial or activity monitoring had been carried out.

Contributions - Project Results

Information on results was inadequate in half the projects

11.305 We expected to see adequate performance measurement procedures in place, in keeping with HRDC's approach to "managing for results" and in keeping with the requirement to account for results achieved with the funds approved by Parliament. However, we found that results at the level of individual projects were not being tracked adequately.

11.306 Information on the results of half of the projects was missing or inadequate. In almost all of these cases, project activity reports were not submitted as required by the provisions of the contribution agreement. In addition, in 32 percent of these projects we found no evidence on file that HRDC had received a final project report.

Grants - Project Selection and Approval

Grant proposals not assessed in all cases

11.307 With one exception, all grant proposals were on file. The large majority of files (80 percent) included the information needed to assess the project against the eligibility and selection criteria. However, 58 percent of the files contained no clear evidence that proposals had been assessed before they were recommended for approval.

11.308 Among those that had been assessed, we had a number of concerns about the adequacy of the assessments. For example, assessments did not explicitly address all eligibility criteria; they did not address the reasonableness of the amount requested; and they did not include a check on recipients for other contracts with HRDC and any amounts they may still have owed the Department.

No rationales for recommending the approval of projects

11.309 All recommendations for approval of the grant applications were on file and the Minister had approved all the grants, as required. However, in no case was a rationale provided for recommending that the grant application be approved.

Grants - Financial Management and Control

Grant payments were made appropriately

11.310 Persons with the appropriate delegated authorities approved all grant payments. We found also that grants were paid in compliance with Treasury Board policy, which requires a certain minimum number of instalments, depending on the total amount of the grant.

Improvement needed in reviews to determine continued eligibility

11.311 According to the terms and conditions set by the Treasury Board, each grant can be approved for a period of up to three years, subject to an annual review to determine continued eligibility. We identified several areas that required improvement. There was a lack of evidence that reviews were conducted on an annual basis, as required. When evidence of reviews was on file, we found that the reviews did not adequately address all of the eligibility and selection criteria. Finally, 42 percent of the grant recipients failed to submit complete reports substantiating their continued eligibility.

Effectiveness Measurement and Reporting

11.312 Over and above the required tracking of ongoing performance at the project level, program evaluation is intended to provide a longer-term perspective on the results of the program overall and on the achievement of program objectives.

The program has not been evaluated

11.313 The Social Development Partnerships Program has not been formally evaluated to date. Program management has undertaken some preliminary work, identifying the activities of the organizations that SDPP assists, and assessing their linkages to the program's objectives.

11.314 During our audit, program management made arrangements with the Department's Evaluation and Data Development Branch to prepare an evaluation framework for this program. The work is scheduled for the current fiscal year, and preliminary work had begun when we completed our audit. We believe it is important that the evaluation framework include an examination of the rationale for apportioning program expenditures between grants and contributions (see paragraphs 11.280 and 11.281).

Concluding Comments

11.315 We concluded that there were deficiencies in the management control framework of the Social Development Partnerships Program in the period up to December 1999. These weaknesses were more serious in the management and administration of contributions, but they extended also to grants. However, in both the grants and contributions components of the program, signing authorities were exercised in compliance with the requirements of the Financial Administration Act.

11.316 The program's objectives are broad and had not been defined in terms of outcomes and measurable results. Results at the project level were not being tracked. An evaluation framework was being developed, but an evaluation of the program has yet to be carried out. Overall, little information on performance was available to support accountability and managing for results.

Corrective action addresses deficiencies

11.317 We audited SDPP projects that had been completed by the end of December 1999. Therefore, we did not expect to see the effects of the corrective action the Department took under its Six-Point Action Plan and under other initiatives following its 1999 internal audit. However, our assessment is that the corrective action the Department has taken and plans to take addresses the deficiencies we found in our audit of SDPP projects. We report the results of our detailed examination of the Department's corrective action in Part I of this chapter.

Sectoral Partnerships Initiative

Program Description

11.318 The Sectoral Partnerships Initiative (about $30 million annually) is a program launched in 1993 to fundamentally change human resources practices in Canada. It would do that by building, through partnerships, the capacity of the private sector to address strategic, sector-wide issues of national human resource planning, sectoral adjustment and job skills development. The initiative consists of eight activity streams and one subprogram (see Exhibit 11.19) managed from HRDC headquarters.

11.319 The objectives of SPI are to develop permanent, industry-led partnerships that co-ordinate human resource management in major industrial sectors, improve the relevance of the learning system, foster a learning culture, support interprovincial mobility, and contribute to labour market information. Since 1986, the government has provided over $200 million to support sectoral partnerships in some 40 industrial sectors.

Program Design

11.320 SPI has used two sources of funding for projects. Agreements signed before 1996 were negotiated under terms and conditions approved by the Treasury Board for the Canada Jobs Strategy (CJS) and funded from annual appropriations. In 1996, Parliament passed the Employment Insurance Act, enabling the Minister of HRD to use the Employment Insurance (EI) Account for projects benefiting insured participants. As of July 1996, SPI has used authority under terms and conditions approved by the Treasury Board for the Labour Market Partnerships component of Employment Benefits and Support Measures (EBSM) to fund projects from the EI Account. Since November 1997, SPI has also had authority under the Human Resources Partnership Initiative to use annually voted appropriations to fund projects that benefit persons not insured under EI, but this authority has never been used.

11.321 Since July 1996, most SPI projects have been funded from the EI Account. However, HRDC has also continued to use Canada Jobs Strategy authorities since 30 June 1996 to fund new projects from annual appropriations (29 percent of our audit sample).

11.322 Under the Canada Jobs Strategy, sole authority to approve Sector Study, Sector Council, Occupational/Skills, Skills Upgrading and Nationally Managed projects lay with the Minister. Authority to approve Delivery Assistance, Rural and Remote, and Industrial Adjustment Service projects (renamed National Sectoral Adjustment Service or NSAS) was delegated by the Minister to senior departmental officials.

Some activity streams have clear objectives and criteria but others do not

11.323 There are three fundamental eligibility criteria for all SPI projects: the organization must be national in scope, it must involve a partnership of all key sector stakeholders (business, employees, subsectors, regions), and the project must address a labour market issue. SPI assistance to a national sector usually begins with a sector study to define sectoral human resources problems while enabling HRDC to build a partnership with key sector participants. Often, this leads to stakeholders developing a consensus on solutions and priorities, identifying further development work in collaboration with HRDC, and creating permanent sector councils to take responsibility for human resource issues in their sectors. HRDC funds these partnerships until they become financially self-sufficient. Subsequent development work for the sector may result in Occupational/Skills and Skills Upgrading Agreements with councils or other sector groups. SPI supports interprovincial mobility in provincially regulated professions, through Labour Mobility Partnership Agreements negotiated with sector consortia. The objectives and core eligibility criteria specific to these five activity streams are clear.

11.324 Nationally Managed Projects Agreements have no specific eligibility criteria beyond the three fundamental criteria for all SPI activities; nor do they have stated objectives. Agreements under the National Sectoral Adjustment Service (NSAS) subprogram made up 51 percent of our audit sample. The objectives of NSAS are not clear, and its eligibility criteria are fairly open-ended.

Project Selection and Approval

11.325 Section 58 of the Employment Insurance Act specifies that partnership arrangements are to be developed and used for the conduct of project work. For SPI, the Department has translated this requirement into the sector council concept. A committee of sector stakeholders oversees the sector studies that define the nature of the government's involvement in the sector. Program managers advised us that sector studies inspire subsequent projects with partners and form the basis for assessing them, and that these studies also form the basis for the SPI annual work plans for each sector.

11.326 Our audit focussed on individual project files and their administration. We did not assess the validity of the sector studies themselves. Even allowing for the use of sector studies to guide other project work, we found weaknesses in the project selection and approvals process.

Project proposals not well documented

11.327 In 42 percent of the projects we audited, there was no identifiable proposal on the project file. This was true for both lower-value and higher-value projects. Project officers advised that in some cases proposals would not be identified as such because they were submitted in the form of draft agreements. When proposals are in this format, it is difficult to tell who originated them.

11.328 Of the files that contained identifiable proposals, few had adequate supporting documentation and analysis; 86 percent were missing the required cash flow forecast; and 43 percent had no timelines. Other gaps included information missing on objectives, budgets and work plans.

Missing or inadequate assessments

11.329 SPI management advised that before March 1998, project officers negotiated agreements with sponsors and brought SPI proposals to the Agreement Review Committee (ARC) for Branch review. SPI management found this process unsatisfactory because sponsors and project officers became overly committed to proposals - given their heavy investment of time during the project development phase - and directors then found it difficult to turn down projects.

11.330 In March 1998, SPI established a new project review process whereby project officers submit proposals along with their own assessment (usually in the form of a concept paper) for vetting by an extended management team. These teams, which include other project officers and directors, may recommend that the sponsor develop the proposal further or that it be approved.

11.331 We found that a little over 60 percent of the project files contained documented assessments. Among the files that did contain documentation, none had a timely risk analysis or a mandatory environmental impact prescreening. Although many sponsors had multiple agreements with SPI, we found that sponsors had not been checked for outstanding debt to HRDC.

11.332 Among the projects that we audited, only one assessment referred explicitly to the project's having been identified in a previous sector study.

Loose application of eligibility criteria

11.333 Applying the core activity stream criteria to the projects we reviewed, we found that core eligibility criteria had been applied only loosely; few projects clearly met all the core eligibility criteria of the specific activity stream selected to fund them. We found that 16 percent of the projects in our sample (totalling $662,000) were not eligible for funding under the criteria of the particular SPI activity stream selected for each.

11.334 Activity streams selected to fund projects were often interchangeable; some projects could have been funded under a number of different activity streams. In some cases, when one agreement ended, the project activity would continue to be funded under a different SPI activity stream or subprogram.

Some approvals not authorized; some not documented

11.335 SPI management advised that until 1998, the Agreement Review Committee assessed proposals and approved projects where authorized, or recommended their approval by the Minister. As of 1998, the extended management team recommends a project's approval to a directors' meeting. Directors then approve the project (where authorized), reject it, or recommend its approval to an assistant deputy minister, the Deputy Minister or the Minister as appropriate, according to the Minister's delegation of program authority.

11.336 It was not apparent from the documentation on file that the process operated as described. Of the projects requiring the Minister's approval, only 40 percent had clearly obtained it. Another 40 percent either had been approved by unauthorized officials or had no record of who actually approved the project. For the remaining 20 percent, officials believed they had continuing authority from the Minister when agreements that had been funded under annual appropriations were converted to new agreements funded from the EI Account.

11.337 Of the projects that did not require the Minister's approval, the actual approval of only one in five was documented. SPI management advised that the Agreement Review Committee had documented its decisions but a computer crash in 1998 wiped out those records. The directors' committee that replaced the Agreement Review Committee did not document decisions until very recently.

Financial Management and Control

Agreements often missing key information

11.338 Written contribution agreements were in place for all projects. SPI used standard formats that complied with Treasury Board guidance on specifying clear roles and responsibilities, terms of payment, eligible expenses and other relevant matters.

11.339 We found that only 41 percent of agreements were fully adequate. Fifty-four percent were missing key information (for example, funding periods, treatment of interest, mandatory separate bank accounts, and adherence to rules on conflict of interest) or included contradictory information. Authorized officials had signed 95 percent of the agreements.

Amendments were rarely made properly

11.340 When an agreement is amended, it is essential that both parties understand and agree on roles and responsibilities under the new conditions. In two thirds of amended agreements, the reasons for the amendments were not stated clearly. Furthermore, we were rarely able to identify who had actually been responsible for approving the amendments.

11.341 Most amendments consisted only of a new financial commitment form signed by a director or, occasionally, by unauthorized project officers. Only 20 percent of amended agreements contained proper articles of amendment signed by both parties and indicating their mutual agreement. Thirty-eight percent of amendments were made inappropriately to extend the funding period after the agreement had already ended.

Advances not always handled appropriately

11.342 In 78 percent of projects where advances were made, not all advances were based on projected or revised cash flows provided by sponsors. Most commonly, initial advances were based on cash flow estimates developed by HRDC officers for financial commitment forms.

11.343 In only 33 percent of cases did the payment of advances clearly follow Treasury Board rules on amounts that could be advanced. Of those projects that received multi-year funding, only a third clearly met Treasury Board rules on providing advances into a new fiscal year. We found in all but one case that advances were cleared by claims before further advances were issued. All requisitions for advances were certified by authorized officials under section 34 of the Financial Administration Act.

Failure to comply with terms of contribution agreements

11.344 SPI officers did not always hold sponsors to the terms and conditions of their contribution agreements or of the activity streams under which the projects were funded.

11.345 Separate bank accounts. SPI managers advised that they frequently waived specific contractual obligations to maintain a separate bank account for each project. Such accounts facilitate the tracking of expenses and revenues and, more important, they allow for the tracking of bank interest paid on federal funds held by sponsors and preclude the claiming of the same expenses under other agreements. In our audit, we were unable to determine the proportion of projects that had maintained separate bank accounts.

11.346 Eighty-seven percent of SPI agreements specified that interest earned by sponsors on funds advanced by the government had to be accounted for as part of the HRDC contribution. In 69 percent of the projects where this was required, it was not clear that interest had been handled appropriately - especially when sponsors had declared no bank interest. Our interviews with project officers provided no assurance that, where appropriate, sponsors were accounting for the interest earned on federal funds in their care.

11.347 Cost-sharing capital purchases. The terms and conditions of the umbrella Employment Benefits and Support Measures program under which most SPI agreements are negotiated require sponsors to cost-share any capital purchased. We found no cases that met this requirement. In the EBSM-funded portion of the dormant SPI projects we sampled, we found the value of capital purchased exclusively with federal money to be $250,000.

11.348 Eligible costs. Among other things, Schedule B of contribution agreements specifies eligible costs and the dollar amounts to be allocated to each eligible expenditure category. Reallocation among major cost categories requires HRDC approval. We found that agreements were rarely amended to authorize sponsors to exceed the agreed cost categories. As a result, 69 percent of the projects had overages in cost categories. In the remaining 31 percent it was not possible to tell whether there were overages, usually because the categories specified in Schedule B did not match the categories on the claim forms that sponsors used.

11.349 Detailed expense claim information on file was very limited, especially for older projects whose sponsors had been required to submit only the most rudimentary supporting documentation - often containing only summary figures by approved cost category. On the basis of the limited supporting information available, we found that for the 49 dormant SPI projects we sampled:

HRDC paid at least $284,000 for costs incurred before or after the funding period covered by agreements (see Exhibit 11.20);
expenses amounting to $463,000 had been allowed through amendments made inappropriately after agreements had ended;
HRDC paid $59,000 for costs in categories not covered by the agreements - such as administration costs that the agreement required the sponsor to pay, equipment rentals, ineligible overhead expenses and profits; and
HRDC paid $179,000 for ineligible expenses in approved cost categories.

(The treatment of these kinds of administrative practices is discussed in paragraphs 11.77 to 11.80.)

11.350 In 92 percent of projects at least some expense claims were certified under section 34 of the Financial Administration Act by officers who did not have the necessary authority.

11.351 Reimbursement of GST. HRDC does not reimburse sponsors for GST that they can recover from the Canada Customs and Revenue Agency (usually 50 percent of GST paid). Sponsors are required to account for the amount of GST they can get back. In 56 percent of projects, it was not clear that GST had been treated appropriately. In many claims, the sponsor did not state the total amount of GST that had been paid. We found other claims that charged HRDC the full amount of GST paid. There are items on which a sponsor might not pay GST (for example, on wages) or an unincorporated group might not be able to get a GST refund. However, few project officers we interviewed could assure us that sponsors had properly accounted for reimbursable GST.

11.352 Cost-sharing of projects. A significant feature of sector participation in SPI is that partners share in project costs. Partner cost-sharing contributions can be in kind (for example, partners' time or facilities) or in cash (directly paid to cover eligible expenses). The agreements specify a maximum dollar amount that HRDC will contribute. Most also specify a maximum percentage of total project costs as the federal share and stipulate that the federal contribution will be the lesser of the percentage share and the maximum dollar amount. For clarity, many agreements also specify the otherwise implicit minimum contribution of partners, in dollar amounts and in percentage share.

11.353 We found that in 29 percent of projects, partners failed to deliver the minimum contribution they had agreed to. In the 49 sampled projects, HRDC had paid $889,000 more than the maximum specified without formally amending the agreements. SPI management asserts that the requirement for minimum contributions by partners is not meant to be enforced and is only leverage to ensure their participation.

Inadequate financial monitoring

11.354 The SPI Program Guidelines (1994) state that an initial monitoring visit should take place within four weeks of the start of a project. Interim visits should take place as necessary, depending on the project's complexity and size, sponsor experience, and political volatility. A final visit must be made at the end of the project and before final payment is requisitioned. We expected to see a monitoring effort commensurate with the assessed risk of individual projects.

11.355 We found that there was no monitoring plan for 96 percent of projects, nor were the projects monitored in a timely manner, consistent with their risk. Although desk monitoring of claims is useful, it is primarily through on-site financial monitoring visits that officers are able to examine detailed receipts and truly verify claims. In 29 percent of all projects (45 percent of projects under $150,000) there was no on-site financial monitoring visit by project officers. Of projects where financial monitoring visits were made, 59 percent had only one visit, after the end of the agreement and after the final payment had been requisitioned. Exhibit 11.21 shows the on-site monitoring done for the three highest-value projects in our sample.

11.356 Program managers told us that in light of the ongoing relationship between project officers and sponsors, SPI relied heavily upon desk reviews of individual claims (often supported by a copy of the sponsor's general ledger) as an alternative to on-site financial monitoring visits during the project. Given the nature and extent of the financial management problems we identified in our audit, we believe that desk monitoring of claims coupled with a single financial monitoring visit after the project has ended and all payments have been requisitioned is insufficient to ensure that moneys are used for intended purposes.

11.357 The Department's monitoring discovered some inappropriate payments and some were corrected. However, in many cases the sponsor would merely be warned against claiming ineligible expenses in future projects or advised to reimburse HRDC through another agreement. In our view, this practice is not appropriate. Files on 67 percent of the projects that required a minimum contribution from the sector contained no detailed documentation or verification of the in-kind and in-cash claims made by the sponsor. None of the projects were closed properly (that is, advances reconciled with claims, claims and revenues verified, ineligible expenses identified, overpayments recovered, all reports completed), and only 18 percent had any activity monitoring reports on file.

Project Results

Project results not documented

11.358 The agreements we audited did specify planned outputs. However, we found that information on results was inadequate in 91 percent of the projects. In 68 percent, we found no indication that the required outputs had been delivered; in 85 percent, sponsors failed to provide the mandatory final project report.

11.359 We found on file very few final assessment reports written before the events of January 2000, when management tightened reporting requirements. There was little evidence on file that SPI management knew what results project sponsors were achieving.

Effectiveness Measurement and Reporting

The Department has taken some action in response to a 1997 evaluation

11.360 An evaluation report on the SPI was published in November 1997. The evaluation methodology involved interviews, a search of the literature, case studies and surveys. In our view, the methodology was appropriate to the nature of the program results examined by the evaluation.

11.361 The evaluation found that programs sponsored by sector councils were bringing about systemic change - developing a training culture, increasing employers' interest in training and standards, increasing skill levels in the workforce, and facilitating school-to-work transition. There was some evidence of limited improvements in productivity and profitability. The evaluation found that the approach resonated well with the general public, that employers and employees supported the sectoral partnership approach, and that the rationale for SPI had not diminished.

11.362 However, the evaluation also pointed out that SPI agreements contained few statements on intended impacts and effects, and there were few provisions in the agreements for ongoing data collection and evaluation. Indeed, the evaluation noted that sponsors did not see themselves as accountable for the intended impacts and effects of the SPI; there was a lack of sector support for accountability and a lack of relevant data on results. The evaluation called for stronger reporting requirements to help clarify commitments to results, and for strategies to measure the performance of sectors in contributing to SPI's objectives.

11.363 The evaluation stressed the need to communicate the activities and experiences of individual sector councils in order to avoid duplication and to strengthen the efforts of all sector councils. Marketing of sector council products and services was notably weak and too often not timely. The evaluation found that both awareness and the take-up of the program were still low, and the participating sector councils had made only slow progress toward self-sufficiency. The three-year time frame for achieving core operational self-sufficiency appeared to be unrealistic for most sectors.

11.364 SPI subsequently attempted to resolve the cross-sectoral communications issue, in part by further encouraging the work of the Alliance of Sector Councils. To address the self-sufficiency of sector councils, SPI restructured its funding model by eliminating the formula of declining funding (thereby providing full funding in the start-up and early operational phases), by providing core funding for a longer period (up to five years instead of three years), and by requiring councils to build reserve funds that would eventually be the basis for their financial independence. SPI also made provision in subsequent agreements for the marketing of sector council products. Responses to the evaluation recommendations on results measurement and the accountability of sponsors are still at the planning stage.

Concluding Comments

11.365 We concluded that there were serious deficiencies in the management control framework of the Sectoral Partnerships Initiative program. We also found lack of compliance with the legal requirements of section 34 (certifying expense claims) of the Financial Administration Act and with the Treasury Board's policy on the treatment of advance payments.

11.366 The Department's evaluation of SPI found progress in the development of partnerships but significant weaknesses in sponsors' provision of information that is needed to measure project results. Our audit supports this evaluation finding. Although SPI has been planning improvements in collecting information on results from sponsors, these improvements have yet to be implemented in agreements.

Corrective action to address deficiencies

11.367 We audited SPI projects that closed by the end of December 1999. Therefore, we did not expect to see in those projects the effects of the corrective action that the Department took under its Six-Point Action Plan and under other initiatives following its 1999 internal audit. However, our assessment is that the corrective action the Department has taken and plans to take addresses the deficiencies we found in our audit of SPI projects. We report the results of our examination of the Department's corrective action in Part I of this chapter.

11.368 SPI management states that in managing the program it must balance three factors: developing and sustaining partnerships, ensuring quality products and projects, and controlling costs. However, we believe that SPI management must also address the continuing challenge of achieving and maintaining an appropriate balance between creating, advocating and sustaining partnerships in this program and meeting the responsibility for prudent management of public funds. The findings of our audit show that this latter balance had not been achieved in the period to December 1999.

Conclusion

11.369 Human Resources Development Canada operates under direction from Parliament through its Minister, from Cabinet and Cabinet committees such as the Treasury Board, and from a number of other centres in government, and guidance from the Clerk of the Privy Council as the Head of the Public Service. Decisions to downsize, reduce budgets and restructure HRDC were made outside the Department. Directions to carry out specific programs and policies come from Parliament and Cabinet. Directions for financial administration come from Parliament and more specific directions from the Treasury Board. The factors that contributed to problems in the management of grant and contribution programs were not solely within the Department's purview.

11.370 In the 1990s central direction, encouraging managers to innovate and improve service by reducing red tape, often resulted in conflicting messages to public service managers. In their reports to the Prime Minister, Clerks of the Privy Council spoke frequently of improving service, improving communication and improving staff morale - but rarely of financial management. The Treasury Board Secretariat moved away from detailed policies, relying less on pre-transaction approvals and more on general policies and guidelines and the sharing of best practices. The Secretariat received less detailed information on the activities of departments.

11.371 Initiatives within HRDC to emphasize service delivery, empower local employees, and reduce the plethora of manuals providing direction to staff were consistent with messages and actions of the Clerk and the Secretary to the Treasury Board. Neither advocated a lack of care and responsibility in spending public funds. Yet these fundamentals of public administration did not get sufficient attention. With its reduced monitoring role, the Treasury Board Secretariat was not immediately aware of the problems at HRDC in managing grants and contributions.

11.372 Over the decade of the 1990s, HRDC's capacity to manage grant and contribution programs properly declined. It lost experienced staff; guidance was not up to date; staff were not properly trained and supported; and key controls and accountability structures were weakened. Current management recognizes these weaknesses and is taking a number of actions to correct them.

11.373 Our audit of four programs confirmed and extended the findings of HRDC's 1999 internal audit. We concluded that in the period up to December 1999 there were serious deficiencies in the management control framework of each of the four programs audited. In particular, there were weaknesses in key aspects of project selection and approval processes and in the financial management and control of projects, including lack of compliance with Treasury Board policy and the requirements of the Financial Administration Act. In all of the programs, poor administrative practices resulted in inappropriate handling of project payments and advances, and in insufficient attention to project monitoring.

11.374 We found that we could rely on the 1999 internal audit to describe the systemic weaknesses in HRDC's management of grants and contributions. The internal audit was not designed to generalize its findings to program expenditures or to the management of specific programs.

11.375 Earlier in the 1990s, internal audit had twice advised management that there were serious weaknesses in the Department's approach to managing grants and contributions. But senior management did not take corrective action. This lack of management attention to reports of control weakness resulted in a more severe and widespread problem later.

11.376 Internal audit is a valuable tool for management to identify weaknesses in control systems and provide a guide to appropriate action. It is a function that management needs to protect and support. Internal audit needs to be able to work independently and to a high professional standard. And management needs to pay close attention to the findings of internal audit.

11.377 Management of HRDC is making good progress in meeting the commitments made in the Six-Point Action Plan and other plans formulated to correct identified problems. The Department is implementing the plan for staffing and equipping staff. It has developed new organizational models for project management at the field level and is currently working on filling the new positions. However, as management recognizes, effort will have to be ongoing and sustained, with careful monitoring, to effect long-lasting improvement. An innovative performance tracking system has been implemented to monitor performance and progress.

11.378 Managing grants and contributions for results needs to improve. None of the programs we audited had consistently measured project results. In TJF/CJF and YIC, project results were defined in measurable terms, but there was a lack of follow-through in their measurement and reporting. In the other two programs the objectives are broad, and have not been defined in terms of outcomes and measurable results. Although the SPI has plans to improve the collection of information on results from sponsors, these improvements have yet to be implemented.

11.379 The information that is available needs to be better used to make adjustments and improvements in program delivery and design. Program evaluations had been completed for three of the four programs we audited, although the evaluation of the TJF was preliminary and a planned further evaluation of TJF and CJF did not proceed. An evaluation framework for the SDPP is being developed, but an evaluation has yet to be carried out. The findings of the TJF and SPI evaluations were used to make some program changes. Evaluations of these programs need to examine the difficult questions of program rationale and effectiveness in order to properly inform the public debate about the value of the programs.

11.380 The Standing Committee on Human Resources Development and the Status of Persons with Disabilities asked that we address how to balance the need for efficient and flexible program delivery with the need for sound financial management. As part of the Action Plan, the Department itself is addressing this issue.

11.381 A fundamental part of dealing with the often-conflicting demands placed on public servants is ensuring that the capacity needed to deliver the programs is in place. The Department recognizes that the capacity to deliver its grant and contribution programs was not in place. It has defined a number of actions to improve its capacity. Questions remain about the appropriate level of staffing needed to deliver grants and contributions.

11.382 It is important that HRDC collaborate with the Treasury Board Secretariat to carry out workload studies and establish benchmarks for the delivery of these types of programs.

11.383 Managers and project officers need to be given flexibility to carry out their duties. But with discretion comes the need for clearly defined expectations, so there is a common understanding of what is to be delivered, and how. Increased discretion brings the requirement of strong accountability for the discretion exercised and the results achieved.

11.384 The most difficult part of balancing efficiency, flexibility and control is the judgment that public servants are asked to exercise. Not all grant and contribution programs are of equal risk, nor are all projects in a program. Judgment is needed to assess risk and determine how it should be managed. For grant and contribution programs, we expect the key controls to be in place for all projects. Yet some proposals can be assessed in a few minutes while others can take many days. In either case, public servants are responsible for the decisions they make, and those decisions should be part of the record of a project. Documenting smaller, low-risk projects would take little time; for larger projects it would involve much more. In March 2000 we published a Risk Assessment Framework for Grant and Contribution Programs to provide managers and staff with a tool for assessing and managing risk, and to provide parliamentarians with an approach to reviewing the performance of these programs.

11.385 There is no simple answer to what the ideal balance should be. As the results of our audit show, it is essential that key controls be in place and minimum standards observed. That these controls are a requirement needs to be spelled out clearly and understood by staff. Then public servants have the foundation to innovate in the delivery of programs and to strike the most appropriate balance between efficiency and flexibility in program delivery and sound financial management.

Department's summary comment: HRDC takes seriously all the issues raised in the Auditor General's report and in the 1999 internal audit of grant and contribution programs. The Department is determined to build on the improvements it has already made. HRDC will fulfil its Action Plan. It will complete the other supporting initiatives undertaken following the launch of the Plan. Furthermore, HRDC is undertaking additional steps to strengthen these actions in response to this report. HRDC will continue to work with all interested parties to ensure that it achieves the right balance between responsive client service and prudent management of taxpayers' money.

The Department's responses to specific recommendations and its overall comment are included in Part I of the chapter, following paragraph 11.149.

About the Audit

Objectives

Our objectives were to determine the following:

the extent to which reliance can be placed on the results of HRDC's 1999 internal audit and its performance tracking system;
the Department's progress in implementing its corrective action plan;
the adequacy of the Department's management control framework for the grant and contribution programs we selected for audit;
how HRDC has measured and reported the results achieved by the selected grant and contribution programs; and
the Department's compliance with financial authorities such as the Financial Administration Act and Treasury Board policies.

Scope and Approach

Our audit focussed on the management of grant and contribution programs in HRDC. We carried out a detailed audit of four programs: the Transitional Jobs Fund (TJF) and its successor program, the Canada Jobs Fund (CJF); Youth Internship Canada (YIC); the Social Development Partnerships Program (SDPP); and the Sectoral Partnerships Initiative (SPI). In addition to examining the management of individual projects under each program, we assessed the design of the programs and the measurement of project and program results.

We selected projects that, with few exceptions, were completed on or before 31 December 1999. This allowed us to audit the full project life-cycle and to make an independent assessment of the nature and extent of problems that the Department's corrective action, taken in response to its 1999 internal audit, needed to address. We selected all of the high-value projects and stratified random samples from the remaining projects in the populations we audited. Our results are accurate within plus or minus 10 percent, 19 times out of 20.

In all four programs, we reviewed the project files in light of our audit criteria. We supplemented our file reviews and confirmed our findings on each file by interviewing project officers and others involved in managing the projects. The TJF/CJF and YIC programs involved projects managed by Human Resource Centres of Canada (HRCCs) across the country. We visited most of the HRCCs responsible for the projects in our samples. For the locations that we could not visit cost-effectively, we asked that the project files be forwarded to the nearest HRCC that we would be visiting, or to Ottawa. We also visited six regional offices in eastern Canada and requested information from British Columbia.

We assessed the corrective action taken by the Department in response to its 1999 internal audit in order to determine whether that action would adequately address our audit findings in each program. We reviewed the Department's progress in implementing its Six-Point Action Plan and other initiatives that form an integral part of its corrective action.

In assessing the corrective action the Department has taken or plans to take, we reviewed HRDC documents, interviewed officials, and examined the methods the Department is using to track and report its performance in improving the management of grant and contribution programs.

Finally, we interviewed a number of former and current senior managers in the Department to obtain their views on the underlying reasons for the problems at HRDC. We also reviewed previous internal audit reports, documents on various management initiatives and minutes of key departmental committees.

Criteria

Our audit was based on the following criteria:

The objectives of grant and contribution programs should be clearly stated. Appropriate program eligibility criteria should be established that are consistent with the program objectives, the Department's legislative mandate and the requirements of central agencies.
Appropriate procedures should be established to ensure that projects selected for funding are those most likely to contribute to achieving program objectives. These procedures should be documented and followed.
Roles and responsibilities should be clearly defined and transparent.
The Department should have in place the capacity to implement the programs.
The Department should appropriately monitor approved grants and contributions and report the results. As part of this, overpayments and repayments should be identified on a timely basis.
Financial and management control of program expenditures should comply with relevant government policies and legislation.
A program accountability framework, including performance measurement and reporting, should be developed and clearly communicated. It should include assignment of responsibility for achieving results and complying with program requirements.
The Department should measure, monitor and report to Parliament on performance. Program evaluation activities should be closely co-ordinated with internal audits.
The designation of transfer programs as grants should be reserved for those situations where it is demonstrably appropriate for transfers to be unconditional.

Audit Team

Assistant Auditor General: Maria Barrados
Principals: Theresa Duk and Henno Moenting
Directors: Jim Blain, Robert Cook, Lilian Goh, Sylvie Paré and Suzanne Therrien

Denise Battala
Julie Charron
Doreen Deveen
Susan Gomez
Jayne Hinchliff-Milne
Josée Labelle
Maurice Laplante
Michelle Lavallée
Jo Ann Little
John Pritchard
Francis Séguin
Don Smith
Frances Smith
Chantal Thibaudeau
Daniel Thompson

For information, please contact Theresa Duk or Henno Moenting.