This Web page has been archived on the Web.

1986 Report of the Auditor General of Canada

Overview

6.1 The law governing employment and immigration was amended in 1977, bringing together the Department of Manpower and Immigration and the Unemployment Insurance Commission to form the Canada Employment and Immigration Commission (CEIC) and the Department of Employment and Immigration.

6.2 CEIC is responsible for employment, unemployment insurance and immigration programs, while the Department is responsible for the effective development and implementation of federal policies and programs relating to the labour market and to immigration, and for informing the public about them.

6.3 The administrative headquarters of the Commission and the Department is in Hull, Quebec. The Commission's operations are decentralized, with 10 regional offices and more than 800 permanent, temporary and mobile service bureaux. The Commission uses approximately 24,000 person-years annually.

6.4 The Department comprises two divisions, which are located at headquarters: Strategic Policy and Planning; and Public Affairs. Although most of the activities of Employment and Immigration are carried out by the Commission, the Department was set up so that there would be a minister with responsibility for the portfolio and so that the usual links with central agencies would be maintained.

6.5 In addition to a president and vice-president, Commission membership includes two commissioners, one appointed in consultation with labour organizations, the other in consultation with employers' groups. The structure allows labour and management representation in the development, approval and implementation of programs affecting them. Commission headquarters is made up of seven groups, four of which are responsible for programs providing services to the public: Employment, Insurance, Labour Market Development, and Immigration. The other three units are Personnel, Finance and Administration, and Systems and Procedures, which provide professional, technical and administrative services.

6.6 The Commission administers three programs: Employment and Insurance, Immigration, and Corporate and Special Services. The objective of the Employment and Insurance program is to foster achievement of Canada's economic and social goals by ensuring that human resources are made fully productive. The Immigration program regulates the entry of immigrants and visitors to Canada in light of Canada's economic, social and cultural interests. The objective of the Corporate and Special Services program is to provide support services to the Department and the Commission and to administer government annuities, contracts and the issuing of social insurance numbers.

6.7 Parliamentary appropriations for the Department and the Commission come from the government's Social Affairs envelope. In 1985-86 CEIC received appropriations of over $6 billion. These funds administered and disbursed by the Department and the Commission in fiscal 1985-86 are set out in Exhibit 6.1.

Exhibit not available

Audit Scope

- Employment Creation Grants and Contributions;

- Local Employment Assistance and Development; and

- National Institutional Training.

6.10 For the sub-activities selected for our audit, we examined planning, program development, implementation, monitoring and control, and evaluation. We also looked at the systems used to transmit information to Parliament and management. We wanted to determine whether these activities were being carried out economically and efficiently and whether systems were in place to measure and report on effectiveness.

Employment Creation Grants and Contributions Program

Background

6.12 In addition to being used to realize projects under this initiative, the ECGC program also served for projects under the LaPrade Fund. The government established this fund in 1982 for financing activities undertaken to counter the effect of closing the heavy water plant in LaPrade, Quebec.

6.13 Between 1982 and February 1986, CEIC funded 2,146 projects through the ECGC program at a total approved cost of $204.7 million. Under the program terms and conditions, projects could be funded by either grants or contributions. By the end of February 1986, 96 per cent of the amount disbursed to projects under the program was in the form of grants.

6.14 The ECGC program will continue in effect in 1986-87 to allow completion of some 20 projects.

6.15 Of the total amount approved for the program, 92 per cent went to construction projects where the main beneficiaries were municipalities, community organizations, non-profit groups and religious organizations. Among the projects were the construction or improvement of sewers, city halls, bridges, roads, sidewalks, parks, industrial parks, sports and recreation centres and churches. The balance of the funds went to tourism development and establishing community centres, youth employment centres, counselling services and assistance centres for the unemployed.

6.16 Exhibit 6.3, shown later, presents the approval process for ECGC projects. The government had designated certain regional ministers to look after the interest of various regions. Contrary to normal CEIC practice in job creation programs, these regional ministers submitted projects to the Commission for review and later played an important part in their approval.

Exhibit not available

6.17 The purpose of our audit was to assess the process and the management controls in place for administering the ECGC program to determine whether it had been managed with due regard for economy and effectiveness.

6.18 The audit included the examination of a sample of project files in five regions: New Brunswick, Quebec, Ontario, Manitoba and British Columbia. Projects in these regions accounted for 91 per cent of the total amount approved for ECGC projects.

6.19 We chose 61 files from the selected regions. We ensured that the sample was representative of the overall population of ECGC projects. Total grants and contributions made to the projects examined amounted to $32 million.

6.20 We examined the files to assess the procedures used by CEIC before and after project approval. We visited eight of the projects in the sample and met the project sponsors.

6.21 Objectives of the program. The specific objectives of the ECGC program were not clearly defined. Parliament approved the overall objective of the ECGC program in Vote 15 of the Main Estimates, which states that the amount spent by the program was for the purposes of "providing employment to unemployed workers and contributing to the betterment of the community". Moreover, other CEIC documents and the agreements with recipients confirm that the objective was to create jobs for the unemployed.

6.22 In developing and implementing the ECGC program, however, CEIC did not translate this broad objective into specific and measurable goals, and failed to establish clear operational definitions. Although the Commission does set specific goals such as the number of jobs or work-weeks to be created for its other direct job creation programs (Canada Works, Local Employment Assistance and Development, Job Corps and Career Access), it set no such goal for ECGC. Nor did the Commission establish a target with respect to the cost of creating these jobs.

6.23 Measurement of results. The audit showed that CEIC did not establish a mechanism to assess whether intended objectives had been achieved or to measure program results. Further, the Commission did not collect data on the number and cost of jobs actually created for unemployed workers by the projects that were approved.

6.24 After spending approximately $200 million on the ECGC program, CEIC did not know how many jobs or work-weeks it had actually created or what their average cost had been.

6.25 CEIC should set clear objectives in operational and measurable terms for all its programs and should establish mechanisms to evaluate the results achieved.

Commission's response: It is the normal practice for CEIC to set clear objectives in operational and measurable terms. These objectives were not set in respect of Employment Creation Grants and Contributions because the intention of the government was not to utilize the funds in the same way as with the normal programs, a fact which the Report highlights. The Commission made its position quite clear in its submission to the Treasury Board dated 14 September 1982. It requested and received (Minute No. 784672) authority to provide the funds in the form of grants, as it believed that it could not meet the detailed twin objectives of carrying out the type of projects being submitted and ensuring the control that payment by contributions required. It is important to recognize the mechanism was put into place and operational at a time, in 1982, when Canada was deep in a recession. Extraordinary measures, like Employment Creation Grants and Contributions, were called for to deal with the various kinds of labour market dislocations which were occurring at that time.

Program Design

6.27 In the following sections, we explain how the program terms and conditions and the assessment and monitoring procedures were designed to provide flexibility in project selection and speed in distributing funds. Little attention was paid to project results in terms of job creation for the unemployed and cost per job created.

6.28 Terms and conditions. Although projects had to meet certain criteria to be eligible for grants, the criteria, in our view, were not such as to ensure that a project would create jobs economically and effectively. The three main criteria proposed by the Commission and approved by Treasury Board had the effect that virtually all proposed projects qualified for grants. Once a regional minister had approved a project, it had to meet only one of the following criteria:

- the project was receiving substantial sums ($10,000 or 10 per cent of the total cost, whichever was higher) from a source other than the federal government;

- the duration of the project was 26 weeks or more; or

- the amount requested did not exceed $250,000.

6.30 The eligibility criteria for contributions were more stringent in terms of using the funds to create jobs. Project sponsors had to submit employment plans showing that at least 50 per cent of those employed on the project would come from the ranks of the unemployed.

6.31 The definition of the term 'job' was so flexible, however, that a project might offer little work to the unemployed yet still be eligible for support. The following is an excerpt from the Program Implementation Procedures:

To provide the maximum flexibility, it has been agreed that the 50 per cent rule will apply only to the total number employed over the life of the project. It is not essential that at least 50 per cent of those employed must have come from "the unemployed" at all times, or even each month. In addition, it is agreed that the count will be made solely on the number employed without reference to any period of time that a person may have worked. To give an extreme example, a project that hired two unemployed persons for one day and one person for the life of the project would satisfy the requirement as 2 out of the 3 persons employed over the life of the project would have come from "the unemployed".

6.33 According to some CEIC officials, requiring that an employment plan be drawn up and that progress be monitored against the plan would have imposed onerous conditions on the project sponsors that would have been difficult to enforce on construction sites. It would also have been difficult to estimate the number of unemployed that might be used on construction projects lasting 26 weeks or longer. But the Commission requires that similar procedures be followed in its other job creation programs where construction is involved.

6.34 In 1977, the Public Accounts Committee recommended that "financial assistance of a capital nature be classified as contributions (not grants) and administered as such, including the requirements for the submission of financial statements and reports on the use of funds".

6.35 Following that recommendation, Treasury Board altered its policy on the use of grants and contributions. In Chapter 9.4 of the Guide on Financial Administration for Departments and Agencies of the Government of Canada, the policy stipulates that "All assistance to projects of a capital nature must be classed as contributions, unless otherwise directed by Treasury Board".

6.36 On 20 September 1982, Treasury Board approved the principle proposed by CEIC that ECGC projects be funded through grants. As a result of this decision, 96 per cent ($196.5 million) of the amount approved for ECGC projects was disbursed in the form of grants, and only 4 per cent ($8.2 million) in the form of contributions.

6.37 In the ECGC program, the Treasury Board has not followed its normal practices. For other job creation programs involving construction projects, the Treasury Board normally requires that they be funded by contributions.

6.38 In our view, given the nature, size and duration of the projects, the use of grants was inappropriate. Paragraphs 6.214 through 6.224 deal with this subject and contain our recommendations about it.

6.39 Operational guidelines. The program terms and conditions gave CEIC officials responsibility for assessing project funding requests submitted by regional ministers to assure compliance with the stated criteria. The procedures established by CEIC for this assessment and for administering project files were set out in operational guidelines. However, these guidelines offered little operational guidance in the way of interpreting the eligibility criteria, reviewing projects, monitoring and following up actual results. They did not call for - nor did they prohibit - the monitoring and follow-up procedures that were normally used in job creation programs. As a result, we found that each region interpreted the guidelines in its own way.

6.40 The operational guidelines encouraged officials to accept proposed projects and recommend them for approval by the Minister of Employment and Immigration. They stated, for example, that "inasmuch as the Government wishes to proceed with these projects, recommendations to the contrary must be well-founded", and "the Government had indicated its intention to fund all of the projects put forward if possible...".

6.41 The result was that funding requests were assessed superficially, with little regard for the number and costs of jobs and work-weeks forecast.

6.42 The operational guidelines did not encourage officials to monitor achievement of the objectives Parliament had been told would apply to projects receiving grants. For example, they stated that "Treasury Board does not require monitoring of projects funded by grants".

6.43 Regional offices, therefore, were not obliged to require reports from sponsors on the actual number of jobs or work-weeks created for the unemployed and others, or to measure actual results. Regional offices did, however, monitor other aspects of the projects, such as progress on construction.

6.44 The operational guidelines for the program gave little guidance to the officials responsible for assessing ECGC projects on how to interpret the eligibility criteria or evaluate the projects in terms of their economy and effectiveness in creating jobs. None of the criteria governing eligibility for grants was defined clearly in the guidelines. The guidelines did not specify how to assess whether employment levels set out in project budgets were acceptable and did not set out procedures for determining whether qualified unemployed people were available for proposed projects. In this way, in the summer of 1985, one project received $500,000 even though unemployed workers were not available in the area.

6.45 For each of its programs, CEIC should issue clear operational guidelines that facilitate the achievement of objectives and the measurement of results.

Commission's response: It is the normal practice of CEIC to issue operation guidelines that facilitate the achievement of objectives and the measurement of results. These were not issued to the same degree as normal in the case of Employment Creation Grants and Contributions, because of the unusual nature of the program and the difficulties envisaged at the commencement of the program and set out in the submission to Treasury Board dated 14 September 1982. As the Report indicates, the instructions to CEIC from the government were quite clear. It wished to have funded the projects identified by Regional Ministers.

Employment Creation Grants and Contributions (or the Immediate Employment Stimulation Component, as it was initially called) was not set up as a program in its own right. It was set up to provide a mechanism for those projects which the government wished to implement under the Special Employment Initiative fund that could not be carried out under a then-existing authority or program of any federal department. Therefore, CEIC was unable, for example, to allocate funds to regions on a formula based on unemployment, as would be the normal practice of the Commission. The allocations to regions were determined by the government at the higher level of the Special Employment Initiative fund. The amount of Employment Creation Grants and Contributions funds provided to each region was merely a reflection of the type of project submitted by Regional Ministers.

Similarly, with the normal programs, projects are usually totally funded by the Commission and start within a short period after approval. Many of the projects submitted under Employment Creation Grants and Contributions involved the payment of funds from other levels of government, which the federal government was anxious to trigger in order to help combat the severe unemployment conditions of the early 1980s. The program was meant to be a fast non-bureaucratic response to these exceptional conditions, and it was to be delivered with minimal resources. Few human resources were, accordingly, provided for implementation.

The operating guidelines to regional offices stated: "On occasion, it may be necessary for the federal government to be seen to be accepting their part of the commitment before others will participate". In Quebec, in particular, the then-prevailing political situation precluded the early commencement of some projects, some of which, as the Report points out, did not start for several years. These long time delays created situations where funding decisions on the method of payment made at the time of approval were rendered invalid by later developments, but could not then reasonably be changed.

6.46 Distribution of funds. Given that ECGC is a job creation program for the unemployed, it would be reasonable to assume that program funds would be distributed among the various regions of the country according to each region's share of overall unemployment.

6.47 CEIC was not able to provide information about the criteria used to distribute funds among regions because it was not involved in this process. Cabinet had allocated Special Employment Initiative (SEI) funds to regional ministers. We did not pursue the basis of this allocation.

6.48 We compared the portion of ECGC funds received by each province in fiscal years 1982-83 through 1984-85 with their share of national unemployment (Exhibit 6.2). The analysis showed some significant discrepancies. Although Quebec, on average, had less than one-third of the country's unemployed between 1982-83 and 1984-85, the province received about half the funds allotted to ECGC. By contrast, Alberta had about nine per cent of the country's unemployed but received almost nothing under the program. There was also a significant gap in the case of British Columbia.

Exhibit not available

Program Delivery

Exhibit not available

6.50 At the beginning, the ECGC program, its eligibility criteria and ways to gain access to the program were not public knowledge. To judge from statements in the House of Commons at the end of 1983 and the beginning of 1984, opposition Members were no more aware of the program than the general public. This, in addition to questions asked in the Standing Committee on Labour, Manpower and Immigration, showed that there was confusion about the objectives for SEI funds and the ECGC program, eligibility criteria and application procedures for obtaining project funding.

6.51 It was not until 1 February 1984, following these questions - and 20 months after the program was introduced - that the House received some information about objectives and eligibility criteria. The information did not explain clearly the criteria used to assess projects or the procedure for applying for funds.

6.52 Our audit showed that project proposals originated with regional ministers and Members of Parliament. We could not discover what criteria regional ministers used to select the projects that they submitted to CEIC; these criteria were never communicated to all Members of Parliament and to the public.

6.53 Review and approval of funding requests. In our view, once regional ministers submitted projects, CEIC headquarters did not require regional offices to look closely enough at projects to ensure that they would achieve the program's stated goals in an economical manner. In the following sections, we show the effects of these weaknesses on the process for reviewing and approving funding requests.

6.54 Projects were approved without sufficient attention to their likely effects on job creation for the unemployed. In 51 of the 61 project files we examined during the audit, there were no forecast data on the number of work-weeks for the unemployed or the cost per work-week (that is, the amount of the grant or contribution per work-week). When a sponsor indicated how many work-weeks a project would give the unemployed, project officers did not analyse this figure and the cost per work-week to see whether they were reasonable.

6.55 In 13 of the 61 files examined, there was no information on the total number of work-weeks to be created, whether for the unemployed or for others. In the other 48 projects, we found information in the file about the total projected number of work-weeks to be created. For these projects, we were able to calculate the projected cost per work-week. In almost half the projects (23 out of 48), the estimated cost per work-week was more than $1,000. In 9 out of 48 projects, the cost was more than $2,000 per work-week. The data did not allow us to estimate the cost of jobs for the unemployed. We were also unable to determine the actual cost of job creation for these projects.

6.56 Because the job creation effects of projects were not considered or analysed, projects with relatively small labour components were approved. As a result, the cost per work-week created was high (see Exhibit 6.4).

Exhibit not available

6.57 These costs are much higher than the cost per work-week created through other direct job creation programs administered by CEIC. For example, the cost per work-week created in fiscal 1985-86 by the Canada Works program - the program most similar to ECGC in terms of its objectives and target clientele - was $311. Following are examples:

- A municipality obtained a grant of $250,000 to expand an arena. The project was to create 82 work-weeks, 45 of them for the unemployed, at a cost of $5,556 per work-week created for the unemployed, and $3,049 per week for the unemployed and others taken together.

- CEIC gave a grant of $75,000 to a non-profit organization to build a softball field. The project was to create 15 work-weeks, at a projected cost of $5,000 per week. We found no information on the number of work-weeks to be created for the unemployed.

- A non-profit group received a grant of $835,000 to renovate a building. The file indicated that 225 work-weeks were to be created, at a cost of $3,711 per work-week.

6.59 For example, CEIC paid a university students' association a contribution of $375,000 and a grant of $1 million to fund a project to create part-time jobs for students during the school year. After examining the application, the regional program director advised CEIC headquarters that the project did not meet the eligibility criteria because the participants were full-time students and could not properly be called 'unemployed'.

6.60 Headquarters then advised the regional office that students should be considered "part-time unemployed", and asked for a new assessment and recommendation in light of these new instructions. The regional office recommended approval of the project.

6.61 In this way, CEIC broadened its own definition of unemployed to make the project eligible for the program. In our view, the nature of the project was such that it met neither the objective approved by Parliament nor the eligibility criteria of the program and should not therefore have been approved.

6.62 We observed during the audit that the awarding of some grants was announced publicly or confirmed by Members of Parliament to sponsors before CEIC officials had completed their review of project proposals. The government therefore had only a summary description of the projects, and no analysis of forecast costs had been done. CEIC officials thus found themselves in the delicate position of having to assess projects and make recommendations for approval, knowing that their Minister had already agreed to fund the projects.

6.63 For example, three Members of Parliament announced, on behalf of the Minister of Employment and Immigration, the awarding of a grant of $1.5 million to a non-profit group to build a network of bicycle paths and some boat launching ramps. On another occasion, the Minister of Employment and Immigration confirmed to a Member of Parliament that a parish council would receive a grant of $500,000 to build a community centre. In both cases, officials had not completed their assessment of the applications and had made no recommendations.

6.64 Other observations. We also noted that 6 of the 61 projects examined were approved, even though they failed to meet the eligibility criteria for grants. Headquarters did not require an assessment of the sponsor's financial ability to complete the project. This deficiency was reflected in the fact that in six projects we audited, additional funds were approved to allow the sponsors to complete the projects.

6.65 In our view, however, the financial procedures and controls established by CEIC were adequate to ensure that the amounts disbursed to project sponsors were in fact spent on approved project activities.

Accountability

6.67 ECGC was a special program, strongly influenced by elected representatives, that sidestepped normal administrative procedures so that job creation projects could be initiated more quickly than usual.

6.68 Elected representatives certainly have the responsibility to develop policies. They are also ultimately accountable to the electorate for the implementation of these policies and the related programs. We believe, however, that in exercising this responsibility, they must not jeopardize the achievement of a program's objectives or its economical, efficient and effective operation. The role of CEIC in achieving the objectives of this program was limited to that of an intermediary. Significant decisions about the allocation and use of funds were taken by several players outside CEIC that were not obliged to account for the use of program funds.

6.69 We have shown how the program's eligibility criteria opened the door to approval of virtually any project proposed by a regional minister; how headquarters encouraged regional officials not to monitor the projects; how it directed them to accept project proposals rather than issuing clear operational guidelines on evaluating projects in relation to program objectives; and how the role of CEIC was limited to assuring communication with regional ministers, reviewing the project proposals they initiated, and administering the files for projects they approved.

6.70 According to the Commission, it served as a intermediary and had no decision-making authority about the quality of projects that were approved. The public was not informed about the program's objectives, the procedure for applying for funds, or the program's eligibility criteria. The information provided to Parliament about the funds appropriated and the results of the program did not inform Members adequately, as described later in paragraphs 6.225 through 6.231.

6.71 Regional officials did, however, perform reasonably well the program delivery role they had been assigned. In some cases, they even went beyond established procedures to monitor progress on projects. However, they were not called on to monitor projects in terms of their effects on the program's job creation objectives or the cost per job created.

6.72 Given CEIC's limited role in decisions about the use of funds allotted to the ECGC program, we could not establish the extent to which CEIC should be held accountable for the results of the program. CEIC was not particularly vigilant in reviewing project proposals and following up on the projects to ensure that public funds were being spent properly and prudently. In the following cases, each project illustrates several of the problems we have described.

Case No. 1 - An Industrial Park

Description and Observations

6.74 The industrial park project was to create 105 jobs and provide 1,308 work-weeks, at an estimated cost of $2,905 per work-week. The file contained no information on the number of work-weeks forecast for the unemployed.

6.75 Despite serious reservations on the part of another federal department about the need for this type of infrastructure project in the region in question, CEIC recommended the project for approval without demonstrating or justifying the need for it.

6.76 After lengthy negotiations between the provincial and federal governments, it was agreed that the scope of the project would be reduced. The federal government signed an agreement on 29 March 1985. According to our analysis, at the time of signing the agreement the project met none of the three eligibility criteria necessary to qualify for a grant. As well, the sponsor provided only a superficial estimate of project costs, which CEIC accepted without further assessment.

6.77 The modified project consisted of purchasing a piece of land and preparing a site plan. It was supposed to cost $695,000, but was later raised to $773,088 to cover a cost overrun.

6.78 In April 1986, 28 months after the project was initially approved, a visit to the project sponsor revealed that no development work had been completed and the project had yet to create any work for the unemployed.

Case No. 2 - A Museum

Description and Observations

6.80 So the sponsor received confirmation of the grant even before incorporation, when the government had only a preliminary description of the project, and there had been no analysis by CEIC of the construction and development costs of such an undertaking. Program officers had much difficulty in obtaining the information necessary to determine if the project was acceptable. These officers found themselves in an awkward position, because their Minister had already agreed to the project.

6.81 It was not until August 1983 that the analysis was completed. At that time, the project was to involve construction of a 12,000 square-foot building with two exhibit halls and a 150-seat auditorium. According to the information supplied by the sponsor, the project was to create 560 work-weeks, at a cost to taxpayers of $2,143 per week. The file contained no information on the number of work-weeks for the unemployed. We found no indication in the file that the sponsor's estimates had been analysed; that the feasibility of completing the project within the proposed budget had been assessed; that the potential costs of operating the museum, once completed, had been estimated; or that the sponsor was able to assume those costs that had been considered.

6.82 On 2 December 1983, when the agreement was signed with the sponsor, the sponsor had already acquired land, at a cost of $368,110, and signed construction contracts worth $260,810. Although a clause in the agreement stated that no payment would be made for expenses incurred before the agreement was signed, the sponsor was later reimbursed for these expenditures.

6.83 Following cost increases, CEIC, on authorization from the regional minister, altered the original agreement in July 1984, increasing the amount of the grant to $1.35 million. During the summer of 1984, however, the sponsor realized that the revised amount would still be insufficient to carry out the project in accordance with the plans and specifications. The sponsor therefore decided to cancel some of the work specified in the construction contract, including landscaping, woodwork finishing, painting and floor coverings, so as to stay within budget.

6.84 It was only in the spring of 1985 that departmental officials became fully aware of the situation. Although the project was far from completion, CEIC had already given out $1,287,828 of the grant. Without the estimated $400,000 needed to complete the work (excluding the cost of operating the museum), the sponsor wanted to abandon the project and was ready to rent the building or to give it up entirely.

6.85 Following hasty negotiations with the sponsor in the summer of 1985, the Government of Canada, through the Department of Public Works, agreed to acquire the museum for the amount of the corporation's accumulated debts - about $75,000. Public Works is now examining various options to allow other federal departments to use the building. Departmental officials told us that significant costs would be involved in modifying the building so it could be used as something other than a museum.

Case No. 3 - Store Signs

Description and Observations

6.87 Under the program terms and conditions, the project required and received Treasury Board approval. Because of the jurisdictional conflict between the federal and provincial governments, raised by the fact that the federal government was funding a municipal project, the sponsor was unable to accept the funds for the project. The sponsor then proposed another project to CEIC so it could use the grant instead toward replacing commercial signs that were no longer suitable because of the new shelters. The Commission accepted the proposal and signed a modified agreement on 28 March 1985.

6.88 Although the nature of the project had changed fundamentally, CEIC did not judge it necessary to inform Treasury Board, believing that the changes fell within the scope of the project as originally approved.

6.89 CEIC did not estimate the number of jobs and work-weeks that would be created by the revised project for unemployed workers and others, or the cost per work-week created. Given that most of the work involved manufacturing the signs, which was done in a factory, it would appear that the project could not provide many jobs for the unemployed. CEIC and the sponsor were not able to demonstrate that the project did in fact provide jobs for the unemployed.

6.90 The signs have now become the property of the merchants. Among the merchants on the street are branches of banks, national and regional chain stores, federal and provincial agencies, and local merchants. In short, CEIC spent $1.2 million to defray the cost of replacing commercial signs that enhanced the value of private property, but it did not demonstrate that the funds had been used to create jobs for the unemployed.

Case No. 4 - Building an Arena

Description and Observations

6.92 The Minister of Employment and Immigration had presented a request to Treasury Board for authority to fund construction of the arena on 15 June 1983. That proposal had stated that the total estimated cost of building the arena was $1.5 million. According to the information in the file, CEIC already knew when it made the submission to Treasury Board that construction costs would be higher than the amount stated. The regional office had sent headquarters a detailed estimate of construction costs totalling $1,700,262 on 9 May 1983.

6.93 CEIC signed a grant agreement that required the project sponsor to contribute $500,000 toward the cost of the project, even though the regional office had already determined that the sponsor could not raise that sum.

6.94 Our review of the file showed that CEIC became involved in a project that was much broader in scope than that envisaged in the original proposal. With a grant of only $1.5 million and no other source of funds, the sponsor signed an agreement on 7 July 1983 with a construction contractor for $2,629,100. Under the contract, work was to begin on 11 July 1983 and be completed in two stages - the arena and a portion of the central building ($1.825 million), and the balance of the central building and the community meeting rooms ($804,000).

6.95 In the fall of 1983, construction of the arena was stopped because of delays in delivering materials. The sponsor took it for granted that the portion of the $2 million grant already approved by Treasury Board, but not yet spent on the arena, could be used to build the meeting rooms; it began constructing them. The sponsor was therefore in the position of having spent the entire $2 million grant, but neither the arena nor the meeting rooms were finished. Treasury Board then approved the funds necessary to complete the project.

6.96 According to documents in the file, the Member of Parliament for the riding and members of his family played an active role in initiating and running the projects. After leaving public life, the former MP was appointed president of the corporation and now administers the sports centre.

Local Employment Assistance and Development Program

Background

6.98 Community development organizations in communities of fewer than 50,000 residents were eligible for LEAD benefits through LEAD "corporations" or LEAD "projects".

6.99 In the case of LEAD corporations, development organizations representing community interests received grants and contributions to stimulate the creation of permanent jobs. The corporations provided planning, technical and professional assistance to local businesses. The corporations received contributions to offset their salary and other operating costs. They also received grants that were to be used for making loans or equity contributions to local businesses. These loans were to be recovered by the corporations and used for reinvestments. Thus, the grants fund was to provide the community, through the LEAD corporation, with a growing pool of investment capital that could be accessed by local business on a revolving basis.

6.100 For LEAD projects, community organizations received contributions to carry out projects of three types: planning projects to analyse the current situation in a community and to establish an action plan for the future; enterprise projects aimed at increasing the number of permanent jobs by establishing new businesses; and infrastructure projects designed to provide facilities that would promote permanent jobs in the area.

6.101 LEAD program expenditures, over the two and a half year duration of the program, amounted to over $188 million. However, the LEAD program will continue for some years to come.

6.102 We examined the program terms and conditions and operational guidelines, as well as the management process and controls in place for the LEAD program. We audited a sample of 71 of the 737 LEAD corporations and projects, representing $36.5 million, or 19.4 per cent of the total program expenditures. The sample was selected at random, taking into account important characteristics of the group as a whole. We met with eight recipients.

Evolution of the Program

6.104 In 1983, three programs were grouped to form components of LEAD. In 1985, CEIC announced a program entitled Community Futures. Two of its five components are designed to carry on the work of parts of LEAD. The other three components were in early stages of development and implementation at the time of our audit. The program is being implemented in 1986. Exhibit 6.5 summarizes the objectives of the programs and the components of them that proceeded and succeeded LEAD.

Exhibit not available

6.105 Each time, the new programs were backed up by major public relations efforts and prompted considerable implementation and restructuring activity, both at CEIC headquarters and in the regions. Sometimes, these major efforts to introduce the changes caused funds to lapse. According to our analysis of the programs preceding LEAD, despite appearances, there was no fundamental change to the objectives and operational procedures. It was more a question of a new format where three programs became components of another program. Two of the components of the new Community Futures program show in their terms and conditions strong similarities with two of LEAD's components.

6.106 Our examination of the files showed that a substantial number of LEAD projects and corporations had benefited from one or two earlier programs that were also aimed at increasing the number of permanent jobs. We found that 16 of the 21 LEAD projects and 8 of the 18 LEAD corporations we examined had received money from previous programs. As a result, several projects were still modelled along the lines called for in the earlier programs - for example, they were oriented mainly toward target groups, such as native people, that were the focus of those programs. Under the regional program plans for recent years, significant portions of the budget were devoted to supporting prior commitments to these older projects. The 1985-86 plans for Quebec and New Brunswick, for example, allotted 70 per cent of their budgets to funding the renewal of projects and associations. In Saskatchewan, the figure was 100 per cent.

6.107 Our examination of program terms and conditions revealed few changes in the shift from the job creation activities of one program to the next.

6.108 However, we noted an evolution toward greater flexibility with the introduction of the new program components, with fewer control requirements and more generous conditions for grants and contributions.

6.109 The only changes in the terms and conditions of the new program to replace LEAD is that they are more flexible and the financial terms more generous for recipients. For example:

- only projects connected with businesses have to create jobs;

- there is no upper limit on contributions in one program component;

- there is no upper limit on operational expenses, except for salaries, which are limited to 50 per cent of total costs;

- a corporation can now get an annual grant of $400,000 rather than $350,000 in its fourth and fifth year;

- a corporation can make loans of up to $75,000 instead of $25,000; and

- audited financial statements are no longer required.

6.111 We saw no evidence of program analysis or evaluation that would support the decision to make these recently implemented local job creation components more flexible or to allot more funds to them, as is now being done.

6.112 CEIC should ensure that any changes in programs are backed up by evaluations and analysis of results achieved in previous programs.

Commission's response: CEIC rejects the analysis on which this recommendation is based, namely that there were only minor changes between LEAD and previous and subsequent programs, with these minor changes not being backed up by evaluations and analysis of results. CEIC believes the Auditor General's analysis is particularly misleading in its evaluation of LEAD and the successor program, Community Futures. The analysis does not make it clear that there were three quite distinct elements to LEAD, only one of which was continued, in a modified form, in Community Futures. LEAD infrastructure and enterprise projects were dropped, while LEAD Corporations, which were successful, were incorporated in Community Futures. Community Futures represents a major departure from earlier programming in terms of local responsibilities for deciding among program options. Of the five Community Futures program options, only one, the Business Development Centre which is modelled on LEAD Corporations, is a continuation of a LEAD program element.

The cost per job created for LEAD infrastructure and enterprise projects was very high. Ongoing monitoring and studies pointed out major problems, some of which are referred to in the Auditor General's Report. In addition, analysis of similar programming in other countries was undertaken, and a study was conducted by the Nielsen Task Force. These all pointed clearly in the same direction - that parts of LEAD were not effective (enterprise and infrastructure) and other parts were very promising (Corporations). Community development is a difficult and risky business, and when LEAD was introduced, CEIC believes it made sense to try out the three approaches. It also believes that it was right to discontinue two of those elements when experience demonstrated that they were less effective than other program directions.

Job Creation: Operational Definition

6.114 The terms and conditions and operational guidelines state several times that the jobs created should be of a permanent nature. For example, grants to LEAD corporations could be used only to assist business endeavours that create either full-time or part-time continuing jobs. The operational guidelines emphasize the creation of avenues of employment that are "viable over the long term," increasing the number of "continuing jobs." The meaning of these terms is not further clarified. Another example of unclear definitions is where permanent jobs were defined as jobs where people work full-time or part-time for an indeterminate period or at regular intervals.

6.115 The absence of clarity about defining the program's purpose was evident in the fact that there was uncertainty in the definition of the term "permanent job" and in the accounting for the number of such jobs. This was reflected in the analysis of project proposals and in the assessment of project results.

6.116 In the regions, we observed that there was confusion about the main objective of LEAD. We saw that the first priority of program managers was not to fund proposals on the basis of their potential to create permanent jobs. For example, there was no distinction made between the creation or the maintenance of a job, whether or not of a permanent nature. For an infrastructure project, we were informed that the program conditions did not require that permanent jobs be created directly. We were also informed that managers did not have to follow up on the permanent jobs created by these projects, and that permanence in the jobs created by planning projects was not an objective.

6.117 There was no performance indicator in the management information system relating to the number of permanent jobs created. The indicators in place related to the number of work-weeks. Although using work-weeks was a useful measure, a more appropriate indicator for the LEAD program would have been one providing information on the number of actual jobs created and how many of them were of a permanent nature.

6.118 LEAD corporations reported to the Commission the number of jobs that had been created indirectly through their efforts. We observed that CEIC had no way of verifying the validity of this information. Its management information system did not capture the number of these indirect jobs.

6.119 To ensure that implementation of job creation programs is directed toward their primary objective, CEIC should provide clear operational definitions of what constitutes permanent job creation, and should establish performance indicators to provide information about the actual number of permanent jobs created.

Commissions's response: CEIC does not accept the premise of this recommendation. Defining objectives was not a problem in LEAD. The Report itself lays out the definition of permanent jobs in paragraph 6.114.

In LEAD, CEIC was trying to work with three entirely different approaches (components) to achieve the objective of permanent job creation. It was doing this in communities which, by definition, had chronic problems of high unemployment. Government programs had to create jobs both directly and indirectly. The jobs created would include part-time and seasonal jobs as well as full-time year-round jobs, and the definition necessarily had to include all these types of employment. CEIC plans to improve its measurement of the direct employment impacts under Community Futures. But, it must also be noted that the rigorous measurement of the permanency of jobs created directly and indirectly would be very costly and impractical because it would involve repeated surveying of the individuals holding the jobs over many years.

Continuity and Self-sufficiency

6.121 LEAD corporations. The terms and conditions approved by Treasury Board for LEAD corporations stated that CEIC could provide money in the form of grants and contributions for up to five years, or to a maximum of $2.1 million. The 1986 program also involves funding for a maximum of five years. Under the terms and conditions of this initiative, many LEAD corporations can continue their existence. By retaining the five-year funding period, it appears that the Commission believes that this is an appropriate period for a corporation to establish an investment portfolio and a regular clientele, so as to assure continuity and self-sufficiency.

6.122 Our audit showed that the terms and conditions and the operational guidelines did not clearly address the question of the continuation and self-sufficiency of these corporations beyond the five-year or $2.1 million limit for grants and contributions. The following are among the issues requiring clarification:

- Is it expected that corporations will continue to operate as non-profit organizations under their existing articles of incorporation after the financial or time limits have been reached?

- Should the corporations become self-sufficient by virtue of revenues arising from the funds they invest given to them as grants and/or through financial contributions from other sources?

- Once CEIC has ceased its involvement with a corporation, what steps can it take to ensure that investment funds are not used inappropriately and that funds continue to be used for job creation?

- Are corporations obliged to continue to operate under existing arrangements approved by CEIC, or can these be changed once CEIC withdraws? For example, can boards of directors approve larger loans than CEIC allows or can corporations become for-profit organizations and declare dividends for shareholders?

6.124 Enterprise projects. The conditions were that an enterprise project "must demonstrate that it will progress toward viability and increase ongoing employment within the framework of a community plan" in terms of its potential for profit, its cash flow and its projected revenues. Before renewing annual funding, the Commission evaluated the viability of enterprises.

6.125 Our audit of 21 enterprise projects showed that 9 of them had not demonstrated their viability, as shown in Exhibit 6.6.

Exhibit not available

6.126 We observed that, even over the five years when these nine enterprises were receiving contributions, their viability was questionable. Additional contributions were approved for them each year. Approval was based on projections and forecasts supplied by the recipients that were neither realistic nor reasonable. These cases show that there was an inadequate capability within CEIC to conduct realistic evaluations of proposals to renew funding for enterprise projects. A regional assessment committee, including both CEIC and non-CEIC members, had a responsibility for assessing project submissions. This mechanism also did not ensure that realistic evaluations took place.

6.127 We observed that large sums of money were given to non-profit organizations that had no previous business experience. The design of the program provided for five-year funding, which acted as a disincentive for these organizations to become self-sufficient.

6.128 Infrastructure projects. The rules were that infrastructure projects should eventually lead to permanent jobs when the activities that the projects were designed to support were carried out. To achieve this objective, however, there should have been some assurance at the outset that the projects would become viable once they were developed; that is, that the private or public sector would use the facilities to create permanent jobs. Nevertheless, there were no eligibility criteria specifying that this assurance be obtained. We found no evidence that, once funding of an infrastructure project ceased, any follow-up was done to see whether the project survived or whether permanent jobs were created.

6.129 Causes of problems. We attribute the problems described above to three main deficiencies that affect the achievement of the self-sufficiency of the projects and corporations: the project assessment and selection process; inadequate management of projects by the recipient; and a lack of requisite knowledge on the part of program officers. Some examples of problems noted are as follows:

- Project proposals did not include an analysis of long-term financial forecasts or of the potential for creating long-term permanent jobs.

- There was a lack of challenge by CEIC of recipients' overly optimistic sales projections.

- The program officers did not have sufficient technical and financial background to monitor and follow up on the corporations and projects.

CEIC, in introducing Community Futures, has been precise about the duration of funding to Business Development Centres and existing LEAD Corporations. Under Community Futures, for example, a Business Development Centre could receive funding for up to 5 years, subject to annual review and approval, after which federal funds will be withdrawn.

It is expected that, in that period, the community will determine the degree of benefit which the Business Development Centre has provided and will continue to operate the Centre by finding other sources of funding for its administration. CEIC looks at the continuation of the service as a matter for local initiative. Whether the Centre maintains the same level of service, grows or diminishes will be largely a matter of the degree of commitment by the community.

With respect to the second point, CEIC's agreement will, as before, require the Centre to be incorporated as a non-profit corporation. This requirement has the effect of ensuring that, in the event of a wind-up, the Corporation's assets cannot be distributed to individuals. Assets must go to another non-profit corporation. However, the matter of a Centre's use of grant funds after the agreement has been concluded will be reviewed by CEIC.

CEIC would like to make the general observation that the kind of community development undertaken by LEAD is a risky business. This is the experience of other countries and the experience of the Canadian private sector in monitoring new initiatives. There is no simple formula that can determine the self-sufficiency of projects.

Monitoring and Follow-up

6.132 The projects and corporations we examined were monitored, proposals were properly prepared, and the necessary approvals were generally obtained. The allowable limits on grants and contributions and the terms and conditions governing payment methods were generally respected.

6.133 The audit showed instances of non-compliance with respect to financial statements, project visits, control of expenses and revenues, and projects with other sources of funding:

- Half the enterprise project files we examined contained no audited financial statements, as required by the program terms and conditions.

- In 11 of the 71 files examined there were no reports on the required financial monitoring visits by program officers.

- Reports on these financial visits did not indicate which expenses and revenues were examined by the visiting program officer, and reviews did not indicate the scope of the expenditures verified. There was therefore no assurance that funds provided under LEAD were being used appropriately.

- In 11 of 21 enterprise project files examined, reports on financial visits gave no indication that the CEIC project officers had verified that all revenues were recorded on the sponsors' books. In these cases it was not possible to establish whether unrecorded revenues might have reduced the amount of the contribution required from LEAD.

- We observed LEAD projects receiving funds from other CEIC programs, other federal and provincial departments, and other organizations. In these cases there was the risk that the jobs created by the project were reported more than once.

- In 16 of 46 enterprise and infrastructure project files, although there was funding from other sources, it was not mentioned in the sponsor's proposal or in the agreement between the sponsor and CEIC. Therefore, CEIC had no appropriate means of monitoring the impact of its contribution on the overall project and what results it achieved.

Commission's response: CEIC will review its conformity with established project control and monitoring requirements. Mistakes have been made and the requirements may not have always been implemented uniformly across CEIC's large, decentralized organization.

With respect to the second point, CEIC makes contributions for specific costs identified in a budget, and payments are made on a reimbursement basis. Under Community Futures, even greater emphasis will be placed on shared responsibilities for local development and, at a minimum, program information must clearly identify the activities of CEIC partners.

Case No. 5 - Coffin Manufacturing

Description and Observations

6.136 Over a four-year period, CEIC paid out $395,976 to the company under three different programs and from its departmental reserve.

6.137 The objective of the Enterprise Projects component of the program was to create businesses that were likely to become viable and thus to create continuing employment opportunities.

6.138 Given the information in the project file, there is reason to question the quality of the initial and annual evaluations of the project proposals, both from a financial perspective and in view of the potential success of this kind of enterprise in what is essentially a tradition-bound industry.

6.139 In 1984-85, the first year of actual operations, funding of $197,292 was provided. Sales of $49,910 had been forecast; the sponsor was supposed to contribute $51,250. However, the only revenue recorded over the year was interest in the amount of $5,689. The company encountered all kinds of production and marketing difficulties.

6.140 In January 1985, CEIC officials recognized that the project was in serious financial difficulty. Nevertheless, when the company presented a request for an additional funding period, the minutes of the meeting of the Regional Assessment Board for 27 February 1985 showed that the Board, "in reviewing the proposal, expressed no major concerns and recommended (name of company) for a second year of funding". CEIC gave $132,036 in additional funds. The regional program managers did, however, specify monthly budget analyses, as well as sales reports. The agreement specified that the expenses amounting to $403,024 would be offset in part by sales of $256,320. We found only one sales report in the file, for April 1985; only one coffin was sold at a price of $400. This project was found not to be viable in June 1985 and, in January 1986, CEIC ended its contribution.

6.141 When we visited the project sponsor in early 1986, only 2 employees remained, out of 10 permanent jobs the project was supposed to create.

National Institutional Training Program

Introduction

6.143 In 1985-86 alone, 151,000 Canadians full-time, and 55,000 part-time, benefited from the program. Approximately 14 million training days were purchased at a cost of $521,042,000. This expenditure makes up 56.5 per cent of the Institutional Training Program; the remainder is made in the form of payments of benefits to trainees either as training allowances ($153,623,000), unemployment insurance benefits under section 39 of the Unemployment Insurance Act ($234,800,000), or trainee travel allowances ($12,177,000).

Current Status

Scope

- the planning and budget allocation processes;

- negotiations of course prices and payment for purchases;

- procedures to measure and report program effectiveness; and

- selection of trainees and payment of allowances.

6.147 In examining the selection of trainees for training under the program, and the payment of income support to eligible trainees, the main focus of our work was the payment of income support. In Nova Scotia, Quebec, Ontario and British Columbia, we visited the regional headquarters of CEIC and three training Canada Employment Centres.

Program Objectives

6.149 The Commission reports on the success of its program particularly as it relates to training in occupational skills, since this type of training is most directly related to the program's objectives. In 1984-85, 37 per cent of full-time trainees took courses in occupational skills and 30 per cent in apprenticeship. The other trainees were in courses such as basic skill training, language training and job orientation training.

6.150 We noted some limitations in the measures used by the Commission (see Program Evaluation section). The measures used for 1984-85, during a period of high unemployment, indicated some difficulties in attaining the objectives:

- Measures of success in meeting labour market needs:

- 27 per cent of the skill trainees took training in occupations that were designated of national importance because they are, or are potentially, in serious short supply.

- The most recent program evaluation estimates that over 50 per cent of skill training is in occupations where there are surpluses at the time the trainee graduates.

- 44 per cent of the trainees in skill training were not working 90 days after completing their course. Of those working, half were working in areas directly related to their training.

- Measure of success in increasing individual earning and employment potential:

- The program evaluation for 1983 graduates found, on average, that there was no significant increase in earning or employment potential for a group of trainees in occupational skill and basic skill training in contrast to a comparison group of non-trainees.

6.152 While provincial governments share the Commission's objectives of meeting labour market needs and increasing individual earning and employment potential, they also have an objective of maintaining a stable and continuing institutional training system. To maintain these institutional structures, governments have come to rely on federal training purchases to provide a significant portion of the funding in the provinces and territories. It is estimated that federal course purchases are equivalent to approximately one-quarter of the operating costs of community colleges and vocational training institutions.

Planning and Budget Allocation

6.154 In all regions, we found that the labour market analysis and the success of previous graduates were used to some extent in choosing specific courses for purchase. However, the usefulness and the impact of this input on course purchases were not assessed.

6.155 As set out below, a number of factors were identified that made it difficult to ensure that the funds provided for purchasing training were being used effectively. These include the method of budget allocations, the effect of perceived purchase commitments on current year budgets, and a lack of clear and consistent central direction on purchasing training for jobs of national importance.

6.156 Budget allocation. We examined the budget allocation process used in 1985-86 to determine the extent to which program objectives influenced the allocation of funds. We found that the minimum guarantees of 90 per cent of the previous year's allocations in the agreements with the provinces are the most important influence on budget allocations; the Commission's discretion in allocating funds is limited to any excess over the guaranteed minimums. The amount of this excess is shown below:

6.158 Effect of perceived purchase commitments on current year budgets. In some regions, the funds available for purchases each year are further constrained by perceived "moral commitments" by program managers to purchase courses from specific training institutions that had received funding under the Skills Growth Fund. This was a separate program under the National Training Act to provide funds for building facilities and purchasing equipment for training in jobs of national importance. There was no legal commitment to purchase courses from institutions that had received funding, unless the training program criteria for course purchases were met.

6.159 We found that the greatest inconsistency between current labour market needs and training purchases existed in purchases being made from some of the institutions that had originally been established under the Skills Growth Fund during 1982-85. We found a number of examples where there was no longer a labour market need for training in areas where previously there had been an occupational shortage or where similar training could be obtained elsewhere at a lower cost from other institutions. But purchases continued to be made because of this perception of a moral commitment to the institutions. Some examples are noted in the following paragraphs.

6.161 A technical training institute in Saskatchewan was incorporated as a non-profit society and received funding from various federal sources, including the Skills Growth Fund ($2 million) and the Employment Creation Grants and Contributions ($115,000). It was built by native people for training native people in the oil and gas industry. The demand for occupations in the oil and gas sector has dropped significantly. The training at the institute in 1985-86 was almost four times the cost of similar training elsewhere due to costs incurred for course development; projected costs for 1986-87 are twice the cost.

6.162 A technical training centre in Manitoba was established through the Skills Growth Fund with training provided by a private company but administered by the province. The centre was established to meet anticipated needs for training in computer technology. The requirements for such training have changed, resulting in the delivery of comparatively expensive training that costs $29.00 more per day than similar training offered at other provincial training institutions.

6.163 We found that maintaining these "moral commitments" by some regions resulted in uneconomical purchases and unnecessary reductions in purchases that could otherwise have been made to meet current labour market needs.

6.164 The Commission should ensure that the purchase of courses meets current purchase criteria and that perceived moral commitments are not the basis for purchases that do not meet these criteria.

Commission's response: The Commission agrees with the intent of this recommendation. Under the Canadian Jobs Strategy, the problem of perceived moral commitments will be largely resolved, since there is no provision for capital support similar to what was available under the Skills Growth Fund.

It should, however, be noted that equal access to training by members of target groups forms part of the current purchase criteria, and "moral commitments" are often, in reality, connected to equity considerations. While transitional measures are continuing through fiscal 1986-87, to allow time for institutions to adjust, in future there will be compliance with the criteria applicable to Canadian Jobs Strategy programs, including equity considerations.

6.165 Lack of clear and consistent central direction. These specific purchase decisions are determined by the final budget allocation, the joint federal-provincial process of identifying training needs and the directions from national headquarters to program managers for use in the negotiations. The National Training Act defines occupations of national importance as occupations where there is expected to be a shortage of workers sufficiently serious to justify special action. In 1984-85, 27 per cent of skill trainees were taking training in occupations that were designated of national importance.

6.166 The original list in 1982-83 had 47 occupations. A specific target, defining the proportion of trainees that should receive training in jobs of national importance, was never provided by national headquarters, although an incentive to purchase this type of training was provided in the budget allocation formula.

6.167 Regional CEIC managers did not have a common understanding of the priority to be placed on the use of the designated list to purchase skill training. In Quebec, for example, the lists were one component of the input in making purchase decisions; in Ontario, a specific regional target was set by the joint committee. The lists were used differently in the regions, resulting in different levels of purchases, despite the priority placed on them.

6.168 Up to March 1984, the national list of occupations was sent to the regions at the beginning of each fiscal year when the planning for the year had just been completed. As a result, the designated occupations on the new lists could not be included in the purchase plans until a year later. In 1985-86 CEIC managers in the regions were uncertain about the status of the list of occupations, because it had been sent to the regions in the form of a preliminary list incorporating revisions to the 1984-85 list. No final approved list was sent.

6.169 Lists of designated occupations, defining where shortages are expected to be sufficiently serious to warrant special training initiatives, should be current, fully approved and issued on time for use in planning purchases. The Commission should provide managers with clear central direction on the use of the lists of designated occupations.

Commission's response: CEIC is already implementing the substance of this recommendation in new procedures under the Canadian Jobs Strategy. The new designation process focuses on regional designation of occupations.

Negotiation of Course Prices and Payments to Provinces

6.171 Within a prescribed period (12 to 18 months) after the end of each fiscal year, each province and territory is required to submit a certified financial statement that provides the basis for subsequent negotiations of prices and, for some provinces, provides the basis for adjusting underpayments or recovering overpayments.

Negotiation of Course Costs

6.173 In our 1978 Report, paragraph 14.37, we recommended that the Commission strengthen its negotiation of course prices by obtaining better pricing information and strengthening financial expertise. In 1980, under the Adult Occupational Training Act, the Commission established guidelines on the annual negotiation of prices and directed regional financial officers to take a more active part in the negotiations as outlined in the guidelines included in the Commission's Financial Administration Manual. In the agreements signed in 1982 under the National Training Act, a provision was included that the province should supply relevant cost information and planning documents to the Commission.

6.174 We found that the 1980 guidelines that were prepared under the Adult Occupational Training Act have not been updated. We also noted that they were not adhered to by all the regions. For example, some regions, including Newfoundland, Quebec, and British Columbia, limited their review of the per diem prices to whether the increase requested by the province appeared acceptable. Further, there was no information available to the regional offices on approximate prices and the component costs that could be included in them, which would serve as a benchmark against which to assess whether costs requested by the provinces were reasonable.

6.175 The Commission should update its guidelines relating to the negotiation of prices and administration of federal-provincial training agreements and ensure they are complied with.

Commission's response: The Commission plans to update the guidelines, particularly in light of the new agreements currently being negotiated.

6.176 The Commission should include in its guidelines benchmark information on prices for each category of training course to provide a basis for each regional office to assess whether costs requested by the provinces are reasonable.

Commission's response: The Commission has refrained from setting benchmarks for course price negotiations with the provinces because of the lack of similarity between provinces and the acknowledgement in the training agreements that the cost-causing elements inherent in course offerings be under provincial jurisdiction. The vast diversity of teaching processes, teachers qualifications, class loads, etc. all but preclude national-level standardization, but statistical information on actual course costs, by type of training in each region, is put at the disposal of all regional officials in this context. CEIC, instead, is gradually moving in the direction of routing an increasing percentage of the funds for institutional training via project sponsors and private sector partners, in order to bring market forces more to bear on course purchases.

Non-compliance with the Agreements

6.178 Receipt of cost certifications. In our 1978 Report, paragraph 14.161, we noted that there were significant delays in receiving cost certifications and, in some provinces, costs were not always certified by an independent auditor. Some of these weaknesses still exist. As a result, in many cases, the Commission does not have a basis for ensuring the accuracy and eligibility of the costs charged by the province.

6.179 CEIC should ensure it complies with the terms of the agreements made with the provinces in regard to the National Institutional Training Program.

Commission's response: While it is acknowledged that complete compliance with all the details of every training agreement was not always achieved, the implications of such lack of compliance were, in CEIC's view, marginal. Some features that gave rise to departures from agreed methodology have been removed from the agreement model currently under negotiation. Generally, the new agreements are shorter and simpler than the old ones. From 1986-87 onward, course prices will be firm and final in all provinces (territories), which eliminates much of the significance of cost certification.

Procedures to Measure and Report Program Effectiveness

Ongoing Program Monitoring

6.182 Program managers, particularly on the provincial side, expressed concern that the 90-day follow-up period was too short to capture labour market success accurately. A program evaluation by the Commission has also concluded that the 90-day follow-up was not a good predictor of employment success in the longer term. For the same people, the evaluators found that the results of the 90-day follow-up survey showed little relationship with the results of the evaluation survey approximately 16 months after course completion.

6.183 Further refinement of the indicator, or a lengthening of the follow-up period, is required for it to be used as a measure of long-term employability for reporting and program management, particularly since the indicator is used in decisions about the types of courses to be purchased and about budget allocations.

6.184 For ongoing program monitoring, the Commission should consider extending the follow-up period after the completion of training to evaluate labour market experience.

Commission's response: CEIC feels that a better approach than that recommended would be to have both a shorter-term and a longer-term follow-up survey. While some of the criticism levelled at the 90-day follow-up survey has merit, this survey still represents an indispensable tool for relatively short-term decision processes. An extended period coverage simply delays the answers. Twelve-month follow-up surveys have in the past been used to augment the intelligence provided from short-term monitoring, and these will also become a regular feature for monitoring the results of Canadian Jobs Strategy programs. The results will be a comprehensive approach to monitoring that is without precedent in Canada.

Program Evaluation

6.186 The evaluation was limited to an examination of whether the Commission's objectives were met on a national level. Treasury Board guidelines recommend an examination of all possible evaluation questions prior to defining the area for evaluation. However, the scope of the evaluation was determined without a prior assessment of all the effects, intended or unintended, that the program might have; for example, an assessment of the savings in unemployment insurance attributable to training. Also, since the evaluation was initiated shortly after the National Training Act was passed, there was no identification or analysis of the relationship between the legislated objectives and other objectives of the program; for example, those embedded in the federal-provincial agreements.

6.187 Provincial program managers were concerned that the program evaluation had not recognized their additional objectives to maintain stable vocational training institutions and to educate a more highly trained labour force. These provincial objectives, which influenced many of the course purchases as sanctioned in the agreements, were not included as evaluation questions.

6.188 The operation of the program is based on a federal-provincial partnership. The scope of the evaluation, based on the assessment of federal objectives, did not fully capture the program activity as described in the agreements and operated on a daily basis in the regions.

6.189 The conclusions in the evaluation are generally applied to all Commission trainees across Canada. There is no indication in the report of the extent to which there may be specific exceptions to the main findings. For example, the evaluation study's assessment of occupations with surplus supplies of labour is based only on Canadian aggregate results that do not take into account local area labour market information. The analysis is also based on broad classifications that can include specific occupations, some of which can be in short supply while others are not. Similarly, the conclusion that there is no significant increase in earnings or employment due to training is generally applied to all types of training for all trainees. The analysis in the report does not attempt to isolate the effects of institutional training for particular groups of trainees in each province who may have benefited from the program.

6.190 CEIC should ensure that further program evaluations are broad enough to include an assessment of program objectives, a complete identification of program effects and an analysis that captures the regional differences in the program.

Commission's response: CEIC agrees that an ideal evaluation should be both broad and specific, as long as it adds to the understanding of program strengths and weaknesses, points to remedial action in a reasonably conclusive manner and does so justifying the cost of the process. CEIC is committed to the process and will take the auditors' recommendations into account for improvements of it. In CEIC's view, the recent evaluation is an excellent example of an approach that was thorough and that was also sufficiently timely to make a difference. The results of the evaluation influenced the new agreement.

Selection of Trainees and Payment of Allowances

Compliance with Criteria for Selection of Trainees

6.194 We found that locally developed monitoring processes were usually in place and that initiatives to improve monitoring had been taken. The eligibility criteria stated in the National Training Act were communicated to counsellors and were generally understood and complied with.

Evidence Required to Support Trainee Declarations

6.196 We found that the only testing usually done of the validity of the trainee's declaration is a check on whether he or she is receiving unemployment insurance benefits. It is largely left up to the trainee to advise the Commission of any changes in economic or family circumstances that might affect allowance entitlements.

6.197 If the trainee states that care is required for dependants, a dependant care declaration is also filled out and signed by the trainee, and supplementary allowances will be paid. In 1984-85, $17 million was paid out to trainees in dependant care allowances.

6.198 We found that some CECs require proof of the existence of the dependants, usually through production of a medical card; but this requirement was not universally applied. Further, no check is done of whether the trainees actually have to pay for the care of the dependants.

6.199 The above findings show that the Commission requires very little evidence from trainees to support these two declarations and it does few checks of their accuracy. Therefore, there is potential for misrepresentation, intentionally or unintentionally, by the trainee.

6.200 In a 1980-81 study on fraud and abuse of non-unemployment insurance income support payments, the Commission found no evidence of trainees defrauding the system intentionally; but it did find that 16 per cent of the trainees sampled were overpaid. These overpayments represented 8 per cent of the total dollar value of the sample in a year when $104 million was paid out in non-UI income support to over 100,000 trainees. The overpayments were mainly because either the trainee or the counsellor misunderstood or misapplied the rules for determining eligibility for and levels of allowance. The Commission has improved administrative controls in the area of trainee declarations, but it has not followed this study up to determine whether these improvements have reduced the incidence of overpayments.

6.201 The Commission should conduct periodic follow-up of its 1981 study. It should also introduce an ongoing or periodic check of the validity of trainee declarations of economic and family status.

Commission's response: CEIC is planning to introduce ongoing checks, and it agrees that follow-ups of the 1981 study would determine how valid trainee declarations are generally and how significant the abuse is. The positive findings of the 1980-81 study and the strengthened payment system now being introduced indicate that serious problems are not likely. However, more emphasis could be put on clients documenting their declarations for review by the counsellor.

Accuracy of Payments to Trainees

6.203 The findings do not overlap significantly with the fraud and abuse study performed by CEIC in 1980-81. That study primarily investigated the accuracy of trainee declarations; we focused on the processing and administration of trainees' pay (we accepted trainee declarations as long as they were consistent internally and with other documents available at CEIC).

6.204 We identified three major causes for the errors in pay (see Exhibit 6.7):

- inadequately defined roles and responsibilities of the Department of Supply and Services, regional headquarters, CECs (22 per cent of sample errors) and training institutions;

- lack of quality control over clerical pay adjustments (43 per cent of sample errors); and

- poor communication between Training and Unemployment Insurance sections (27 per cent of sample errors).

6.205 We also found that, as a result of the lack of specificity in national guidelines on the detailed procedures of the system, regions and training CECs have had to develop their own methods for maintaining and monitoring the systems of trainee documentation, pay adjustment, and follow-up.

6.206 Payment to trainees is made by the Department of Supply and Services (DSS) through a service contract. We did not attribute any errors directly to the DSS pay system, because CEIC retains the final responsibility for accuracy of payments. However, it is our view that the DSS system contributed to the occurrence of errors by increasing the workload of CEIC clerical staff. For example, each CEC has developed its own system for monitoring the accuracy of DSS pay actions, and much time is spent on this monitoring and in adjusting and resubmitting pay documents. We noted that there were no performance standards in the service contract for accuracy of the pay service provided, nor were there appropriate mechanisms for monitoring that service.

6.207 The Commission should take measures designed to improve the accuracy of payments to trainees by:

- defining clearly the roles and responsibilities of the various participants in the system;

Given the pressure on human resource levels, the accuracy of payments will continue, to a degree, to compete with the effectiveness of the counselling service itself. While we have taken steps to improve the former, priority will continue to be on the needs of the client.

Evaluation of the Effects of Income Support

6.209 Some research has been done whenever levels of allowances have been reviewed, but the only systematic and formal analysis was conducted as part of the most recent program evaluation on the Institutional Training Program. We reviewed the evaluation findings and the analysis supporting the most recent changes in allowance structure.

6.210 In 1985, substantial changes were made in the allowance structure. The changes resulted in an increase by 50 per cent in regular allowances, an increase in the number of people eligible for allowances, and an increase by an estimated $63 million in the total standard annual payout. While the changes led to an increase in the economic incentive to train and eliminated certain inequities in the system, the analysis supporting them left a number of questions unanswered.

6.211 For example, the Commission did not address the reasonableness of the total support that can be available to trainees. Trainees can be eligible for regular allowance or UI benefits, dependant care allowance and commuting or living away from home and travel allowances. Taken together, these can form a substantial income support package and, indeed, may be significantly higher than the wage potential in the occupation being trained for. In our sample of trainees, we found more than 5 per cent being paid more than $300 a week, with an average weekly rate in that group of $373.

6.212 The 1985 program evaluation briefly examined "whether existing income support levels constitute an incentive or a disincentive to take training ..." and concluded that the existing levels were at least adequate to provide an incentive to take training. The two key findings presented were that only 5 per cent of trainees that discontinued courses cited financial reasons, and that 72 per cent of trainees stated they would have taken training for less or no income support (27 per cent would have taken it for less, 45 per cent for none). These findings are inconsistent with arguments made in the subsequent review that there was a need for increases in the economic incentive provided by allowances. The Commission has not followed up, validated or expanded these findings, which raises questions about the whole income support structure.

6.213 The Commission should follow up the 1985 evaluation findings and evaluate more fully the effects of providing income support to trainees.

Commission's response: The 1985 evaluation was a major Canadian contribution to the evaluation of a labour market program. Future evaluations will pay even more attention to the income support aspects. It is important to note that the recent evaluation indicated that income support was adequate for existing participants. The objective of the increased allowances was, however, to encourage training of new categories of recipients, notably those who are presently under-represented and who are most in need of the training.

Grants and Contributions

6.215 During this year's audit, we observed that, before 1981-82, CEIC used mostly contributions to fund job creations projects. The Commission later asked for and obtained approval to use grants, first in the ECGC program and later in the LEAD corporations program. Exhibit 6.8 shows the growth of grants in recent years.

Exhibit not available

6.216 We found some justification by the Commission to support the use of grants to fund ECGC projects and LEAD corporations. However, it did not address the question why grants were more advantageous than contributions in terms of achieving program objectives.

6.217 Under Treasury Board policies, grants and contributions are payments made to recipients from whom the government receives neither goods nor services. Payments in the form of grants are unconditional and include no audit requirement; contributions are conditional payments, subject to audit, that are covered by agreements setting out the terms and conditions of payment.

6.218 The definition of grant set out in Chapter 9.4 of the Treasury Board Guide on Financial Administration is open to interpretation. Interpreted broadly, Treasury Board directives would allow the payment of grants without any obligation on the part of recipients and without any audit of funds thus distributed to ensure that they were spent for the purposes intended. Interpreted narrowly, controls to ensure that grants are spent for the purposes intended could be the same as those for contributions. The directives do not oblige the parties to sign a grant agreement and to provide for an audit to ensure that expenditures conform with the agreement; but neither do they prohibit such an arrangement.

6.219 In recent years, because of the flexibility in interpreting the unconditional nature of grants, the distinction between grants and contributions has all but disappeared. CEIC has adopted different positions on ECGC and LEAD with respect to monitoring and evaluation by its officers of projects funded with grants. For example, for ECGC the Commission had not specified whether the grant agreements should be enforced, whereas enforcement was required for LEAD. We also noted that some LEAD corporations refused to give CEIC access to certain information, claiming that grants were unconditional transfers and include no obligation to account for the use of funds.

6.220 In the case of the ECGC program, the procedure for evaluating the effectiveness of the grants programs in meeting CEIC's objectives was not established.

6.221 In our view, because grants require no monitoring and follow-up, the use of grants should be minimized. Although grants may be less costly to administer, we believe that departments and agencies should be authorized to use this funding mechanism only where it can be demonstrated clearly that this is a more effective way of achieving program objectives. If the department or agency believes some monitoring and follow-up should be imposed, it should use contributions as the funding mechanism. However, the specific procedures required should be tailored to each program.

6.222 Should grants be used, control mechanisms, such as proper eligibility criteria, payment by installments and program evaluation, should be specified.

6.223 CEIC should minimize the use of grants in funding its job creation programs. Where grants are to be used, CEIC, in collaboration with Treasury Board, should demonstrate that this mechanism is a more effective way of achieving program objectives.

Commission's response: The choice of grants under the LEAD Program for the purpose of providing investment funds to LEAD Corporations was to establish an arm's-length relationship with ultimate clients.

Although monitoring and audits are not required for grants, these funds have in fact been monitored under the LEAD program, and CEIC is refining and further tightening controls for the use of these funds under Community Futures.

In the case of the ECGC Program, grants were used because the scope and nature of project activity were such that the requirements associated with contributions were seen to be too restrictive from an operational point of view.

With both the LEAD and ECGC programs, the Treasury Board was consulted and approved the use of grant funds.

CEIC agrees with the Report's suggestion that there exists some confusion about the appropriate uses of grants versus contributions as alternative forms of payments.

CEIC will assess the use of grants versus contributions as part of its review of the delivery of the Canadian Jobs Strategy during the latter part of the fiscal year 1986-87. It will carry this out in close consultation with the Office of the Comptroller General and the Treasury Board.

6.224 CEIC should specify the evaluation method to be used to assess the effectiveness of grants programs.

Commission's response: CEIC has an excellent record with respect to evaluations. For some years now, CEIC has had a formal evaluation cycle.

All programs are formally evaluated and, indeed, such an evaluation is already started for the LEAD program.

Unlike programs with more easily measured objectives such as short- term jobs created or training courses completed, LEAD and Community Futures have objectives which require a more in-depth analysis of results and tracking over time to determine the nature and durability of results. The program also aims at more intangible results which can only be assessed by in-depth evaluation designed to make both quantitative and qualitative assessments.

Information for Parliament

6.226 Under Treasury Board directives, grant recipients or categories of recipients must be specifically authorized by Parliament. In the Main and Supplementary Estimates presented since the inception of the ECGC and LEAD programs, CEIC has not shown the programs' intended classes of recipients in the table of grants. Members therefore did not know - and thus did not approve - the classes of grant recipients under the two programs.

6.227 Part III of its Estimates and CEIC's annual reports contained no detailed descriptions of the ECGC program. There were, however, descriptions of other job creation programs, including LEAD. We observed that all job creation programs are given the same objective in the Estimates; that is, "projects undertaken... for the purpose of providing employment to unemployed workers and contributing to the betterment of the community". This description, which applies to all programs, does not identify the differences - which are sometimes significant - in the objectives, mechanisms and target groups for the various programs.

6.228 Part III of the Estimates and annual reports are expected to contain information about the anticipated and actual performance of programs. In the case of ECGC, we found that these documents said nothing about program results. There was performance information about LEAD, but there were serious gaps in it. Because it had difficulty in collecting and analysing information from the projects, CEIC had to rely on estimates of program results for its presentation to Parliament. In the case of LEAD corporations, some of the information that was available was not reported.

6.229 In setting out the 1984-85 performance of LEAD projects and corporations in Part III of the 1986-87 Estimates, CEIC describes "the creation of 320,021 work-weeks of employment and 7,294 jobs" as "actual performance against plans for 1984-85". The narrative indicates that these figures are "lower than the established goal" even though the figures reported as actual were estimates. As well, the number of jobs and work-weeks do not represent separate achievements; they are two different methods of measuring the same thing. We looked at how CEIC had arrived at the figures stated. In the case of projects, CEIC estimated that 120,521 work-weeks or 3,304 jobs had been created. The number of work-weeks had been based on a sample of signed agreements and assumed that all agreements had created work-weeks in the same proportion as the sample. To convert these work-weeks into jobs, CEIC established that the jobs varied in length according to the program component. However, the Commission has not been able to substantiate these estimates to actual results, either for the number of work-weeks or for job length.

6.230 In the case of corporations, the estimate of 3,900 jobs or 199,500 work-weeks was based on the assumption that the number of jobs created during the year in question was proportional to the number of jobs created in the preceding year and that all the jobs were full-time. CEIC had no idea, however, whether the number of jobs in the preceding year was a reliable basis for the calculation. In making the calculation, CEIC assumed the existence of 85 corporations. A document obtained from the Commission, which stated that the number of jobs had been "carefully tracked", showed, however, that the number of corporations was 37 in 1984-85 and the number of jobs created was 1,094, of which 168 (15 per cent) had subsequently terminated. Net job creation was therefore 926, compared to the estimate of 3,900.

6.231 The primary purpose of LEAD was to create permanent jobs. It is surprising that CEIC did not provide information to Parliament about the permanence of the jobs created, whether by LEAD corporations or by projects. Thus Parliament was not informed about progress toward achieving the main objective of the program.