This Web page has been archived on the Web.

1992 Report of the Auditor General

Assistant Auditor General: Robert Lalonde
Responsible Auditor: Louis Lalonde

Main Points

9.2 We believe that the ongoing performance measurement system for some components audited should be enhanced to ensure that timely and complete effectiveness information is available at all management levels.

9.3 Given the large number of agreements at EIC, managers cannot be expected to exercise the same degree of control for each agreement. Control should vary with the level of risk and complexity of the agreement. In this regard, initial project approval is a crucial step for managers. However, vague project objectives, coupled with imprecisely defined local labour market needs and priorities, prevent EIC from selecting projects on the basis of merit. This situation increases the need for subsequent monitoring.

9.4 EIC does not respect its own minimum requirements for financial and activity monitoring practices. Furthermore, the Department does not have a strategy for choosing projects to be assessed at the end of the agreement. Limited information is derived from the monitoring of projects. More complete information resulting from the close-out assessment of projects would allow EIC to identify lessons learned that would improve subsequent project selection.

9.5 Although the Community Development Program relies on community-generated initiatives to improve the labour market situation, we found that few of the five-year strategic plans were truly strategic and that the data necessary to measure the impact of implemented initiatives were neither identified nor collected.

9.6 The eventual establishment of local labour boards could help EIC strengthen its mechanisms to ensure that funds available under the Unemployment Insurance Act for training are used effectively.

9.7 Parliament requires reliable information about EIC's activities and their effects on the labour market. It has been informed of EIC's mission, activity objectives and major lines of business. But it has not received evaluation evidence of the effectiveness of EIC's efforts to achieve its overall objective.

Employment - Monitoring Performance Against Expectations

Introduction

9.9 EIC, in co-operation with other government departments and the private sector, is responsible for developing, co-ordinating and administering federal policies and programs that are directly linked to Canada's labour market and immigration requirements. EIC's objectives are set out in its official Mission Statement:

"At EIC, we help people find jobs or collect Unemployment Insurance benefits between jobs. We manage the entry of immigrants, refugees and visitors. We help workers and employers change as the workplace changes. We also help people learn new skills or start businesses."

9.10 The programs that EIC delivers are the Employment and Insurance Program, the Immigration Program and the Corporate Management and Services Program. The mandate and the objective of the Employment and Insurance Program are described in Exhibit 9.1 .

Background

9.12 The Canadian economy, like those of other industrialized countries, is evolving. The service sector is increasingly becoming the dominant force in our economy. The policy paper notes that matching workers to jobs will be a difficult challenge because of the inherent diversity of the service sector. Unemployment rates can vary considerably from one region to the next, depending on whether the regional economy is diversified or has an industrial infrastructure based on natural resources.

9.13 Statements in EIC's 1992-93 Estimates Part III about the external factors influencing the Employment and Insurance Program are summarized in the next three paragraphs.

9.14 EIC emphasizes the need for workers to continue their training throughout their entire careers so that they can keep their skills current or widen their fields of competence. The kinds of occupations practised are changing, and the demand for workers with more specialized skills is rising considerably. Workers who are unskilled and have little formal education are very vulnerable in these times of change in the labour market and in occupations. The situation is exacerbated by Canada's drop-out rate (almost 30 percent) from secondary schools.

9.15 The inadequate skill levels of the Canadian work force could be attributed in large part to the insufficient training provided by the business community. Formal training is offered to employees by only about 25 percent of businesses, mostly large companies. The training offered by small businesses is informal, for the most part, and is often curtailed by their lack of financial resources and high rate of employee turnover. Some of Canada's major competitor countries allocate a large share of their resources to training and developing the work force.

9.16 To meet the challenges of the 1990s, Employment and Immigration Canada sought to change its method of dealing with Canadian workers by adopting the Labour Force Development Strategy (LFDS). Prior to its introduction, the main purpose of federal labour market programs was to provide temporary income support to unemployed workers. The LFDS allows a larger share of funds in the Unemployment Insurance Account (UIA) to be used for training and offers new programs and services for Canadians facing serious difficulties in the labour market.

9.17 The Canadian Labour Force Development Board (CLFDB) was established in 1991-92 to provide advice and recommendations to the Minister of EIC on developmental uses of the UIA. The National Aboriginal Management Board (NAMB) was also created in 1991-92 to monitor and review programming initiatives as they apply to Aboriginal people.

9.18 Employment and Immigration Canada (EIC) has adopted new structures for the Employment activity to improve its service to clients. Built on the National Employment Services and the Canadian Jobs Strategy (CJS), these structures stress flexibility and greater decentralization, as well as participation by the private sector and by local organizations.

9.19 The Commission introduced major changes to its systems in order to rationalize employment programs and make their delivery more flexible and to simplify administrative procedures. The implementation of on-line systems for the management of Employment activities and funds was completed, and an expanded computerized inventory of worker clients and employer job orders was developed. With the advent of LFDS in 1989, extensive automation of services to the public was initiated for the Employment and Insurance Program.

9.20 The new Boards (CLFDB and NAMB) require additional consultation with EIC management to ensure that they understand EIC's programs and services and are provided with labour market information. In addition, information from the planning and budget management systems has to be integrated into planning documents and business plans.

9.21 Employment and Immigration Canada operates in co-operation with other federal and provincial/territorial departments and agencies, as well as with individuals and organizations in the private sector. EIC also participates in a large number of sectoral co-operation agreements administered by other federal departments. While Canada Employment Centres (CECs) and other labour market stakeholders have been given more leeway in delivering EIC programs, they should take local needs into consideration. Local organizations work as full-fledged partners to help develop and carry out certain activities.

9.22 According to the Estimates Part III, the number of person-years allocated to the Employment activity was 7,691 for 1991-92 and 7,886 for 1992-93. As shown in Exhibit 9.2 , funds available under the Unemployment Insurance Act have practically tripled from $560 million to $1.4 billion, while program funds have slightly decreased to $1.4 billion. Estimates for 1992-93 call for a further increase of funds under the UI Act to approximately $2 billion and a decrease of program funds to $1.2 billion. Participants in CJS programs have averaged about 400,000 for the past three years, and some 300,000 more are expected to benefit from the additional funds available under the Act.

9.23 To support the Employment and Immigration Program, the Employment activity seeks to:

• facilitate decision making by individuals, employers, organizations and institutions concerning labour market needs;
• improve the employability of selected individuals and facilitate their successful integration into appropriate employment;
• encourage more employers to assume responsibility for meeting the changing skill needs in the workplace; and
• develop local employment opportunities and help communities to both assess their employment problems and implement programs to solve these problems.

Objectives, Scope and Criteria of the Audit

9.26 We selected projects and agreements for audit from components of three Employment sub-activities: Information and Special Initiatives, Labour Market Adjustment and Community Development. We also audited the Work Sharing and Job Creation projects financed under sections 24 and 25 of the Unemployment Insurance Act. Where appropriate, our audit also relied on the work and reports of the Internal Audit Bureau and the Program Evaluation Branch.

9.27 The components subjected to audit totalled close to $719 million or 25 percent of total program funds for 1991-92; these are shown in Exhibit 9.2 . We excluded the Employability Improvement sub-activity from our selection because of possible changes resulting from the multilateral meetings on the Constitution, as well as the implementation of the new structure and the federal/provincial agreements under negotiation at the time of our audit.

9.28 The conceptual framework used in our audit is presented in Exhibit 9.3 . Parliament gives legislative direction and approves funds to support relevant programs. It expects the funds that it allocates to EIC to be spent for the purposes specified. EIC has an obligation to obtain value for money and is accountable to Parliament for the results the Department achieves.

9.29 Feedback on results is provided at two levels: on outputs (for example, a contribution to alleviate skill shortages) and on outcomes. Initial outcomes, or impacts (such as the number of jobs created), would be followed by the effects of programs (such as improvement in the match between supply and demand in the labour market). Performance measurement and program evaluation are directed at both outputs and outcomes. Performance measurement focusses on the ongoing measurement of outputs (whether internal or external to operations) and is essential to improving program management. Program evaluation is more oriented to the periodic re-examination of the program in the light of its legislative mandate, its success and its cost-effectiveness. Effectiveness measurement serves to inform the operational levels of the organization and to periodically provide information to senior management and to Parliament.

9.30 Effectiveness measurement means the monitoring of performance against expectations. Through this formal mechanism, lessons are learned that lead to program improvement. Performance is monitored to determine what is being accomplished and how this compares with targets, whether they are focussed at the local community or at the national level. Measurement is an essential management tool for keeping activities on track and ensuring that program mandate and objectives remain relevant. Exhibit 9.3 underlines the importance of having results fed back into operations and into program planning to improve program design.

9.31 EIC requires appropriate assurance from its operations throughout the country that funds allocated to local projects are well spent and used for the planned objectives. In addition to monitoring individual programs, senior management also needs to ask questions about relative effectiveness. Are the results of various strategies such as direct expenditures different from those of partnership-directed expenditures? Is one strategy more cost-effective than another? At the level of overall program effects, is management satisfied with its program design and can it give assurance that direction from Parliament is being followed?

9.32 The common theme of this audit is the adequacy of the Department's monitoring performance against expectations. The questions we ask with respect to global objectives, program components, individual projects and partnership agreements are: Does management obtain, in a cost-effective way, reliable information on the results of its efforts? Does management use such information to continually improve programs and, where appropriate, discontinue them or recommend appropriate program changes to the minister or Cabinet?

Observations and Recommendations

Program Evaluation

Overall, the quality of evaluations was acceptable

Evaluation has not yet addressed the overall program objective

9.36 Evaluating progress toward this national policy objective is of significance to government and to Parliament. About $3 billion dollars are now spent annually by EIC on labour market programming. Is the money being spent in the best way? Answers, even approximate answers with identified margins of error, should be sought and brought forward for debate.

9.37 The program evaluations we examined, although of acceptable technical quality within their scope, did not address important cross-linking Employment activity issues that considerably influence the Department's success. Examples of issues not yet addressed by evaluations include the impact of immigrants and of the education system on the functioning of the labour market.

Important questions about serious labour market problems also need to be addressed

Evaluation scope should be re-examined

9.40 The Department should design evaluations to probe more fully the cross-linkage influences on the success or failure of the program and the impact of its activities on the functioning of the labour market.

Department's response: EIC will continue to strategically allocate its program evaluation resources to maximize their effectiveness in meeting the evaluation needs of the Commission's management and the Treasury Board. It will expand its program evaluation analyses, where cost-effective, to take into account cross-linkage influences and the impact of its activities on the labour market.

Ongoing Performance Measurement

The production of performance measurement reports is not always timely

Data are not formally interpreted

Some effectiveness performance indicators are missing

The performance measurement reports are of limited use to Canada Employment Centres

9.46 We recognize that there are at least two approaches for monitoring performance. One approach relies on indicators within a performance measurement system. Another approach relies on information obtained from project control measures. Both approaches are valid, and one may complement the other. In light of deficiencies that we identified in project control and of EIC's decentralized organization, we believe that management at all levels would benefit from an increased emphasis on effectiveness indicators in its performance measurement system.

9.47 The Department should ensure that:

• indicators are complete and measure the key effectiveness elements of program objectives;
• information is gathered for all indicators selected;
• ongoing performance measurement reports are timely and appropriate for all decision-making levels; and
• CEC managers are provided with information on the degree to which results are achieved through their interventions.

Employment performance measurement systems do not yet report on all possible aspects of program performance, but they do provide a comprehensive overview of measures reflecting the efficiency, effectiveness and quality of EIC interventions. Developmental work on additional measurements is continuing. Supporting management information systems are gradually coming on stream.

As well, in 1993-94, the Executive Review of Results Report (ERRR) will be enhanced by the inclusion of additional key measures. Outcomes on Community Development activity will also be available towards the end of 1992 through the Community Futures Committee and Business Development Centre reports as well as participant survey results for Self-Employment Assistance (SEA) and local general projects.

These measures, combined with recent and planned enhancements to the management information systems, will provide increased and more timely access by regions and CECs to on-line performance feedback. They will also facilitate the ongoing processes (Executive Committee and National Operations and Programs Committee) whereby senior management analyze and act on emerging performance issues.

EIC will ensure that all performance measurement information is made available on a timely basis. Although delays have occurred in incorporating all performance measures into regular corporate reports, work is under way within the context of survey development to provide CEC managers with selected results information. This information would be provided periodically to each CEC in order for them to track progress. The cost of providing regular ongoing results to every CEC based on survey results is prohibitive for EIC to consider.

Control of Projects

9.49 We examined over 220 agreements with respect to the following components of the Employment activity: Community Futures, Local Projects (Job Development - General Projects and Job Creation projects, section 25 of the UI Act), Workplace-based Training, Work Sharing (section 24 of the UI Act), Labour Market Adjustment Assistance (Training Trust Funds), and Labour Market Innovations. In 1991-92, the total number of agreements for these components was 25,500.

9.50 Control of projects is an integral part of the management of program operations. Managers must ensure that resources are put to optimum use with prudence and for the stated purposes. To that end, control measures are normally exercised throughout the life cycle of a project, at the following stages:

• analysis of project proposals;
• monitoring of the project; and
• closing-out of the project.

9.51 Given the large number of agreements at EIC, managers cannot be expected to exercise the same degree of control at each of the above-mentioned stages for every agreement. Control should be appropriate to the circumstances and vary with the level of risk, such as the complexity of the agreement and the ability of sponsors to meet agreement requirements. In this regard, initial project approval is a crucial step for managers; it allows them to choose projects on the basis of merit and the likelihood of achieving the targeted results. A careful analysis of proposals can reduce the need for subsequent monitoring.

9.52 In the following paragraphs, we describe the weaknesses we observed at each stage of the life cycle of projects.

A more strategic analysis of local labour market needs and priorities is required to select the most deserving projects

For three of the components audited, project objectives were not sufficiently precise

9.55 We found that project objectives were not sufficiently precise for the following components: Local Projects, Workplace-based Training and Labour Market Innovations. In one region, of the 18 Local Projects (Job Development - General Projects) agreements we examined, only two specified the probability that participants would find permanent employment on the local labour market or pursue their occupational training. In two regions, 22 of 45 agreements (49 percent) for Workplace-based Training did not stipulate the way in which training was expected to support the objectives and priorities of the program.

9.56 Another example of vague objectives is observed in the Job Creation projects. Section 25 of the UI Act provides for the use of UI benefits for the creation of new employment opportunities for laid-off workers during periods of economic downturn where no other productive activity exists. Under the authority of the UI Act, EIC allows individuals who are receiving UI benefits to participate in projects designed to enable them to retain their skills or to gain new skills. Local managers approve these projects, after taking into consideration the recommendation of a Member of Parliament. In general, we noted that project proposals did not set sufficiently precise goals for participating UI recipients. In the 25 project files we audited, only two proposals clearly specified the objectives that UI recipients were to achieve. The other 23 project proposals contained no description of the skills to be maintained or acquired, effectively making it impossible to measure whether project objectives for recipients were achieved.

9.57 Agreements examined under the Labour Market Innovations component had vague objectives. The component was developed as an instrument for gathering and analyzing information on the feasibility, acceptability, effectiveness and efficiency of different approaches to solving labour market problems. It can succeed only insofar as projects focus on major labour market problems. The Program Evaluation report revealed that most of the projects had not been designed to test innovative methods, and that those methods were not very well documented.

Eligibility criteria for the Training Trust Funds and Work Sharing components are not applied consistently

9.59 Although viability is the main eligibility criterion, EIC does not evaluate the potential for Training Trust Funds to become self-sustaining. The objective of a TTF is to assist employed workers to adjust to labour market changes by contributing to the increase in the quality and volume of training in the workplace. EIC contributes to a TTF for a specific period of time, with the objective that the fund will then continue to function and provide necessary training assistance without federal participation. Since the Department did not have aggregate information on the extent to which the self-sustaining objective of the component was being met, we requested confirmation from 60 TTFs as to the continued existence of their funds. The answers revealed that 25 (or 42 percent) were dissolved following the end of EIC financing, thus not achieving the self-sustaining objective. EIC contributed approximately $5 million over a period of five years to these dissolved TTFs.

9.60 The eligibility criteria for the Work Sharing component were used inconsistently in two respects: the information requested from employers as a basis for applying the criteria and the analysis of the actual proposals. These inconsistencies stem from the different interpretations of the EIC's operational procedures. These procedures stipulate that the employer must demonstrate that the shortage of work is significant enough to warrant the use of work sharing. The duration of the agreement is based on the length of time needed for the company to recover fully. Some local offices requested a great deal of information from employers and specific proof that work sharing was necessary. The information was then carefully analyzed; the duration of the agreement was based on the actual decrease in the employer's activities. Other local offices requested minimum information, and the duration of the agreement was based on the recovery plan submitted by the employer. Finally, some local offices requested minimum information and performed a limited analysis, yet the maximum duration of an agreement was granted.

The minimum monitoring requirement set by EIC is not respected

9.62 A strategy on the monitoring of agreements has been developed by Headquarters; although it represents a minimum requirement, it is not respected. This illustrates a lack of commitment by CECs. Managers are under pressure to deliver programs and to spend within their authorized budget levels. They therefore allocate their resources accordingly.

9.63 The following examples, drawn from Work Sharing and Job Creation project files, illustrate weaknesses in the monitoring of agreement activities.

9.64 Operational procedures stipulate that the monitoring of Work Sharing agreements should include control visits with sampling of employer payroll records. Officers adhered to these procedures in only one of the 10 centres we visited. However, most of the centres visited were trying to cope with the high demand: The total number of agreements went from 1,800 in 1989 to 10,900 in 1991.

9.65 For Job Creation project agreements, operational procedures require a minimum of three visits per project. This requirement does not take into account the amount of funds allocated to the project, risk analysis, and activity reports (if any) submitted by the sponsor. We also noted that the monitoring of agreements was inconsistent. For some projects there were two or three monitoring visits in addition to the activity report by the sponsor, while for no discernible reason, others of equal importance received no attention.

9.66 Although it is no substitute for on-site monitoring, the review of project activity reports represents a minimum level of control. EIC often did not request activity reports. Even when these reports were received, there was no documented evidence that project officers had carried out an effective analysis of activities in progress.

9.67 The Department has recognized weaknesses in the financial monitoring of agreements. The Internal Audit Bureau of EIC conducted several audits over a two-year period and made the following observations regarding financial monitoring of agreements:

• The documentation of project files is incomplete: Key control documents, such as project risk evaluations, monitoring plans and reports, were either incomplete or non-existent.
• Projects are monitored by various groups, but information is not presented in a uniform way that would make possible the effective exchange of results.
• Most of the project management problems observed could not have been identified by simply reviewing files; file monitoring cannot, therefore, be used as a substitute for on-site monitoring to ensure proper project management.
• Monitoring visits are carried out by the same project officer who recommends approval of the project, approves monthly payments and provides guidance and support to the sponsor on the management of the project.
• Staff responsible for financial control lack training.

For three of the components audited, there is no valid close-out process

9.70 For Local Projects and Workplace-based Training components, there was no valid close-out process that could be used for ongoing assessment and could reorient the program if necessary. As stated previously, the project objectives are not sufficiently precise. Therefore, project officers cannot do a valid close-out assessment. At present, such assessments simply do not exist, or, if made, are brief and incomplete. Because there are no complete assessments by which to gauge the achievement of objectives, managers cannot know the immediate results of their activities.

9.71 Since the vast majority of project sponsors receive program funds year after year, it would be appropriate to consider project merits and past results when selecting projects. This information was generally lacking in the files we audited. Thus, project results cannot be shared and lessons cannot be learned that could improve subsequent project selection.

9.72 For example, the Labour Market Innovations component can succeed only insofar as the results are reliable and are communicated to other stakeholders. The Program Evaluation report revealed that sponsors generally did not assess the results. Even when they did, their assessments did not address the question of effectiveness. The little information obtained on results was never published; its usefulness was limited.

9.73 Given the high volume of agreements at EIC, managers cannot be expected to perform a close-out assessment for each project. The strategy for selection of projects to be assessed should include risk analysis and should also be appropriate to the circumstances.

9.74 Excessive controls can be counter-productive. However, we believe that the control of projects is both essential and cost-effective. The feedback it provides on results achieved is an important contributing factor in the sound management of program funds.

9.75 To improve the control of projects, Employment and Immigration Canada should:

• ensure that project objectives are clearly stated;
• select projects based on a more strategic analysis of the local labour market needs and priorities;
• perform monitoring in a more efficient and effective manner;
• develop a strategy for the selection of projects to be assessed at the close-out of an agreement; and
• use the aggregate information generated by the close-out assessments as a basis for identifying lessons learned that would improve future project selection.

We are reviewing our reference and program materials to ensure that project objectives are clearly stated in agreements.

The Department established a Planning and Accountability Process for the local level in the last year and a half. The implementation of this process is now well under way. The guidelines will be reviewed to ensure that the necessary linkages to the selection process are established.

EIC's expectations in terms of project monitoring are demanding. Although managers at all levels remain committed to these principles, other service delivery requirements have made it more difficult to find the resources required to fully implement them. Nevertheless, EIC recognizes that monitoring efforts need to be improved. A Monitoring Task Force has several major initiatives nearing completion. These initiatives include the redrafting of our manuals, working tools and the development of a training package. The scope of monitoring is also being extended to better cover contracting and project close-out.

A selection strategy has been drafted and is now being finalized. This selection process is based on program objectives, criteria, local needs and priorities. Information gathered in this exercise will be fed back into the planning process.

Partnerships

9.77 EIC offers services in 10 regions and 487 Canada Employment Centres. These in turn, can enter into partnerships to carry out their mandates. The Community Development sub-activity relies heavily on such partnerships with, for example, some 220 Community Futures Committees and 200 Business Development Centres.

9.78 Our audit focussed on how the Department manages its activities when program success depends on the actions of its partners. We examined the activities undertaken through some of the partnerships on which EIC relies for Community Futures. In regard to the Labour Force Development Strategy (LFDS), we reviewed the implementation of the proposed partnerships.

Community Futures

9.80 EIC's Community Development sub-activity relies on community-generated initiatives to improve the labour market situation. It supports the development of local employment opportunities and helps communities facing severe labour market problems to effectively organize their resources to:

• assess local problems and opportunities;
• establish realistic objectives;
• formulate appropriate plans; and
• implement strategies that are critical to the success of their plans and that promise significant impacts on the labour market.

9.82 A community is defined as a geographical area that includes a local government unit, or group of units, with common economic and employment characteristics or problems. To be eligible for EIC's support, communities must face serious labour market difficulties, lack adequate mechanisms to address the unemployment problem on their own, and normally be outside metropolitan areas and distant from more buoyant labour markets.

Relationships between Canada Employment Centres and Community Futures Committees are not fully established

• develop an economic development and adjustment plan or strategy;
• secure and maintain the co-operation and participation of stakeholders and experts in implementing the plan; and
• verify the strategy's appropriateness to local needs.

9.84 Community Futures Committees share their territories with Canada Employment Centres, although the boundaries may vary. Both organizations are required by EIC to prepare strategies that address local needs. In its 1989 directives on planning in Canada Employment Centres, EIC management stipulates that, through the planning process, CEC managers are responsible for:

• liaison with existing provincial/territorial and local agencies, in order to co-ordinate service delivery to the extent possible;
• identification of significant labour market issues that can and should be addressed by EIC;
• prioritization of labour market issues; and
• development of a strategy for the type and size of response appropriate to identified labour market issues;
• co-ordination of existing programs and services to provide the most effective response to identified labour market issues.

9.85 In five of the nine communities visited, we found that there was no liaison between the Community Futures Committees and the Canada Employment Centres concerned. The Community Futures program consultants responsible for assisting the communities generally report at the regional level. Relations between CFCs and EIC are thus established with program consultants, rather than with CEC representatives. Consequently, the co-ordination of strategic plans that involve shared territories is not assured.

9.86 The establishment of a Community Futures Committee and its membership are contingent upon the Minister's approval. The Community Futures options can be implemented only if the CFC recommends them as part of its local strategy and only if they are approved by the Minister. EIC enters into agreements directly with the sponsors of initiatives called for in the Community Futures Committee's plan. ( see photograph )

Few of the strategic plans examined were truly strategic in nature

9.88 The Community Initiatives Fund (CIF) is an option that is meant to be used to finance particularly worthwhile new initiatives. CFCs must consider whether such initiatives are vital in implementing the chosen strategy to stimulate long-term growth in employment and renewal of economic activity.

9.89 An analysis of the CIF projects approved since 1987 shows that more than 130 focussed on the development of sports or tourism facilities and 70 on the manufacturing sector as shown in Exhibit 9.4 . Twenty-three of the 24 projects where funds were allocated to pay for necessary infrastructure were in these two sectors. The Exhibit also shows the distribution of CIF projects by types and sectors of activity, and by number of projects and dollars approved.

9.90 We examined 22 projects that were supposed to create some 2,000 long-term jobs. Some projects paved the way for subsequent job-creating projects -- for example, road construction or the installation of municipal infrastructure that would facilitate the promotion of future projects.

9.91 In a community that had proposed three initiatives to EIC, we found that all three had been designed before the five-year strategic plan was completed and submitted to EIC for approval. Two of these initiatives were approved before the strategic plan. This is not in keeping with the eligibility criteria for CIF proposals, which require that initiatives be integral to the implementation of the chosen strategy. Investments of more than $5.6 million were required, and EIC contributed $1.6 million towards these initiatives.

9.92 In another community, EIC has spent more than $6 million for the construction of two sports centres in two localities about 20 kilometres apart. In each case, the amount of private investment was small. Other sources of financing were the provincial government and the municipalities. The total cost of the projects was $20.8 million. The two municipalities agreed to absorb the anticipated deficits once the centres were operational. Neither project was completed at the time of our audit. Although the projects did create jobs and the five-year strategic plan of the community had given priority to the tourism sector, the plan had not identified measurable goals for long-term growth in employment against which progress and the results of these interventions could be assessed.

9.93 In most CIF projects examined, the data necessary to measure the impact on the long-term level of employment were neither identified nor collected. Consequently, the impact of these projects on employment in the community could not be established. EIC is not in a position to assess the proposals and evaluate the results of these activities.

9.94 The expected results of the interventions of the Business Development Centres, another option of Community Futures, are expressed in terms of jobs created and maintained. A visit to ten of these centres revealed that the systems used by six of them to measure results were unreliable. These results do not enable the CFCs to assess the appropriateness of their strategic plans, since measurable objectives have not been set. In spite of this, the results are used by the Community Futures Committees to assess the option and recommend its continuation. The results are also included in their annual reports to EIC.

Employment and Immigration did not provide enough information on the program to Community Futures Committees

9.96 A 1981 evaluation of the Community Employment Strategy highlighted several important lessons for the design of employment programs and the involvement of local elements in the process. The strategy represented an attempt by the federal and provincial governments to involve communities in improving the design and targeting of services to the chronically unemployed. The evaluation report observed, among other things, that:

• the delineation of roles and responsibilities was not clear;
• the organizational structures supporting the program's objectives should have been matched to operational requirements in terms of their capacity to co-ordinate services and support locally based planning processes;
• the analysis of community readiness was inadequate in terms of organizational skills, knowledge of government programs and the type of support and development time required; and
• the planning horizons were too short; communities would have benefited from instruction in planning methods and from technical assistance.

9.97 As we have noted, the harmonization and complementarity of activities between Canada Employment Centres and Community Futures Committees is not assured. Few of the five-year strategic plans provide a basis for EIC to assess the initiatives to be financed by them. Appropriate mechanisms for measuring results by EIC or by the CFCs are not in place.

9.98 Employment and Immigration Canada should ensure that:

• relationships are established between the Canada Employment Centres and the Community Futures Committees;
• future strategies provide a basis for assessing the initiatives to be financed by them; and
• appropriate mechanisms to measure the results of the interventions proposed by the Community Futures Committees are in place.

Current guidelines emphasize the need for initiatives to correspond to the objectives identified in the strategic plan of the community. In one of the examples mentioned, it is noted that a community strategic plan prioritized development in the sport and tourism sectors and proceeded to construct two sport centres. In the analysis of these two projects, it was considered that a large number of jobs would be generated during the construction of the centres, and that 63 permanent jobs would result in a community sustaining a high level of indefinite layoff in the forestry sector. As well, the need to encourage growth in tourism was a significant factor in the strategic plan.

There are numerous projects under way at this time that are expected to specifically address the concerns raised, including:

• a community training project, under which a training program is being designed. This program will be given to all CFC and BDC board members and staff to enhance their ability to design and implement a strategic development plan;
• the training program that is currently being implemented for volunteers and staff of Community Futures Committees, which will increase their knowledge in the area of access to other federal and provincial programs and will also clarify EIC's expectations with respect to the CFCs' obligations;
• the Goals and Measures II project, which is designing work tools and procedures for CFCs to negotiate with EIC staff a set of observable performance measures that are consistent with the CFCs' strategic goals and will provide a measure of the CFCs' success in reaching those goals;
• a project to enhance the regularity and accuracy of reports of the CFCs' activities and their impacts; and
• the Goals and Measures project, which will affect all aspects of reporting and performance assessment by the CFCs and thus will result in more appropriate mechanisms to measure the results of CFCs' interventions and strategic plan.

Future Partnerships

9.100 Since its establishment, the Board has made recommendations to the Minister of EIC concerning the productive use of Unemployment Insurance funds. It has also recommended more counselling services and an increase in EIC administrative personnel and resources. Accountability mechanisms between CLFDB and federal and provincial/territorial governments have yet to be defined.

9.101 The creation of provincial/territorial and local boards is planned for 1992-93. Local boards will provide guidance and strategic direction to the operation of training programs in their local areas, within broad guidelines established by the CLFDB and their provincial/territorial boards, and with reference to the economic and structural influences that have an impact on their local labour markets. The establishment of local boards will obviously be dependent on the establishment of their respective provincial/territorial boards.

9.102 It is expected that 60 to 70 local boards will be established during the next few years. Although none has yet been established, estimates of funds available under the Unemployment Insurance Act have practically tripled from $560 million to $1.4 billion since 1990-91. However, EIC management says it can ensure the productive use of these UI funds by several means. These include EIC's planning guidelines, the strategic plans of Canada Employment Centres and Community Futures Committees, federal-provincial/territorial agreements and sectoral studies.

9.103 We have observed that some of these means do not enable EIC to ensure that additional funds available for training are used efficiently and effectively. As we have already noted for the components audited, Canada Employment Centres do not set sufficiently precise and measurable objectives and priorities in their strategic planning. They do not evaluate the degree to which their objectives and priorities are achieved. We have observed similar problems with respect to the strategic planning of Community Future Committees. The quality and effectiveness of training under federal-provincial/territorial agreements have seldom been evaluated jointly by federal and provincial/territorial governments.

9.104 How can EIC ensure that the training it funds meets the needs of the labour market? Who will be accountable for measuring results achieved? Significant additional funds from the Unemployment Insurance Account have been available since January 1991, yet these questions remain unanswered.

Conclusion

9.106 We have also reported important deficiencies in the Department's monitoring of results with respect to overall objective, program components, individual projects and partnership agreements. Some can be corrected directly by EIC - but not all, particularly those dependent upon jointly undertaken activity. Where EIC works with other government departments and/or the private sector to improve the functioning of the labour market, its control task must recognize such multiplicity of interests. Nevertheless, the complexity this introduces to the accountability relationship does not remove it from parliamentary scrutiny.

9.107 Parliament requires reliable information about EIC's activities and their effects on the labour market. It has been informed of EIC's mission, activity objectives and major lines of business. But it has not received evaluation evidence of the effectiveness of EIC's efforts to achieve its overall objective. To what extent do EIC's expenditures contribute to the more efficient and effective functioning of the labour market? This important question remains unanswered.

9.108 This Office has, on several previous occasions, voiced its concern as to whether Parliament is receiving adequate information from departments on their management of public funds designed to achieve national objectives. We have underlined the need for measurable expectations by which to monitor the use of public funds transferred to other jurisdictions. We continue to be of the opinion that national objectives must be defined and mechanisms must be established to relate federal expenditures to national goals.

9.109 Regardless of the level of government that actually carries out the labour market programs, Parliament (as explained in Chapter 6) requires stewardship information, including the program mission; the major lines of business, the structures, the instruments and objectives designed to achieve the mission; and performance information showing to what extent public funds have been successfully used.