This Web page has been archived on the Web.

1999 November Report of the Auditor General of Canada

Main Points

32.2 The Department of Finance has gone to considerable effort in trying to address and resolve the issues we raised in our 1997 chapter about its equalization program. The changes enacted through Bill C-65 had a significant financial impact on the federal government and resulted in payments being more consistent with the principles of the program.

32.3 At Human Resources Development Canada, the Department expended considerable effort to improve service to clients and manage the Canada Pension Plan Disability program more effectively. It took action in each area discussed in our 1996 chapter.

32.4 The RCMP Public Complaints Commission has undertaken initiatives to address our previous audit concerns about reducing its backlog of cases and streamlining its complaint review procedures. However, it needs to further reduce its backlog, develop a communications strategy, restructure its information system and work with the RCMP to meet its review turnaround target.

32.5 Our follow-up of Canada's Export Promotion activities found that Industry Canada and Foreign Affairs and International Trade have made significant progress on many operational matters; however, progress is much less evident in areas that would improve accountability for results. As well, Foreign Affairs and International Trade advised the Standing Committee on Public Accounts that it would propose a realistic and feasible service charging scheme, but the departments have decided not to implement charging for services. While the departments believe that the measures they have taken are sufficient, we are concerned that the progress we have observed will not be sustained if our other recommendations are not implemented.

32.6 In our follow-up of financial management and control at Foreign Affairs and International Trade, we noted a lack of urgency in implementing a number of our recommendations; however, where the Department had focussed its efforts, it had made considerable improvements.

32.7 The Treasury Board Secretariat has taken certain steps to respond to our recommendations on Renewing Government Services Using Information Technology. The Secretariat has made good progress in its monitoring and measurement of government-wide information management and information technology initiatives. However, progress over the past three years has been slow and, to date, full implementation of action to address our recommendations has been limited.

32.8 A number of the original chapters were reviewed by the Standing Committee on Public Accounts. We feel that this has been instrumental in bringing about necessary changes.

Introduction

32.10 Although some recommendations can take several years to implement, our general policy is to follow up progress after two years, unless there is a good reason to follow up on some different basis. Each year, we provide a running list of previous chapters that tracks the status of our follow-up activity for those segments that are not included in this chapter. Exhibit 32.1 provides the list of chapters due for follow-up, but not included in this chapter.

32.11 It is important for readers to understand what a follow-up is, and is not. Apart from a few unique situations, it is not a second audit of the same issues. Rather, it is a report on what management tells us, or can demonstrate to us, about the progress it has made toward meeting our recommendations since our initial report on the subject. We do not exhaustively seek or examine additional evidence to support or refute what management has told us, but we do review its claims for reasonableness and report to Parliament accordingly.

32.12 In our 1999 Report, we review progress made by entities in response to recommendations contained in 23 previous chapters. This chapter contains the results of our review of 11 of these chapters. The others are reviewed in individual chapters of our April, September and November volumes of the 1999 Report.

32.13 In a marked change from previous years, the majority of our reviews this year depict an improvement in the efforts made by management to take positive and effective remedial action. Getting departments to undertake improvements is not easy, but through our recommendations and the interest shown by the Public Accounts Committee we are seeing progress. Examples where considerable effort has resulted in changes being made include Human Resources Development Canada - Canada Pension Plan: Disability, and the Department of Finance - Equalization Program.

32.14 In other cases, we are concerned with the slow progress on fixing the problems - for a variety of reasons, as outlined in the various segments. Examples where we note some encouraging developments but feel that greater effort is required include Indian and Northern Affairs Canada - Funding Arrangements for First Nations - 1996, Chapter 13 (reported in Chapter 10 in our April 1999 Report), and Foreign Affairs and International Trade - Financial Management and Control.

32.15 The Indian and Northern Affairs Canada follow-up chapter in our April 1999 Report was subsequently reviewed by the Public Accounts Committee. We believe that the Committee's continued interest in this area has acted as a positive catalyst, resulting in a specific commitment to action to address our 1996 recommendations. In fact, we feel that the Committee has, through its reports to the House, contributed to changes in a number of important areas.

32.16 Because this is a follow-up of actions taken on previous recommendations, we do not normally make new ones. However, in the segment Renewing Government Services Using Information Technology, our review indicated a need for a follow-up recommendation due to changed circumstances.

Treasury Board Secretariat - Renewing Government Services Using Information Technology - 1996, Chapter 16

Background

32.18 We made several observations and recommendations concerning the development of a government-wide electronic infrastructure and the move toward shared administrative systems as key initiatives in service renewal. In support of the government's implementation of its technology document entitled "Blueprint for Renewing Government Services Using Information Technology", we also recommended that the Secretariat track policy implementation and assess results against government-wide objectives.

Scope

Conclusion

Observations

Government-wide electronic infrastructure

32.22 In the follow-up, we found that progress continued to be slow on moving toward common network management services. The cluster network arrangement endorsed in 1995 by Treasury Board ministers was abandoned soon after completion of the 1996 audit. Although the Secretariat advised us in the 1996 audit that it was in the process of defining and articulating a future vision of a common network infrastructure, the follow-up showed little progress in its formulation, despite the continuing and growing need for government-wide interoperability.

32.23 The situation has improved since early this year. In April 1999, the Secretariat put forward plans for a government-wide strategic IM/IT infrastructure and received endorsement from Treasury Board ministers. The proposed infrastructure included a strategic framework for implementing a common IM/IT infrastructure, a governance framework, a network strategy and procurement approach for telecommunications and initial funding to begin its implementation.

32.24 Implementing such an infrastructure can prove to be a real challenge. In this current effort, there are a number of existing factors that should facilitate successful implementation. The major factors include the following:

• Presence of a business driver. The desire for putting in place a common IM/IT infrastructure is driven by program needs to support an electronic and more integrated delivery of government services.
• Acceptance of the proposed architecture. The proposed federated architecture, developed under the direction of the CIO Branch, was accepted by senior management across major departments and agencies.
• Identification of initial projects. Four domains of logically grouped technologies have been identified as initial projects to serve as building blocks for the progressive implementation of the common infrastructure.
• Recognition of funding needs. The need for ongoing central funding to support the common government-wide components of the IM/IT infrastructure has been recognized and initial funding was approved to start the implementation.

32.26 Information technology standards. A critical element of ensuring a government-wide IM/IT infrastructure is to adopt and comply with standards that permit various components to communicate and interoperate. In the 1996 audit, we noted that the implementation plan for the refocussed Treasury Board Information Technology (TBIT) standards program that was endorsed by Treasury Board ministers remained at the policy level. This continued to be the case in our follow-up, with little progress being made in the adoption of standards to ensure interoperability. The few standards initiatives that have been undertaken in the past three years were instigated mostly by departments and not by the Secretariat staff that were responsible for the TBIT standards program.

32.27 The adoption of the current IM/IT infrastructure initiative provides an opportunity to address the TBIT standards issue. Standards are an integral part of the implementation framework. There is also a proposed governance framework involving an Architecture Review Board that can oversee the adoption of and compliance with standards.

32.28 With respect to monitoring for compliance with the TBIT standards, we found some evidence at a high level for large projects requiring Treasury Board approval. For example, the Secretariat placed limitations on the acquisition of long-term telecommunication services until such time that it completed a review of the management and governance of government-wide telecommunication services.

32.29 Management of technology funds. In 1996, we examined the management of two technology funds - the Informatics Partnership Fund and the Informatics Infrastructure Fund. We concluded that lessons can be learned from the experience and applied in administering other technology funds to maximize the benefits of using information technology in government.

32.30 Subsequent to the audit, the Informatics Partnership Fund ended but no reviews were conducted of individual projects to promote knowledge sharing, although that was a general criterion of that Fund. The Informatics Infrastructure Fund ended during 1999-2000, with funding of $41 million provided to 18 projects. A review was initiated to evaluate those projects against their objectives but it was not completed.

32.31 In addition, we commented in 1996 on a lack of clarity in the terms and conditions for the Informatics Infrastructure Fund. At that time, the Secretariat indicated that it was reviewing the terms and conditions of the Fund, in particular the payback condition, which was the main point of contention. During the follow-up, we found that, although some work had been initiated by the CIO Branch, the terms and conditions of the Fund were never amended. Only one in 18 initiatives that were awarded funding was required to pay back the loan in accordance with the stated terms and conditions of the Fund.

32.32 In our view, the lack of clarity in the terms and conditions of the Informatics Infrastructure Fund continued to limit its effectiveness as an investment vehicle for government-wide infrastructure initiatives. The $60 million budget was to span over four years. Even with an extension to five years, only $41 million was used (see Exhibit 32.2). Moreover, while the Fund was intended for use by all departments and agencies to move toward a government-wide infrastructure, the prime user of the Fund was the Treasury Board Secretariat. Fifteen of eighteen initiatives that received funding originated from the Secretariat.

Shared Systems Initiative

32.34 In 1999, the CIO Branch conducted a review of the Shared Systems initiative to determine its success to date and to identify any remedial action required of the Secretariat and the participants. The review identified the need for business cases by departments that would include full costing for the individual shared system. It did not specifically address the issue of costs.

32.35 In briefing major cluster groups of various shared systems, the Secretariat indicated that it would develop criteria by which the effectiveness of shared systems will be measured. It also suggested that the performance of the Shared Systems initiative be reviewed annually by an oversight group under a newly constituted Information Management Board.

32.36 Shared accountability. In 1996, we observed that a new management framework, called the "shared accountability triad", carried risks that could undermine the benefits of the Shared Systems initiative. In our Report, we recommended that the Secretariat continue to provide a leadership role by retaining oversight, monitoring progress, and taking joint action with departments, as appropriate, to further the interest of the government as a whole.

32.37 In seeking endorsement and funding support for the Shared Systems initiative, the Secretariat defined its role as providing leadership, policy, and strategic and operational direction to cluster groups of various shared systems. The draft report for the 1999 Shared Systems review showed that the Secretariat has not fulfilled this role. The review recommended that the CIO Branch be re-established as the champion for the initiative and that it create a Shared Systems management function that would be responsible for strategic planning, policy and co-ordination of functional domain activities and issues. It further recommended that the functional heads within the Secretariat provide more active leadership by communicating and planning their policy changes with the corresponding cluster groups.

32.38 In early August 1999, the CIO Branch was finalizing the report for the Shared Systems review. The findings have been shared with major cluster groups of various systems and the Branch has developed an initial plan to respond to the issues raised in the review.

32.39 The Treasury Board Secretariat should finalize the report for its Shared Systems review, complete its action plan and implement the plan on a timely basis in order to benefit ongoing cluster groups of existing shared systems and any new shared systems domains. In particular, the Secretariat should develop and put in place a governance and accountability structure that will provide cluster groups of the various shared systems with strategic and business planning at a corporate level, monitor the overall performance of initiatives and facilitate ongoing investments.

Treasury Board Secretariat response: The Shared Systems review provided the Treasury Board Secretariat and government departments with a comprehensive assessment of the government-wide Shared Systems initiative. The findings and the subsequent recommendations will serve all stakeholders in restructuring and refocussing this valuable government initiative. Having conducted extensive consultations in the government community, the report and action plan has been finalized. As detailed in the action plan, the Secretariat is currently working with departments in the restructuring of the governance, accountability and management frameworks that support the Shared Systems initiative. The action plan will help ensure that the government-wide investments in administrative systems are well managed and optimize the corporate investments of both individual departments and the government as a whole.

Implementing the technology Blueprint

32.41 We found that the Secretariat has recently taken steps to establish an oversight body for its strategic IM/IT infrastructure initiative. In June 1999, the Secretariat put in place a governance structure with members from senior management across government. The Information Management Board (IMB), chaired by the CIO, has a mandate for making decisions and overseeing the adoption and implementation of a government-wide IM/IT infrastructure.

32.42 The Treasury Board Secretariat expects the IMB to review and endorse proposals that seek central funding for advancing a government-wide IM/IT infrastructure. The Secretariat proposes to adopt a benefits-driven approach for investment decisions related to initiatives for the government-wide IM/IT infrastructure. The Secretariat has developed a draft set of value/benefit criteria to assess the various opportunities proposed by individual departments.

32.43 Tracking implementation at a strategic level and defining results relative to government-wide objectives. In 1996, we recommended that where policy and direction for the use of information technology have been set and promulgated, the Treasury Board Secretariat should track implementation at a strategic level to measure the results against government-wide objectives and take corrective action as appropriate.

32.44 We noted good progress in the Secretariat's approach to evaluating government-wide IM/IT initiatives.

32.45 The CIO Branch has become more proactive in analyzing and measuring the results of government-wide initiatives. For example, it completed a baseline study in 1998 to assess its Enhanced Management Framework used by departments in managing large IT development projects. In 1999, it commissioned the Shared Systems review. Since 1998, the CIO Branch has been monitoring and tracking Year 2000 progress for departmental systems that support government-wide mission-critical functions.

32.46 The Portfolio Management Division of the CIO Branch reviews departmental business plans to ensure that they are congruent with government-wide objectives and strategies for information management and information technology. Submissions to the Treasury Board that relate to IM/IT are reviewed for compliance with the Enhanced Management Framework to help departments and agencies meet budgets and time lines that they set. Departmental perspective documents, departmental management assessments and Memoranda to Cabinet are used as the basis for monitoring departmental performance in the IM/IT.

Human Resources Development Canada - Canada Pension Plan - Disability - 1996, Chapter 17

Background

Scope

32.49 In addition, we examined random sample files to evaluate progress made by the Department in reassessing Disability files and estimating mispayments.

Conclusion

32.51 We note that the Department has expended considerable effort to improve service to clients and manage the CPP Disability program more effectively. It took action in each area discussed in our 1996 chapter. The Department completed implementation of certain recommendations and revised the applicable administrative policies and practices. Its work is not finished, however. Numerous initiatives are still under way and it is too early to determine whether the expected results will be achieved.

32.52 Since the Department has not completed implementation of all the initiatives undertaken in response to our recommendations, we recommend that it continue to apprise Parliament of its progress in the coming years.

Observations

The Department has implemented a trend analysis system but has not identified any early-warning indicators

32.54 The Department has not developed and implemented early-warning indicators to enable it to take action when it notes significant variations in Disability benefit data. Consequently, it is not in a position to react to changes that occur in the profile of existing and potential clientele.

32.55 The Department needs to be able to explain trends noted in the profile of its clientele and in Disability benefit expenditures and assess the consequences. To this end, it needs to not only measure changes, but also explain the causes of such changes.

The Department is close to meeting its actuarial expertise needs

32.57 We have examined the latest version of the memorandum of understanding between the Department and the Chief Actuary of the Office of the Superintendent of Financial Institutions. In our view, the memorandum still does not clearly set out their respective roles and responsibilities regarding reliability and integrity of data used by the Chief Actuary for actuarial forecasts of CPP Disability benefits.

The Department is able to assess the costs of proposed legislative changes

32.59 The Department, in collaboration with the Office of the Superintendent of the Financial Institutions, has developed a simulation model, Dynacan, by which to estimate and assess costs of proposed legislative changes. This model could be used to compare projected costs with current costs.

The Department has taken significant steps to improve its management information

32.61 For example, the Department now knows the total costs of administering the benefits, but it is not yet able to determine the full cost of processing an initial application for Disability benefits, including costs associated with appeal procedures. Moreover, although the Department compiles data on grant and denial rates for Disability benefit applications, it has not yet analyzed trends or their causes.

32.62 The Department has begun developing a Management Information Framework that will incorporate many of the currently collected measurements and identify any remaining gaps in key information needs. The Department needs to give priority to the completion of the Framework and use the information to understand changes and to manage performance.

A quality assurance program is being implemented to assess the adjudication of applications for Disability benefits

32.64 The Department has been assessing the quality of Initial Decisions and Reconsideration in two regions since September-October 1999. It plans to extend the initiative to the other regions in March 2000. Development of the program for the other streams of the administrative process began in September 1999. Testing is expected to take place in July 2000. According to the proposed timetable, performance reports for the quality assurance program will not be ready until April 2001. However, management reports could be available as early as September 2000.

32.65 We have reviewed the Department's plans for the development and implementation of its quality assurance program. The plans appear reasonable. We urge the Department to rigorously adhere to them.

Management of reassessment activities has improved significantly but the Department needs to continue its efforts

32.67 The Department has significantly increased the number of reassessments since 1996. In the past three years, it has conducted 36,600 reassessments of Disability cases and terminated benefits in 8,300 cases. According to the data for the last three years, the Department reassesses the files of approximately four percent of the Disability benefit beneficiary population each year. An analysis of cease rates, by type of reassessment conducted, indicates that the Department is doing well in its selection of files for reassessment and gives priority to cases that have the highest probability of no longer being eligible for benefits.

32.68 The Department has advised us that the effort to decentralize the reassessment work in the regions will result in a temporary reduction in the number of reassessments conducted. Experienced assessors will have to devote time to training their colleagues in the regions, which means that they will have that much less time to conduct reassessments. As a result, the Department expects to be able to conduct only 8,000 reassessments in 1999-2000. We have further noted that the number of reassessments has declined in each of the last three years, dropping from 14,000 in 1996-1997 to 11,800 in 1997-1998, then to 10,700 in 1998-1999.

32.69 We acknowledge that the Department's current efforts to regionalize reassessment are an investment in the long-term success of the reassessment program. Nevertheless, we urge the Department to not allow too great a decline in the number of reassessments conducted each year; this will help the Department to ensure that it can identify, within a reasonable period of time, those beneficiaries who are no longer eligible for benefits, and to avoid a major backlog of work.

The procedures used to estimate mispayments now meet our requirements

32.71 The Department has since revised its procedures in this area. Its review of the files selected has improved and its handling of overpayments identified in the samples is appropriate. Officers medically review most of the Disability files in the samples. Where a file was put into pay during the year being sampled, the initial decision is reviewed. Moreover, clear rules have been established for deciding whether a mispayment should or should not be considered in calculating the total estimated value of mispayments in the population sample.

32.72 As a result of these changes, the Department is now able to provide more realistic estimates of mispayments. For example, the study based on the payments made in 1996 is nearing completion. On the basis of this study, the Department estimates that the most likely value of undetected mispayments for 1996 is $60 million. A similar study relating to payments made in 1994 had resulted in an estimate of mispayments of only $26 million. Specifically, Disability benefit overpayments for 1996 were estimated at $29 million, compared with $14 million for 1994.

32.73 In our view, the discrepancy between the above-noted amounts is not a reflection of a higher incidence of mispayments in 1996 than in 1994. Rather, it is the result of the improved procedures used by the Department to estimate mispayments.

Some measures have been taken to implement a vocational rehabilitation program but progress is slow

32.75 Considerable attention has been given to designing a program that will maximize the success of individual clients who are identified as ready for vocational rehabilitation. According to the Department, preliminary results indicate that 26 percent of clients who participated in the program in 1998-99 returned to work. Currently, some 300 clients are participating in the vocational rehabilitation program. This is an increase of more than 100 percent over the previous year, the first full year of regionalized program delivery.

32.76 Although most beneficiaries cannot return to work due to the nature of their severely disabling medical conditions, the Department recognizes that many individuals with a disability want to work or to do volunteer work. The Department is currently studying other ways to encourage these individuals to take an active role in the work force.

32.77 We urge the Department to continue its efforts to implement measures to assist those CPP Disability beneficiaries with demonstrated potential to reintegrate into regular work life.

The exchange of information with the main stakeholders has begun

32.79 Between June 1998 and March 1999, the Department entered into five exchange-of-information agreements with Workers Compensation Boards. Three other provinces have also expressed an interest in entering into such an agreement. In most cases, the agreements provide for the payment to the Workers Compensation Boards of retroactive benefits due to beneficiaries, where such amounts are to be recovered by the Boards. Discussions are also under way to develop more collaborative arrangements to support mutual clients, such as joint vocational rehabilitation.

32.80 To date, information has been exchanged with two provinces. With regard to the other agreements, system compatibility problems will need to be resolved before any information can be exchanged and the agreements become operational.

32.81 The Department has entered into discussions with an association of insurance companies to improve relations between CPP Disability and private insurers and improve client service. The matter has been placed on the agenda of the association's standing committee and a sub-committee has been set up to work on the issues raised.

32.82 Pilot projects have been carried out in one province to pursue methods of integrating common vocational rehabilitation activities. One pilot project has now been completed and its evaluation suggests that efforts to eliminate barriers to the re-entry of beneficiaries to the labour market should be continued. The evaluation also shows that it is important to maintain the relations established between the different partners - CPP, the province and the private sector - throughout the project.

The Disability components of the Canada and Quebec Pension Plans were compared within the context of the 1996 program evaluation

32.84 In September 1996, the Department released the results of the evaluation of the Disability component of the CPP. The evaluation compared the CPP and QPP and identified the differences between the two disability programs. It concluded that bringing the CPP into line with the QPP would not necessarily result in long-term savings for the CPP.

32.85 The evaluation report nonetheless noted a significant difference between the two programs in their respective roles regarding the provincial Workers Compensation Boards. If the CPP were to play a similar role to that of the QPP with regard to the Commission de la santé et de la sécurité du travail (Québec), provincial Workers Compensation Boards would be placed in the position of first payer and fewer persons with disabilities would be referred to the CPP. Although the recent legislative amendments have made it possible for the CPP to adopt such a position, no steps have yet been taken in this regard.

The Department has begun to collect data to assess its performance in managing the CPP Disability program

The Annual Report contains more information on CPP Disability activities, but it is released too late

Canada's Export Promotion Activities - 1996, Chapter 25

Background

32.89 In our 1996 chapter, we concentrated on services provided by two departments - the Department of Foreign Affairs and International Trade, and Industry Canada. These departments together had the largest expenditures in international business development (about 85 percent of total government expenditure of about $375 million).

32.90 We reported that there was a need for improved information on costs, revenues and benefits and made several recommendations for managing the services with greater regard to economy and efficiency. We recommended that Foreign Affairs and International Trade, in consultation with its government partners, table in Parliament a performance report on the overall performance of the 22 departments and agencies involved in the government's export promotion efforts. We also updated our 1987 observations to the Standing Committee on Public Accounts in support of cost recovery and recommended that charging would provide useful input for the development of good performance indicators and help the service providers to concentrate on businesses with the greatest potential for export success.

32.91 In August 1997, responding to a request from the Public Accounts Committee, Foreign Affairs and International Trade and Industry Canada submitted a detailed status report containing plans to implement the recommendations in our chapter. In the spring of 1998, when we would normally have undertaken a follow-up review, implementation of the planned initiatives was in a preliminary phase. We decided to delay our follow-up for a year so that we could observe the changes implemented in the field, rather than report to Parliament only on departmental plans.

32.92 In 1997, the Team Canada approach to exporting evolved to become Team Canada Inc in order to reflect the participants' vision of a "virtual" trade agency. Team Canada lnc linked the 22 federal Team Canada members with provincial and territorial departments and agencies that deliver export development services. For example, the various individual plans are linked in the Team Canada Inc Business Plan, which is a document designed to promote consistency of approach and focus. The collaborative approach also involves 10 Regional Trade Networks and 12 Trade Team Canada Sector teams that are in the process of defining how they will pool resources to develop common products and services to assist Canadian businesses in finding international business opportunities.

32.93 In 1997, Foreign Affairs and International Trade also began the Performance Measurement Initiative (PMI), aimed at achieving operational improvements in the Trade Commissioner Service. These improvements are designed to better manage client expectations and to help trade officers abroad focus on activities that, from the perspective of Canadian exporters, are the most likely to provide added value and results.

Scope

Conclusion

32.96 Progress is much less evident in those areas where we recommended actions that would improve accountability for results and provide knowledge useful for managing international business development activities with greater regard to economy and efficiency. The departments have decided not to implement a framework for charging for services. Although there are plans in place to improve the situation, neither department currently has systems in place to enable them to manage with adequate knowledge of the cost and commercial worth of the individual export promotion services and products they deliver. The new annual report that we recommended on government-wide efforts in international business development emphasizes "achievements" and anecdotal success. It does not report on expenditures, costs, revenues and performance indicators that would shed light on whether the funds are being spent on matters that will best advance Canada's overall economic objectives; nor does it clarify the extent to which the specific goals of the program are being met. The departments' view is that our evaluation of progress did not take into account ``the natural progression of steps required to implement an adequate accountability and reporting framework."

32.97 The departments believe that the measures they are planning or have already introduced will be sufficient to ensure that the operational improvements they have achieved will be sustained. We remain concerned that the significant progress we have observed on many operational matters will not be sustained if our other recommendations are not implemented.

Observations

Service delivery

• market prospects - general assessments of market potential and barriers;
• key contacts search - identification of additional local sources of more detailed information;
• local company information - information about local companies identified by the potential exporter;
• visit information - help in timing and organizing a visit;
• face-to-face briefing with a trade officer; and
• troubleshooting - help with issues related to market access.

• general information;
• skills development;
• export counselling;
• market entry support; and
• trade financing.

32.100 Defining and communicating to clients what core services will be provided and what clients should do for themselves is an important advance; however, further work is needed. In our visits to foreign posts and International Trade Centres, we found general agreement on what the core services are, but variable opinion on how much "core" service should be provided. In particular, it was not clear when a "core" service starts to become "enhanced" as a result of a very high level of officer effort.

32.101 According to officers that we interviewed, particularly demanding or interesting clients receive 20 or more hours of individual officers' time over the course of a year, compared with the hour or so typically provided. Depending on the officer, this higher level of service costs between $2,000-$4,000 of time and overhead per officer per year. The variation in levels of service is also consistent with the results of Foreign Affairs and International Trade's 1998 client survey in which 54 percent of the business respondents complained that service quality and levels varied from one post to another.

32.102 It is not clear that the extra resources devoted to clients who demand more service are justified by improved results. The departments need to define more clearly what level of effort is acceptable in providing core services. In the present situation, it is possible to provide, free of charge, extensive core services to an experienced and/or new "export-ready" client that are not made routinely available to all clients.

32.103 A strenuous effort is being made to better manage client expectations. In our 1996 Report, we observed that officers believed their time was not used productively at posts abroad when business clients were not ready to export. Their dilemma was that they had difficulty saying "no" because of the understanding that all clients were equally entitled to a certain level of service.

32.104 One of the Performance Measurement Initiative's clearest accomplishments is the establishment of a system to assist officers to communicate with business clients and to provide templates for referring non-export-ready clients to services available in Canada. Other key elements of the system include a well-designed Intranet site, thorough training, documentation of best practices, regular employee and client surveys, and the establishment of a support unit at headquarters to help officers abroad deal with specific issues where client expectations may need to be managed. Trade officers claim that they can now focus better on clients with the most export potential and on activities most likely to provide added value. There are also plans to extend the PMI to non-business clients, such as other government departments and other levels of government. However, the lists of core services will not be finalized until after a consultation process.

32.105 Implementing more uniform practices can produce operational improvements. Trade officers now have a common set of guidelines and list of services for services to be provided abroad. However, we also found instances where more uniform practices would improve performance. For example, in 1996 we observed that too often Canada-based staff arrive in a new location and do not have the benefit of the experience of their predecessor, in the form of transition notes and files. Given that it costs, on average, $300,000 per year for an overseas posting, it is important that new officers be up to speed in the minimum time possible.

32.106 We recommended to Foreign Affairs and International Trade that preparation of a transition package be required and become part of the performance appraisal process. However, most of the officers that we interviewed had not received an adequate set of transition notes and files. Recently, a set of guidelines for handover notes was sent to all officers, but preparation seems to remain a matter of individual choice. We hope that where more uniform practices are desirable, guidelines will continue to be introduced and their implementation monitored.

32.107 Defining appropriate levels of service for trade missions and high-level visits still needs attention. Expectations for levels of service can be very high, and inappropriately so. One result can be that business clients must sometimes wait for service until the visit is over. Are these visits worth the expense? No one really knows because the departments do not have adequate cost information or comparative evaluations of the visit outcomes. What is clear is that the level of service provided and costs covered are far more extensive than for other types of events. We recommended in 1996 that an informal "contract" be used to help define roles, responsibilities and who pays for what when a trade mission comes to a post. Foreign Affairs and International Trade has indicated that after consulting with Team Canada Inc partners, it will implement such a contract for most missions and high level visits. With respect to the scheduling of high-level provincial and federal trade missions abroad, the Department noted that a mechanism is being developed with the provinces that aims to achieve a more co-ordinated and rationalized approach. As we note (paragraph 32.113), there will still be a need to obtain and provide information on the costs of these missions.

32.108 Support tools for trade staff have improved. The Performance Measurement Initiative is an especially effective support system for trade officers abroad. Domestically, implementation action plans are in place for the core services defined so far, and officers and their clients are both using Export Source, a web search engine with hot links to other export-related sites, and the 1-888 Export Information Service. In 1998, Team Canada Inc was one of 20 federal teams and individuals to receive the first Head of the Public Service Award. If the departments maintain their current level of high commitment, trade officers and the clients they serve will benefit.

Information on costs and results

32.110 More information is needed on officers' activities and use of time. One of the most valuable export promotion resources is officer time and expertise. Foreign Affairs and International Trade recently conducted an employee survey that included questions on how officers spend time; the Department intends to repeat the survey annually. This is a major and commendable innovation for the Department. However, employee surveys have limits because of their infrequency and inability to continuously track activities. Moreover, the Department's first survey did not inquire about the new core services in terms of cost, workload or impact. As a result, the workload indicators derived from it do not give information about the relative value and cost of the core services. The Department told us that future surveys will give such information.

32.111 Industry Canada has less information about how its trade officers spend their time, but has plans to conduct focus groups in the fall of 1999 on officers' perceptions about effective use of their time, best practices, and workload.

32.112 Time reporting is needed, in addition to employee surveys. In our 1996 Report, we observed that by abandoning the elements of the old Trade Tracking System, which tracked the above-noted sort of information, Foreign Affairs and International Trade had no way of knowing, on an ongoing basis, which activities are most time-consuming and therefore most costly. We also noted that Industry Canada did not have a tracking system of any kind. We continue to recommend that since trade officer time is one of Canada's most valuable export promotion resources, the departments institute a time reporting system so that they can track, on a continuing basis, how officer time is spent. Foreign Affairs and International Trade believes that the employee survey, together with other information, is a preferable tool for measuring workload and assessing costs of individual services. Industry Canada has informed us of plans to ascertain, through an activity-based costing study, the cost of the core and enhanced services offered by International Trade Centres. It is important to note that both departments agree that without information on how officers spend their time, it will not be possible to know the cost of services they provide.

32.113 Greater transparency is needed on the costs of high-level missions and visits. When a post absorbs costs for transportation, communications, spousal programs, and so forth, the cost of a visit may appear to be lower than it actually was. The total costs to the federal government of these visits, however allocated to departments, should form part of the performance report to Parliament that we have recommended.

32.114 The departments have better information about the commercial value of the services they provide but improvement is still needed. Knowing costs is not enough. It is also important to know how much the service is worth, in dollar terms, to clients. In the spring of 1999, Foreign Affairs and International Trade completed a client survey. The Department intends to repeat this survey annually. The survey provides valuable information about levels of client satisfaction and dissatisfaction that could be broken down by such factors as size of client, location of post and type of service. However, because of confidentiality considerations, the survey cannot provide feedback about specific transactions and products delivered by officers at specific sites; nor does it ask questions about what businesses believe the products and services they received were worth.

32.115 Industry Canada reported to the Public Accounts Committee in 1997 that "templates have been developed to allow International Trade Centres to measure the value of particular services to their clients." The Department informed us that these templates were never adopted because of the work in progress to define core services. Officials stated that when definitional work is completed, the Department may begin to track activities. The Department also has plans to conduct focus groups with clients.

32.116 While client surveys and focus groups provide useful feedback on a periodic basis, willingness to pay for export promotion services, as measured by actually charging for them, would be an excellent ongoing performance indicator of the commercial value of the different products and services.

32.117 An appropriate framework for charging for services has not been implemented. We continue to believe that charging for services would help filter out business clients who are not seriously interested in exporting and would help give valuable feedback on the perceived commercial worth of the services provided. In its status report to the Public Accounts Committee, Foreign Affairs and International Trade agreed that there are potential benefits that can be derived from charging. They pointed out that the benefits "include measuring the effectiveness of the services provided as well as their commercial value to business clients."

32.118 Concerning an approach toward deciding whether to implement charging, Foreign Affairs and International Trade said, "The Performance Measurement Initiative will guide the Department's approach to these issues [and] will propose a realistic and feasible service charging scheme. A proposal for such a scheme should be available by the end of March 1998." It pointed out that because there are many departments involved, "A common framework for charging will be closely associated to a common framework for services delivery." Industry Canada added that it is "currently evaluating the merits of charging service fees. A formal report [that may never have been completed] on the issue of service fees for ITCs is due in the fall [1997]."

32.119 Such a realistic and feasible charging scheme was never presented. At the end of the follow-up, the departments informed us that in late 1998 they had rejected charging "for the foreseeable future." Where feasible, clients might pay for or share costs for items such as travel, food, exhibit space or hotel rooms, because these were costs that did not involve charging, either directly or indirectly, for officer time. In the departments' view, this would be a more effective vehicle to address the question of charging for export promotion services. However, we were not able to find a written record of this decision; nor was it communicated to the Public Accounts Committee.

32.120 The departments offered several reasons why charging should not be implemented. They include the following:

• Charging is inconsistent with the government's objective to double the number of small and medium-size enterprises that export.
• Informal consultations with some business clients revealed little or no support for such a scheme.
• Both departments are concerned that unless there is agreement among all Team Canada Inc partners on charging, the scheme could not be implemented. Furthermore, they believe that such an agreement would be costly and difficult to negotiate.
• According to Foreign Affairs and International Trade, instituting charging would involve organizational and cultural transformations that the Department fears would involve lengthy disruptive transition periods for clients.
• Departmental accounting systems and business rules would have to be changed to allow for efficient costing, charging and collection of funds.
• The departments concluded from a review of the charging practices of many of Canada's major trade partners and competitors (U.S., Britain, Australia, France, Norway, Sweden, Finland, Germany, and four others who charge for services) that the charging regimes of these global competitors could not realistically be duplicated in Canada. The major reason was that the organizational structure for delivering export promotion services in Canada is "within a large and complex ministerial structure," as compared with the more specialized agencies and organizations in the other countries.

32.122 We believe that the departments' reasons for not charging for officer time include several challenges for implementation, but they do not constitute a compelling case for continuing the status quo. Charging offers opportunities for delivering export promotion services more economically and efficiently and with a greater knowledge of what clients want, need and are willing to pay for.

32.123 Indeed, we observed a wide variety of cost-sharing arrangements and instances where Canadian businesses already pay to attend certain government-assisted, export-related events and courses. The willingness, or lack thereof, to pay for these services in these instances can provide useful feedback to Team Canada Inc.

32.124 We also note that Canada's major competitors, including the U.S., Britain and Australia, all continue to charge for various export promotion services and products that Canada still provides free. We plan to continue to pursue this issue and to raise with both departments the merits of our recommendations to implement a charging scheme.

Information technology

32.126 WIN Exports, a key information technology tool, is not working properly. A simple example demonstrates that it is essential to have a model that makes clear which services are best provided abroad and which are better provided in Canada. For example, the added value of locally engaged staff comes from their knowledge of and access to foreign companies and governments. To help foreign companies interested in acquiring Canadian goods and services, they are expected to use WIN Exports, a potentially powerful on-line information system with extensive data on Canadian exporters. WIN Exports is designed to generate business leads for these companies, track the services they receive, and generate information useful for management. Consequently, locally engaged officers spend considerable time searching WIN Exports, at the expense of gaining or imparting further information and intelligence about the foreign market. How can the system be best configured and maintained so that officers at foreign posts use their time more productively? An IT plan based on an appropriate model would address this and similar issues.

32.127 Apart from the above-noted observation that WIN Exports is not used in an optimal fashion, the tool has a more serious immediate problem. It causes officers' computers to hang up or crash, and officers are therefore reluctant to use the system. This means the data in WIN Exports are becoming increasingly out-of-date. The consequence is that officers waste time when they try to use the system, and foreign posts and International Trade Centres may not have accurate information about companies or whether they are receiving services elsewhere. WIN Exports' inability to function properly seriously impacts on the ability of departments to deliver a high-quality export promotion service. Foreign Affairs and International Trade has indicated to us that it is well aware of these issues and is committed to resolving them.

Reporting to Parliament

32.129 The third, and most recent report, in 1999, on "achievements" of Team Canada Inc, is a clear improvement over the situation that prevailed in 1996, but it still does not meet the commitments that both departments made in 1997 to the Public Accounts Committee. The departments need to provide a report that:

• relates the reported achievements to the costs of providing the service;
• combines information on macro, governmental and departmental performance targets and whether they have been achieved (for example, reporting on the number of small- and medium-sized enterprises served by the government that have become new exporters);
• provides measures of client satisfaction that compare services provided on a cost-shared or charged basis; and
• reports on measures for the success of the service as a whole.

Acquisition Cards - 1997, Chapter 7

Background

• The Treasury Board Secretariat (TBS) should work with other federal organizations, acquisition card companies and suppliers to improve the acquisition card program and to strengthen control over card use.
• Departments and agencies should review their strategies and plans for the use of acquisition cards, analyze the use of cards and determine what controls best meet their needs to manage the risks associated with the use of acquisition cards.

Scope

Conclusion

32.133 However, it is important to note that the extent and use of acquisition cards has grown substantially since they were first introduced in 1991. By their nature, acquisition cards involve a higher degree of risk because the control of card use rests with an individual, and monitoring by entities takes place after the transaction has been completed. As a result, there will always be concerns about errors and/or abuse going undetected when using acquisition cards for government business.

32.134 Departments and agencies need to continue to ensure that appropriate controls for acquisition card use are in place, operating as intended, reviewed periodically, and conducted with prudence and probity and within the policy and guidelines established by the Treasury Board Secretariat.

Observations

Continued growth in the use of acquisition cards

32.136 The use of acquisition cards within the federal government has continued to grow since the inception of the card program in 1991. The number of cards and transactions and the volume of expenditures have increased significantly since our audit in 1997:

• There are now approximately 26,000 acquisition cards issued to government employees (30 percent increase).
• Annual transactions on acquisition cards exceed 1.3 million (221 percent increase).
• Expenditures on acquisition cards have risen to more than $355 million (207 percent increase).

Acquisition card contractors

32.138 The two companies offer a variety of payment options involving rebates, with one option involving fees. In 1998-99, rebates paid to departments and agencies were approximately $1 million of a potential $1.8 million. Rebates valued at about $800,000 were lost due to late payments by departments.

Instances of misuse of acquisition cards continue to be reported

Other observations on credit card use

Central agency initiatives

32.142 The policy states that the Departmental Acquisition Card Co-ordinators are responsible for authorizing the issuance of cards and for monitoring the program designed to ensure that reliable controls for the cards are in place and working. In addition, the policy provides detailed guidelines on such things as overall program management, card issuance, restrictions and obligations regarding the use of acquisition cards and how to handle disputed items.

32.143 In 1998 the Treasury Board Secretariat issued version 1 of the Acquisition Card Program - Best Practices Guide. The Guide provides entities with examples of leading practices in the management and monitoring of the use of acquisition cards. Version 2 of this Guide was released in the fall of 1999 and renamed Acquisition Card Program - Management Guide. It contains useful information on the history, use and benefits of acquisition cards, describes the acquisition card control framework, discusses recommended practices for acquisition cards, and highlights internal controls that will assist managers in monitoring card activities.

Management review of acquisition card transactions

• reducing acquisition card limits in order to reduce the departments' exposure in the event of misuse;
• using information technologies to provide more timely monitoring of acquisition card transactions;
• implementing an Automated Acquisition Card Registry (AACR) and developing a user guide that describes how to make the best possible use of SMART DATA and AACR to monitor card use; and
• developing a Web-based course on procurement that includes a component on acquisition cards.

The work of internal audit

32.146 In March 1998, Natural Resources Canada published a follow-up audit on the MasterCard-Electronic Data Interchange initiative and has reviewed acquisition cards as part of other functional and organizational audits. Internal Audit continues to monitor acquisition card purchasing levels on a quarterly basis.

32.147 The RCMP's 1999-2000 Review Plan includes an audit of acquisition cards, and PWGSC is currently undertaking a credit card audit that includes travel cards, acquisition cards and fleet vehicle cards to determine the adequacy of the management control framework for the use of these cards.

Department of Finance - Equalization Program - 1997, Chapter 8

Background

32.149 In Chapter 8 of our April 1997 Report, we reported the findings of our audit, which focussed on the systems and practices that the Department of Finance has in place to assess performance, to keep Parliament informed and to administer the equalization program.

32.150 Our intention in reporting in April 1997 was to give the federal government and the provinces an opportunity to examine our findings in light of the need to obtain parliamentary approval for the continuation of this program beyond 31 March 1999.

Scope

Conclusion

Observations

Achieving comparable revenue-raising capacities

32.154 We also proposed that changes be made to the treatment of property taxes. While the federal and provincial governments were unable to arrive at an alternative measurement of the revenue capacity for this revenue source, they agreed to continue examining ways to improve the method of calculating this source.

Maintaining a program that is sustainable for the federal government and provides stability for the provinces

Remaining accountable

32.157 Based on our observations of Bill C-65, this did not happen. Although MPs were well briefed about this program and the proposed changes, the legislation was not introduced until early February 1999 and the Bill needed to have Royal assent before 1 April 1999. While the Department indicated that it would respond to any request of Parliament, we continue to believe that the initiative must begin with the Minister and the Department for providing MPs with material well before the date the new legislation is required.

32.158 As part of accountability, we also indicated that meetings with provincial officials should be held on a continuous basis, if the difficult issues like measuring the revenue-raising capacity from property ownership are to be solved. We understand that this process has begun.

Administration of the program

32.160 The government discussed the matter in depth with provincial officials and they agreed that this was an area that needed further attention. They further agreed that it would be reviewed in conjunction with the tax collection agreements since those agreements are also subject to temporary over and underpayments.

Foreign Affairs and International Trade - Financial Management and Control - 1997, Chapter 9

Background

32.162 We concluded that where the Department had emphasized improving practices and controls in specific areas, it had been successful. However, there were other areas where similar efforts were needed. We made nine recommendations, four of which dealt with property management issues.

Scope

Conclusion

Observations

Financial management

32.166 Our follow-up found that this was not being accomplished. Although the senior financial officer reminded missions to carry out the analyses, there was no headquarters follow-up to the directive to verify compliance. Our review found that managers have continued to operate as before. We also determined that the configuration of the new financial and management information system (IMS), which was being implemented by the Department in 1998 and 1999, does not yet provide accounting information on a summary basis as before. This will make the type of analysis we recommended more difficult. (The Department plans to implement the capability by April 2000). We remain concerned that this basic budgeting and expenditure control has not been effectively implemented.

32.167 The 1997 audit found problems with the financial information supporting the original costing of the $25 consular fee charged to passport holders. The follow-up found that the Department has kept its costing current and is using an acceptable methodology.

32.168 In 1997 we recommended that the Inspector General implement a system to follow up and report on action taken on the many valuable recommendations contained in inspection reports. Our follow-up audit found that the Office of the Inspector General is attempting to co-ordinate internal audits with inspections so that they occur at the same time. This has led to fewer detailed recommendations in the inspection reports. In early 1999, the Inspector General also indicated to the Departmental Audit and Evaluation Committee that all inspection reports would be followed up, six months after a mission inspection. The Committee subsequently approved the follow-up strategy.

Property management

32.170 Our follow-up found considerable improvement in financial planning. Although the planning formats have not changed substantially, the information in plans has been made more realistic and senior management has been better informed on financial planning alternatives.

32.171 In a 1994 chapter, we had noted that the Physical Resources Bureau was beginning to address information technology needs for property information. Our 1997 audit observed that the Department still lacked adequate information to manage its property. We recommended that the Department develop a database that includes property maintenance history, develop performance measures to improve property management and link property information to the financial system.

32.172 Our follow-up found some activity but little progress on that recommendation. In particular, the way the new financial system was set up has so far not improved the Department's capacity to link its property information to financial information. Bureau officials told us that the Department still has plans to develop a property information module in its new IMS, once the financial module has been implemented at missions.

32.173 Our 1997 audit commented on maintenance spending, focussing on official residences. We recommended better co-ordination between Bureau and mission inspections and better co-ordination on budgeting and analyzing spending for individual properties. We also recommended that new inspections be done where missions planned significant spending on renovations or repairs not signalled as required in previous inspection reports.

32.174 The Department has improved co-ordination of efforts between missions, geographic bureaus and the Physical Resources Bureau in managing budgets for renovation and repair and in analyzing spending on properties. In 1998-99, Physical Resources Bureau maintenance managers and geographic financial advisers met regularly to co-ordinate the spending of their respective budgets at missions.

32.175 We also found that the Bureau had reminded missions that any maintenance spending over $5,000 needed Bureau approval. All the missions had received copies of the draft Property Manual that provides guidance on conducting inspections. The Bureau developed a more comprehensive inventory of major maintenance requirements at missions on an in-house database and made a submission to the Treasury Board for funds to address deferred maintenance.

32.176 Inspection reports we reviewed did not always comment on work previously carried out or whether it had been correctly entered into property records; however, officials told us that Bureau inspectors do address these points and report on an exception basis. We were also told that rather than conducting new inspections, the Bureau often preferred to rely on its improved co-ordination efforts together with other less costly means to assure itself that major unforeseen expenditures were justified.

32.177 We commented in our 1997 audit that program managers at missions were generally not meeting the guidelines on representational housing. We recommended that the Department re-examine its continuing need for representational housing. In the fall of 1998, the Department contracted for an independent study on a number of property issues. The study included questions on the use of representational housing. At the end of our follow-up, departmental representatives had begun to review the study's preliminary findings.

32.178 We reviewed the use of representational housing since 1997 at the Washington D.C. mission, which we visited during the original audit, to see if the Department had addressed our specific concerns with that mission. As in 1997, we found that none of the employees allowed the $500 U.S. per month higher rental ceiling for representational housing was meeting the conditions for maintaining a representational house. The mission conducted a study of its hospitality requirements. In 1998, for all new postings, it began using a property's location as a criterion for rent ceilings instead of the representational/ non-representational criteria. At the time of our audit, negotiations were taking place with the Treasury Board Secretariat to gain acceptance of the new methodology.

Foreign service directives

Household Goods Removal Services of the Federal Government - 1997, Chapter 21

Background

• the value obtained under contracting arrangements;
• the extent to which household goods removal was being managed economically and efficiently; and
• the controls in place to guard against the risk of overcharging.

32.182 In their joint response to our audit chapter, National Defence, PWGSC and the RCMP outlined a plan to achieve their management objectives - to strengthen the contracting framework, to improve management efficiencies and to pursue potential additional savings. In 1998, National Defence, in co-ordination with other members of the Interdepartmental Committee on Household Goods Removal, reported to the Standing Committee on Public Accounts on the status of actions taken to answer our recommendations.

Scope

Conclusion

Observations

Getting better value from a fairer and more effective contracting arrangement

32.186 The first initiative was to improve the method of assessing contractors' performance. The Committee developed a model that places more emphasis on relocated employees' satisfaction with the move, while still considering the extent to which the contractor has complied with the Committee's moving regulations. The IDC indicated that in the current contract period, it will validate the model on a trial basis, using a redesigned customer survey questionnaire and data on contractors' compliance. It also indicated that for the subsequent contract, it plans to use the validated model to reallocate business during the life of that contract.

32.187 The IDC's second initiative was to introduce, in collaboration with PWGSC, new contracting arrangements for the 1999-2001 contracting period. These arrangements eliminated the previous practice of requiring all qualified bidders to contract for services at the price offered in the lowest bid submitted. The new arrangements mean that each potential contractor can submit a competitive proposal for different percentages of government business for which it elects to submit bids. Bidders currently holding a household goods contract with the federal government can bid to a maximum of 50 percent, while new bidders are limited to a maximum of 15 percent of government business. The Request for Proposals for the 1999-2001 household goods removal contract included these new contracting arrangements.

32.188 The Statement of Work for the 1999-2001 contract included a clause that allowed for piloting new ideas. It specified that innovative service delivery methods would be actively explored during the life of the contract. It committed the government to considering alternatives to the existing service delivery method and to implementing them, where feasible. Contractors were asked to forward ideas and to participate on a voluntary basis.

32.189 In 1997 we reported that the Competition Bureau had started an inquiry in May 1996 into allegations that one or more of the van lines had breached a 1983 Supreme Court of Ontario order by preventing affiliated carriers from providing services to the federal government except through the van lines. We commented that until the inquiry had been completed, the appropriateness of the existing system of allocating business would remain an open question. At the time of our follow-up, the Bureau indicated that the inquiry had not been finalized. Notwithstanding this, the IDC has since introduced a new method of allocating business to contractors for the current contract.

Cost and efficiency of the household goods move-management function

32.191 Back in 1996, National Defence had begun to experience problems with its information system for managing household goods removal. These problems contributed to slowing down management operations and communications with contractors. The RCMP was also using National Defence's system at the time and these problems led it in May 1997 to transfer its household goods removal operations to the PWGSC system. The problems also caused National Defence to begin using the PWGSC system in June 1998, although the plan remained to centralize all operations once AMMS was operational. In October 1998, however, National Defence decided to discontinue work on AMMS due to difficulties integrating it with other internal systems that were being developed. The availability of alternative systems could not be guaranteed in time for the 1999-2001 contract period. In view of this, the members of the Interdepartmental Committee agreed that the most effective solution available was to use the PWGSC's automated system for all moving activities.

32.192 The PWGSC system has essential functionality to permit the management of moves, distribution of business and communications with contractors. According to PWGSC, it is now also Y2K-compliant. The use of the same information system by all three members of the IDC permits some improvement in efficiency but less than the AMMS had been expected to provide.

32.193 In 1997, we noted that the household goods management control activities were conducted unevenly across the three member departments, and a number of them were not carried out at all. Currently, National Defence continues to perform quality control inspections and reweighs, and the RCMP continues to perform only quality control inspections. PWGSC does not perform quality control inspections and reweighs. The RCMP and PWGSC indicated that the lack of resources contributed to limiting the performance of these controls. All three departments rely on the verification of contractors' invoices.

32.194 During the 1997-1999 contract, using the same reweigh procedures, National Defence performed 50 percent more reweighs than in the previous contract period. However, in July 1999, it initiated a centrally funded quality assurance program to ensure that the removals policy, including its reweigh program, would be applied consistently across all National Defence units. The effectiveness of this program will have to be determined in the future.

Managing the risk of overcharging

32.196 For the 1999-2001 contract, the scaled weight remains the basis for invoicing all government shipments. However, in June 1999, the IDC introduced a new process on a trial basis for determining the weight of household goods as the basis for invoicing a move. This new process uses a standard table of average weights and photographs of goods to be moved to determine the weight of a shipment. The IDC believes that such a process with appropriate controls could significantly reduce the risk of overcharging due to weight bumping. Two of the three contractors are using it in a trial to validate the table of weights and the process. If the trial succeeds, the Interdepartmental Committee intends to institute the process in the next contract period.

Systems Under Development - 1997, Chapter 23 and 1996, Chapter 24

Background

32.198 In Chapter 23 of our December 1997 Report, we reported on our risk assessments of three in-house information technology projects. We made a recommendation that had six components. They were that departments reduce their risks on systems under development by properly identifying and involving clients, preparing detailed project plans, developing business cases, using project charters, establishing independent quality assurance, and supplementing subjective measures of progress with objective measures such as time tracking.

Scope

32.200 In our follow-up of Chapter 23, we reviewed progress on the recommendation we had made after examining the Self-Serve Systems at Human Resources Development Canada (HRDC), Standardized Accounting/Corporate Tax at Revenue Canada, and Strategis at Industry Canada.

Conclusion

32.202 The Treasury Board Secretariat has received 16 of 20 action plans from departments for the implementation of the EMF, and 4 of the 16 departments have submitted updates for their plans. The Secretariat has created a public Web site and an internal Web site where other departments can view these plans and share lessons learned and ideas for future action.

32.203 Our follow-up with the three departments audited in 1997 indicates that in two cases, HRDC and Industry Canada, all six components of our recommendation have been implemented. The third department, Revenue Canada, has implemented fully all but two of the components. The two components remaining have been implemented in part.

Observations

32.205 Overall, project management of TCCCS has made good progress since last year; however, because of the significant cost and technical risks, the Department believes that the project is at a medium risk level.

32.206 Canadian Forces Supply System Upgrade. Since its initiation in 1981 by National Defence, the Canadian Forces Supply System Upgrade (CFSSU) project has undergone major transformations. The latest one affects the contract that was signed in January 1995 to deliver a system in February 1999 that would have met DND's requirements for the inventory and distribution of its supplies, as well as the planning tools necessary for the logistical support of its operations.

32.207 In November 1996, our chapter highlighted the serious problems affecting the project. Two years of intense activity by the project management team and the contractor made it possible to switch from a custom-made solution to one combining two existing commercial packages. The overall budget of $295 million remained the same but the target date slipped to September 1999; at the time of our follow-up report in December 1998, it had slipped further to February 2000.

32.208 Since our 1998 Report, delivery problems escalated to the point that, by mutual agreement, the prime contractor and National Defence proposed to bring total system responsibility back to DND. With this proposal, as of 1 November 1999, DND will effectively become the prime contractor for the rest of the project. To stay within the limits of the remaining budget of $63 million, the project management team is recommending working with one commercial package only, using a reduced staff, and delivering the revised acceptable functionality. Some functional capability that was part of the original project will be added after the current project is completed. A new target date of December 2001 is being proposed.

32.209 Because of the very tight budget remaining, the resulting resource constraints, and the major reorganization of the development effort, DND has developed a new implementation strategy with the goal of reducing the overall risk of what has been to date a high-risk project. Given the significant difficulties encountered by the project (close to a three-year schedule delay and the associated cost overruns), the Department estimates that the CFSSU project will be delivering 80 percent of the original requirements, with the remaining 20 percent being implemented as part of normal in-service upgrades. The new management and contractual framework that has been developed, with the contractor's acceptance, will see the Crown take over as prime contractor except for arrangements made within specific work tasks. Moreover, the Department says that the Crown will then be in a position to conclude the project successfully.

32.210 Treasury Board Secretariat. The work of the Treasury Board Secretariat in obtaining action plans for implementation of the EMF, monitoring their implementation and consulting with departments has been moving forward. The 20 largest departments were asked and agreed to share their plans for implementing the first of four plateaus of the EMF by March 1998. We reported last year that by September of 1998, nine plans had been received. Up to September 1999, seven additional action plans have been submitted. Four of these sixteen contributors have submitted updates to their plans, demonstrating further commitment to the EMF implementation efforts. Four departments have not submitted any plans to date.

32.211 The Treasury Board Secretariat has recently set up a Web site for contributors to facilitate sharing. At the site, departments can post their plans as well as lessons learned, suggestions for further action, plan updates and other topical items. A Web site that is open to the public has also been created.

32.212 The EMF has been expanded to include best practices for managing all investments in the Information Management/Information Technology portfolio.

32.213 Self-Serve Systems. The management of the Self-Serve Systems projects at Human Resources Development Canada has continued to pursue the initiatives that we noted in our December 1997 chapter. However, activities are very reduced so that priority can be given to the year 2000 problem. Processes are being improved though, and governance issues such as business planning and priorities are being addressed. Managers are using time tracking to supplement subjective measures of progress. Business cases and project charters are being prepared for all projects. The quality assurance function, which was well established when we reviewed the projects in 1997, has been strengthened and is now involved in all projects. More effort is now going into identifying and involving users. The Department is also implementing improvements to its systems development processes in line with the Secretariat's EMF.

32.214 Standardized Accounting/ Corporate Tax. The Standardized Accounting and the Corporate Tax Systems (SA/T2) projects at Revenue Canada have experienced staff turnover and some other problems. These problems have caused delays that have pushed final implementation from the original target of April 1998 to June 2000. The overall budget is expected to rise to $160 million from the $106 million that we reported in 1997. Revenue Canada, as we reported in 1997, has done a good job of identifying and involving clients, planning, and using business cases and project charters. Although there is no independent quality assurance function, as we noted in 1997, there are quality control steps throughout the Department's development processes. Also, while there are not yet objective measures of progress to supplement subjective judgments at the task level, time tracking is used above that level. We noted also that project progress has been measured by the delivery of some significant intermediate products, for example, the new corporate tax forms, the revenue ledger and standardized accounting for scanning and imaging. The Secretariat's EMF is being applied to all new projects.

32.215 Strategis. The Strategis project at Industry Canada has made significant progress in implementing all components of our recommendation. Client surveys are being done four times a year to get information and feedback on services. Project planning has been improved and business cases for new services are used to set priorities for implementation, as was the practice in phase 1. A quality assurance (Q/A) function has been established and is involved in each project from start to finish. The Strategis Content and Product Committee, chaired by the head of Q/A, reviews the business cases and project charter information. This committee monitors each project as it proceeds through its implementation. Objective measures of progress have been introduced, including time tracking to supplement subjective assessments. Strategis continues to innovate by improving its development processes as described above, by renovating its Strategis Web site as new features and facilities become available, and by exploring new Web-based services, such as electronic transactions, for its users. Although innovation in such fast-changing technology as the Web and the Internet involves taking risks, the judicious use of feedback, learning, and tried and true methodologies can mitigate these risks. Industry Canada has told us that it uses the Secretariat's EMF for more traditional systems development projects, as well as selected best practices from the EMF for Strategis, its Web-based publishing operation.

Agriculture and Agri-Food Canada - Prairie Farm Rehabilitation Administration - 1997, Chapter 24

Background

Scope

Conclusion

32.219 To maintain credibility with its stakeholders, PFRA needs to fully implement the changes set out by its new strategy. The next steps for management entail realigning the organization and its resources. Although PFRA has begun determining the most cost-effective manner to deliver services, more work remains to change the organization's resources to meet new challenges. PFRA has a long history of change and adaptation. The interests of stakeholders are intricately tied to how successfully PFRA positions itself for the future and efficiently runs its current and planned activities.

Observations

Land and water management issues are being defined and prioritized

Geographical risk and assessment of needs are being built into programming

Delivery of sunset programming has been analyzed

No consistent regime of monitoring or inspection was found

Reassessment of PFRA's resources is just beginning

There has been little change in cost recovery from two years ago

32.226 Based on our findings, we concluded that while PFRA's previous strengths in the area of cost recovery have continued, the organization has not fully addressed our 1997 findings. The strengths still existing at PFRA include good communication with people charged fees, ties to sustainable development, fees that tend to be market-priced and assurance by PFRA that financial controls are in place. In addition, we found that the Branch is putting in place necessary systems to allow it to track costs that could be recovered. However, we still found significant problems, such as the following three examples:

• There was little evidence of cost recovery being incorporated into strategic planning with ties to higher-level departmental goals. Existing cost recovery regimes, such as the community pastures program, tend to remain the organization's focus. Although not at the implementation (or program) stage, the development of major new efforts such as the Hog Environmental Management Strategy, did not include an early analysis of the appropriateness of applying user charges and other aspects of cost recovery.
• Some parts of PFRA have endeavoured to incorporate cost recovery into annual work planning, while other sections have not. There is a lack of leadership on this issue. Regional offices feel ill prepared to deal with this complex and sometimes difficult area.
• PFRA does not yet collect and allocate costs and overhead for each of its current and prospective services and programs. As explained in our user charge audit, costing is one area where PFRA, like other organizations in the agriculture portfolio, still needs to improve.

A good start has been made in developing a performance measurement framework

The Office of the Correctional Investigator - 1997, Chapter 33

Background

32.229 The objective of our 1997 audit was to determine the extent to which the policies and procedures of the Office of the Correctional Investigator have enabled it to carry out its mandate effectively. We observed weaknesses in five areas: workload management, communications, operational policies and procedures, resolving systemic issues with Correctional Service, and reporting results. We concluded that while the practices of the Office of the Correctional Investigator were often helpful in resolving individual complaints, they were not efficient and consistent, and had contributed, among other things, to an adversarial relationship with Correctional Service.

Scope

Conclusion

32.232 However, further progress is required in some other areas. For example, work remains to be done in further developing the management information system, reporting results and implementing the communications action plan. It is essential that the Office implement the applicable recommendations related to these areas.

Observations

Managing the workload

32.234 Officials from both organizations have indicated that there is now less overlap between the Correctional Investigator complaint process and Correctional Service inmate grievance process. The Office continues to reinforce an internal policy that, under normal circumstances, investigators should not initiate an investigation when the same complaint is still being examined within the Correctional Service's inmate grievance process. In addition, the Office has worked with Correctional Service to improve the inmate grievance process in such areas as timeliness and the use of lessons learned from upheld grievances in its management decisions.

32.235 In the past, investigators lacked clear guidelines concerning which cases to investigate and the level of effort to invest given the case circumstances. As a first step, the Office has provided its investigators with a new, approved policy and procedures manual to assist in differentiating between the requirements for an inmate contact, an inquiry and an investigation. This can potentially reduce the level of work required per inmate concern and thus further facilitate overall workload management.

32.236 Our 1997 Report indicated that the demand for the services of the Correctional Investigator was incessant and that both the demand and complexity of issues involved have increased in recent years. The number of individual complaints reported by the Correctional Investigator in his annual report has decreased significantly from about 6,800 (1995-96) to 4,500 (1998-99). However, Office management has indicated that during 1998-99, more time was dedicated to other tasks such as review of the Corrections and Conditional Release Act, review of Correctional Service Canada policy and procedures, and increased complexity of issues associated with women and Aboriginal inmates. There is now also the ongoing requirement that all videotapes and supporting documentation on Emergency Response Team Interventions and Use of Force be forwarded to the Office for review. During 1996-97, the Office received videotapes and supporting documentation for 23 incidents. The following year, the number increased to 198, and for 1998-99, the number was 214. Investigators also estimate that the new Case Tracking System will require more detailed information input that, over the short term, will increase the time required to process each case.

Communication strategy

32.238 Interviews with officials from the Office and Correctional Service indicate that both inmates and Correctional Service staff still misunderstand the role of the Office. Their recent meetings with inmate committees and outside stakeholders have reinforced this observation.

32.239 To address this concern, the Office has designed a poster and brochure for future distribution to all institutions and parole offices. The task has not yet been completed. Work continues on editing the content of these documents as well as acquiring a toll-free telephone number to facilitate inmate access to the Office.

Resolving systemic issues

32.241 Over the past two years, officials from the two agencies have worked together to develop a memorandum of understanding that provides a structure for interaction between the two parties for dealing with investigations and resolving issues. Key sections of this document include a description of the role of the Office of the Correctional Investigator, the process for communications, parameters for closure of investigations, and a non-binding dispute resolution mechanism.

32.242 We understand that both the Correctional Investigator and the Commissioner of the Correctional Service have agreed to the memorandum and are expected to sign it later this year.

Complaint investigation policies and procedures

32.244 By the end of 1998, the Office had implemented an approved policy and procedure manual and briefed all staff. The manual was a first major step toward handling inmate problems and reporting Office performance results in a consistent manner. As part of the new manual, the Office plans to implement its designed quality control regime in the fall of 1999 to ascertain the level of investigator compliance with approved policy.

32.245 The Office is now proceeding with further improvements and enhancements that will build on the progress already made. Using the content of the manual, the Office is creating two needed by-products: a more readable policy and procedure manual for investigators; and better-quality information on its existing Internet Web site for external stakeholders.

32.246 We also noted in 1997 that the Correctional Investigator lacked procedures that could help flag systemic issues. Both the existing procedures manual and future planned revisions will address part of this concern. Investigators will also need an approved summary of the Office's publicly stated position on recurring inmate issues. If provided, this will ensure that investigators communicate the same policy position for each recurring systemic issue to all stakeholders.

Reporting results

32.248 In April 1999, the Office implemented a new management information database, designed to support its new policy and procedures manual. Historical data have been downloaded and the system is functional. However, consistent and timely new data input by investigators is still a challenge. More work is now required to integrate the system with its hardcopy files and operational requirements. The new system also has an audit capability to assure data input quality and compliance with approved procedures. This capability will not be tested until the fall of 1999.

32.249 The new system must have the capability in the future to report results in terms of contacts, inquiries and the disposition of investigations as proposed in Exhibit 33.5, 1997. Currently, the database has not evolved sufficiently to be able to provide such performance information. The next phase of the planned system development will involve identifying performance reporting needs and building the capability for the Correctional Investigator's annual report in 2000.

RCMP Public Complaints Commission - 1997, Chapter 34

Background

32.251 The purpose of our 1997 audit was to determine whether the Commission had adequately organized itself to discharge its mandate. The audit focussed on such areas as planning, communications, the review process, terms of reference and member training for public hearings, management information systems and reporting of results. We concluded that while the Commission had established systems and practices to fulfil its mandate, a number of areas in the Commission's management practices needed to be strengthened: communications strategy; action plans for the Commission's review process and public hearings; and performance reporting.

Scope

Conclusion

32.254 However, further improvements are still required in important areas. Although the Commission has processed an increased number of complaints, a large backlog of 400 cases remains. The Commission now needs to eliminate this backlog and achieve its revised turnaround standard for processing complaints, as planned by 2001.

32.255 In addition, the Commission needs to restructure its management information system with a view to ensuring reliable data quality and an enhanced ability to report its results annually to Parliament.

Observations

Communications strategy

32.257 The Commission has still not approved a long-awaited communications strategy and plan. However, public awareness of its role was heightened greatly by the recent Public Interest Hearing into the conduct of RCMP officers at the Asia Pacific Economic Co-operation (APEC) conference at the University of British Columbia in November 1997.

32.258 Commission officials expressed that they still need to communicate their role better, particularly in small communities, with natives and visible minorities. They indicated that they will hire a communications officer whose first task will be to develop a communications plan for implementation in early 2000.

Streamlining the review process

32.260 The Commission has recently streamlined its review process in three key areas: simplifying the quality control phase of its review process; making some critical policy changes to ensure that each review is better focussed and managed; and implementing administrative changes such as simplifying letters and reports. During the 1998-99 fiscal year, the Commission had completed about 350 of its 500 backlogged cases. This was the first time that it had processed more complaints than it received (260 complaints) during the same time period.

32.261 Currently, the Commission has an inventory of about 400 outstanding complaints. It has targeted March 2001 to eliminate the entire backlog and implement fully its 120-day case turnaround standard.

Public hearings

32.263 In order to provide a sharper focus to its public hearings, the Commission included written terms of reference in its Notice for the November 1997 APEC Public Interest Hearing. The Commission has indicated that it intends to repeat this practice for all future public hearings.

32.264 Most of the current members have received public hearing training over the past year and a half from one or more of the following sources: a conference on the basics of tribunal management; a workshop concerning current issues in public hearings; and a one-week training program in administrative tribunals. The Commission is currently developing an in-house training program that, among other things, will address critical and emerging issues related to public hearings.

Measuring and reporting results

32.266 In 1999, the Commission implemented a new management information system. Data quality remains a concern as some old case records have not yet been converted to the new system and, in some instances, are incomplete or of poor quality. In addition, the Commission has not yet undertaken a review of system data quality against actual file documentation. Staff will not be trained on the new system until priorities such as improving the speed of data input from the Commission's remote site (Surrey, British Columbia) and installing a reporting capacity have been met.

32.267 At present, a report-generating capacity for the new system has not been designed. Performance reports on matters such as case turnaround time are still prepared manually. Commission officials have indicated that they will not adjust how they report results in their annual report until the reporting component has been installed in the new system. The Commission expects to be able to provide enhanced performance information in its 1999-2000 annual report.

32.268 We also recommended two years ago that the Commission develop better measures of turnaround time for its reviews. At that time, the Commission was not calculating turnaround from when it received the request for review but rather from when the required documentation arrived from the RCMP. We stated that it would be more reasonable from both the complainant's and the RCMP member's perspectives to calculate time from when the Commission received the initial request for review.