2000 April Report of the Auditor General of Canada
Chapter 1—Service Quality
1.10—Rethinking, Re-engineering and Restructuring the Service Approach at Citizenship and Immigration Canada
1.14—National Variation in Accessibility of Telephone Services—Canada Customs and Revenue Agency (Taxation)—April 1996 to March 1999
1.1 The purpose of our audit was to see whether the federal government is providing better quality of service to Canadians, after a decade of commitments and a series of initiatives to improve it. In 1996, we provided a midcourse assessment of the government's progress. In the current audit, we revisited the same 13 services delivered by 10 government departments and agencies.
1.2 The most significant improvement since 1996 has been to make telephone services more accessible. Canadians are now able to obtain faster responses to their enquiries once their calls are answered. Despite that improvement, however, we are concerned by the high percentage of calls not answered when lines are busy and calls abandoned while the caller waits on hold to speak to an agent. At the Canada Customs and Revenue Agency (Taxation) the percentage of unanswered calls remains high at 54 percent, although it has improved since 1996, when it was 73 percent.
1.3 For most services delivered by means other than telephone, such as counter service, there was not enough performance information for us to determine whether service has improved. The Passport Office has measured some of its key results and demonstrated significant improvement in its performance. Many departments and agencies have taken initiatives since 1996 to improve service but have not sufficiently measured the results.
1.4 Departments and agencies need to consult more with their clients to identify the aspects of service that matter most to them, as well as the quality of service they expect. This information would help the service providers establish appropriate targets and indicators of performance to measure.
1.5 Communication to clients at points of service has improved since 1996. Canadians visiting these offices are better informed about the level of service they can expect. However, more attention is needed to informing them on what it costs to provide the services, whether targets have been met and how they can lodge and resolve complaints.
Background and other observations
1.6 The demand by Canadians for services of the federal government - such as enquiries about citizenship and immigration, Old Age Security, passports, weather or statistical information - has increased significantly since 1996. Volumes of service are higher in 10 of 13 service lines we audited. At the same time, the environment for delivering services continues to evolve with rapid advances in information technology and growing use of alternative methods of service delivery.
1.7 Telephone services have become increasingly prominent, while the use of counter services has declined. In the government telephone services we audited, the volume of telephone enquires climbed from 36 million in 1996 to 56 million in 1999, an increase of 54 percent. The numbers of written enquiries (mail, fax and e-mail), while still important in some services lines, are generally much lower.
1.8 Although service managers have acted on our 1996 recommendations and sought ways to make continuous improvements, none of the recommendations has been implemented fully. However, we noted a shift among public service managers and staff toward a stronger focus on service and innovation.
1.9 We were discouraged to find slow progress on the project to improve the government's telephone directory listings (the Blue Pages). The need to redesign them was identified in 1990 by the Public Service 2000 Task Force on Service to the Public.
1.10 Over the past three years, departments and agencies have provided Parliament with more information on service quality, and the information they report is more likely to be meaningful. But better information is needed on performance trends and on costs.
1.11 Since 1998, the Treasury Board Secretariat has given departments and agencies more guidance and support to improve service delivery. However, it has not systematically monitored progress across the government, and the information it has reported to Parliament has not been adequate to provide a clear understanding of progress in improving the quality of service to Canadians.
The response of the Treasury Board Secretariat on behalf of the government is included at the end of the chapter. The Secretariat indicates that the new Blue Pages telephone directory format will be implemented according to established publication schedules. The Secretariat agrees with our recommendations on measurement, client satisfaction and reporting. It indicates that it is working with departments to develop an approach designed to promote continuous improvement in service delivery across the government.
Service quality is important to all Canadians
1.12 All Canadians require the services of the federal government at one time or another. These could be services they use regularly in their everyday lives, such as getting weather information, or services they need occasionally. For example, Canadians may have to obtain a passport when travelling to another country. Once abroad, they may need consular services. When returning, they must go through customs and may have to pay taxes and duties on goods they bring back to Canada (see Chapter 5, Canada Customs and Revenue Agency - Travellers to Canada: Managing the Risks at Ports of Entry).
1.13 At certain times in their lives, some Canadians may require services related to Canada Pension Plan benefits and Employment Insurance benefits (see Chapter 2, Human Resources Development Canada - Service Quality at the Local Level). Similarly, Canadians who wish to start a business may look to the federal government for economic and market information.
1.14 Given the pervasiveness of government services and their importance to Canadians, their quality is a matter of wide concern. As clients of specific services, Canadians have the right to expect high quality. At the same time, the government has the obligation to provide high quality at an affordable cost. To reach an appropriate balance between cost and quality, it has to weigh the interests of individual clients against the broader interests of all citizens and taxpayers.
The government has expressed its commitment to service quality since 1990
1.15 In its 1990 white paper, "Public Service 2000, The Renewal of the Public Service of Canada", the government explicitly committed itself to delivering high-quality services to Canadians. The President of the Treasury Board was assigned overall responsibility for this commitment. Starting in 1992, the government required departments and agencies to put in place some key elements to improve service quality. These included targets for delivery, measures of performance toward those targets, cost information and complaint and redress mechanisms. Departments and agencies were to communicate these elements to clients at points of service, along with descriptions of the services and pledges of the quality of service that clients could expect.
1.16 In June 1995, the government launched the Quality Services Initiative. It outlined specific actions to be phased in across the government over three years. While the key elements established in 1992 were an integral part of the Quality Services Initiative, the new approach stressed the importance of client satisfaction, employee involvement, innovation and the celebration of success.
1.17 The government's initiatives to improve service quality have evolved since 1996, when we last reported on their progress. The government remains publicly committed to strengthening the quality of service delivery. It has reaffirmed its commitment on a number of occasions and in a variety of ways (see Exhibit 1.1).
1.18 In April 1998, the government responded to the Fifth Report of the Standing Committee on Public Accounts, which dealt with our 1996 audit of service quality. The government's response clearly emphasized a commitment to service quality, and outlined a framework for organizing services from a citizen's standpoint.
1.19 Other jurisdictions in Canada and abroad are also moving to make government services more client-focussed and to improve their quality. For example, the Province of Ontario is implementing its "Customer-Centred Government", a strategy that aims to deliver service of high quality from the customer's perspective and to increase public satisfaction. The Province of New Brunswick has established a corporation, Service New Brunswick, with a mission to improve the delivery of government services to the public. One improvement is electronic service delivery through a network of commercial service centres.
1.20 The Organization for Economic Co-operation and Development (OECD) recently reported that making service more responsive to citizens is a key objective of public management reform in all member countries. The United States government is implementing the National Partnership for Reinventing Government. A focal point of that reform effort is to put customers first in service delivery. The United Kingdom's "Service First" initiative, built on its Citizen's Charter, established new principles of public service delivery. It focusses on responsiveness, quality, effectiveness and working together with other service providers.
Time to revisit service quality
1.21 In 1990, the Public Service 2000 initiative envisioned that public servants by the year 2000 would be operating in a much more flexible and service-oriented environment. Our 1996 audit assessed the government's progress at midcourse toward establishing such an environment. Our report made several recommendations to improve service, many of which the Standing Committee on Public Accounts also made in its Fifth Report (see Appendices A and B).
1.22 After a decade of government commitments to service quality, and a series of initiatives to improve it, we believe that now is a good time to look at the results.
Focus of the audit
1.23 We audited the same 13 service lines covered in our 1996 audit, including the same 6 telephone operations (see Exhibit 1.2). These 13 service lines represent highly visible, frequently used services to the public. Our purpose was to see whether their quality has improved since 1996, and by how much. We looked at whether the departments and agencies that deliver the services have measured their results and reported them to Parliament. We also examined what the Treasury Board Secretariat has done to help them improve service quality. Our audit covered the delivery of the services between 31 March 1996 and 31 October 1999.
1.24 As this audit focussed on results, we asked departments and agencies to provide us with their performance information. However, we did not audit its accuracy. We assessed the information using the criteria developed for our audit.
1.25 This chapter provides a general assessment of progress made in improving service quality government-wide. It is not intended to give a full account of quality in each service line we covered. Where appropriate, we used specific examples to illustrate general points.
1.26 Further details on our objectives, approach and criteria are presented at the end of this chapter in About the Audit.
Observations and Recommendations
The Environment for Service Delivery in the Federal Government
1.27 Fiscal restraint and downsizing of the public service in the 1990s forced the government to find smarter and more cost-effective ways to do business, including the way it delivers services to Canadians. A number of factors have provided both opportunities and challenges for the government in meeting its service quality commitments.
1.28 Advances in information technology. Information technology has grown exponentially. The Internet has become a major tool for providing information, and Canadians can now turn to the Web sites of all departments and agencies. An example is the spectacular growth in the use of the Internet for specific information on national parks, national historic sites and canal and national marine conservation areas - from a negligible volume in 1995-96 to around 25 million "hits" in 1998-99.
1.29 When Canadians were asked recently how they wanted to obtain government services, many favoured electronic service delivery. A quarter of the respondents were comfortable with kiosks and almost 20 percent preferred Internet-based services. In the 1999 Speech from the Throne, the government said it would become "a model user of information technology and the Internet." Its "Connecting Canadians" initiative has a goal of making all government information and services available to Canadians on-line by 2004.
1.30 Call centres have become widespread. The telephone has become a preferred mode of service for many clients and the public. Several departments and agencies are using call centres as a more cost-effective method of service delivery. Examples are Canada Customs and Revenue Agency, Citizenship and Immigration Canada and Human Resources Development Canada. The private sector's use of call centres has also grown. For example, the Amex Canada Inc. call centre has grown in the past three years from fewer than 200 agents to nearly 800. This call centre provides telephone services to clients in Canada and the United States and, through its outsourcing division, to clients of other businesses.
1.31 Alternative and integrated service delivery. The federal government has established a number of new ways to deliver public services. These include service agencies, like the Canadian Food Inspection Agency (created in April 1997), Parks Canada Agency (December 1998, formerly in the Department of Canadian Heritage) and the Canada Customs and Revenue Agency (November 1999, formerly Revenue Canada). The government has also set up collaborative and partnering arrangements among departments, and with other governments, the private sector and not-for-profit organizations. It is using them increasingly to provide more cost-effective services. An example is Industry Canada's network of Canada Business Service Centres, which involves joint ventures with other federal departments, provincial ministries, municipal governments and private sector organizations. (See photograph)
1.32 Calls for "single window" service delivery have required that departments and agencies look for ways to manage their services jointly and sometimes to deliver services with other levels of government. For example, in partnership with the Ontario Ministry of Community and Social Services and the Regional Municipality of Hamilton-Wentworth, Human Resources Development Canada has established a single window where the public can go for services related to income support, employment preparation and job searching. As we completed our audit, there were also a number of pilot projects under way through Service Canada, a government-wide initiative to provide Canadians with one-stop access to a range of services (see Exhibit 1.3).
Most service volumes have increased since 1996
1.33 Since our last audit, one of the most noticeable changes in the service lines has been the sheer volume of services they provide. Volumes are higher in 10 of 13 service lines we examined (see Exhibit 1.4). Telephone services have become increasingly prominent, while the use of counter services has declined. And numbers of written enquiries (mail, fax, e-mail), while still important in some service lines, are generally much lower.
1.34 In 1996 we noted that direct deposit payments could be a cost-effective way to reduce unnecessary telephone calls. Since then, as the current audit found, direct deposit of major payments has increased significantly. At 31 March 1999, the government was using direct deposit for 81 percent of Old Age Security payments and 77 percent of Canada Pension Plan payments - each up 17 percent since 1996. Direct deposit of Child Tax Benefit payments was at 81 percent, an increase of 24 percent over 1996.
1.35 Nevertheless, the volume of telephone enquiries overall has continued to grow. In the government telephone services we audited, the volume climbed from 36 million in 1996 to 56 million in 1999, an increase of 54 percent in three years (see Exhibit 1.5). As a result, a number of departments and agencies had to expand their telephone services. Taxation had to restructure some of its telephone operations: it now has a call centre in Toronto dedicated to telephone operations only, with counter services available at other locations. In Canada, Citizenship and Immigration Canada now provides most of the information on its services by telephone. It has consolidated 23 small telecentres into three large call centres in Vancouver, Toronto and Montreal.
1.36 Departments rely increasingly on methods of service delivery that do not require direct contact with the client. For example, improved telephone technology allows automated voice response systems to handle a higher share of simple enquiries. In 1998-99, almost 60 percent of enquiries to the telephone services we examined were handled by automated systems. Another means of service delivery is the Internet. Contacts to obtain weather information from Environment Canada's Web site jumped from fewer than two million in 1996 to well over 50 million in 1999.
Some demonstrated improvements in service quality
1.37 We asked the departments and agencies to provide us with performance results for the 13 service lines from 1 April 1996 to 31 March 1999. Where possible, we tracked trends in service performance since 1996. In some cases, we were able to assess performance in meeting service delivery targets (for example, timeliness and accessibility) established by the departments and agencies.
1.38 Four of the six telephone operations and one other service provided performance data that demonstrated improvement in at least some aspects of service quality. The four telephone operations were in:
- Canada Customs and Revenue Agency (Taxation);
- Citizenship and Immigration Canada;
- Human Resources Development Canada (HRDC-Employment Insurance); and
- Human Resources Development Canada (HRDC-Income Security).
The other service was the Passport Office of Foreign Affairs and International Trade.
1.39 It is important to note that these cases reflect measured progress in only some aspects of service quality. Departments and agencies do not always set targets for, or measure, all aspects of service quality and client satisfaction that may be relevant.
1.40 Telephone services: progress but some concerns. In 1996, we noted serious problems with accessibility of service in large telephone operations. This area shows the most significant improvement in service quality.
1.41 A caller's first priority is to gain access to the telephone system. Busy telephone lines were cited as the most common problem by respondents to the Citizens First survey (carried out for the Citizen-Centred Service Network, see paragraph 1.65). Even callers who do not get a busy signal may abandon the call in frustration if the telephone continues to ring with no answer. Some may get an answer but abandon the call if they have to wait too long on hold (see Exhibit 1.6).
1.42 So an important indicator of the quality of telephone services is the proportion of calls that go unanswered (whether busy or abandoned). As Exhibit 1.7 shows, over the past three years this proportion has fallen in the four largest of the six telephone services we examined.
1.43 Nevertheless, the proportion of calls that go unanswered remains high - in 1998-99 at 10 percent and higher in the six telephone operations we examined. By comparison, good practice in the private sector puts the proportion at three to five percent.
1.44 In 1998-99, 28 percent of calls to Citizenship and Immigration Canada were not answered. In Taxation, it was 54 percent, compared with 73 percent three years earlier. The fact that five of every 10 calls go unanswered does not mean that half the callers do not get service. It does mean that many may have to make several calls before they are served.
1.45 Some telephone services have established targets for the proportion of answered calls that are answered within a certain number of rings or a certain number of seconds. The largest of the operations with such targets and with performance data for several years (HRDC-Income Security) showed significant improvement. Its target for answered calls is to answer 95 percent within three rings. From achieving only 54 percent in 1994-95, this service improved to 98 percent in 1998-99. HRDC-Employment Insurance also exceeded the same target and achieved 99 percent in 1998-99.
1.46 In the other two operations with like targets (Reference Canada and the Canada-Ontario Business Call Centre), performance has declined since 1995-96. Each service has a target for answered calls of 85 percent answered within three rings. In 1995-96, Reference Canada reached 93 percent but dropped to 83 percent in 1998-99. The Canada-Ontario Business Call Centre achieved 90 percent in 1995-96, compared with 84 percent in 1998-99. In the latter case, Industry Canada officials attributed the decline in performance to problems with staff turnover in 1998-99, which they indicated have been resolved.
1.47 Another measure of the quality of telephone service is the length of time a caller waits on hold or in a queue after the call is answered (see Exhibit 1.7). Here, the available data show mixed performance. HRDC-Income Security, the Canada-Ontario Business Call Centre and Reference Canada met their performance targets in 1998-99. In the Canada-Ontario Business Call Centre and Reference Canada, however, the wait time in queue has increased since 1995-96. The other three telephone services (Taxation, Citizenship and Immigration Canada, HRDC-Employment Insurance) did not meet their wait-time targets. However, Taxation reduced its average wait times from 111 seconds in 1995-96 to 80 seconds in 1998-99.
1.48 Service lines have generally improved telephone services by extending their hours of service. Of the telephone operations we examined, all but Reference Canada make basic information accessible around the clock, seven days a week, through interactive voice response systems or automated telephone information systems. Reference Canada does not use automated systems because it provides information on many government services, and needs to interact with callers to assess their requirements. To focus more on this personal service and to support initiatives such as Service Canada, Public Works and Government Services Canada is replacing Reference Canada with a new "1-800-O-Canada" service.
1.49 Other services. In other service operations, we noted that the Passport Office has significantly improved its performance in issuing passports by mail within its target of 10 days from application. From 63 percent of passports issued within 10 days in 1995-96, it improved to 97 percent in 1998-99.
Some services lack information needed to demonstrate progress
1.50 Six of the 13 service lines have not provided performance information in a form that demonstrates whether they have improved service quality since 1996. In most cases, they have not tracked performance results systematically against targets or over time, or have not rolled up results annually, or have no systems to collect results measured at local levels and report national performance. The six service lines are delivered by:
- Customs (customs inspection)
- Environment Canada (providing weather information)
- Foreign Affairs and International Trade (consular services)
- Parks Canada (services to national park visitors)
- Royal Canadian Mounted Police (providing police assistance)
- Statistics Canada (providing statistical information)
1.51 These same service lines may indeed have improved service quality, given the initiatives they have taken. For example:
- Customs introduced changes to speed up the clearance of low-risk, frequent travellers entering Canada at major border crossings and at one airport.
- Environment Canada now provides more precise five-day weather predictions.
- Consular Service now operates an after-hours emergency line to Ottawa from posts abroad, supported by a computer system that allows access to case information worldwide.
- Parks Canada initiated a toll-free telephone service for reservations in national park campgrounds of Atlantic Canada, the Prairies and British Columbia. Last year it also introduced a national 1-800 number where callers can ask for general information.
- The RCMP developed a differential response model - a systematic way to identify priorities in responding to calls for police assistance. Several detachments across Canada now use this model.
- Statistics Canada introduced a client "helpline" for users of its electronic products, including Internet commercial services.
1.52 Departments and agencies should measure the results of service performance and track them over time to identify changes in service quality and be able to demonstrate progress.
Unsatisfactory progress with the Blue Pages
1.53 In 1990, the Public Service 2000 Task Force on Service to the Public identified the need to improve the Blue Pages - the government's telephone directory listings. The purpose was to make it easier for Canadians to find the right telephone number for the government service they wanted.
1.54 In 1996, we noted some changes in the structure of the Blue Pages but said that further improvement was possible. In 1997, the Treasury Board Secretariat partnered with Public Works and Government Services Canada (Government Telecommunications and Informatics Services - GTIS) to redesign the Blue Pages.
1.55 The objective was to make the Blue Pages easier to use by listing the services of all levels of government by type of service instead of by provider (department or agency). Adding new information, such as hours of operation and Web site addresses, was to improve access to services. A new electronic platform was to be developed by June 1999 to improve the collection of information and allow for the integration of provincial and municipal information. Detailed plans were prepared, and in March 1999 the government gave the Secretariat approval to go ahead with revamping the Blue Pages. The project was to be completed by the end of 2000.
1.56 Progress has been slow. In the current audit, we were discouraged to find that this project has made little progress. By the end of 1999, only seven cities across Canada had completed pilot projects. The City of Kingston piloted a directory listing of three levels of government (see Exhibit 1.8). The Province of Ontario has agreed to a province-wide roll-out of the new Blue Pages in 2000, beginning with Kitchener-Waterloo. However, it is not clear to us how the roll-out in the rest of Canada can be achieved by the end of 2000.
1.57 GTIS told us that in September 1999 it suspended work on developing the electronic platform, due to lack of funds. At the time of our audit, it was looking for alternative funding to resume the project. In the meantime, the Secretariat is continuing with its project to secure the participation of other levels of government and start negotiations with telephone companies.
1.58 The purpose for the original collaboration between the Secretariat and GTIS was to take advantage of the expertise of each. Both have to work in tandem to move the project forward. A co-ordinated effort is needed now to ensure that the benefits of the redesign are not deferred, and to minimize the project's cost. Currently, only partial cost information is available. As part of the management control framework for this project, and in order to carry out proper business-case and cost-benefit analysis, project authorities need to track total costs.
1.59 Learning from the American experience. A blue pages redesign project for federal listings has been under way in the United States since 1995. The U.S. Vice-President has called it an important government-wide service initiative. The project's milestone reports indicate that by the end of 1999, 37 states were using the updated directories.
1.60 The Treasury Board Secretariat took the initiative to learn from the U.S. model for its electronic platform, which was provided to Canada. The GTIS project plan includes determining the extent to which Canada can make use of the U.S. platform. In the meantime, the Ontario government has begun "Canadianizing" the U.S. model.
1.61 Given a decade of activity already spent on the Blue Pages redesign project, the government should set realistic target dates for its timely completion and ensure a co-ordinated approach by the organizations responsible for the project.
A shift toward a culture that fosters a focus on service and innovation
1.62 In 1998, Consulting and Audit Canada surveyed 55 services provided by 23 departments and agencies. The survey showed the emergence of a client-oriented service culture, "with service managers appearing to have a genuine desire to implement client-oriented service standards and satisfy clients." The 1999 Public Service Survey supported this perception, with 75 percent of respondents agreeing that their work units had client service standards.
1.63 Our own observations and interviews during our audit found a concern for service quality among public service managers and staff. Some initiatives designed to develop a stronger focus on service indicate a cultural shift. Several departments and agencies have found innovative ways to deliver services, ranging from technology applications to improvements in process. Exhibit 1.9 shows some examples. (See photograph)
1.64 The search for more cost-effective ways of providing services has led some departments and agencies to rethink, re-engineer or restructure their approaches. Since 1996, for example, Citizenship and Immigration Canada has completely re-engineered its service operations. It has phased out most of its counter services and adopted telephone operations as its primary channel of information on service delivery. This has included consolidating 23 telecentres into three large call centres (see Exhibit 1.10).
Measures and Targets to Judge Performance
Accessibility and timeliness of service are most widely measured
1.65 In July 1997, the Canadian Centre for Management Development established the Citizen-Centred Service Network. This is a network of more than 200 officials from federal, provincial and municipal governments, as well as academics and experts in public sector service delivery. One of its research initiatives, Citizens First, was an extensive survey of Canadians to see what they think of the quality of services their governments provide. Some 2,900 Canadians responded, and the October 1998 survey report identified timeliness as their most important gauge of service quality.
1.66 In the 13 service lines we examined, we found a considerable emphasis on measuring both the accessibility and the timeliness of service delivery, especially telephone services (see Exhibit 1.11). As we have already noted, all six telephone operations have targets for accessibility and timeliness, often measuring performance by percentage of total calls answered, percentage of calls answered within a certain number of seconds or rings, and wait times in a queue (see Exhibit 1.7).
1.67 Our 1996 audit report noted that service quality has other important dimensions. These include accuracy or reliability, courtesy and the service environment - the physical facilities where clients obtain service. In the current audit, we found that fewer service lines use targets and measures for these dimensions of service quality than for timeliness and accessibility.
1.68 Among the 11 services delivered by means other than the telephone, only three measure the quality of the service environment and six measure accuracy or reliability. Among the telephone services, we found that Human Resources Development Canada has made little use of its accuracy measures. However, Taxation does have a target for accuracy; it has used an independent firm to measure the accuracy of responses provided by the General Enquiries and the Business Window telephone services. In 1999, for the first time in the past few years, General Enquiries had an accuracy rate of over 80 percent. (See photograph)
1.69 All of the large government telephone services we audited have some form of internal monitoring methods such as silent monitoring of live or taped calls, "mystery shopper" techniques and peer reviews. However, these are not yet fully developed monitoring systems. Most are not used systematically to monitor telephone contacts or provide feedback to call agents. Some rely on the voluntary participation of call agents.
1.70 Departments and agencies should develop and implement systems to measure results for all aspects of quality that are important in delivering a particular service.
Better consultation with clients needed to establish targets and measures
1.71 We recommended in 1996 that service managers ensure that delivery targets reflect clients' priorities. In the current audit, we found that both the RCMP and the Consular Service of the Department of Foreign Affairs and International Trade consult with clients before establishing targets. Three other services do this in part.
1.72 Managers in 8 of the 13 service lines, however, have not explicitly considered client priorities in setting particular delivery targets. For example, telephone call centres generally set service targets for such aspects as accessibility and wait time in queue without first determining the specific needs and preferences of the service line's clients.
1.73 We also found that clients are not consulted enough to ensure that measures and targets are established for the dimensions of service quality that matter most to them. As an example, although call centres' systems can readily provide information to measure performance in several aspects of service, measuring those aspects alone may jeopardize other dimensions of service that are not measured as easily but are important to clients. These might include courtesy, for example, or fully meeting the client's needs in a single contact.
1.74 In some cases, the amount of time spent on each call (talk time) was used as an internal measure of performance. We believe that using this measure may put pressure on call agents to hurry the contact in order to reduce talk times. If so, clients may have to call again to deal with unresolved issues. Amex Canada Inc., which the National Quality Institute has recognized for its success in call centre operations, has consulted its clients and now focusses on resolving problems during the first call rather than measuring talk time. The Canada-Ontario Business Call Centre recognized the need to respond more fully to client enquiries and has eliminated its talk-time target while maintaining service at established levels.
1.75 More effective service by one agent on the first call can improve the client's view of service quality. However, this requires that call agents have on-line access to databases with up-to-date information on clients' claims or applications. We found that such access is currently limited in all of the large government telephone services we audited. Call agents thus are not always able to answer enquiries about, for example, processing times. (See photograph)
1.76 Departments and agencies should consult with clients to help establish delivery targets and to ensure that they manage and measure aspects of service that matter most to clients.
Some improvement in providing key information at points of service
1.77 Since 1996, service lines have considerably improved their communication of pledges and delivery targets to clients, in brochures and pamphlets made available at points of service (see Exhibit 1.12). The brochures and pamphlets usually describe the service, make pledges for matters such as fairness and courtesy, and set out their targets for service delivery. The service lines have also posted pledges and commitments to quality of service on their Web sites.
1.78 In communicating other key information to clients at points of service, however, the service lines we audited have made less progress (see Exhibit 1.12). Although some service lines provide information on how clients can lodge complaints and obtain redress, very few provide performance results against targets, or information on service costs.
1.79 Departments and agencies should communicate to clients at points of service the results each service has achieved, the cost of providing it, and the complaint and redress mechanisms available to clients.
Managing for Continuous Improvement in Service Performance
1.80 We examined service managers' efforts to continue improving service quality. These included:
- conducting surveys of client satisfaction;
- conducting audits and evaluations;
- developing action plans for continuous improvement;
- establishing complaint and redress mechanisms; and
- analyzing and using performance data.
1.81 We found that the use of these techniques for continuous improvement varies (see Exhibit 1.13). Several service lines - the Passport Office, Taxation, Customs, Environment Canada and the RCMP - have made consistent efforts using almost all of the techniques to identify areas that need to improve.
Increased use of surveys and reviews to help identify areas for improvement
1.82 We found that client satisfaction surveys have been widely used at varying intervals and on a national, local or case-by-case basis. Apart from Customs, Taxation and Statistics Canada, however, service managers generally were not using the surveys to ask clients about their priorities or to help identify the dimensions of service quality that are key to client satisfaction.
1.83 Parks Canada has monitored visitor satisfaction for several years by such means as visitor feedback cards, formal surveys and, occasionally, public consultation. However, client satisfaction surveys in the 9 parks we examined had uneven coverage of service dimensions. The surveys covered courtesy in 6 of the 9 parks and the service environment in 7, but none covered all dimensions of service. Parks Canada recognizes the need to improve its measuring of visitor satisfaction. It has piloted a series of standardized surveys that it is now introducing in all parks.
1.84 Ten of the 13 service lines we examined have carried out reviews (audits or evaluations) of service delivery. However, we noted that most of the reviews have not adequately assessed systems and practices for measuring service performance; nor have they reported the results they have achieved toward established targets.
More action plans for continuous improvement
1.85 We recommended in 1996 that service managers develop and follow action plans to improve service. In the current audit we found some progress: 9 of the 13 service lines have some form of action plan for continuous improvement. However, some of the plans take the form of fairly general information on future activities, reported in departmental or business line plans. Those plans lack specific details on their implementation, timelines, responsibilities and budgets.
1.86 We are encouraged to note that the interdepartmental Assistant Deputy Minister Advisory Committee on Service and Innovation, led by the Treasury Board Secretariat, undertook a project in 1998 to assist departments in designing and implementing service improvement plans.
Little use of benchmarking
1.87 Three of the service lines make systematic use of benchmarking to compare their practices and performance with similar organizations in other jurisdictions or in the private sector. We noted, however, that service managers in call centres we examined have not compared their performance with "best in class" results elsewhere.
1.88 The use of effective scheduling tools can improve the management of call centres. Unlike Amex Canada Inc., the call centres we examined have not adopted computerized scheduling systems to forecast workload and deploy call agents. In service lines where the demand for telephone services is volatile, scheduling can be critical. Exhibit 1.14 shows accessibility (calls answered) in Taxation over the past three years. While it is encouraging to note that the underlying trend in the proportion of calls answered climbed steadily from April 1996 to March 1999, the proportion continues to fluctuate widely.
1.89 Departments and agencies should use benchmarking as a source of learning about good practices to help identify opportunities for continuous improvement.
Complaint and redress mechanisms need improvement
1.90 In 1996 we recommended that service managers systematically collect and analyze complaint data and other feedback from clients, and devise methods to prevent mistakes from recurring.
1.91 The current audit found that complaint and redress mechanisms are still inadequate. Clients need to know the names and addresses of officials responsible for receiving complaints about service quality. For example, at certain border ports of entry, Customs has provided complainants with addressed envelopes. However, more than half of the service lines do not provide adequate information to clients at points of service on how to lodge a complaint. Further, without good mechanisms to collect data on complaints systematically, there is little opportunity for analyzing complaints as a way to identify opportunities for improvement.
1.92 Departments and agencies should demonstrate their commitment to service quality by informing clients at points of service on how they can lodge and resolve complaints.
Some progress in analyzing and using information on service performance
1.93 We recommended in 1996 that service managers collect, analyze and use information on service and cost performance to determine the highest quality of service they can provide at an affordable cost. We found that some have made progress since then. The majority of the 13 service lines were collecting at least some data on service performance and analyzing it to help identify opportunities for improvement.
1.94 We also recommended in 1996 that service managers (including telephone centre managers) collect and use performance data to analyze persistent problems of accessibility and to devise appropriate remedies. In the current audit we found that only three of the six telephone operations - Taxation, and Human Resources Development Canada's Employment Insurance and Income Security programs - have made satisfactory progress in responding to this recommendation.
1.95 Service managers have established credible performance measures for some aspects of service quality. Frequently, however, they lack enough good information on costs to assess the cost effectiveness of the level of service quality they have achieved.
1.96 We found that where a service involves user fees or cost recovery, better cost information is generally available. For example:
- Parks Canada established a revenue policy in May 1998 requiring that direct, indirect and capital costs be included in determining the cost of services.
- The $25 service fee included in the cost of a passport is based on both direct and indirect costs of consular services.
- The Passport Office is self-funding: it recovers its costs from the fees it collects for passport services. It also uses accrual accounting in its financial reporting systems.
- Statistics Canada collects information on direct and indirect costs and analyzes it to determine the cost of its products.
1.97 We are concerned that without good cost information, service managers do not have the information they need to provide the highest quality of service at the lowest possible cost.
1.98 For example, we noted that because Public Works and Government Services Canada pays for their facilities, departments and agencies do not always include facility costs in the cost of providing their services. Departments or agencies with telephone operations have their own call centres, and they may have overlooked opportunities to save costs by sharing facilities. Furthermore, major government call centres we visited in Toronto and Vancouver are located downtown, where costs are high. By comparison, Amex Canada Inc. has located its only call centre well outside of downtown Toronto.
1.99 Departments and agencies should collect, systematically analyze and use performance information, including cost information, to manage performance and continuous improvement.
Reporting to Parliament on Service Quality
1.100 In 1996 we found that departments and agencies needed to improve their reporting of service performance. The Standing Committee on Public Accounts also recommended that in their Performance Reports, departments and agencies include information on their service performance.
1.101 In its 1998 response to the Public Accounts Committee, the government committed itself to "ensuring that solid performance information on service delivery [is] available to Parliament and the public through Departmental Performance Reports and the Reports on Plans and Priorities." In our view, this means that departments are to report clearly on how well they are serving Canadians.
1.102 Departments and agencies are also required to report on how they are implementing the Quality Services Initiative. In particular, guidance from the Treasury Board Secretariat states that they should "tell the performance story" of service delivery. They are expected to make this a primary focus of their Performance Reports. Departments are also asked to report their progress toward focussing on the needs of citizens and clients.
1.103 We examined the performance information published on the 13 service lines in Reports on Plans and Priorities and Departmental Performance Reports for 1997 through 1999. We also examined the President of the Treasury Board's annual report to Parliament for those years. We classified the service quality information in those reports as expectations (or objectives), descriptions or results. We distinguished between general information on results and information linked directly or indirectly to previously stated expectations.
1.104 We expected that departments would tell in their Reports on Plans and Priorities how and to what extent they intended to improve service quality. We also expected that they would focus on the key service accomplishments and report them in their Performance Reports against previously stated expectations.
Reporting to Parliament on service quality has improved
1.105 The Reports on Plans and Priorities and Departmental Performance Reports were introduced only recently. Managers told us that performance reporting is still evolving and they plan to improve their information on service performance in the coming years.
1.106 Some improvements are already evident. In particular, the amount of information reported to Parliament on service performance increased significantly between 1997 and 1999. In our judgment, the information is also more meaningful. As Exhibit 1.15 shows, the proportion of information on results that we could link to expectations increased from 37 percent in 1997 to 64 percent in 1999 (although it decreased somewhat from 1998 to 1999). However, the links between results and expectations were not always explicit and we had to analyze the documents closely to identify some of them.
Several areas need more attention
1.107 We identified areas where departments and agencies still need to improve their reporting to Parliament on service quality.
- To show the progress they have made, departments and agencies need to report trends in service performance and adequately explain changes over time. We found that they have reported very little information on trends, although slightly more in 1999 than in 1997. The departments and agencies delivering 6 of the 13 services we examined had included some limited information on trends in their 1997 Performance Reports. By 1999, 8 of the services were reporting such information.
- Balanced reporting - reporting that shows both strong and weak performance - enhances credibility. We found that this area needs attention. Negative aspects of service performance or service quality initiatives are seldom reported. The RCMP Performance Reports, which discuss weaknesses in tracking client satisfaction, are an exception.
- The reporting of cost information is an area of significant weakness. We found almost no information on the costs of specific initiatives to improve service quality, nor cost information that would help put results into perspective.
- Information on service quality must be understandable and placed in context to be useful to parliamentarians. We found that departments and agencies tend to present general information about services clearly, but are less clear in explaining expected and actual results. For example, the impact of external factors on service outcomes is often not explained.
1.108 We recognize that performance reporting is evolving, and that service quality is only one element that departments report to Parliament. Nonetheless, the government has committed itself to improving service quality and is accountable to Parliament and the public for results.
1.109 Departments and agencies should clearly show in their Reports on Plans and Priorities the results they expect in service performance. In the corresponding Performance Reports, they should link those expected results to clear statements of the results they have actually achieved.
1.110 Departments and agencies should provide more balanced and complete information to Parliament on the results of their service performance. This should include information on both strong and weak performance, on trends, on costs and on external factors that affect service outcomes.
Guidance and Support by the Treasury Board Secretariat
1.111 In our 1996 audit report we outlined the Treasury Board Secretariat's responsibilities for the government's quality service initiatives. The Secretariat's approach at the time involved providing departments with support in meeting their performance objectives, acting as a catalyst to remove constraints, and celebrating innovation and success. We recognized the value of the Secretariat's activities in providing departments and agencies with guidance that clearly presented the quality management approach.
1.112 We recommended that the Secretariat encourage departments to take key steps toward a focus on clients and improved service quality. The Standing Committee on Public Accounts recommended that the Secretariat publish a framework for implementing the Quality Services Initiative, set a final date for its completion and provide departments with guidance and incentives to ensure its successful completion. In its April 1998 response to the Committee, the government did not accept the recommendation to set a final completion date. It stated that it was "continuously improving the quality of services to Canadians, [which] cannot receive attention for a fixed period of time only."
1.113 In 1997, a progress report from the President of the Treasury Board described the Quality Services Initiative as a three-year strategy to improve client satisfaction with the quality of service delivery. In its April 1998 response to the Public Accounts Committee, the government made a commitment to citizen-centred service delivery that would "broaden the former quality services initiative...to include new integrated approaches to service delivery such as single windows and partnerships."
The Treasury Board Secretariat's guidance and support has been uneven
1.114 The Secretariat has supported service quality initiatives in a number of ways since our last audit, and has continued some actions that were under way at that time. It continued its guidance and support to departments but was generally less active in 1996-97 and 1997-98 than since early 1998.
1.115 The following paragraphs set out some of the key guidance and support activities of the Secretariat since 1996. Exhibit 1.16 provides additional information.
1.116 In late 1997, the Secretariat and the National Quality Institute together developed a self-assessment tool called "Achieving Citizen/Client Focused Service Delivery, A Framework for Effective Public Service Organizations - The Quality Fitness Test." Its purpose was to help public servants and departments assess how well they were applying principles of service quality. Secretariat officials told us that in 1998-99 approximately 600 public servants were trained to use the Framework.
1.117 The Secretariat has continued since 1996 to support interdepartmental co-ordinating and working committees. It provides advice and guidance through an advisory committee of assistant deputy ministers. The committee met infrequently in 1996-97 and 1997-98. Since September 1998, it has met regularly and has co-ordinated a number of projects.
1.118 The Secretariat has given support to other quality networks, including the specialized Interdepartmental Call Centre Network and the Interdepartmental Service and Innovation Network. The latter was established to provide leadership for service improvement by sharing expertise among departments and fostering regional input.
1.119 The Secretariat has continued to sponsor regular events (such as "quality month"), to publish newsletters and to arrange visits and exchanges with other jurisdictions. These activities have contributed to interdepartmental communication, identification of best practices and celebration of success.
1.120 In early 1998, the Secretariat established a Service and Innovation Sector, with a mandate to focus on government-wide approaches. It then developed an action plan to advance citizen-centred service delivery, based on increasing Canadians' access to service and improving service performance.
1.121 In December 1998, Service Canada was established as a unit to provide Canadians with one-stop access to a range of government services in person, by telephone or electronically through the Internet (see Exhibit 1.3). At the time of our audit, the Secretariat was also developing InnoService - a Web site designed to inform public servants on how to improve service delivery.
1.122 Because departments' actions to improve service quality form part of a government-wide initiative, central co-ordination is required. Service Canada was established in response to this need. The Treasury Board Secretariat, as the responsible central agency, has accepted a role in providing guidance and support to departments and agencies. It needs to sustain a high level of attention to improve Canadians' satisfaction with the services they receive.
Reporting on Government-Wide Progress
1.123 In our 1996 audit report, we noted that the Treasury Board Secretariat was responsible for assessing progress in improving service quality and for reporting that progress to Cabinet and Parliament. At the time, it had provided little information to Parliament and we recommended that it report clearly to Parliament on the government's progress. The Standing Committee on Public Accounts also recommended that the Secretariat report progress annually to Parliament.
1.124 To report at a government-wide level, the Secretariat needs to monitor the progress of departments and agencies in implementing initiatives and improving service quality. In its response to the Public Accounts Committee, the government stated that it was important to monitor progress regularly and that the Secretariat would be "studying appropriate ways of monitoring departmental performance and reporting on progress to improve service delivery for the government as a whole."
Government-wide progress has not been monitored systematically
1.125 Until recently, the Treasury Board Secretariat relied on departmental business plans for information on service quality. However, the nature of the information it requested varied from year to year. To obtain a government-wide view, the Secretariat identified service delivery as one of several issues to be given special attention in the 1998-99 business plans. It also set out its expectations in greater detail than it had in previous years. However, annual business plans cannot be used to monitor progress now that departments are no longer required to submit them.
1.126 In the spring of 1997, Statistics Canada repeated a survey it had done in 1995 on behalf of the Secretariat. The survey showed that the federal public service was now making more use of practices designed to improve or maintain the quality of services. Consulting and Audit Canada carried out a survey for the Secretariat in the winter of 1998. That survey found that 48 of the 56 service providers who responded had standards in place for their services, and another 5 planned to have them later that year. These surveys, though potentially useful elements of a monitoring strategy, did not provide the information needed to assess departments' improvement of service quality.
1.127 In our view, the Secretariat has not adopted a systematic approach to monitoring the government's progress in improving service quality. This lack of systematic monitoring has impeded the Secretariat's ability to report results to Parliament.
1.128 Information on the progress departments and agencies are making is required not only for reporting but also to co-ordinate the government-wide initiative on service quality. Specifically, information on progress is needed to assess whether the government is meeting its commitments and to provide a sound basis for corrective action where progress is inadequate. In addition, the Secretariat needs to monitor progress in order to provide departments and agencies with appropriate guidance and support.
Treasury Board Secretariat needs to improve its reporting to Parliament
1.129 In March 1997, the President of the Treasury Board published a progress report on the Quality Services Initiative. The report provided an overview of the government's progress and made recommendations to departments. It stated that the Secretariat would monitor progress on those recommendations and would report in 1997-98. The Secretariat did not table this progress report in Parliament and has produced no further reports of this nature.
1.130 The Secretariat's own annual Performance Reports have provided Parliament with some useful information on service quality - for example, summaries of surveys on service delivery to the public. In the Performance Report it tabled in 1997, the Secretariat committed itself to include in future reports its analyses of the extent to which departments have integrated service quality into their operations. Although it reiterated this commitment in its Performance Report the following year, neither its 1998 nor its 1999 report presented any such analysis.
1.131 The President of the Treasury Board's Annual Report to Parliament, "Managing for Results 1999", noted the need to improve reporting to Parliament on matters that involve many departments. The Report also says, "Progress in achieving the goal of improved citizen satisfaction with service delivery will be measured through surveys of citizens' expectations and satisfaction undertaken every two years in co-operation with provincial governments." Results from such a survey are expected to be available in the fall of 2000.
1.132 Although these developments are promising, in our opinion the information so far reported by the Secretariat has not been adequate to give Parliament a clear indication of the government's progress in improving service quality.
1.133 The Treasury Board Secretariat should develop effective ways to monitor systematically and report to Parliament the government's progress in improving the delivery of services to Canadians.
1.134 Nature and extent of progress. We concluded that service quality has shown some improvement since our 1996 audit. Improvements were most evident in telephone operations, where performance is more likely to be measured systematically. Despite the improvements in service, however, telephone operations still need work. The proportion of calls that are not answered remains high, and delivery targets for aspects of service like wait time in queue are not always met.
1.135 We were discouraged to find slow progress on the project to improve the government's telephone directory listings (the Blue Pages). In our view, it will be difficult for the government to meet its target of completing the project by the end of 2000.
1.136 With the exception of the Passport Office, which has improved the timeliness of its service since 1996, we were unable to obtain the information needed to measure changes in the quality of services provided by means other than the telephone. These other services have taken steps to deliver services in more cost-effective and innovative ways but have not adequately measured the results.
1.137 Measuring and reporting service results. All telephone operations we examined have established measures and targets for accessibility and timeliness. However, departments and agencies do not always set targets for other aspects of service quality or measure client satisfaction.
1.138 We concluded that clients were not consulted sufficiently to ensure that measures of performance would be in place for those aspects of service that matter most to them, or that delivery targets take clients' expectations and priorities into account. There has been some improvement since 1996 in communicating key information to clients at points of service. However, more needs to be done to inform clients about whether results have met targets, how much it costs to provide a service, and how to lodge a complaint and obtain redress.
1.139 We found that since our last audit, Parliament has been given more information on service performance and the information is more likely to be meaningful and to help put results into perspective. However, improvement is needed in reporting performance trends, providing more balanced information and reporting costs.
1.140 The role of the Treasury Board Secretariat. The Treasury Board Secretariat has provided departments and agencies with guidance and support that has become more active since 1998. However, we concluded that it has not systematically monitored the government's progress in implementing service quality initiatives and improving service quality. As a result, the information it reports to Parliament has not been adequate to clearly indicate the government's progress in improving the quality of services to Canadians.
1.141 Implementation of 1996 recommendations. The Treasury Board Secretariat and other departments and agencies have acted to some extent on all of the recommendations we made in 1996. However, the response has been uneven and none of the recommendations has been implemented fully.
1.142 Managers of the service lines we examined have used several techniques for continuously improving service delivery. These include conducting client satisfaction surveys and reviews and establishing action plans. We concluded that more effort is needed to collect, analyze and use performance information (including cost information) in managing performance and continuous improvement. In addition, benchmarking would be a way to learn about good practices in other organizations, and better use could be made of data on complaints.
Government's overall response: We agree that client satisfaction is an essential element for the provision of services offered by the government to the public. The government is committed to meeting client needs and expectations and we trust that this chapter will further our joint goals to improve the quality of service offered to Canadians.
Service improvement in all areas of government operations continues to be a major priority of the Treasury Board Secretariat and all government departments. The Secretariat emphasized this imperative with the creation in 1998 of the Service and Innovation Sector as a Policy Centre within government, to lead government-wide initiatives to improve the delivery of services to Canadians. The Sector is building on the initial work done under the Quality Services Initiative to develop a broader, citizen-centred approach to service improvement. For this purpose, the Sector is working closely with the leaders of service delivery across the government.
The chapter indicates that the number of telephone calls received by government in the 13 service lines examined rose from 36 million in 1996 to 56 million in 1999, a 54 percent increase in three years. We are pleased to note that the audit revealed that "the most significant improvement in service quality" has occurred in the area of telephone accessibility.
An important element of the government's commitment to make service more citizen-centred is Blue Pages redesign. Canada has become a leader in the field of redesigning government telephone listings in public directories by successfully integrating all levels of government into one easy-to-read directory format. Implementation of the new directory format will occur according to publication schedules established in co-operation with the telecommunications industry and with provincial and municipal governments.
The Secretariat agrees with the Auditor General's recommendations on measurement, client satisfaction and reporting. Much good work has been done by many departments in these areas. The Secretariat will continue to work with departments to assist them in monitoring and reporting on service quality in an integrated way, together with other information on results and performance, through the annual planning and reporting process.
The Secretariat is also working with departments to develop an approach to planning and implementing service improvement that will promote continuous improvement in service delivery across the government.
About the Audit
The objectives of the audit were to determine:
- the nature and extent of progress made by departments and agencies in improving the quality of their services since our 1996 audit;
- whether departments and agencies have measured results and reported them to Parliament;
- whether the Treasury Board Secretariat has assisted departments and agencies in implementing the government's commitments to service quality, and whether it has reported to Parliament on the government's progress in improving service; and
- whether departments and agencies have acted on our 1996 audit recommendations and on related recommendations made by the Standing Committee on Public Accounts.
The scope of the audit included the Treasury Board Secretariat and 10 departments and agencies, focussing on the 13 services that we examined in our 1996 audit.
We placed particular emphasis on performance results. The audit covered the period from 31 March 1996 to 31 October 1999.
We first collected performance data and supporting documentation from departments and agencies, and then analyzed that information. We reviewed the performance information and assessed it in light of our audit criteria; we did not audit its accuracy. Where necessary, we discussed the data and reports with service managers and staff and carried out follow-up interviews.
We conducted on-site visits to observe the front-line operations of several high-volume service centres. In addition, we reviewed information reported to Parliament from 1997 to 1999 on the performance of the 13 service lines. We also examined the part the Treasury Board Secretariat has played in promoting and supporting government-wide initiatives for service quality.
We expected that:
- departments and agencies would have made improvements over time in providing cost-effective services to Canadians;
- departments and agencies would have delivery targets that address key aspects of service delivery, reflect client priorities and are communicated to clients;
- measures of the quality of services delivered, and of costs incurred, would be credible and allow a determination of the extent to which performance targets have been met;
- departments and agencies would tell clients what the service costs, and use service and cost performance information as a basis for determining the highest-quality service that can be provided at an affordable cost;
- departments and agencies would measure performance against delivery targets, report it to clients and use performance information to improve service results;
- departments and agencies would establish adequate complaint and redress mechanisms;
- performance information reported by departments and agencies to Parliament would be relevant, understandable and balanced, and would include associated costs;
- the Treasury Board Secretariat would provide guidance and support to departments and agencies in implementing the government's commitments to improve the delivery of services to Canadians; and
- the Treasury Board Secretariat would monitor progress and report to Parliament on the government's progress in improving service delivery.
Assistant Auditor General: Maria Barrados
Principal: Henno Moenting
Director: Lilian Goh
For information, please contact Henno Moenting.