1989 Report of the Auditor General of Canada

Main Points

20.1 Statistics Canada is a central statistical agency charged with providing statistical information and analyses on Canadian society. Expenditures in 1988-89 were $246.5 million; there were 4,200 full-time employees and approximately 1,600 part-time interviewers (paragraphs 20.8 and 20.9).

20.2 In the areas we examined, we noted a dedication on the part of Agency staff to maintaining good statistical practice. Statistics Canada managers, however, are faced with the challenge of making choices among many competing demands in the context of budgetary constraint. For planning and accountability purposes, more specific information should be included with the statement of activity objectives on the data requirements to meet the needs of the main users (20.17, 20.19, 20.23).

20.3 Statistics Canada's independence in statistical matters is recognized. Its obligation is thus even greater to report to Parliament, through the Minister, and to users of its statistics, the consequences of the choices it makes. More information should be provided on the impacts associated with program changes. An example would be the reinstatement of the 1986 Census with a reduced budget, which resulted in a reduction of the quality of the population count (20.16, 20.44, 20.46).

20.4 Budgetary restraint has resulted in a number of reductions in sample sizes and survey coverage and difficulty in meeting established levels of service to the public. Because objectives were not always clearly defined, the consequences of these changes could not be readily assessed against stated requirements. There is an ongoing need to provide more information to users, for example, on the implications of the changes in the data being put into the System of National Accounts (20.61, 20.66, 20.57, 20.82).

20.5 Steps need to be taken to clearly define the authority under which Statistics Canada is selling publications. We found that the procedure for setting prices of publications was different from that used by the Canadian Government Publishing Centre at Supply and Services Canada. We believe there is a need for appropriate approvals to be obtained for the pricing system for its publications (20.85, 20.88).

Audit Scope

20.6 We directed our audit to assessing how Statistics Canada has adapted its practices and programs to budgetary restraint. We selected five statistical programs for examination: the Labour Force Survey; the Annual Survey of Manufactures and the planning of the Business Survey Redesign; the System of National Accounts; Health Statistics; and the 1986 Census of Population. In addition, we examined Marketing and Information Services for statistical products. The audit examined the systems, procedures and management practices in place.

20.7 We did not assess the quality of the Agency's statistical methodology. We used the Agency's own measures of the quality of its statistics to identify the impact of budgetary constraint on programs and practices.

Background

20.8 Statistics Canada is a central statistical agency that has been in operation since 1918. Its broad mandate under the Statistics Act is to collect, compile, analyze and publish statistical information. Decisions made on the basis of this information affect the social and economic affairs of practically all Canadians. Statistics Canada also plays an important role in developing statistical expertise in Canada and abroad.

20.9 The Agency employs approximately 4,200 full-time employees and 1,600 part-time interviewers. The statistics program is divided into six main activities, four broad subject areas and two functional areas that cut across the others (Exhibit 20.1). The Agency also maintains a network of regional offices across Canada which employ over 300 permanent employees and most of the part-time interviewers. Total forecast expenditures for 1988-89 were $246.5 million.

(Exhibit not available)

20.10 Statistics Canada is a highly professional organization with over 900 professional and scientific staff. It provides data to meet a very broad range of requirements. To determine the specific types of data that are required, the Agency supplements its own professional expertise with communication links it has established with stakeholders in governments, business and labour unions, the academic community and the media. It faces a practically limitless demand for more and better data. Statistics Canada recognizes that meeting all users' needs is virtually impossible.

20.11 Since our last comprehensive audit in 1983, the Agency has put in place a National Statistics Council and a network of advisory committees for specific activities. The Council and the advisory committees provide advice on program priorities and relevance. The Chief Statistician has placed a priority on human resource management. Statistics Canada has been recognized for its innovative Corporate Assignments Program, designed to provide staff with a wide range of experiences in the Agency. Some additional initiatives include improved in-house training, an annual awards program, and a no-lay-off policy despite significant reductions in person-years.

20.12 The Agency's expenditures, excluding the census, have remained relatively unchanged during the last six years. In terms of 1988 dollars, they were $226.7 million in 1982-83 and $228.6 million in 1988-89 (Exhibit 20.2). During the same period, it made an 11 percent reduction in its person-years from 4,740 in 1982-83 to 4,225 in 1988-89. A further reduction to 4,103 person-years is planned for 1989-90. Thus, during this period, the Agency has been required to reduce person-years and operate on a budget that has not grown in real terms, with an increased proportion of the funds coming from revenue-generating activities.

(Exhibit not available)

20.13 At the same time, it has been attempting to meet the demands for new data and improvements in existing data, maintain its position of professional leadership and continue to contribute to innovations in statistics. The Agency has put a great deal of effort into improving its productivity through increased automation and integration of production activities. Initiatives have also been taken to increase vote-netted revenue from $12.7 million in 1982-83 to $24.5 million in 1988-89 (in constant 1988 dollars).

(Photos not available)

20.14 In order to provide funds to carry out the 1986 Census of Population, Statistics Canada reduced its net expenditures by approximately $68 million. It did this through a combination of reducing expenditures and increasing revenues over a five-year period. At the time of our audit, the Agency was faced with the management challenge of coping with continuing resource constraint.

Observations

20.15 There is a recognition within government that Statistics Canada needs to maintain its independence in statistical matters in order to carry out its mandate in an unbiased manner. Although commonly referred to as an agency of government, Statistics Canada is formally a government department, currently reporting to the Minister of Regional Industrial Expansion. The reporting and accountability requirements are those of a government department, not a Crown corporation.

20.16 All government managers are expected to account for the responsibilities they are given and are answerable for their actions and stewardship. Because of its accepted independence in statistical matters, there is an even greater obligation on the part of Statistics Canada to report to Parliament, through its Minister, on program choices that have been made and the effect of those choices on the statistical products concerned. Statistics Canada has an additional obligation to the users of its products to provide full and complete information on the quality of those products, any changes that may have been made to them and the consequences of the changes on their main uses.

20.17 It is in the nature of an audit report to emphasize negative findings, and this chapter is no exception. As outlined in the audit scope, we did not examine all of the Agency's activities. While we report here on areas for improvement, we observed a widely held respect by many data users for the Agency's professionalism. In our examination, we noted a dedication on the part of Agency staff to maintaining good statistical practice. We also found that most managers satisfactorily carried out their statistical activities in conformity with the Agency's guidelines for maintaining quality. Because of the nature of our reporting, these examples have not been included.

Planning

20.18 Since our 1983 audit a formal planning process has been established in the Agency. In 1987 the overall planning priorities and directions for the Agency were outlined in the Strategic Overview. The current planning process is built on the input of program managers, which is consolidated for review by more senior levels of management. Each level of management meets and discusses the long-term planning proposals, which include proposals for meeting efficiency targets and contingency plans for coping with further reductions. Managers bring to the planning meetings a wide range of inputs, as described in paragraphs 20.8, 20.10 and 20.11, as well as the results from federal-provincial consultations, internal analysis of social and economic issues and program evaluation.

20.19 A key task in statistical management is the setting of priorities, since there will always be competing demands for, and limits on, resources. Effective allocation of the Agency's increasingly scarce resources rests on making appropriate choices from among a multiplicity of user demands and pressures for increasingly timely and reliable statistics within the context of its mandate.

20.20 The Strategic Overview states that the planning process is based on the assumption that the current program mix can only be changed at the margins. The outcome of this planning approach is that many new needs have a lower priority, in the short term, than existing programs.

20.21 The Chief Statistician has identified a number of serious data gaps. Cost savings have been made through program reductions that limit the amount of data collected, the frequency with which they are collected or the size of samples. The Agency intends to fund new initiatives judged to have the highest priority primarily through these marginal reductions and through savings that result from increased efficiencies.

20.22 The current planning process deals with large program areas which it does not formally link to the operational plans of specific surveys or data gathering activities. For example, because of the level of aggregation in corporate planning documents, the Labour Force Survey and the Annual Survey of Manufactures cannot be identified except where they are offered as possible areas for savings. Program profiles used in an Agency-wide review did not provide enough detail to make these two surveys identifiable.

20.23 In a statistical agency, choices about the priority to be given to a particular data series are based on assessments of the costs involved and of the type and precision of data required to meet legislative requirements and the needs of the main users. However, in our examination we found that information with the statements of activity objectives in the planning process did not consistently specify the type and precision of data required to satisfy the main users and the legislative requirements. As a result the process could not consistently provide accountability statements of the effects of the marginal changes in response to budget reductions in terms of activity objectives.

20.24 In 1983 we reported that, in planning or managing data series, management did not appear to be making explicit, wherever possible, the required level of reliability of data which could be related to their main uses. The Agency stated that it agreed with the underlying intent of our recommendation that these requirements should be made explicit. However, it favoured a less "mechanistic" approach because it had no objective basis for assessing the relative importance of the various uses. It also argued that technically it was not possible to measure many of the factors affecting reliability.

20.25 Other central statistical agencies have established processes that deal with the problem of specifying the type and precision of data requirements to meet the needs of main users and legislative requirements. Explicit statements of data requirements for data series are made for planning and accountability purposes. These statements are not always quantified but are as specific as is technically possible.

20.26 In the continuing development of its planning processes, Statistics Canada should provide more specific information with the statement of activity objectives of the data required to meet the needs of the main users.

Management's response: In addition to the long-term planning process, a variety of mechanisms are used to keep the total program of the agency under regular review: Ministerial directives; systematic monitoring of Cabinet memoranda; consultations with ministerially appointed National Statistics Council; formal interdepartmental committees; an active internal analytic program; a variety of federal-provincial mechanisms; formal program evaluations undertaken through contract by leading external experts; some 15 professional advisory committees; client contacts with thousands of users; market feedback through sales; contacts with trade associations, Chambers of Commerce; networking with academia; and international consultations with a view to facilitate international comparisons by our clients.

Formal documentation of main users and uses is maintained at the subactivity level. Extending this to some 415 distinct statistical activities would involve unproductive additional paperwork.

1986 Census of Population

20.27 Every five years a census of population is carried out in Canada. The census every tenth year, at the beginning of the decade, is required by the constitution; it provides the basis for determining representation in the House of Commons. The mid-decade census is constitutionally required for only two of the western provinces, but it is required by the Statistics Act for all of Canada. In 1956 the mid-decade census became a national census.

20.28 Statistics Canada prepares options and recommendations on the content of the census. Cabinet makes the final decisions on its content, and on the size of the expenditure, based on this advice.

20.29 Traditionally the census at the beginning of the decade has been larger than the mid-decade census. In both cases, questions are included to provide basic population counts and personal information about the total population. Questions are also included to obtain additional information from a sample of the population -- 20 percent in 1986. These questions, in both the 1981 and 1986 censuses, give cross-classified data for small geographic areas, providing information on detailed relationships, for example, between occupation, industry, income and education. According to Statistics Canada, these data on small sub-provincial areas are unavailable from any other source.

20.30 A census of population is a major undertaking. All Canadian adults are involved. For the 1986 Census about 42,000 people carried out the enumeration and other field activities at a total cost of approximately $150 million. The projected budget for the 1991 Census including estimates for population growth and inflation, is $265 million, depending on the number and types of questions that are recommended to, and approved by, Cabinet.

(Photo not available)

20.31 Published population figures are required for payment of funds under a number of federal acts and regulations. Under the Federal-Provincial Fiscal Arrangements and Established Programs Financing Act, the Chief Statistician is required to provide a certificate stating the population counted in each province in each census year and an estimate of the population in the years between.

The requirement to reduce census costs

20.32 Census planning typically starts as soon as data collection for the previous census has been completed. Statistics Canada began a review in 1982 of federal government requirements for 1986 Census data. There were bilateral consultations with federal and provincial government users, as well as with other census data users such as municipalities, businesses and universities. In November 1984, the government announced that the 1986 Census of Population, as planned, would be cancelled to generate savings of $100 million in the period 1985-86 to 1987-88.

20.33 On 21 December 1984, the Minister announced the reinstatement of the 1986 Census, since Statistics Canada had developed an alternative plan to realize the $100 million savings. The savings, according to the Minister's announcement, were to be achieved by the following methods:

• reducing the data gathered, resulting in a smaller census operation;
• increasing the emphasis on cost recovery. Users of census and other data would be expected to bear a fairer share of the costs;
• establishing the 1986 Census as a Federal Government Student-Youth Employment Program; and
• phasing out some programs, unless cost recovery was realized. Some programs would be expected to take a place behind the census.

Risks associated with the change in the census were not identified

20.34 In his announcement reinstating the census, the Minister expressed confidence in the ability of Statistics Canada to sustain a high level of statistical quality. The Chief Statistician advised Treasury Board that the commitments associated with the reinstatement of the census posed rigorous challenges, but these would be met without damaging the technical infrastructure of the Agency or its statistical programs. Compared to the usual four-year planning period for a census, Statistics Canada was required to make sudden cost-cutting changes in only five months. In the planning of the "revised" census, we saw no documentation of analysis of the risks associated with certain of the choices that had to be made to achieve savings.

Savings were achieved through operational changes in the census-taking

20.35 In addition to using summer students instead of traditional census representatives, all paid advertising and postal checks were eliminated. There were also substantial reductions in telephone assistance services available to census respondents. Many of the operational procedures that were changed in this fashion had been developed originally to ensure maximum inclusion of the population in past censuses.

Little reduction from the 1981 census in the amount of data gathered

20.36 One of the ways, suggested by the Minister, of achieving the $100 million savings was a reduction in the amount of data gathered, which would result in a smaller census operation. We were advised by Agency officials that there was a small reduction in the size of the originally approved census questionnaire. Two questions were eliminated, and the level of detail in a third was reduced. The size of the mid-decade census in 1986, however, required a significantly larger number of responses to questions than the previous mid-decade census in 1976. The 1986 Census was only 4 percent smaller in terms of the amount of data gathered than the 1981 decennial Census.

Reductions limited the benefits from one census question

20.37 One of the cost savings introduced, for example, was a change in processing the answers to the question about address of place of work (question 28). The incremental cost of collecting the data was estimated in 1983, at the planning stage, to be $2.2 million; approximately $1.5 million of this would be for coding the responses. To save the cost of coding, it was decided that the information from the question would only be provided to municipalities that paid for the coding. As a result, this information has only been coded for the province of Ontario, which paid for the coding of the responses. The responses from the rest of Canada for this question remain uncoded.

The quality of the population count was reduced

20.38 The changes in operational procedures had a significant negative impact on the quality of the population count. Statistics Canada has developed a statistical technique, "the reverse record check", to estimate the size of the "undercount" -- that is, the number of people not included in the census count. For 1986, the undercount was estimated to be 3.21 percent which was 60 percent higher than in 1981. This means that an estimated 839,000 people were not included in the population count. The estimate of the undercount in 1986 is the highest recorded since undercoverage was first measured in 1966. The Agency, in its review of past census operations, considered the 1981 census an "unprecedented success" in terms of factors such as the operations, cost, timeliness and completeness of the population count. In that census an estimated 2.01 percent of the population was not included in the count.

20.39 Population size, as certified by the Chief Statistician, is used to determine the payments to provinces and territories from the federal government for established program financing (health, post-secondary education) and for equalization. These annual transfers in 1988-89 were approximately $25 billion. Relatively small changes in population figures can have a significant impact on the amounts of funds that are transferred. Although a completely accurate census is an unattainable goal, a number of procedures have been established to minimize the error in the census count and the likelihood that a reported change in population is the result of errors in measurement.

20.40 The Chief Statistician considers it professionally indefensible to link a dollar value to the undercount because, in his view, no adequate methodology has been developed in Canada to estimate the size of possible double-counting (overcount). The Chief Statistician notes that the statistical profession, both in Canada and the United States, is deeply divided about the reliability of these estimates. Statistics Canada has made no estimate of the net error in the census count. In the United States, estimates of net undercount are made. Officials in Canada have expressed the opinion that the size of the underestimate is larger than the size of the overestimate.

20.41 While some error in population count is inevitable, maintaining a consistently high quality is important to ensure fair and equitable distribution in the main federal transfer programs. Population figures from one census influence the determination of the payments over a six-year period. Changes in figures can result in significant changes in the payments.

20.42 Statistics Canada should continue to investigate ways and means of developing a basis for estimating net census error.

Management's response: The agency for many years has investigated, developed and implemented improvements in methods used to measure the quality of census data, including measures of coverage. An explicit program to continue this research in conjunction with the 1991 Census has been in place since 1987.

Users need more information on the impact of the change in undercount

20.43 The Chief Statistician is responsible for furnishing statistics to users and providing advice on their quality. In general, census results are provided to users through the Agency's standard dissemination channels. The Chief Statistician is also required to provide certification of the population numbers for specific programs as part of their legislation. None of the certificates that were based on the 1986 Census includes any reference to the quality of the data.

20.44 In July 1988, the Agency issued a user information bulletin that reported the undercoverage rates of the 1986 and 1981 censuses by province. None of the catalogued publications on the 1986 Census we reviewed, however, that provided population numbers, referred to the increase in the undercount numbers since 1981. Consistent with Agency guidelines, users of census data need to be provided with this information in census publications.

20.45 Users of census data should be provided with more information about the size of changes in the undercount in both publications and certifications.

Management's response: Consistent with its policy of informing data users about the quality of its statistical products, the agency will continue to provide census data users with information about coverage errors, including the estimated undercount. Output plans for the 1991 Census will be reviewed to determine whether additional information can usefully be incorporated in publications and other forms of output. It must be noted, however, that information on the estimated census undercount is not available until after the main census results are published.

Unspecified effects on other agency programs

20.46 Half of Statistics Canada's targeted savings of $100 million were made by 1986-87, primarily from savings in census operations. The remainder was to be recovered over the next three years. This commitment has placed a financial burden on Statistics Canada. Savings targets, even with further financial restraint, have been met. The impact on other programs attributable to the required savings for the census is difficult to determine, as described in paragraphs 20.51 to 20.59 and 20.80 to 20.82. However, we believe that Statistics Canada has an obligation to advise Parliament whenever significant program reductions and other changes are made, such as those that are associated with going ahead with the 1986 Census.

The alternative of a smaller mid-decade census

20.47 The Statistics Act requires that a mid-decade census be taken; however, the amount of information that should be included in the census has not been legislated. Statistics Canada has estimated that 70 percent of the cost in 1981 and 78 percent of the cost in 1986, was explicitly required by statute. Over time, censuses have been used to collect an expanded range of data beyond the base requirement. From 1976 to 1986, the mid-decade census changed from a smaller census to almost a complete replication of the full decade census, including a larger number of discretionary questions than previous mid-decade censuses.

20.48 Census taking is a complex and expensive undertaking. All of the discretionary items in a census can also be collected in other household surveys for smaller numbers of people. The increasing size of the mid-decade census has been justified by Statistics Canada because of its uniqueness as a source of small-area detailed data and demands for the data by users. Statistics Canada argued that this change was warranted because of changing data requirements and the need for nationally available small area data and because the required information was not available from any other source.

20.49 Changes are continually being made in the development of new data sources. Alternatives to meeting the discretionary portions of the census may be available from the development of new data sources to respond to priority needs. Examples of such sources would be the selective use of other sample surveys and projections built on the updated sampling frame from the census.

20.50 Statistics Canada should continue its examination of options for meeting the discretionary mid-decade data requirements in the most cost effective manner.

Management's response: The agency continuously seeks to meet information requirements of users in the most cost-effective manner. Such an examination was carried out prior to the 1986 Census, and another is already under way for 1996.

System of National Accounts

20.51 The System of National Accounts provides an integrated system of information on Canada's economic activities, both domestic and international. It produces well known economic measures and tables, including gross domestic product, gross national product, balance of international payments, financial flows, input-output tables, price indices and productivity measures. The forecast expenditure for National Accounts in 1988-89 was $12.86 million and 219 person-years.

Need to define the risks associated with delaying research and development on the system of national accounts

20.52 In a 1980 review of Agency methodology, Sir Claus Moser observed that research and development are essential to maintaining high statistical standards. For a statistical office, these functions provide the basis for adapting to changing technological and institutional situations. There have been a number of major changes in the economic environment in Canada, including a shift toward service industries and the increasing use of technology. If it is to reliably reflect the changing economy, the System of National Accounts needs to reflect these changes.

20.53 The statistics from the National Accounts play an important role in the development and analysis of economic policy. There is therefore an ongoing concern about the quality of the data in the accounts and the relevance of the accounts to current economic conditions. Statistics Canada has done several reviews and evaluations to assess the System of National Accounts and to identify the type of development work that needs to be done to ensure that the overall credibility and relevance of the system is maintained.

20.54 This research and development work is recognized by the Agency as important. However, although an additional pool of staff was provided to improve the accounts, many of them were used to meet more pressing operational requirements. For example, more than half had been assigned to work on the implementation of the harmonized system and the Business Survey Redesign Project.

20.55 An assessment should be made, for internal planning purposes, of the risks associated with delaying research and development in the identified areas in the National Accounts.

Management's response: The consequence of not doing this work has been recognized and acted upon. In recent years research and development topics included: total factor productivity; satellite accounts for tourism and health; the underground economy; personal savings; chain price indexes; and expanded measures of industry output by province and deflation of the main components of final expenditures in the provincial accounts. Particular attention has been paid to improving coverage of services activity, both within Canada and in trade with other countries. This has entailed a re-examination of concepts and definitions, as well as development of new surveys and publications. During the recent historical revision, new price indexes were introduced to reflect the rapid growth of computing capacity. Additional work is, of course, underway.

Need to improve documentation on national accounts for the users

20.56 As we pointed out in 1983, it is essential that users be provided with enough information on the quality of the data in the National Accounts to allow them to determine the data's appropriateness for their specific purposes. The National Accounts are derived from complex data sources and there is a necessary arbitrariness in some of the steps used in constructing the accounts, including imputations and adjustments of data.

20.57 Users require complete and up to date information on the data and methods used in the System of National Accounts. Documentation of sources and methods is frequently postponed under pressure to meet deadlines. More published information on data sources and methodology employed in the System of National Accounts is needed by users.

20.58 Furthermore, the Business Survey Redesign Project will change the way many of the economic series that make up the National Accounts are collected. In December 1988, the new sampling frame was used for the monthly Wholesale and Retail Trade Surveys. In our 1983 audit we noted that problems with the sample frame at that time resulted in an underestimation of the value of retail trade for input into the National Accounts. To compensate for this, upward adjustments to the results were made. The new frame is designed to improve the estimates from the survey. However, these changes will in turn affect the data in the accounts and potentially the use and interpretation of them by users. The Chief Statistician describes the transition period as a time when "the data will likely be more volatile". Users will require more information about the changes in data going into the accounts and how these may affect the results in the accounts.

20.59 The Agency should provide users with more published information on the process of National Accounts construction and the quality of its output.

Management's response: A large quantity of information is already readily available for interested users. Additional material is currently in preparation or has been recently released. This includes:

• A User Guide to the Canadian System of National Accounts; to be released October 1989.
• A Guide to the Financial Flows and National Balance Sheet Accounts; published February 1989.
• A methodology document for monthly estimates of GDP by industry; first draft to be available by the Spring of 1990.
• A methodology document for service producing industries in the Input-Output Accounts; completed November 1987; a similar document for goods producing industries; in preparation. Publication scheduled for 1990.
• A new sources and methods publication for National Income and Expenditure Accounts as series of separate handbooks covering major components; to be released periodically over the next two years.

Labour Force Survey

20.60 The Labour Force Survey produces the official monthly estimates of unemployment and characteristics of the labour force. The survey is based on a complex sample design covering 48,000 households. The data are usually released 13 days after the last questionnaires have been completed, or about 3 weeks after the questionnaires are sent into the field. In 1988-89 the survey cost $11.6 million and used 85 person-years and 800 part-time interviewers. Some of the main users of the survey are the Canada Employment and Immigration Commission and other federal and provincial government departments.

No clear statement of survey objectives defining data requirements

20.61 As part of the overall requirement to reduce expenditures, the sample sizes of some surveys, such as the Labour Force Survey, were reduced. The statement of survey objectives is not supported by a specification of the level of reliability needed by the main users of the data. As a result, Statistics Canada could not state the effects of these reductions on the data, in terms of the survey's objectives.

20.62 In 1986, 5,217 households were dropped from the survey as part of the general budget cut in Statistics Canada. This change has affected the precision of some of the statistical estimates. Users were informed of these changes in precision. Contingency plans for further reductions in sample size that would have resulted in the provision of only national and provincial data were developed during the planning process. These plans were not implemented.

20.63 Statistics Canada is defined as the provider of data under the Unemployment Insurance Act. There is no statement of what statistical data are required under the Act. The findings from the Labour Force Survey are used to estimate unemployment rates by unemployment insurance economic regions. These rates, in turn, are used in criteria for determining eligibility for, and durations of, certain benefits. A difference of one percentage point is used to determine entrance requirements and half a percentage point the weeks of benefits. Reductions in sample size increase the likelihood that this difference will come about as a result of sampling variability rather than a real difference.

Annual Survey of Manufactures

20.64 The Annual Survey of Manufactures dates back to 1917. In 1987-88, it cost over $2.8 million and used 76 person-years. The outputs of the survey are an integral part of the wider economic statistics program, including the System of National Accounts.

20.65 The data collected by the survey provides information on the manufacturing sector of the economy. This sector accounts for more than one-fifth of Canada's real domestic product, roughly 17 percent of total employment, and some 65 percent of the nation's total exports of goods and services. The data can also be used in the analysis of the changes arising out of trade liberalization, including the effects of the Canada-United States Free Trade Agreement. The two major sets of data produced are principal statistics providing operating data such as financial and employment figures and detailed commodity statistics of Canadian manufacturing industries.

No clear statement of survey objectives defining data requirements

20.66 As part of the overall requirement to reduce expenditures, greater reliance has been placed on estimation than on direct surveying. As in the other surveys examined, there was no statement of the data requirements of the main users of the Survey of Manufactures as part of the objectives. As a result, Statistics Canada could not state the effects of the changes in this survey, in terms of the survey's objectives.

Increased use of estimation

20.67 Several changes have taken place in the survey since 1983. In 1987 the name of the survey was changed from a census to a survey of manufactures, in recognition of the increased reliance on sample surveys to collect data from smaller manufacturing establishments. The level of resources assigned to it has been reduced. There has been a shift from long- to short-form questionnaires, with greater use of taxation data and increased use of estimation procedures. The long-form questionnaire covered about 93 percent of manufacturing shipments in 1986. For manufactures not covered by the long-form questionnaire, there is a reduction in the detailed commodity statistics collected, and estimates are provided instead. The use of the sampling frame produced by the Business Survey Redesign Project will produce further changes.

Lack of timeliness of publications

20.68 The Agency recognized that publications releasing the results of this survey were not timely; in 1984, the publications were reviewed to improve timeliness. There was some improvement in the release of the principal statistics for 1985, but further slippage for 1986. The main timeliness problems, however, are with the commodity statistics. For the reference year ending in 1985 (including the 31 March 1986 year-end), the first commodity publication was released in January 1988.

20.69 Commodity statistics for 1987 were not edited because of a lack of resources. There were 36 people working on editing the 1986 data; only 15 were available to do this work for the 1987 data. As a result, the 1987 data will not be published. The frequency of publication of the commodity statistics is being reviewed.

Health Statistics

20.70 Health statistics refer to series on vital statistics, information about health care institutions and their use, disease registries, and surveys related to health status, knowledge and attitudes. The forecast budget for health statistics in 1988-89 was $5.8 million and 94 person-years.

20.71 Health care and its costs are an issue of increasing national concern. In 1987, the total cost of health care in Canada was nearly $47.8 billion, shared by all levels of government and individuals.

Need to ensure the ongoing cost-effectiveness of Statistics Canada gathering data on hospital operations

20.72 In its Strategic Overview, Statistics Canada defines improvement in information on the health of individuals as one of its program priorities. The Chief Statistician has personally taken a lead role in several initiatives designed to bring about substantial improvements in health information. Several of the statistical series related to health care institutions have been collected by the Health Division for several decades. In our review of the data collected by the Health Division, we found there was little evidence of a continuing need for Statistics Canada to provide some of these health care institutional data. Examples of activities with uncertain importance for Statistics Canada are described below.

Data on hospital operations

20.73 Information about the operation of hospitals is a major effort of the Health Division ($200,000 non-salary expenses and 20 person-years in 1987-88). The bulk of operational data is still being collected and published as it was in 1976-77. The two main data bases produce the following detailed information:

The Quarterly Hospital Information System is largely a data base used for making comparisons among hospitals. There are approximately 435 indicators per hospital, including type of service provided, ownership, selected diagnostic and treatment services, number of meal-days, kilograms of laundry, etc. In 1986 the Advisory Committee on Health Statistics recommended that this system be put on a cost-recovery basis. The Committee recommended that, if hospitals and provinces did not purchase the data, the program should be discontinued; the recommendation was not accepted. A federal-provincial working group on hospital information, at Statistics Canada's request, considered the need for Statistics Canada to collect these data. They raised questions about the need for Statistics Canada to continue to collect them but did ask that the system be maintained.

Annual hospital statistics. Data from provincial ministries make up the annual hospital statistics data base. This contains information similar to the quarterly data but in more detail. The Advisory Committee on Health Statistics questioned the need for this degree of detail. The data are produced with considerable delays -- publication is expected two years after the end of the reference period. The Canadian Hospital Association has developed a new hospital management information system that will standardize the input from all provinces. With common reporting across the provinces, comparisons can be made directly. This raises further questions about the need for Statistics Canada's involvement.

Morbidity data

20.74 These data are obtained from institutional records at a cost to the division of $227,000 (non-salary expenses) and eight person-years. Annual statistics on hospital admissions and separations are provided by the provincial ministries. The Hospital Medical Records Institute is developing a similar data base for management information. It is now processing about 75 percent of the admissions/separations from Canadian hospitals. Statistics Canada processes 100 percent of the data for statistical purposes but doesn't produce it as quickly as the Hospital Medical Records Institute. This raises the question of the continuing need for Statistics Canada to allocate scarce resources to this area.

20.75 Statistics Canada should reassess the ongoing requirement to provide hospital-based statistics.

Management's response: Disagree, given the fact that the federal-provincial working group on hospital information concluded that the data continue to be needed. Moreover, the recommendation as presented places inadequate weight on the comprehensive initiatives currently under way to improve the entire program of health statistics. Statistics Canada has assumed a leadership role in the reassessment of the existing program of health statistics. A new National Health Information Council, reporting to the Chief Statistician and the Conference of Deputy Ministers of Health, provides a forum for reviewing the requirements for health information of all kinds, whether derived from hospital or other administrative files, population-based surveys or other sources. Statistics Canada is also collaborating with the Canadian Institute for Advanced Research with the objective of developing a new paradigm for health information to help guide future statistical and analytic work.

Business Survey Redesign Project

20.76 The Business Survey Redesign Project is one of the largest and most complex projects Statistics Canada has undertaken. The major planned output is a more complete and up-to-date business register to which all redesigned business surveys will be linked. The project has important implications for a variety of operational activities, including the Annual Survey of Manufactures and the System of National Accounts. It also responds to some of the observations made in our 1983 Report.

20.77 At its peak in 1987-88, the project involved direct expenditures of $9.2 million and 175 person-years. Cumulative expenditures from 1984-85 to the end of October 1988 have been $27.2 million. It is not yet complete; actual testing of the new register had just begun at the time of our examination.

Need to state the implications of resource allocation to a major new project in a period of restraint

20.78 Statistics Canada undertook the Business Survey Redesign Project at considerable internal cost. It justified the priority placed on the project by the need to improve the current business surveys, reduce the response burden on businesses and provide more complete coverage of economic activity. In the spring of 1984 Treasury Board approved approximately 45 additional person-years for each year of the three-year planned duration of the project. Although it was recognized at that time that internal reallocations would have to provide the major part of the necessary funding, the negative implications of these reallocations for other activities were not documented in plans and proposals.

20.79 Overly ambitious targets were set for the project. At the time of our audit, it was running about a year and a half behind the original plans. Deadlines continued to slip during 1988. Project delays have resulted in additional pressure on resources and delays in documentation for other surveys such as the Annual Survey of Manufactures. Project milestones were set out in Part III of the Estimates but the failure to meet them has not been consistently reported.

Data Dissemination

20.80 The Statistics Act requires Statistics Canada to make public the information it collects. The data gathered by Statistics Canada are made available in a variety of forms, including printed publications, electronic data bases, and other electronically readable data products. The Agency also operates public information and inquiry services and a communications program. New Agency policies on data dissemination, a requirement to increase cost recovery, and overall budgetary constraint have changed the level of service being provided to the public by Statistics Canada.

There is a need to define clearly the difficulty in meeting established levels of service as a result of changing demand and budgetary constraints

20.81 The Agency guarantees public access to its general information. A recently developed Agency marketing policy makes the commitment to free access to information through a variety of dissemination mechanisms. These include a toll-free telephone inquiry service, regional reference centres and the provision of complimentary publications to provincial and territorial governments and the media. Also included is the Depository Services Program of the Department of Supply and Services, which provides complimentary publications to selected libraries and to federal Members of Parliament.

20.82 The Agency carries out ongoing reviews of service quality. These reviews, and our own audit work, show that resources have not kept pace with significant increases in the demand for some services. As a result, there has been a lowering of the level of service. For example, more telephone inquiries by users go unanswered in Toronto, Montreal, Vancouver and Ottawa offices. And the consolidation of publications has resulted in less free data going into the Depository Services Program.

Publishing

20.83 Our 1983 audit noted that "most directors choose to disseminate information through printed publications". Our current audit concluded that publication is still the most common means of dissemination. This year Statistics Canada will produce more than 450 titles covering a broad range of statistical information. Over 1.2 million copies of publications were distributed in 1988-89. Twenty-one percent of these were distributed free through library exchange programs, distribution for provincial statistical focal points, and the media. Four percent were distributed internally. Forty percent were distributed through the Depository Services Program. Supply and Services Canada pays incremental costs to Statistics Canada for this distribution through the Depository Services Program. Treasury Board has temporarily increased estimate reference levels in order to make up for shortfalls in revenue from this program. The remaining 34 percent of publications are sold (see Exhibit 20.3).

(Exhibit not available)

Lack of proper authority to set prices and sell publications

20.84 The Statistics Act does not give the Agency specific legislative authority to set the selling prices of its publications. Before 1984, the Department of Supply and Services had this responsibility. In 1984 a memorandum of understanding was signed confirming and describing publishing responsibilities between Statistics Canada and Supply and Services. Statistics Canada was given all publishing responsibility for "fully priced" and "supported price" general publications, responsibility to maintain and manage inventories, as well as the right to retain publications sales revenue. Upon signing the memorandum, Statistics Canada took on its new responsibilities and also set its own publication prices.

20.85 Treasury Board approved the arrangement in the memorandum of understanding for a two-year period. At the end of the two years, an evaluation by the two departments was required before the arrangement could be made final. Although the evaluation was never undertaken, the agreement was extended by the Deputy Ministers of Supply and Services and Statistics Canada. Treasury Board has not been formally advised of this arrangement and has not given its approval. Without Treasury Board approval for the extension of the memorandum of understanding, Statistics Canada is setting prices and selling publications without proper Board authority.

20.86 The arrangement for assuming responsibility for publications set out in the memorandum of understanding between Statistics Canada and the Department of Supply and Services should be evaluated by the two departments as required by the Treasury Board, and resubmitted to the Board.

Management's response: Statistics Canada has been discussing with the Treasury Board Secretariat for some months an articulation of the role the agency should have in the management of its publications. These arrangements will be formalized.

Absence of required approvals of current prices

20.87 The Department of Supply and Services obtains Treasury Board approval of its approach to publication prices. It does this by obtaining approval of a costing model and pricing system for all its publications. All Government of Canada publications that are published and distributed by the Publishing Centre are priced using the costing model. The Publishing Centre normally sets prices based on the costs. There is also a provision to adjust prices to a "market value" if it is higher than the cost.

20.88 Statistics Canada has not obtained approval of its pricing structure for publications; its policy is to recover the cost to the government. We reviewed Statistics Canada's publication pricing practices. Its current prices are not based on a costing model and pricing system similar to the Publishing Centre. We found that the procedure for setting prices included additional factors beyond those included in the Publishing Centre approach.

20.89 Statistics Canada should obtain appropriate approval for the pricing system for its publications.

Management's response: When Statistics Canada obtained full delegation for publishing its own publications, it sought guidance from the Department of Justice and was advised that it had the power to set prices for its publications under the agreement.

Publications information system does not provide adequate cost information

20.90 There have been delays in implementing management information systems to meet the needs of the Publications Division. Information is currently not available in the system to estimate the full cost of publications sold.