This Web page has been archived on the Web.

1983 Report of the Auditor General of Canada

Synopsis

12.1 Since its creation in 1973, a major role of the Secretariat has been to advise the Solicitor General in relation to his statutory responsibilities respecting the mandates and activities of the Royal Canadian Mounted Police, The Correctional Service of Canada and the National Parole Board. The Secretariat has also played a role in promoting improvements in the criminal justice system which suffers from jurisdictional fragmentation and lack of co-ordination. The Secretariat has attempted to improve communication and consultation among the various parties in the system - Department of Justice, provinces, municipalities, international organizations, private sector, etc. - with the objective of promoting equity and cost-effectiveness in administering justice throughout the country.

12.2 Examining the role of the Secretariat, we found there had been some initial uncertainty, both internally and outside, about what role it should play. Should it have a leadership role in improving the criminal justice system in general? Should it act only as a policy adviser for the Minister? Or should it establish some balance between these two roles? This confusion was rooted in the Secretariat's broad mandate, its changing and complex environment and its need to respond to the immediate needs of the Solicitor General. It is only in the last few years that a clearer role for the Secretariat has emerged that encompasses both advice to the Minister and leadership in developing policy initiatives relating to criminal justice and national security issues. Other departments and agencies seem to be in agreement with the role. As well, the Secretariat has started addressing management, planning and control issues.

12.3 The Secretariat's 1982-83 budget was $19 million, and it had 262 authorized person-years. Exhibit 12.1 shows a summary organization chart. In Part III of its Estimates, the Secretariat refers to a number of factors currently affecting the criminal justice system. These include public concern about rising crimes against persons and property, rapidly escalating costs, increased emphasis on accountability of public institutions, and increased pressure for greater intergovernmental co-operation to reduce duplication of services and co-ordinate efforts. Exhibit 12.2 shows the increasing cost of the criminal justice system and the increasing incidence of crime. Exhibit 12.3 shows the fragmentation of Canada's criminal justice system.

Exhibits not available

12.4 Until the late 1970s, there was little Secretariat-wide planning and co-ordination of activities. More recently, following internal reviews in 1980 and 1981 and Policy and Expenditure Management System (PEMS) requirements, greater emphasis has been placed on developing a more systematic approach to planning activities. Although longer-term Secretariat goals and shorter-term objectives have recently been developed for most of its activities, some objectives need to be more fully developed and translated into programs and projects.

12.5 Strategic overview documents could be improved by showing alternative program strategies with related resource implications for the agencies (the RCMP, the CSC and the NPB), and by providing program evaluation information as required by PEMS. An operational planning process is in place, and both a Multi-year Operational Plan and an annual operational plan are prepared. These plans, however, are based primarily on prior years' resource allocations, revised, until recently at least, only to reflect additional resources required for new activities.

12.6 Although the need for carrying out program evaluation was recognized when the Secretariat was established in 1973, the responsibilities and the means for doing so were never clearly identified. The Secretariat is only now attempting to define how its programs should be evaluated.

12.7 The policy areas in the Secretariat, Policy Branch and the Police and Security Branch have experienced difficulties in achieving an appropriate balance in allocating resources between their dual roles of responding to the immediate concerns of the Minister and developing major policy initiatives. This has been due to a lack of adequate planning and control procedures, as well as a lack of information for assessing workloads and allocating resources. The Secretariat has not had proper work plans setting out activities, budgets, resources and milestones for monitoring progress.

12.8 A great many demonstration projects have been funded by the Secretariat's Consultation Centre since it was established in 1973. In spite of the importance to policy makers of having the results of these projects evaluated, very few project evaluations have been conducted. It was not until January 1982 that a policy was established requiring all projects to be evaluated. This policy has not yet been fully implemented.

12.9 A review by internal auditors indicated a need for performance measurement and reporting with respect to the regional operations of the Consultation Centre, as well as a lack of adequate management and control systems in other divisions of the Programs Branch.

12.10 Our examination of financial management and internal audit functions indicated that the Secretariat was carrying out the activities normally associated with these functions in a satisfactory manner.

Audit Scope

12.12 Because of the Secretariat's policy development and analysis role and because of its size, the relative importance of key functions and processes, which are the areas in which we tend to concentrate our audit efforts, differ from those of an operating department or agency. For example, efficiency and economy are not major audit issues within the Secretariat, whereas questions relating to measuring and reporting the effectiveness of its activities are of considerable importance, depending, as they do, on the extent to which its role and objectives have been clearly defined. Thus, a full understanding of the past, present and future role of the Secretariat was a natural beginning for our audit.

12.13 We reviewed the evolution of the Secretariat's role and examined the operations in the Policy, Research and Consultation Centre areas. We also examined the major management processes of planning and program evaluation. These affect all the Secretariat's operations and, in some instances, the other entities in the criminal justice community. We also examined the internal audit, financial management and electronic data processing functions.

12.14 As a result of internal corporate reviews and an internal audit, the Secretariat was making changes in its organizational structure, management processes and information systems at the time of our audit. Where appropriate, we used the results of these internal studies to supplement our own examination.

Observations and Recommendations

Role of the Secretariat

12.16 Between 1973 and 1978, there was some confusion about the Secretariat's role and mandate. At the time the Secretariat was created, the role of the Department of Justice with respect to policy development and analysis under Section 4 of the Department of Justice Act was not well developed. As a result, the Secretariat stepped in to fill a void by dealing with the more general administration of justice policy matters in addition to the specific law enforcement and corrections policy matters related to the different agencies.

12.17 The assumption of this major role soon led to conflict with the Department of Justice which, by the mid-1970s, had also started to assume greater responsibility for justice policy matters. In 1975, with direction from the Prime Minister, the roles and responsibilities of the two Ministers concerned were clarified. A formal Joint Justice Committee was also established to resolve issues of duplication, overlap and conflict. Creative tension between the two Departments was established; the Departments believe that it has a certain amount of merit.

12.18 Since 1978, the role of the Secretariat in enabling the Solicitor General to fulfil his statutory obligations has been strengthened by the creation of a separate Police and Security Branch. Also, with the introduction of PEMS by the Government in 1980, another dimension was added to the Secretariat's role in strategic planning for all entities in the Department of the Solicitor General. The Secretariat appears to have clarified its role and this seems to have been accepted by other departments and agencies. It has now started to identify and address planning, management controls and evaluation issues which had largely remained unresolved.

12.19 Given the small size of the Secretariat compared to the magnitude and complexity of the criminal justice system, it would be unrealistic to expect the Secretariat to assist in resolving successfully all the multitude of issues facing the system. In the past, the Secretariat has demonstrated its ability to address important issues - gun control legislation, young offenders legislation, crime prevention programs, and so on. Because of inherent limitations of size, there remains a need for the Secretariat to ensure that its efforts are concentrated on the most important areas by identifying the major issues confronting the criminal justice system, ranking them in priority and, with the participation of other parties, working to resolve them.

Planning

12.21 Although some plans were developed in the 1970s, there has, until recently, been little Secretariat-wide planning and co-ordination. For example, in 1976, the Secretariat identified over 100 criminal justice issues in which it was interested. From these, about 20 issues were selected for further work. Responsibilities were assigned and some resources identified. Many of these projects, however, were not completed or did not achieve their objectives. This was due to a number of reasons, including priorities not being identified, problems with staff continuity, lack of workplans with target completion dates and lack of management controls to monitor projects. We note that although many of these issues were identified in 1976 and work was begun then, a number of them are still on the current list of ministry priorities or identified as emerging issues in Part III of the Estimates.

12.22 Following corporate reviews in 1980 and 1981, the Secretariat has given more attention to systematic planning. A Special Adviser on Corporate Planning and Evaluation was appointed in January 1981 and, in November 1982, the Secretariat completed the review and revision of the corporate goals. Current longer-term corporate goals are defined in general terms such as "the development of a more just, equitable and humane criminal justice system that is responsive to public concerns."

12.23 At another level, the Secretariat has started to define shorter-term corporate objectives and translate these into activities and projects to facilitate planning and control. This work is not yet complete. The relationships among the Department's goals, corporate goals and corporate objectives as stated in the Strategic Overview, Corporate Planning Handbook and Part III of the Estimates are not clearly set out.

12.24 In 1980, with the introduction of PEMS, the Secretariat established a process for preparing the Strategic Overview. The agencies are fully involved in this process. Strategic Overviews for 1981-82 and 1982-83 were prepared, with input from all concerned.

12.25 This process has helped the Department in clarifying its role, long-term goals and objectives as well as identifying issues confronting the criminal justice environment. It has not, however, led to any changes being identified or proposed to the existing programs of the Secretariat or the agencies, nor have any alternative programs or policies been proposed for pursuing the objectives that have been identified.

12.26 We also noted that program evaluation information, such as the results of evaluations carried out in the RCMP and CSC, were not included in the strategic overview, as required by the Policy and Expenditure Management System.

12.27 The Department's Strategic Overview should, when appropriate, present alternative programs and policies for achieving the goals and objectives of the agencies as well as the Secretariat and should also include available program evaluation information.

Secretariat's response: It is noted that the Policy and Expenditure Management System Submissions Manual (January 31, 1983) does specify that program alternatives and program evaluation data should be included in the strategic overview. However, the Manual goes on to say, at pp. 12-13, that "Policy Committees establish annual guidelines for the preparation of Strategic Overviews...(and)...may also provide detailed instructions to individual departments and agencies on the content of their Strategic Overviews". The Manual suggests that, where conflicts between specifications of the Manual and those of the Policy Committees' instructions occur, the Policy Committees' instructions are to be followed. It is also noted that the position of the Office of the Comptroller General on this subject appears to be that program evaluation data should only be included in the Strategic Overview where major program/policy changes are contemplated.

The Secretariat will seek direction from the Ministry of State for Social Development to clarify the circumstances under which such information should be provided in future Overviews and will comply with such direction.

12.28 The Secretariat has an operational planning process which leads to the preparation of an annual plan as well as the Multi-year Operational Plan. The plans and process are integrated across the Secretariat and take into account agency and general criminal justice issues. However, operational planning within Branches and Divisions has been deficient. Proper workplans for present and projected activities, which include resource requirements, have not been prepared. Resources identified with Branches and Divisions have been accepted as given and not reassessed or reallocated in line with priorities.

12.29 The Annual Operational Plan and the Multi-year Operational Plan should be based on workplans for the Branches that identify and allocate resources in line with priorities.

Secretariat's response: Agreed. Under the Corporate Planning and Management System, Phase II of which was implemented 1 April 1983, all elements of the Secretariat are required to provide detailed workplans which specify, in addition to a multi-year forecast of human and operating resources: the relationship of the activity or project to Secretariat objectives, long-range goals and other activities; specific outputs that are to be produced and the results these outputs are expected to produce; tasks to be completed and milestones by which progress can be monitored; and the means by which progress and success of the activity or project are to be assessed.

At an Annual Program Review which takes place in the fall of each year, program mandate, long-range goals and short and mid-term objectives are reviewed and major initiatives and ongoing activities are ranked in priority for the next cycle. The Annual and Multi-year Operational Plans will therefore be based on fully developed workplans.

12.30 The Secretariat has written a Corporate Planning Handbook that should be useful to managers in planning and controlling resources. Planning and control will also be improved through the use of data to be provided by a management information system which is still in the early stages of implementation.

Program Evaluation

12.32 Since 1981, some progress has been made with the appointment of a Special Adviser on Corporate Planning and Evaluation, review of departmental goals and corporate objectives and development of a management information system which is in the early stages of implementation. During 1981, 1982 and 1983, the Secretariat has also stated its commitment to program evaluation in Part III of the Estimates, in its Annual Reports and in the Main Estimates.

12.33 In summary, then, the Secretariat has not met the 1977 government program evaluation policy or its own commitments to evaluate its programs. It has not developed a plan for carrying out and reporting program evaluations. Although the appointment of the Special Adviser on Corporate Planning and Evaluation is an important first step in remedying this situation, much more needs to be done.

12.34 As required by government policy, the Secretariat should develop a program evaluation policy and a plan for implementing it as soon as possible.

Secretariat's response: Senior management of the Secretariat has recognized the need for greater attention to effectiveness evaluation. Discussions on ways and means to implement program evaluation in the Secretariat have been held with representatives of the Program Evaluation Branch of the Office of the Comptroller General. In the meantime, the essential prerequisites for conducting effectiveness evaluation have been and are being developed and implemented in the Secretariat, including detailed statements of goals and objectives, program descriptions and workplans and essential management information systems.

A Program Evaluation Policy and Long-term Plan have now been developed and are presently being reviewed by senior management, in consultation with the Office of the Comptroller General.

Policy Areas

12.36 The Policy Branch and the Police and Security Branch have responsibility for policy advice to the Minister with respect to The Correctional Service of Canada, the National Parole Board and the Royal Canadian Mounted Police, as well as for liasing with others involved in the Criminal Justice system. The Policy Branch also co-ordinates the preparation of the Department's Strategic Overview document and both Branches participate in developing and implementing many of the major policy initiatives.

12.37 In 1982-83, the Policy Branch had a budget of $2 million and 32.5 person-years. The Police and Security Branch had a budget of $1.4 million and 27.5 person-years, of which 10 were dedicated to the Police and Law Enforcement Directorate.

12.38 Some appreciation of the work done by these Branches can be gained from the following brief description of their activities:

- Activities related to the Minister's immediate concerns. These include writing letters to the public, media, and Members of Parliament; preparing speeches; reviewing and commenting on submissions by the departmental agencies; and responding to other requests of the Minister.

- Activities related to departmental and Secretariat priorities. These include initiating, leading or participating in major policy initiatives such as Criminal Law Review, Conditional Release Study, Young Offenders legislation, and the Federal Role in Law Enforcement.

- Liaison activities. These include liasing with other organizations and bodies at the federal, provincial and municipal levels, in the private sector, in international organizations and bodies, etc.

12.40 The lack of resources in some units has been cited as a major reason for not giving attention to certain activities. However, in the absence of an information system, data have not been available to assess the workloads of the various policy units relative to Branch and Secretariat priorities.

12.41 We also noted that some major initiatives and projects did not have workplans with defined tasks and activities, budgets and resource projections and target completion dates against which to monitor progress. In the absence of approved workplans, it has been easy to divert resources to day-to-day requirements. The impact of this on developing major initiatives and projects such as the Cost of Criminal Justice System project and the Clemency Review and Release Study, has not been measured, although some delays have been attributed to this problem.

12.42 Workplans for policy areas should be prepared, identifying levels of resources dedicated to the major activities carried out in these areas. The resource requirements established for each of these activities should take into account both ongoing workloads and departmental priorities.

12.43 Workplans should also be prepared for all major projects and initiatives and should contain information on activities, tasks, milestones and budgets. These should be used to monitor progress.

Secretariat's response: Workplans, as noted in the response to a previous recommendation, are required to be prepared and maintained for all areas of the Secretariat; these contain the information noted in the recommendation.

In the case of the policy branches, as well as the Programs Branch, these workplans are prepared for both discretionary and non-discretionary activities and projects, to ensure that program priorities for both types of activity are adequately recognized and resourced. In the Police and Security Branch, for example, the 1983-84 total operating budget of $1.8 million is captured through a series of 24 major activities and 89 projects, for which workplans were filed by 31 March 1983.

It is recognized that some workplans are incomplete or have not been completed to as high a standard as others. This is the first year that such detailed workplanning has been required of managers at all levels and, while some difficulties have been experienced, the workplanning exercise is expected to be completed within a matter of weeks.

Programs Branch

12.45 We examined the operations of the Consultation Centre but, for other Divisions in the Programs Branch, we relied on the work of internal auditors who had completed an audit of the Branch in 1982 and reported their findings in January 1983. We also reviewed the study made by the Research Division to evaluate the Gun Control Legislation enacted by Parliament in 1977.

12.46 The Consultation Centre had a budget in 1982-83 of $5.1 million and 33 person-years. It has a head office in Ottawa and regional offices in Moncton, Montreal, Toronto, Saskatoon and Vancouver. The Centre is the Secretariat's only regionalized component.

12.47 The Consultation Centre plays a unique role within both the Secretariat and the Canadian criminal justice system. Its role is to stimulate improvements in criminal justice through the process of consultation with provinces and the private sector and the funding of demonstration projects to examine alternatives for the criminal justice system.

12.48 In the 1970s, because of a lack of Secretariat-wide planning procedures, the Consultation Centre carried out its activities without reference to the plans and priorities of other Branches. For example, from 1978 to 1981, the Centre committed approximately 50 per cent of its demonstration project funds to diversion projects at a time when diversion had diminished as a priority in the Policy Branch and the Research Division.

12.49 This situation changed in 1981 when the Secretariat introduced Secretariat-wide strategic and operational planning and when the Programs Branch accepted as its primary role the policy support function in relation to established policy priorities.

12.50 The Consultation Centre has funded hundreds of projects since 1973. By their nature, demonstration projects indicate little to policy makers if they are not evaluated in a credible manner. Despite the large number of projects undertaken, very few have been evaluated. Although the need has been recognized since 1975, the Consultation Centre only established in January 1982 a policy that evaluation must be a component of all demonstration projects. This policy, however, has not been fully implemented. The Centre has also not earmarked resources for these evaluations.

12.51 The Consultation Centre should ensure adherence to its policy that demonstration projects will be funded only if an evaluation component has been clearly identified.

Secretariat's response: The Secretariat accepts the spirit of the recommendation that greater attention should be paid to the evaluation of experimental and demonstration projects, and responsibility to carry out this intent has been assigned to the Research and Statistics Divisions of Programs Branch. The Statistics Division is in the process of enhancing its evaluation capability and, with the Consultation Centre, is experimenting with a regionally-based approach to evaluation. A Project Evaluation Committee, including the Director, Statistics Division (as chairman), the Director, Consultation Centre and the Director, Research Division, has been established to review all Consultation projects and proposals, to assess which should and can be evaluated and to plan the evaluations. The Statistics Division and Consultation Centre are also developing self-evaluation kits to offer alternative mechanisms for evaluation. Care will be taken to ensure that the evaluation policy and plan noted in the response to a previous recommendation recognize the need for the evaluation of these types of projects.

12.52 A recent internal audit review of the Programs Branch, including the Consultation Centre, identified deficiencies in the areas of definition of roles and responsibilities, performance measurement and project management and control systems. These are now being corrected by the Branch.

12.53 In the Research Division, we reviewed the planning, conduct and the final draft report of the Gun Control Evaluation Study.

12.54 The main strengths of the Gun Control Evaluation were the Secretariat's management of the project, the fact that new and useful data were collected and that the early implementation of the legislation was monitored.

12.55 The main deficiencies were the relatively weak research design, the lack of a final database in which all the information collected could be assembled, kept up to date and subjected to further analysis, and the lack of sufficient clarity in reporting the limitations of the research.

Financial Management

Electronic Data Processing

Internal Audit

12.59 The internal audit function has been operating in a generally satisfactory manner. Improvements are being made in defining and communicating a clear mandate within the Secretariat, elaborating annual audit plans and ensuring timely reporting of findings and follow-up on implementation.