2011 March Report of the Auditor General of Canada

Main Points

What we examined

Responsibility for health and social services in the Northwest Territories is shared between the Department of Health and Social Services, employing about 140 people, and 8 regional Health and Social Services Authorities, which together have about 1,260 employees. The Department is responsible for securing funding, developing legislation, setting policies and standards, monitoring and evaluating program results, and strategic planning. Each of the Health and Social Services Authorities is responsible for the delivery of services in its region and for day-to-day management and administration of programs and services. In an attempt to establish a more integrated health care system, the Department and Authorities have begun efforts to bring more consistency to standards and types of programs and services delivered by Authorities to all communities.

The Department’s budget for 2010–11, at $326 million, accounts for about 25 percent of the territorial government budget. The Department has allocated about $222 million to Health and Social Services Authorities for the delivery of health and social services to residents.

We examined whether the Department adequately manages health programs and services provided to residents; the audit did not include social services. In particular, we looked at whether the Department identifies health priorities and sets direction for the health care system, develops or modifies and supports programs and services to achieve its priorities, monitors program delivery, and assesses and reports the results achieved. We focused on programs and services that address diabetes, home care and long-term care, medical travel, and recruitment of medical professionals.

Audit work for this report was substantially completed on 30 September 2010.

Why it’s important

Like all jurisdictions across Canada, the Northwest Territories faces financial constraints, rising costs of drugs and technologies, and an aging population. These circumstances challenge the Department’s capacity to provide leadership to and manage the health care system, to maintain health care and services at current levels, and to offer new services.

What we found

  • In A Foundation for Change, the Department’s action plan, the Department has identified the priorities and actions needed to improve the health care system, and it has started modifying programs and services that address its priorities. Although several actions have been implemented or are under way, the Department has yet to implement certain other actions that could improve the health care system. These include establishing performance indicators, working with Health and Social Services Authorities to develop performance agreements that set out results to be achieved with funds, and revising the method of allocating funding to Authorities.
  • The Department does not adequately support and monitor diabetes programming, home care and long-term care programs, and medical travel. The Department has insufficient information to determine whether health outcomes of patients with diabetes are improving. Current standards for home care and long-term care programs are too broad to serve as a basis for monitoring and to ensure equitable access to the programs. The Department has taken action to improve the design and delivery of the Medical Travel Assistance program.
  • The health care system depends to a large extent on temporary health professionals (mostly physicians and nurses) to maintain the delivery of health services. These temporary workers cost more than permanent staff. The Department is working on alternative ways to improve recruitment as part of its action plan.

The Department has responded. The Department agrees with our recommendations. Its detailed responses follow each recommendation throughout the report.

Introduction

Mandate and responsibilities

1. In the Northwest Territories, responsibilities for the delivery and management of health and social services programs are divided among the Department of Health and Social Services, seven Health and Social Services Authorities, and one Community Services Agency—Tlicho (hereafter all eight are referred to as Authorities).

2. The Department of Health and Social Services. The mandate of the Department of Health and Social Services is to promote, protect, and provide for the health and well-being of the people of the Northwest Territories. More specifically, the Department is to

  • promote healthy choices and responsible self-care;
  • protect public health and prevent illness and disease;
  • protect children and vulnerable individuals from abuse, neglect, and distress; and
  • provide integrated, responsive, and effective health services and social programs for those who need them.

3. The Department is responsible for administering 28 pieces of legislation, notably the Hospital Insurance and Health and Social Services Administration Act, the Public Health Act, the Child and Family Services Act, and the Medical Care Act. The Department’s major responsibilities include

  • strategic planning,
  • securing funding,
  • developing legislation,
  • setting policies and standards, and
  • monitoring and evaluating programs.

4. The Department’s 2010–11 budget, at $326 million, accounts for about 25 percent of the territorial government budget. Of the $326 million, about $222 million is allocated to the Authorities to deliver health and social services. The remainder of the budget is allocated to the Department for supplementary health benefits and its oversight and support activities. About 1,400 people are employed full- or part-time in the delivery of health and social services across the Northwest Territories (the Department employs about 140 people and the Authorities employ about 1,260 people).

5. The Health and Social Services Authorities. The eight Authorities are responsible for the delivery of health and social services to people in their respective regions (Exhibit 1), as well as for the day-to-day management and administration of programs and services. They manage health centres and four hospitals across the Northwest Territories that deliver insured hospital and physician services to both in- and out-patients.

Exhibit 1—The Northwest Territories has eight Health and Social Services Authorities

Location of the eight Health and Social Services Authorities on a map of the Northwest Territories

[text version]

Source: Adapted from Department of Health and Social Services, June 2008

6. The Northwest Territories Health Care Plan. The Government of the Northwest Territories provides its residents with a full range of medically necessary hospital and physician services (also called Insured Health Benefits). It also provides some residents with Supplementary Health Benefits, such as prescription drugs, dental services, and medical travel. Annually, Health Canada provides about $20 million in contribution funding to the Government of the Northwest Territories to support Non-Insured Health Benefits for First Nation and Inuit residents and health promotion/disease prevention programs, such as the Aboriginal Diabetes Initiative. In addition, Health Canada provides $7.5 million of time-limited grant funding annually, to support health system reforms and offset costs associated with medical travel.

Operational context

7. The cost of providing health care in the Northwest Territories continues to increase, as in other jurisdictions in Canada. Between the fiscal years 2003–04 and 2008–09, health and social services expenditures increased by about 30 percent, from $238 million to $319 million.

8. Several factors make the delivery of health care in the Northwest Territories challenging. Some of these factors, such as an aging population and the increasing costs of health technologies and drugs, are largely beyond the Department’s control, while other factors, such as the cost of health care delivery, are more within the Department’s influence.

9. Delivering health care in the North is more expensive and complex than in southern Canada. The Northwest Territories is vast, comparable in size to Alberta and Saskatchewan combined. The population of about 43,000 is spread over 33 communities, most of them in remote locations with fewer than 1,000 people. Only five communities—Inuvik, Hay River, Fort Smith, Behchoko, and Yellowknife—have a population of over 2,000. The two major highways in the territory reach fewer than half of the communities; many rural and remote communities can be accessed only by plane or on winter ice roads. Given the remoteness of these communities, recruitment of health professionals is a constant challenge.

10. The territorial government anticipates that the proportion of seniors (aged 60+) will increase from 9 percent in 2008 to 13 percent by 2017, making it the fastest growing age group in the Northwest Territories. As the incidence of chronic conditions generally increases with age, chronic diseases such as heart disease, diabetes, and cancer will likely increase. This will in turn have an impact on the cost and delivery of health care.

11. The Department of Health and Social Services reports that the overall health status of Northwest Territories residents (Aboriginal and non-Aboriginal) has been improving over the past few decades. However, Aboriginal people continue to have poorer health outcomes than other residents of the Northwest Territories. Many of these outcomes can be attributed to lifestyle changes and choices, which may increase the risk of adverse health conditions, such as diabetes, cancer, heart disease, and stroke. Social and economic factors, such as low income, poor housing conditions, and low educational achievement, may also contribute significantly to the poorer health status of the Aboriginal population.

12. The Department has noted that residents of the Northwest Territories have high expectations for health care. They expect to receive the same level of primary health care services as those living in other larger and more densely populated regions of the country. The Department has indicated, however, that the level of care provided within the Northwest Territories cannot be equivalent to that of southern locations because of the number of small communities, geographic barriers, and large travel distances in the Northwest Territories.

13. The territorial government and the Department have long been concerned about the rising cost of providing health care. In recent years, several reviews have been conducted and action plans put forward to modernize and restructure the system (Exhibit 2). For example, in 2004, the Department and the eight Authorities moved to an Integrated Service Delivery Model—a team-based, client-focused approach to providing health and social services. The model was developed to ensure that people across the Northwest Territories had equal access to services and that the Authorities had consistent policies, procedures, and standards. The Department’s 2006 strategic plan, Shaping Our Future, and its 2009 action plan, A Foundation for Change, continue to work within this model and include actions to improve system-wide management and accountability.

Exhibit 2—Strategic and action plans to reform the Northwest Territories health care system

1998 Shaping Our Future—A Strategic Plan for Health and Wellness
2001 It’s Time to Act—A Report on the Health and Social Services System of the Northwest Territories
2002 Health and Social Services Action Plan—2002–2005
2004 Integrated Service Delivery Model for the NWT Health and Social Services System
2006 Shaping Our Future—2006–2010—An Updated Strategic Plan for Health and Wellness in the Northwest Territories
2006 Northwest Territories Health and Social Services Action Plan—2006–2010
2009 A Foundation for Change—Building a Healthy Future for the NWT—2009–2012
Focus of the audit

14. We examined whether the Department of Health and Social Services is adequately managing health programs and services provided to residents. In particular, we examined work that the Department has done to

  • identify health priorities and set direction for the system,
  • develop or modify and support programs and services to address these priorities,
  • monitor program delivery, and
  • assess and report the results achieved.

We focused on programs and services for diabetes, home care and long-term care, medical travel, and recruitment of medical professionals. We did not examine social services as part of this audit.

15. To better understand the issues the Department and the Authorities face in delivering programs and services, and to further validate our observations on the Department’s actions, we visited three Authorities—the Yellowknife Health and Social Services Authority, the Hay River Health and Social Services Authority, and the Tlicho Community Services Agency. We did not audit the delivery of health services or the quality of health services provided by the Authorities; nor did we audit the accuracy of data produced by the Department. More details about the audit objective, scope, approach, and criteria are in About the Audit at the end of this report.

Observations and Recommendations

Setting direction for the health care system

16. Providing clear direction to the health care system is important to

  • achieve better control of the growth of health care expenditures,
  • encourage all parts of the health care system to work together to prevent a waste of resources,
  • encourage Health and Social Services Authorities to adopt a territorial-wide perspective or approach in the management and delivery of services,
  • ensure equitable access to services across the territory, and
  • strengthen accountability to the Northwest Territories Legislative Assembly and residents.

17. The Department’s most recent strategic plan, Shaping Our Future (2006), and action plan, A Foundation for Change (2009), commit to increasing accountability for the delivery of health programs and services in the Northwest Territories and to entering into performance agreements with Authorities to clearly set out expected results. We examined whether the Department has set priorities for the health system and whether performance agreements between the Department and the Authorities have been established.

The Department has identified priorities and actions needed to improve the health care system

18. We examined whether the Department has a planning process to identify priorities and incorporate them into its plans, whether it consults with the Authorities in planning its work, and whether there is consistency between the Department’s plans and the Authorities’ plans. We reviewed the Department’s strategic plan, action plans, and business plans, the plans of three Health and Social Services Authorities—Yellowknife, Hay River, and Tlicho—and the meeting minutes of senior management committees.

19. Identification of goals, priorities, and actions. We found that the Department has identified and incorporated clear goals, priorities, and actions to improve the health care system into its action plans and business plans. In November 2009, the Department released its action plan, A Foundation for Change—Building a Healthy Future for the NWT—2009–2012. The plan sets out the following three goals for the health care system:

  • Wellness. Improve the well-being of individuals, families, and communities (communities, families, and individuals make healthy choices and are protected from disease).
  • Accessibility. Provide dependable, timely access to high-quality health and social services (people get the care they need, and know where and how to find it).
  • Sustainability. Organize the system effectively to ensure long-term sustainability (resources are used effectively and innovatively to ensure the health and social services system will be sustained for future generations).

20. Under each of these goals, the document identifies a series of priorities and actions, which together are designed to assist in the transition of the health care system to provide affordable and high-quality care. Examples of priorities are the management of chronic diseases, such as diabetes, heart disease, and cancer; the enhancement and customization of continuing care; the improvement of community services; and the innovative and maximum use of human resources. Examples of actions designed to improve the management of chronic diseases include the development of a Chronic Disease Management Model for the Northwest Territories and the promotion of nutrition, physical activity, and other preventative healthy choices.

21. The Department’s business plan for 2010–11 incorporates initiatives such as improving chronic disease management, enhancing continuing care, improving the design and delivery of medical travel, improving the recruitment of human resources, and strengthening performance measurement and reporting that are tied to the goals, priorities, and actions set out in A Foundation for Change.

22. Consultation with Health and Social Services Authorities. In response to a commitment made in the Department’s Action Plan for 2002–2005, the Department established two forums to facilitate ongoing consultation with the Authorities: the Joint Leadership Council and the Joint Senior Management Committee.

  • The Joint Leadership Council, chaired by the Minister and comprised of the Chair of each Authority and the Deputy Minister, has the mandate to set priorities and oversee the delivery of the health care system.
  • The Joint Senior Management Committee, chaired by the Assistant Deputy Minister and comprised of senior departmental management and the Chief Executive Officer of each Authority, provides direction to the system at the operational level.

23. Our review of the Council and Committee meeting minutes for 2006 to 2010 showed that the Minister of Health and the Department consult regularly with Board Chairs and Chief Executive Officers of the Authorities on priorities. The Authorities express their views on issues faced by the health care system as well as on the strategies and programs developed to address these issues.

24. Alignment of plans. We reviewed the strategic plans for three Authorities—Yellowknife, Hay River, and Tlicho. We found that these plans, while focusing on specific issues related to each Authority, are generally consistent with the Department’s goals and objectives. All plans were updated in 2009 or 2010. They include goals and objectives that reflect the Department’s goals of wellness, access, and sustainability. They also discuss programs and services such as diabetes, home care and long-term care, and human resources. Finally, they identify some performance indicators to assess the success of their activities. The plans are consistent with discussions of the Joint Senior Management Committee and reflect the intention on the part of all parties for greater system thinking.

25. The Department’s goals, priorities, and actions, and the alignment of Authorities’ goals and objectives with them, help to facilitate the implementation of actions required to strengthen the health care system and improve the wellness of residents, access to health care services, and sustainability of the system.

Performance agreements with Health and Social Services Authorities are not in place

26. We looked at whether agreements between the Department and Authorities set out expected results to be achieved with funds provided. We also looked at whether the Department assesses if Authorities are meeting the objectives of these agreements and whether the Department and Authorities take corrective actions when appropriate. We reviewed agreements for the fiscal years 2006–07 to 2010–11, as well as reports provided by Authorities pursuant to these agreements.

27. We found that the Department and Authorities have contribution agreements in place to fund programs and services. However, we did not find agreements setting out expectations for performance. The Department told us that work on developing such performance agreements had just begun and will not be complete before early 2011.

28. The Department indicated that one issue that needs to be resolved to implement performance agreements is the misalignment of funding with patient flow (movement of patients within a region or between different regions, or both these situations). The Authorities are currently funded on the basis of historical costs, which have not taken patient flow into account. Consequently, some Authorities may incur budget deficits, at least in part because they provide health services to a higher-than-expected number of residents from other regions. The Department indicated in its 2006 strategic plan that it intended to change its funding model. In its 2009 action plan, it indicated that it intends to change the model by 2012—a model that has been in place since 1997–98—to better reflect the needs of each Authority.

29. We found that contribution agreements between the Department and the Authorities included basic financial terms and conditions and corresponding reporting requirements. Current agreements specify that Authorities shall use their contribution funds for categories of expenditures, including hospital services, physician services, medical equipment, and human resources to deliver core services to residents and to manage, operate, and control those health facilities for which they are responsible. The agreements also establish financial reporting requirements that require Authorities to submit to the territorial government quarterly and annual variance reports as well as an annual audited financial statement and an annual budget. We found that Authorities have provided the Department with the reports required in the agreements and that the Department follows up with Authorities as necessary.

30. Contribution agreements are primarily designed to transfer money to Health and Social Services Authorities and to require them to provide financial reports to the Department. The development of separate performance agreements with Authorities would allow the Department to monitor whether performance expectations are met. This in turn would help to develop a consistent territorial-wide system of reporting that allows for better financial planning, increased financial sustainability, and better accountability to residents of the Northwest Territories about the quality of programs and services.

31. Recommendation. The Department of Health and Social Services should follow through on commitments made in its strategic plan and action plan to

  • revise the model to allocate funding to Health and Social Services Authorities, and
  • develop performance agreements that include expected results for key programs and services, and corresponding reporting requirements.

The Department’s response. Agreed. Improving governance and accountability to ensure the delivery of quality programs and services and consistent financial management is a priority in A Foundation for Change, the Department of Health and Social Services’ system action plan for 2009–2012. Key actions identified under this priority include the implementation of a new funding methodology and development of performance and service agreements with the Health and Social Services Authorities. These actions are to be developed during the 2011–12 fiscal year for implementation in 2012–13.

Supporting health programs and monitoring delivery

32. Through our review of diabetes, home care and long-term care, medical travel, and human resources recruitment programs, we examined whether the Department works with the Health and Social Services Authorities to develop or modify programs and services that address its health priorities and plans, whether it supports the delivery of programs and services by Authorities through developing and promoting policies and standards, and whether it monitors results achieved and compliance by Authorities with policies and standards. For each of the programs examined, where applicable, we reviewed related management frameworks, policies, standards and guidelines, assessment processes, evaluation and review reports, and data.

The Department does not adequately support and monitor the delivery of diabetes programming

33. Similar to the rest of Canada, the prevalence of diabetes is increasing in the Northwest Territories. The Canadian Diabetes Association reported that in 2005 more than 2,300 people had diabetes in the territory. The Department estimates that about 200 new cases of diabetes are diagnosed each year. Diabetes management includes diagnostic tests and treatment, as well as education and related programs designed to facilitate lifestyle modification, such as having a healthy diet, losing excess weight, and exercising regularly. For the most part, funding for diagnostic tests and treatment for diabetes comes from the Department, while funding for diabetes education programs comes from the federal government under the Aboriginal Diabetes Initiative. In the 2008–09 fiscal year, the Department spent about $650,000 on education programs.

34. Program support. We found that while diabetes management was identified as a priority in 2009 and research has been done to develop a strategy for the disease, the Department does not yet have a territory-wide strategy or approach designed to prevent or manage diabetes. The Department is currently developing a Chronic Disease Management Model, which is to incorporate diabetes prevention and management. As part of this work to develop the Model, the Department told us that three diabetes education programs are being combined to increase consistency in program delivery. Department documentation states that integration of diabetes management with strategies for preventing chronic disease is critical, as diabetes shares common determinants with lifestyle-related heart disease and cancer. The targeted date for completing the Chronic Disease Management Model is 2012.

35. The Department provides clinical practice guidelines for diabetes to nurses in Authorities’ community health centres. These guidelines reference the 2008 Clinical Practice Guidelines from the Canadian Diabetes Association.

36. Monitoring of program delivery. The Department does not monitor whether Authorities comply with Northwest Territories Clinical Practice Guidelines, whether they deliver and manage diabetes treatment and prevention consistently, or whether patient outcomes are improving as a result of treatment.

37. The Department reports prevalence and incidence data, extracted from its administrative databases, to the National Diabetes Surveillance System (maintained by the Public Health Agency of Canada). The National Diabetes Surveillance System produces data for comparing diabetes prevalence and incidence in the territory with other locations in Canada. It also produces data for comparing mortality, diabetes-associated diseases, and health care use in the populations with and without diabetes. We found that the Department does not use its administrative data to monitor diabetes programming. Nor does it systematically collect data on diabetes prevention and treatment results.

38. Although the Department is addressing diabetes through the development of the Chronic Disease Management Model, weaknesses in monitoring program delivery and the absence of data on results of diabetes treatment and education activities reduce the Department’s ability to measure the success of treatment and education programs and to make changes to improve them.

39. Recommendation. The Department of Health and Social Services, in collaboration with Health and Social Services Authorities, should

  • identify a core set of diabetes education, prevention, and treatment programs;
  • monitor implementation of the programs across the Northwest Territories;
  • identify and collect data to measure program results; and
  • use this data to improve program delivery.

The Department’s response. Agreed. Establishing a Chronic Disease Management Model for the Northwest Territories is currently identified as a key action contained in A Foundation for Change, the Department of Health and Social Services’ system action plan for 2009–2012. The recommendations related to diabetes will be addressed within the overall model of chronic disease prevention and management, delivered through Primary Community Care Teams. The first draft of the model has been developed and is to be completed in 2011–12.

Processes for assessing home and long-term care clients are not yet standardized and monitoring delivery of care is limited

40. Within the Department, home and community care and long-term care are part of a continuum of services, generally referred to as Continuing Care services. These services, based on client need, are intended to maintain or improve the physical, social, and psychological health of individuals who, for a variety of reasons, may not be able to fully care for themselves. The goal of continuing care is to improve independence and quality of life for these individuals and their families.

41. The Home and Community Care Program provides a broad range of services to individuals. These include, for example, respite care, palliative care, foot care, medications management, home management, meals on wheels, and transportation assistance. Services are provided by physicians, nurses, and other health professionals. Departmental statistics show that in 2009–10, about 1,800 clients received home care services through the Home and Community Care Program. Funding for the program comes from the Department and the federal government. In 2008–09, the Department spent about $9.1 million on home care, which included $3.8 million from the federal government.

42. Long-term care provides the opportunity for clients to live full-time in a facility that provides a level of service greater than what can be provided by the Home and Community Care Program. Long-term care facilities provide services for individuals whose needs cannot be safely met in a home setting. These individuals may have chronic and/or complex care needs, including multiple and severe disabilities or health issues resulting in reduced function. Department documentation indicates there are 136 long-term beds across the Northwest Territories, with an occupancy rate of close to 95 percent. Program expenditures amounted to about $13 million in the 2008–09 fiscal year.

43. Program support. To determine whether a patient is eligible for home care services or long-term care, the Authorities currently assess an applicant’s service and care needs. However, department documents indicate that the current assessment process does not adequately address specific client groups, such as persons with disabilities; allow for categorization of clients into distinct levels of care, which would be of significant benefit for program planning, delivery, and administration; or facilitate access to information needed by care providers, as well as by management and governing bodies.

44. In 2009, the Department made a commitment to ensure that clients who need continuing care enter into care through a coordinated referral and assessment process. This was intended to provide consistent and high-quality care to clients across the Northwest Territories. We found that the Department is working on acquiring a standardized and automated instrument for assessing care needed, planning care, and managing the care of clients within home care and long-term care. The Department expects that the instrument will improve patient safety and quality of care and reduce costs.

45. The Department also made a commitment to establish a territorial admission committee to develop and manage access to long-term care and to standardize models of care, direct care hours, and staffing so that they are consistent across the territory. The Department has taken action in this area. Clients seeking placement in long-term care facilities must now apply to the newly created Territorial Admissions Committee. This committee, which began accepting applications in October 2009, replaced six regional long-term care admission committees, thereby streamlining the admission process with one coordinated and prioritized placement list. The committee determines the placement of patients after a review of their application. Once clients have been admitted to long-term care, services are provided by the Authorities or by non-governmental organizations contracted by the Authorities. The Department has developed a management model, which includes an operational plan, functional programming, design standards, and prototype design, and is intended to provide the basis for developing appropriate long-term facilities.

46. Monitoring of program delivery. The Department adopted program standards for home care and long-term care in 2000. However, it does not monitor the Authorities’ compliance with these standards, as it considers them to be too broad to be useful for monitoring purposes. We found that while the Authorities applied some common standards and policies (for example, all Authorities prepare individualized care plans for each client), they also applied others that varied across Authorities (for example, a physician referral is required for access to home services in only some Authorities). We noted that services provided varied in the three Authorities we visited, depending on the community and the availability of qualified staff to provide the service. The Department has been working with the Authorities to update standards and expects the revised standards to provide a basis for monitoring and evaluating services.

47. The Department carries out limited monitoring of the Authorities’ home care activities. For example, it receives data from the Authorities on staff hours spent on each home care activity. With this information, the Department can observe variances in intensity of work performed by each Authority and ask for an explanation of these variances. However, the Department has difficulty assessing and comparing the performance of home care in each Authority because the Authorities have not defined service limits (for example, number of hours per day or per week per client for a given service such as meal preparation).

48. Non-standardized assessments and care management for home care and long-term care clients may result in inequitable access to services and inconsistent quality in the services delivered. As well, the lack of monitoring (against program standards) may result in the quality of services being inconsistent or below standard.

49. Recommendation. The Department of Health and Social Services, in consultation with Health and Social Services Authorities, should

  • implement a standardized process for assessing the service and care needs of all home care and long-term care clients;
  • complete the revision of program standards for home care and long-term care programs; and
  • develop and implement a plan to monitor these programs, including specifying the data to be collected by the Authorities and reported to the Department.

The Department’s response. Agreed. Standardization of the delivery of Continuing Care services across the Northwest Territories is currently identified as a key action contained in A Foundation for Change, the Department of Health and Social Services’ system action plan for 2009–2012. Program and staffing standards are to be updated in 2011–12 for inclusion in the 2012–13 business planning process.

The Department is working to improve the delivery of the Medical Travel Assistance program

50. Across the Northwest Territories, patients sometimes need to travel from one community to another to obtain health services. Some of this medical travel is within the Northwest Territories (usually from smaller communities to regional health centres in Hay River, Inuvik, and Yellowknife); some is outside of the Northwest Territories (usually to Edmonton). The Department’s 2010–11 Annual Business Plan states that just over 11,000 patients travelled for health services in the 2008–09 fiscal year.

51. In 1998, the territorial government established the Medical Travel Policy to ensure that the cost of medical travel was not a barrier to eligible residents who must travel in order to access necessary and appropriate insured health services. The Policy includes provisions for benefits (for example, transportation between communities) and for allowable expenses (for example, accommodation and meals).The Department is responsible for the administration of the Policy. It has delegated responsibility to the Stanton Territorial Health Authority to coordinate medical travel through the Medical Travel Assistance program.

52. Medical travel assistance is vital to residents of the Northwest Territories. The Policy sets out that the objective of assistance is to provide equitable access to care for patients who require treatment not available in their home community. The Department reported in its most recent Annual Business Plan that medical travel cost about $19 million in the 2008–09 fiscal year, an increase of over 30 percent from the 2006–07 fiscal year. According to department documents, factors that influence costs include increasing volumes, the non-availability of some specializations within the Northwest Territories (for example, orthopaedics), and contractor costs (for example, airfare).

53. We examined whether the Department monitors the Stanton Territorial Health Authority’s coordination of the Medical Travel Assistance program to ensure that it is consistent with the Medical Travel Policy. We reviewed department documents and conducted interviews with officials from the Department and from the Stanton Territorial Health Authority, as well as other stakeholders.

54. We noted that the Department commissioned an evaluation of the program to look at several issues, including escalating travel costs. The evaluation, reported in 2009, made a number of recommendations, including improving data sharing within the program and between service providers, and providing guidance for new medical staff on the cost effects of decisions to request medical travel. The Department has since taken action to improve the design and delivery of the program. It has solicited the assistance of experts to review and redesign the program and expects this work to be completed by March 2011. We also found that the Department had requested an audit of the Medical Travel Assistance program by the Northwest Territories’ Internal Audit Bureau. The audit was carried out in 2010; however, at the time we completed our audit work, it had not yet been reported.

55. In the three Authorities that we visited, we found that the Authorities had established their own processes for transferring patient records to other facilities as part of medical travel. Without a formal territory-wide approach for transferring these patient records, there is a higher risk that treatment may not be optimal, which could have a negative impact on patients’ health and increase medical costs. In September 2010, the Department introduced a new “Specialist Services” referral form for all primary care clinicians to use to increase efficiency of medical travel. The use of this common form is intended to help ensure timely booking of appointments, efficient use of resources, and the avoidance of duplicate investigations and unnecessary travel.

56. Although the Department has taken several actions to improve the Medical Travel Assistance program, we found that there are no formal mechanisms to monitor compliance with the Medical Travel Policy, including assessment of the program’s performance. The lack of monitoring by the Department means that it does not know whether coordination of medical travel complies with the Medical Travel Policy. Inconsistent application of the policy could result in clinical decisions that are not cost-effective and could compromise the sustainability of the health and social services system.

57. Recommendation. As part of the review and redesign of the Medical Travel Assistance program, the Department of Health and Social Services should

  • establish mandatory territory-wide protocols to ensure that required patient records are provided to relevant health care providers, and
  • establish reporting requirements for the Stanton Territorial Health Authority, as coordinator of the Medical Travel Assistance program, so that the Department can monitor the performance of the program on an ongoing basis.

The Department’s response. Agreed. The Department is currently undertaking a comprehensive review of the Medical Travel Assistance program, focusing on the clinical decision making and referral process as well as business/administrative processes. The review will be completed in late 2010–11, and implementation plans are to be initiated in 2011–12.

The Department is working to resolve recruitment challenges

58. Recruitment of health professionals is a constant challenge in the Northwest Territories due to factors such as geographical remoteness, socio-economic realities, and the persistent shortage of physicians and nurses across Canada. Vacancy rates for health care professionals in the Northwest Territories remain significant and many health professionals are expected to retire over the next 10 years. The lack of health care professionals puts the Department’s capacity to ensure the delivery of health care services at risk.

59. Responsibility for recruitment of health care professionals is shared among the Department of Health and Social Services, the Authorities, and the Department of Human Resources. The Department of Health and Social Services and the Authorities are responsible for identifying needs for health professionals and ensuring that positions are filled. The Authorities pay the salaries and associated costs for health professionals out of their operating budgets. Both the Department of Health and Social Services and the Authorities work with the Department of Human Resources in the recruitment process.

60. The Department of Human Resources’ mandate is to provide leadership and direction to departments and agencies (including Authorities) in all areas of human resource management. More specifically, the Department is responsible for leading, coordinating, and supporting the territorial public service in recruitment, retention, and human resource management planning.

61. We looked at whether the Department of Health and Social Services has identified its human resource needs and whether the Department of Human Resources has processes to recruit the appropriate number of health professionals with the required skills and competencies to ensure the delivery of health care services. As part of our examination, we looked at selected recruitment strategies in both departments.

62. Recruitment strategies of the Department of Health and Social Services. Identifying human resource needs helps an organization clarify where its efforts should be focused for its human resource practices. Recruitment initiatives are designed to help the organization meet its identified human resource needs.

63. We found that the Department of Health and Social Services has identified its human resource needs. It is also aware of the challenges it faces in meeting those needs. For example, it has identified the need for workers to meet the growing demand for home and community care. It has also identified a training gap for home support workers and resident care aides related to this need.

64. The Department and Authorities seek to recruit medical staff through their network of contacts within the medical profession, through presentations to various health professionals associations, and through an online recruitment tool—PracticeNorth.ca (launched in 2009). As well, resumes for physicians and nurses sometimes come in unsolicited as health professionals network among themselves. The Department and Authorities also rely on recruitment efforts by the Department of Human Resources—for example, the Department developed a pharmacist recruitment initiative.

65. However, recruitment and retention of workers remains a challenge. As a result, both the Department and the Authorities depend significantly on temporary staff (mostly nurses and physicians) to fill vacant health positions and maintain the delivery of health services to Northwest Territories residents. The need for registered nurses or nurse practitioners, particularly in the regions, is mostly met by staff from a relief pool of nurses. The need for physicians is met by locums (physicians hired on a temporary basis). While this arrangement allows the Authorities to maintain the delivery of health services, it comes with a high cost. Department documents indicate that staff under contract cost significantly more than permanent staff, partly because they must be flown in and out of the Northwest Territories and provided with accommodations. In addition, the shorter the terms are for these contracts, the more expensive they are because more flights have to be paid for. Continuity of care is also affected; for example, patients may need to restate their medical history to each new health professional they see.

66. We found that the Department of Health and Social Services had developed a five-year human resource strategy and action plan in 2004. A major goal of the strategy was to help build and develop a skilled workforce in the Northwest Territories. Initiatives included promoting and supporting the delivery of nursing programs and other health professional training offered through Aurora College. The plan, which expired in 2009, was not assessed to determine which components were successful; nor was a new strategy established.

67. The Department has noted that the shortage of health care workers and the challenges of attracting them to work in the North continue to make recruiting and retention one of its highest priorities. It has started work on human resource challenges identified in A Foundation for Change, its 2009 action plan. However, it has not yet moved forward on a key commitment outlined in the plan—managing physician services at a territorial level.

68. A lack of health care professionals has serious consequences for the delivery of health care services. Without adequate human resources to deliver the services, the standard of health care services could be compromised. For example, on several occasions in 2010, some health centres had to be put on “essential services only” status when the centres could not be staffed. The recruitment of health professionals needs to be improved if the Department and Authorities are to provide high-quality health care on a continuing basis.

69. Recruitment strategies of the Department of Human Resources. In 2009, the Department of Human Resources released its Northwest Territories Public Service Strategic Plan, “20/20: A Brilliant North,” outlining a 10-year framework for development of the public service. In early 2010, the Department released an Action Plan and Results Report, which outlines the specific actions the Government proposes to take over a three-year period to realize the vision and goals in the Strategic Plan.

70. We examined a key commitment the Department made in its Action Plan and Results Report—to establish targeted recruitment strategies for hard-to-attract occupations, including health professions. We found that the Department has made some progress but much work remains to be done. The Department has completed its review of current strategies but has not finalized plans or established performance measures and targets.

71. Given that responsibility for recruitment of health professionals is shared among the Department of Health and Social Services, the Authorities, and the Department of Human Resources, we expected to find a service agreement between the organizations. A service level agreement could define roles and responsibilities and formally set out the process for recruitment, including timelines for hiring health professionals in the territory. These are important planning details given the demand for the limited number of available health professionals across the territory and the country. We found that no agreement setting out service levels had been signed between the Department of Human Resources, the Department of Health and Social Services, and Authorities.

72. Recommendation. The Department of Health and Social Services, in collaboration with Health and Social Services Authorities and with the support of the Department of Human Resources, should develop a comprehensive human resource recruitment plan for the Northwest Territories health care system. It should monitor progress against the plan on an ongoing basis.

The Department of Health and Social Services’ response. Agreed. The maximization of human resources is currently identified as a key action contained in A Foundation for Change, the Department of Health and Social Services’ system action plan for 2009–2012. Following the piloting of human resource plans in several departments and the finalization of government-wide tools by the Department of Human Resources, the Department of Health and Social Services will work with Health and Social Services Authorities and the Department of Human Resources to develop system-wide recruitment and retention plans. These plans will be developed during the 2011–12 fiscal year for implementation in 2012–13.

The Department of Human Resources’ response. Agreed. The comprehensive human resource recruitment plan should align with the Government of the Northwest Territories Recruitment Strategy and the redesigned recruitment process.

73. Recommendation. The Department of Human Resources and the Department of Health and Social Services, in collaboration with the Health and Social Services Authorities, should develop and implement a service level agreement for recruitment and retention processes and activities that clearly sets out roles and responsibilities, timelines, and services to be delivered.

The Department of Human Resources’ response. Agreed. The plan to develop and implement a service level agreement, which includes recruitment and retention, is under way.

The Department of Health and Social Services’ response. Agreed. The Department of Health and Social Services supports the Department of Human Resources in its development of a government-wide service level agreement for human resources, including recruitment and retention, with modification of the generic service level agreement to be assessed given the special needs of the Health and Social Services system. The timelines for this project will be established by the Department of Human Resources.

Measuring and reporting on performance

74. Data on the costs, volumes, outputs, and outcomes of health programs and services is necessary to manage and improve health care delivery and health outcomes. It is also important to report this information to the Legislative Assembly and the public to demonstrate progress made in realizing the Department’s commitments and to support accountability.

Limited information is reported on the performance of the health care system

75. We looked at whether the Department has established performance indicators for health programs and services and whether it has planned for and carried out program evaluations or other reviews. We also examined whether it regularly reports performance information to the Legislative Assembly. We reviewed program evaluations and other reports completed by the Department since 2005. We also reviewed quarterly progress reports on A Foundation for Change, as well as annual business plans for the 2009–10 and 2010–11 fiscal years.

76. Performance measurement. Although some performance indicators are reported in the annual business plans, the Department has not yet developed a set of system-wide performance indicators to help it assess the performance of the health system against its three goals outlined in A Foundation for Change—wellness, accessibility, and sustainability. In this action plan and its annual business plans, the Department has acknowledged the need to improve basic performance measurement and reporting, and it has started preliminary work on developing performance indicators. However, the Department and Authorities have not yet agreed upon a set of performance indicators to use to help manage the health care system. The Department is seeking to develop these indicators in consultation with the Authorities to ensure that there is agreement on what is to be measured and that data is available for these indicators.

77. The Department has conducted several evaluations and program reviews in recent years. For example, it completed a review of the Home and Community Care Program in 2006, and assessed the use of services provided by physicians in 2006, hospitals in 2007, and health centres in 2008. It has evaluated the Medical Travel Assistance program and conducted several satisfaction surveys. However, we found that the Department has not yet developed an evaluation plan setting out where evaluations need to be focused and when they are to be completed. The Department has indicated that it needs to implement a risk management framework and identify performance indicators before developing its evaluation plan. It has further indicated that it needs to work with the Authorities to build a common method for gathering data to report on indicators and provide the quantitative information necessary for program evaluation.

78. Performance reporting. Over the last several years, the Department has published several reports that provide information on selected aspects of the health care system. In addition, the Department has informed the Legislative Assembly about its progress in implementing actions it committed to in A Foundation for Change and its annual business plans. Although the reports inform members of the Legislative Assembly about whether the Department is meeting its commitments, they say little about the performance of the health care system.

79. The Department last published the Health Status Report in 2005. This report was intended to inform the public about the well-being of the population in general and about the major determinants of health. Data was reported on health status (for example, self-rated health), health conditions (for example, incidence rates for cancer), mortality (for example, life expectancy at birth), and health determinants (for example, immunization rates). At the time we completed our audit work, the Department had started preliminary work on the next Health Status Report.

80. In summary, the Department has not provided a clear and consistent picture of how the health care system is performing. As a result, it is difficult for the Legislative Assembly to assess the Department’s performance. Performance measurement and reporting need to be improved.

81. Recommendation. The Department of Health and Social Services, in consultation with Health and Social Services Authorities, should

  • develop a set of system-wide performance indicators for the health care system and identify key data requirements,
  • develop a program evaluation plan setting out areas it plans to evaluate, and
  • regularly inform the Legislative Assembly about the performance of the Northwest Territories health care system.

The Department’s response. Agreed. Developing an ongoing system of reporting and evaluation is a priority in A Foundation for Change, the Department of Health and Social Services’ system action plan for 2009–2012. Key actions identified under this priority include introducing a system-wide performance measurement and reporting system, developing capacity for delivering the performance measurement and reporting system, and evaluating client satisfaction regularly. The timeline is to be developed during the 2011–12 fiscal year for implementation in 2012–13.

Conclusion

82. The Department is adequately managing health programs and services provided to residents. However, there are a number of areas where the Department needs to take further action to ensure that it can effectively manage the health care system in the future. These areas include following through on identified priorities, improving the development and support of programs and services to residents, and providing improved reporting on results achieved.

83. The Department has set clear direction by identifying priorities and actions—in both its 2009 action plan, A Foundation for Change, and its annual business plans—that are needed to improve the health care system in the Northwest Territories and to be more responsive to residents’ needs. However, the Department’s capacity to positively influence the achievement of priorities is limited by the absence of a set of system-wide performance indicators and the lack of performance agreements with Health and Social Services Authorities. The system is in transition, and the collaboration of all stakeholders is needed to achieve the Department’s goals of improving the wellness of residents, access to services, and sustainability of the system.

84. The Department has started modifying programs and services that address its priorities. These priorities include developing a Chronic Disease Management Model, updating program standards for home care and long-term care, and improving the Medical Travel Assistance program. However, the lack of specific program standards and data prevents the Department from effectively monitoring program delivery.

85. The Department has published several reports that provide insights into selected aspects of the health and social services system. However, little information is reported on the performance of the health care system.

About the Audit

All of the audit work in this chapter was conducted in accordance with the standards for assurance engagements set by The Canadian Institute of Chartered Accountants. While the Office adopts these standards as the minimum requirement for our audits, we also draw upon the standards and practices of other disciplines.

Objective

The objective of our audit was to determine whether the Northwest Territories Department of Health and Social Services is adequately managing health programs and services provided to residents.

“Adequately managing health programs and services” means

  • identifying health priorities and setting direction;
  • developing or modifying programs and services to address these priorities and supporting and monitoring program delivery, in conjunction with Health and Social Services Authorities; and
  • assessing and reporting the results achieved and modifying programs and services accordingly.

Scope and approach

Our audit included work the Department of Health and Social Services has done to identify health priorities and set direction for the health care system, to develop or modify programs and services to address these priorities, to support and monitor program delivery, and to assess and report the results achieved. We limited our audit to programs and services that address diabetes, home care and long-term care, medical travel, and recruitment of health professionals.

To better understand the issues the Department faces in delivering these programs and services, as well as to further validate our observations on the Department’s management systems and practices, we visited three Health and Social Services Authorities—Yellowknife, Hay River, and Tlicho. We also conducted limited work in the Department of Human Resources on human resource recruitment initiatives.

Our audit work included interviews with officials of the Department of Health and Social Services, the Department of Human Resources, the three Authorities, and other stakeholders. It also included an examination of relevant departments’ and Authorities’ documentation and data.

We did not examine social services or documents containing residents’ personal health information; nor did we assess the accuracy of data collected and produced by the Department.

Criteria

To determine whether the Northwest Territories Department of Health and Social Services is adequately managing health programs and services provided to residents, we used the following criteria:
Criteria Sources
Identifying health priorities and setting direction

The Department of Health and Social Services has an ongoing process to identify health priorities and incorporate them into its strategic and operational plans.

The Department, as part of its work to identify health priorities and develop plans, consults with Health and Social Services Authorities.

The Department works with Health and Social Services Authorities so that there is consistency between the Department’s plans and the Authorities’ plans.

The Department and the Department of Human Resources have human resource plans and processes in place to recruit the appropriate number of health professionals with the required skills and competencies at the right time and at the right place.

  • Northwest Territories Health and Social Services Establishment Policy, 1999
  • Health and Social Services Joint Leadership Council, Terms of Reference, 2006; and Joint Senior Management Committee, Terms of Reference, 2010
  • A Foundation for Change—Building a Healthy Future for the NWT—2009–2012
  • Shaping Our Future—2006–2010—An Updated Strategic Plan for Health and Wellness in the Northwest Territories
  • Financial Management Board Handbook, 2007
  • Integrated Service Delivery Model for the NWT Health and Social Services System, 2004
  • Northwest Territories Human Resources Establishment Policy, 2006
Developing programs (diabetes programming, home care and long-term care, and medical travel) for residents and supporting and monitoring their delivery

The Department works with the Health and Social Services Authorities to develop programs and services that address its health priorities and plans.

The Department supports the delivery of programs and services by Health and Social Services Authorities through developing and promoting policies and standards.

The Department monitors results achieved and compliance by Health and Social Authorities with program delivery policies and standards.

  • Northwest Territories Health and Social Services Establishment Policy, 1999
  • Health and Social Services Joint Leadership Council, Terms of Reference, 2006; Joint Senior Management Committee, Terms of Reference, 2010
  • A Foundation for Change—Building a Healthy Future for the NWT—2009–2012
  • Shaping Our Future—2006–2010—An Updated Strategic Plan for Health and Wellness in the Northwest Territories
  • Financial Management Board Handbook, 2007
  • Integrated Service Delivery Model for the NWT Health and Social Services System, 2004
Assessing the performance of health programs and reporting on results

The Department has agreements with Health and Social Services Authorities that set out the objectives for which funding is provided, expected results, and reporting requirements.

The Department assesses whether Health and Social Services Authorities are meeting the objectives of agreements and takes corrective actions where appropriate.

The Department plans for and undertakes program evaluations or formal reviews of key programs and services it delivers or Health and Social Services Authorities deliver.

The Department regularly reports to the Legislative Assembly information on the implementation of its priorities and plans.

  • A Foundation for Change—Building a Healthy Future for the NWT—2009–2012
  • Shaping Our Future—2006–2010—An Updated Strategic Plan for Health and Wellness in the Northwest Territories
  • Financial Administration Manual, 2006
  • Financial Management Board Handbook, 2007

Management reviewed and accepted the suitability of the criteria used in the audit.

Period covered by the audit

The period under examination was from April 2006 to September 2010. We selected April 2006 as the beginning of the period because the latest strategic plan was introduced at that time. The four-year period represents a sufficient period of time on which to base our observations. Audit work was substantially completed on 30 September 2010.

Audit team

Assistant Auditor General: Jerome Berthelette
Principal: Glenn Wheeler
Directors: Richard Gaudreau, Greg Cebry

Cheryl Derry
Andrew Ross
Ruth Sullivan

For information, please contact Communications at 613-995-3708 or 1-888-761-5953 (toll-free).

Appendix—List of recommendations

The following is a list of recommendations found in the report. The number in front of the recommendation indicates the paragraph where it appears in the report. The numbers in parentheses indicate the paragraphs where the topic is discussed.

Recommendation Response
Setting direction for the health care system

31. The Department of Health and Social Services should follow through on commitments made in its strategic plan and action plan to

  • revise the model to allocate funding to Health and Social Services Authorities, and
  • develop performance agreements that include expected results for key programs and services, and corresponding reporting requirements. (26–30)

The Department’s response. Agreed. Improving governance and accountability to ensure the delivery of quality programs and services and consistent financial management is a priority in A Foundation for Change, the Department of Health and Social Services’ system action plan for 2009–2012. Key actions identified under this priority include the implementation of a new funding methodology and development of performance and service agreements with the Health and Social Services Authorities. These actions are to be developed during the 2011–12 fiscal year for implementation in 2012–13.

Supporting health programs and monitoring delivery

39. The Department of Health and Social Services, in collaboration with Health and Social Services Authorities, should

  • identify a core set of diabetes education, prevention, and treatment programs;
  • monitor implementation of the programs across the Northwest Territories;
  • identify and collect data to measure program results; and
  • use this data to improve program delivery. (33–38)

The Department’s response. Agreed. Establishing a Chronic Disease Management Model for the Northwest Territories is currently identified as a key action contained in A Foundation for Change, the Department of Health and Social Services’ system action plan for 2009–2012. The recommendations related to diabetes will be addressed within the overall model of chronic disease prevention and management, delivered through Primary Community Care Teams. The first draft of the model has been developed and is to be completed in 2011–12.

49. The Department of Health and Social Services, in consultation with Health and Social Services Authorities, should

  • implement a standardized process for assessing the service and care needs of all home care and long-term care clients;
  • complete the revision of program standards for home care and long-term care programs; and
  • develop and implement a plan to monitor these programs, including specifying the data to be collected by the Authorities and reported to the Department. (40–48)

The Department’s response. Agreed. Standardization of the delivery of Continuing Care services across the Northwest Territories is currently identified as a key action contained in A Foundation for Change, the Department of Health and Social Services’ system action plan for 2009–2012. Program and staffing standards are to be updated in 2011–12 for inclusion in the 2012–13 business planning process.

57. As part of the review and redesign of the Medical Travel Assistance program, the Department of Health and Social Services should

  • establish mandatory territory-wide protocols to ensure that required patient records are provided to relevant health care providers, and
  • establish reporting requirements for Stanton Territorial Health Authority, as coordinator of the Medical Travel Assistance program, so that the Department can monitor the performance of the program on an ongoing basis. (50–56)

The Department’s response. Agreed. The Department is currently undertaking a comprehensive review of the Medical Travel Assistance program, focusing on the clinical decision making and referral process as well as business/administrative processes. The review will be completed in late 2010–11, and implementation plans are to be initiated in 2011–12.

72. The Department of Health and Social Services, in collaboration with Health and Social Services Authorities and with the support of the Department of Human Resources, should develop a comprehensive human resource recruitment plan for the Northwest Territories health care system. It should monitor progress against the plan on an ongoing basis. (58–71)

The Department of Health and Social Services’ response. Agreed. The maximization of human resources is currently identified as a key action contained in A Foundation for Change, the Department of Health and Social Services’ system action plan for 2009–2012. Following the piloting of human resource plans in several departments and the finalization of government-wide tools by the Department of Human Resources, the Department of Health and Social Services will work with Health and Social Services Authorities and the Department of Human Resources to develop system-wide recruitment and retention plans. These plans will be developed during the 2011–12 fiscal year for implementation in 2012–13.

The Department of Human Resources’ response. Agreed. The comprehensive human resource recruitment plan should align with the Government of the Northwest Territories Recruitment Strategy and the redesigned recruitment process.

73. The Department of Human Resources and the Department of Health and Social Services, in collaboration with the Health and Social Services Authorities, should develop and implement a service level agreement for recruitment and retention processes and activities that clearly sets out roles and responsibilities, timelines, and services to be delivered. (58–71)

The Department of Human Resources’ response. Agreed. The plan to develop and implement service level agreements, which includes recruitment and retention, is under way.

The Department of Health and Social Services’ response. Agreed. The Department of Health and Social Services supports the Department of Human Resources in its development of a government-wide service level agreement for human resources, including recruitment and retention, with modification of the generic service level agreement to be assessed given the special needs of the Health and Social Services system. The timelines for this project will be established by the Department of Human Resources.

Measuring and reporting on performance

81. The Department of Health and Social Services, in consultation with Health and Social Services Authorities, should

  • develop a set of system-wide performance indicators for the health care system and identify key data requirements,
  • develop a program evaluation plan setting out areas it plans to evaluate, and
  • regularly inform the Legislative Assembly about the performance of the Northwest Territories health care system. (74–80)

The Department’s response. Agreed. Developing an ongoing system of reporting and evaluation is a priority in A Foundation for Change, the Department of Health and Social Services’ system action plan for 2009–2012. Key actions identified under this priority include introducing a system-wide performance measurement and reporting system, developing capacity for delivering the performance measurement and reporting system, and evaluating client satisfaction regularly. The timeline is to be developed during the 2011–12 fiscal year for implementation in 2012–13.

 

PDF Versions

To access the Portable Document Format (PDF) version you must have a PDF reader installed. If you do not already have such a reader, there are numerous PDF readers available for free download or for purchase on the Internet: