DIPLOMA PROGRAM IN ENVIRONMENTAL HEALTH
McMaster Institute of Environment and Health
McMaster University

The Diploma Program in Environmental Health is designed to provide new and/or upgraded skills and knowledge in the principles and practice of environmental health. Students are selected on an interdisciplinary basis and are required to have a University Degree or equivalent. The Diploma Program is suitable for public health unit professionals, physicians, community health nurses, environmental industrial professionals and those in labour and non-governmental organizations dealing with environmental health issues. Students will have the option of enrolling in the campus-based instruction program or through its distance education program.

For further information, please contact :

Monica Anderson, BA, CHRM Administrative Coordinator
McMaster Institute of Environment and Health McMaster University
1280 Main Street West, BSB B150 Hamilton ON L8S 4K1
Phone: (905) 525-9140, ext. 27559 Fax: (905) 524-2400
Email: ecoenvir@mcmail.cis.mcmaster.ca

Epidemiology of Tuberculosis in Ontario: Update 1998

Introduction

At the beginning of the 20th century, tuberculosis (TB) was the leading cause of death due to infectious disease (1). Although improvements in public health and discovery of antituberculous drugs led to a decline in the incidence of TB and to a vision of TB elimination, the incidence of TB began to climb again in Ontario in the 1980s. Factors responsible for this included increased immigration from countries with high TB rates, HIV, and the emergence of drug-resistant TB. Now with the advent of the 21st century, TB has reemerged as an important public health issue.

Currently, the World Health Organization (WHO) estimates that one-third of the world's population is infected with Mycobacterium tuberculosis and, in 1993, the WHO declared TB a global emergency.

Despite the vast amount of information available on the etiology, treatment, and prophylaxis of the disease, modeling data has shown that the global incidence of TB is expected to increase from 8.8 million cases in 1995 to 10.2 million cases by the year 2000 and up to 11.9 in 2005. In addition to this, if it is assumed that the level of effectiveness and availability of treatment programs remain the same, 3.5 million deaths are expected for the year 2000, an increase from 3 million in 1995 (2).

In Canada, while the incidence of TB has declined over the past few decades, this rate of decline has leveled off with approximately 2000 new cases and more than 100 deaths reported annually (3). Although the rate of TB in Canada is one of the lowest in the world at about 7 per 100,000 population per year, the TB rates in sub-populations still vary. While the rate of TB among Canadian-born non-aboriginal people is approximately 1 per 100,000, rates of TB among aboriginal people are still high at 70 per 100,000 (4). In addition to this, there are a large number of immigrants, refugees and visitors arriving in Canada each year from countries with high rates of TB. The risk of reactivation of disease in persons who have had previous TB disease is increased in the foreign-born group (4). The ten countries with the highest incidence rates for TB are presented in Table 1 (5).

In 1996, 76% of TB cases reported in Canada occurred in Ontario, Quebec, and British Columbia (Figure 1). The rate of TB was highest in the Northwest Territories at 53.9 cases per 100,000 which is attributed to high rates among the mainly aboriginal population (3). Ontario accounted for the highest number of cases across Canada representing 41.4% of all Canadian TB cases and 36.2% of the Canadian population.

In order to more closely examine the emergent TB issues in Ontario, a descriptive analysis of the 1996 to 1998 Ontario TB data as well as an in-depth look at the trends in drug-resistant TB from 1990 to 1998 were undertaken. This report is an update of the descriptive epidemiology of TB in Ontario based on the format of previous reports published in PHERO (6, 7, 8).

Methods

Under the Health Protection and Promotion Act (HPPA), all new active and reactivated cases of TB are to be reported to the local medical officer of health. These reports are then transmitted, on a non-nominal basis, to the Public Health Branch (PHB), Ontario Ministry of Health and Long-Term Care. Since 1990, TB cases have been reported electronically through the Reportable Diseases Information System (RDIS) each week. For this report, records for TB cases with episode dates from January 1, 1990 to December 31, 1998 were extracted from RDIS (in December 1999) and analyzed using Epi-Info (Centers for Disease Control and Prevention, Atlanta, Georgia). Cases from 1996 to 1998 were analyzed for general TB trends while all cases from 1990 to 1998 were analyzed for trends in drug-resistant TB.

While this analysis looked at the descriptive epidemiology of TB from 1996 to 1998, for specific variables where the trends have remained virtually unchanged over this period, the data from the most recent year (1998) were presented. Where the numbers were small, particularly for analyses including the aboriginal population, aggregate data from 1996 to 1998 were used.

The RDIS case definition for the reporting of TB is :

a) Mycobacterium tuberculosis complex (e.g., M. tuberculosis, M. bovis (excluding BCG), or M. africanum) demonstrated on culture from sputum, body fluids, or tissues
or
b) without bacteriological proof but with clinical symptoms or signs, radiological or pathological evidence of active pulmonary or nonpulmonary disease, preferably with:
i) a positive tuberculin skin test (as defined by the provincial guidelines) and/or
ii) demonstration of acid-fast bacilli in smears from sputum or other body fluids or tissues and/or
iii) response to antituberculosis treatment

For RDIS reporting, "new active" cases are defined as having no documented evidence or history of previous episodes of active TB; "reactivated" cases are persons with active TB who have documented evidence or history of active TB which had become inactive. "Inactive" cases are those for whom cultures for Mycobacterium tuberculosis have been negative for at least 6 months, or if culture results are not available, where chest (or other) x-rays have been stable for at least 6 months. Cases reported in RDIS as having inactive TB were not included in this analysis.

For this analysis, primary pulmonary and pulmonary TB were combined into one category of pulmonary TB. Respiratory TB includes pulmonary TB and other respiratory (e.g., laryngeal) TB but not pleural or miliary TB.

In the RDIS reporting system, cases are classified by country of origin into three mutually exclusive categories: 1) Canadian-born (does not include aboriginal cases); 2) persons born outside Canada (foreign-born); and 3) aboriginal people, which includes registered Indians, unregistered Indian/Metis, and Inuit. Cases in which the origin was missing or entered as "unspecified" accounted for 32 (1.4%) of all TB cases reported in the 1996 to 1998 period. It was assumed that these cases were comparable to the population analyzed and these cases were not included in any analyses including the origin variable.

1996 population census data for Ontario (Statistics Canada) were used to calculate incidence rates for TB.

Data on drug-resistant TB were based on the information available from the RDIS case reports. These data may differ slightly from drug sensitivity data based on laboratory reports due to delays in reporting or entering data, or differences in case classification by year of isolation or year specimen was received (laboratory reports) versus year of symptom onset (RDIS data). Laboratory data may also reflect isolates rather than individual cases.

Results

Incidence
The trends in the incidence of TB over the past 53 years from 1945 to 1998 are depicted in Figure 2. During the period from 1960 to 1988, a total of 32,351 cases of active TB were reported. The trend during this time period shows a relatively steady decrease in the number of reported TB cases from a peak of 2030 cases of TB in 1960, to 644 cases in 1988. The average number of cases per year declined from 1,100 per year in the 1970s to approximately 700 per year in the 1980s (8).

The increase in cases of TB from 1988 to 1989 was, in part, due to a change in the Ontario reporting system for TB which resulted in cases being classified by year of onset starting in 1990. Prior to 1990, cases reported through the paper-based system were classified by date that the case report was received at the health unit. This change, in part, accounts for the 10% increase in cases of TB from 1988 to 1989, as some cases reported in 1990 were classified as 1989 cases based on their date of onset.

This was followed by a slight increase in rates from 1989 to 1995. From 1990 to 1998, the average number of cases increased to 800 cases annually (7.4/100,000 population).

Within Ontario, the geographic distribution of TB varies among health units (Figure 3). In 1998, the highest incidence rates occurred in the amalgamated City of Toronto (17.0/100,000) followed by Ottawa-Carleton (10.1/100,000) and Hamilton-Wentworth (6.8 cases/100,000). Within the amalgamated Toronto, the highest rates of TB occurred in the former: City of Toronto (20.7/100,000), Borough of East York (17.6/100,000), and City of Scarborough (17.5/100,000).

Consistent with previous years (8), the majority of TB cases reported in 1998 resided in urban centres that tend to have higher concentrations of new immigrants and refugees. As depicted in Figure 4, the 6 health units that now make up the amalgamated City of Toronto, followed by Peel Region and Ottawa-Carleton health units reported the highest number of cases in 1998.

A high incidence of TB was also reported by the Northwestern health unit (9.1/100,000), where all 8 reported TB cases in 1998 were of aboriginal origin.

Demographics
In 1998, 53.8% of TB cases were males while 46.2% were female. Figure 5 shows the number of cases and age-specific rates of TB by sex for 1998. The mean age of TB cases was 47 years.

For males, the age-specific rate of TB ranged from 1.1/100,000 in the 10-14 year age group to 24.9 per 100,000 the 70+ year age group, reflecting the trend in previous years (8). Similarly, for females, the rates ranged from 1.1/100,000 in the 5-9 year age group to 10.9/100,000 in the 70+ age group.

The age-specific rates are slightly higher for females in the 10-29 year age groups, after which the age-specific rates are consistently higher for males. The largest difference between sex-specific rates occurred in the 70+ age group. Of those TB cases aged 0-4 years that occurred between 1996 and 1998, the majority (68%) were Canadian-born children. A prior survey of pediatric cases in Ontario (9) found that the majority of these are first-generation Canadians living in immigrant households (i.e., born to immigrant parents). In the 5-19 age group, the majority of cases continue to occur in foreign-born children.

Figure 6 shows the age distribution of TB cases by origin. The mean age of Canadian-born cases was 47.6 years; for aboriginal cases the mean age was 45.7 years. Similar to previous years, the mean age for foreign-born cases was younger at 41.5 years. For foreign-born cases, the number of cases reported peaked in the 25 to 39 year age groups with a second peak occurring in cases 60 to 79 years of age. For the Canadian-born cases, the highest number of cases occurred in those aged 70 to 79 years with a smaller peak in those under 4.

Between 1996 and 1998, 37.2% of cases were less than 35 years, the age below which there is little concern about isoniazid hepatotoxicity and need for close monitoring. In the period from 1996 to 1998, of the 853 cases in the under 35 year age group, 86.6% were foreign-born, 10.1% were Canadian-born, 2.7% of cases were aboriginal, and 5 cases had missing or unspecified data for origin. A large number of these cases were therefore potentially preventable cases of TB.

Origin
Between 1996 and 1998, 84.6% of all TB cases occurred in foreign-born individuals, 12.9% in Canadian-born, and 2.4% in aboriginal people.

As demonstrated in previous reports, for foreign-born persons, recent arrival in Canada is a risk factor for the development of active disease. In 1998, 36.4% of the foreign-born cases were diagnosed with active disease within the first 2 years of arriving in Canada and a total of 42.9% diagnosed within 5 years of arrival. Another 36.0% had been in Canada between 5 and 15 years of arrival. Overall, the mean time from arrival in Canada to date of diagnosis of TB was 10.4 years for foreign-born cases diagnosed in 1998; the mean age of these cases at time of diagnosis was 46.3 years.

The interval between arrival in Canada and onset of TB, however, varied by age. For the age group up to 39 years, 50.0% of cases were diagnosed within the first two years of arrival and a total of 58.1% of cases in the first five years. In the 40 years and over age group, only 28.9% of cases occurred within the first five years of arrival to Canada. Therefore, the first 5 years after arrival appears to be the highest risk period for development of active TB, particularly for the younger age groups. These statistics reinforce the need for appropriate assessment and treatment or prophylaxis of these high-risk groups.

Figure 7 shows foreign-born cases by country of origin and the mean age of these cases.

Status, Site and Risk Factors
In 1998, 89.5% of all reported TB cases were classified as new active and 9.7% as reactivated; 0.8% of cases were missing data on the staging of disease. This distribution is similar to that of foreign-born cases. For Canadian-born cases, however, a higher proportion of cases are reported as "new active" (93.1%).

For 1998, 61.4% of the cases were reported as respiratory, 38.6% were non-respiratory and 0.5% of cases had the anatomical site listed as "unknown". The distribution of TB cases by site is presented in Figure 8. This distribution is consistent with the sites reported for TB cases over the past 10 years. From 1996 to 1998, respiratory cases of TB represented a larger proportion of cases among the aboriginal (81.8%) and Canadian-born cases (79.2%) compared to the foreign-born cases (57.5%) (Figure 9).

The most common non-respiratory sites of active TB were the lymph nodes (49.7%), pleura (7.3%), and bones and joints (5.2%). Specific sites of non-respiratory TB by origin for 1996 to 1998 are presented in Figure 10.

Tables 2 and 3 present the risk settings and risk factors for TB cases. Risk setting was listed for 99.3% of cases and risk factor for 98.9% of cases. More than one risk setting or risk factor could be reported for each case. Comparable to the results of previous years, travel or living in a TB endemic country remained a significant risk setting and was reported for 71.8% of cases. Risk factor was reported as "unknown" for 52.6% of the cases. Where a specific risk factor was documented, presence of an underlying medical condition and close contact with a case remained the two most commonly reported risk factors for TB (9.5% and 9.3% respectively).

Case ascertainment
For the period 1996 to 1998, 82.2% of TB cases were detected by symptoms, 5.2% by routine screening, 3.5% by immigration surveillance, 3.3% by contact tracing, and 0.6% post-mortem. For the time period from 1990 to 1998, 76.5% of TB cases had a positive culture result.

Hospitalization and Mortality
Between 1996 and 1998, 2291 TB cases were reported. Approximately 45% (n=814) of the cases for which hospitalization data was reported (n=1823), were hospitalized for treatment of their TB. Of the 2291 cases, 186 died, representing 8.1% of all reported cases during this time period and a mean of 62 deaths annually. Categorized by origin, there were 134 reported deaths among foreign-born cases for a case fatality rate of 7.0% over this time period; 35 (11.9%) deaths in Canadian-born cases, and 8 (14.5%) deaths among aboriginal cases. The mean age of the cases who died was 69.5 years (median 74 years).

Drug-Resistance
Sensitivity data were available on 4383 of the 5468 culture confirmed cases (of the remaining cases, 943 had sensitivity results documented as "unspecified" and 142 were missing any documentation). For the period from 1990 to 1998, 580 cases were reported as being resistant to one or more antituberculous drugs. Eight of the 580 cases did not have a culture positive result documented on RDIS; these may represent cases in which resistance status was determined in a jurisdiction outside Ontario. The proportion of the 580 cases with resistance to one or more drugs increased from 4.2% in 1990 to a high of 13.9% in 1997 (Figure 11).

The mean age for cases that were drug-resistant was 42.5 years (median 40 years) compared to 48.6 years (median 43 years) for cases sensitive to all first-line antituberculous drugs.

Between 1990 and 1998, 44.1% of the 580 resistant cases were reported as resistant to isoniazid alone. This represented the vast majority of cases with single drug resistance. During the same period, 34.1% of resistant cases were reported as resistant to two or more drugs.

Trends in single drug-resistance have been most variable for isoniazid (Figure 12). The proportion of TB cases resistant to isoniazid ranged from 1.5% in 1990, to a peak of 5.6% in 1994. This proportion generally increased between 1990 and 1996 but did decline in 1997 and 1998.

Multidrug resistant TB (MDR-TB), defined as drug resistance to at least isoniazid and rifampin, increased between 1990 and 1991 and has consistently represented approximately 1% of TB cases since 1991 (Figure 12).

The vast majority (67.6%) of resistant TB cases reported between 1990 and 1998 resided in the 6 health unit areas which now make up the amalgamated City of Toronto (Figure 13).

Of all drug-resistant cases reported between 1990 and 1998, 91.2% were foreign-born, 6.0% were Canadian-born, and 0.52% were of aboriginal origin. Thirteen drug-resistant cases had origin listed as either "missing" or "unspecified".

The countries of origin for cases with resistance to at least one antituberculous drug are presented in Figure 14. The highest number of drug-resistant cases were of Vietnamese origin, however, the highest proportion of cases with drug resistance were from Kenya.

Discussion

Over the past decade, the epidemiology of TB in Ontario has not changed dramatically. Incidence rates continue to hover at approximately 7 cases per 100,000 population, with approximately 80% of cases occurring in foreign-born individuals. Recent arrival in Canada continues to stand out as a risk factor for active disease in young foreign-born individuals. Consequently TB remains a key issue for selected health units in which there are high concentrations of new immigrants or refugees. These health units also tend to have higher concentrations of other risk groups including the homeless and those with HIV infection. TB also remains a key issue for health units with a high proportion of aboriginal people.

Source

Monali Varia, MHSc, BSc
Project Epidemiologist
Vaccine Preventable Diseases and Tuberculosis Control
Disease Control Service
Public Health Branch

Jill Sciberras, RN, BNSc, MHSc
Nurse Epidemiologist
Vaccine Preventable Diseases and Tuberculosis Control
Disease Control Service
Public Health Branch

Contact

Barbara Kawa, MD, DPH
Senior Medical Consultant
Vaccine Preventable Diseases and Tuberculosis Control
Disease Control Service
Public Health Branch

References:

1. Walkenstein MD. "Tuberculosis". . Accessed January 7, 2000.
2. Pilheu J.A. Tuberculosis 2000: problems and solutions. Int J Tuberc Lung Dis. 2(9):696-703.
3. Long R, Njoo H, Hershfield E. Tuberculosis: 3. Epidemiology of the disease in Canada. CMAJ. 1999; 160:1185-90.
4. Fanning A. Tuberculosis: 1. Introduction. CMAJ. 1999; 160:837-9.
5. Dye C, Scheele S, Dolin P, Pathania V, Raviglione M. Global Burden of Tuberculosis. JAMA. 1999; 282:677-686.
6. Naus M. Epidemiology of Tuberculosis in Ontario in 1989. Public Health and Epidemiology Report Ontario. 1990; 1(7):102-107.
7. Troy CJ. Epidemiology of Tuberculosis in Ontario, 1989-1992. Public Health and Epidemiology Report Ontario. 1993; 5(3):63-71.
8. Kerbel D. Epidemiology of Tuberculosis in Ontario, 1995. Public Health & Epidemiology Report Ontario. 1997; 8(4):81-93.
9. Kerbel D. (unpublished data, Ontario Ministry of Health)
10. Correspondence: Howard Ngoo, Laboratory Centre for Disease Control. Dec 23, 1999.
11. Simone P, and Dooley S. (1994). "Multidrug-Resistant Tuberculosis". Centres for Disease Control and Prevention. . Accessed January 7, 2000.

Disease Control Service Comment

It has been estimated that as many as 50 million people around the world are infected with tuberculosis strains that are resistant to at least one antituberculous drug (10). Groups at risk for drug resistance include persons who have been treated with antituberculous drugs in the past, contacts of persons who are known to have drug-resistant TB, and foreign-born individuals from areas with a high prevalence of drug-resistant TB. This includes Latin America, Asia and Africa (11).

In 1998, MDR-TB cases represented 1.1% of all reported TB cases in Ontario. Increases in drug resistance are in part due to the high number of persons arriving in Canada from countries with a high prevalence of drug resistance, but are also occurring as a result of inappropriate or interrupted treatment here in Canada. Further analysis of this issue is currently underway.

Physicians need to be made aware of the increasing concern around drug resistant TB. Patients with TB should be started on 4-drug therapy; sensitivity results must be followed-up and acted on appropriately if resistance is present. Although the number of cases in which resistance status was reported as either unspecified or missing in RDIS has decreased since 1992, health units need to continually monitor their cases to ensure that sensitivity results are being reviewed and effective treatment is being ordered.

MDR-TB is associated with high mortality rates and extremely high costs due to prolonged hospitalization and use of second line drugs. In Ontario, treatment is available and mortality due to this infection can usually be prevented if the case, and contacts, are detected in a timely manner and followed-up appropriately.

In order to decrease and, ultimately, eliminate TB from Ontario, surveillance, education of health care professionals, high risk groups and the general public, and effective prevention and control programs are necessary.

Awareness is the first step to improving not only education but also the effectiveness of surveillance activities. Physicians need to consider TB in their differential diagnosis not only with young foreign-born individuals but also in the elderly population who may develop reactivated disease, especially if their immune system is compromised. In addition they need to know how to proceed with confirmatory testing (culture and sensitivity), and be aware of the importance of doing so. The importance of timely reporting of cases to the health unit should also be emphasized. These activities will prevent delays in the diagnosis of TB and contact follow-up.

Health care facilities and other occupational settings where staff are at increased risk for TB exposure need to be aware of the value of baseline and ongoing skin testing surveillance programs.

Case management of active disease may be improved by reducing diagnostic delays through awareness, education, and ensuring adherence to treatment by directly observed therapy (DOT). Once a case of active disease has been detected, effective control measures must be implemented as soon as possible. For the most part TB is a treatable and curable disease. Early diagnosis is one of the most important factors in reducing morbidity and preventing the spread of disease to exposed contacts. The second part of this equation is compliance with an appropriate antituberculous drug regimen. Poor adherence to treatment is one of the major barriers to the elimination of TB and the single most important factor in the development of secondary drug resistance.

Directly observed therapy (DOT) remains one of the most effective measures to improve compliance with drug therapy. The success of DOT as an intervention is dependent not only on the individual DOT worker's ability to sustain a relationship with the client for the duration of their therapy but also on the availability of resources to implement this intervention.

The identification and prophylaxis of individuals with latent infection is much more complex. Infected persons may be identified through contact follow-up of a case, routine surveillance of high risk groups, or medical surveillance of immigrants and refugees. These individuals need to be informed of the importance, implications and process for surveillance of latent infection in an effective, culturally appropriate manner. This activity is usually left up to the health unit staff, who may represent the first contact with the health care system for some of these individuals. Activities are ongoing at all levels of government to improve, specifically, the immigration medical surveillance system.

Tuberculosis is a disease which has re-emerged as a significant cause of mortality and morbidity in developed, as well as developing, countries over the last decade. In Ontario, the disease has not been eliminated, but has remained stable in recent years at a rate of approximately 7 cases per 100,000 population. While the rate of TB has not increased lately, the proportion of cases which are resistant to one or more antituberculous drugs has increased over the past decade, highlighting the need for continued vigilance in the early identification of cases and in ensuring prompt, appropriate and complete treatment of cases.

Resource allocation to deal with this issue is in part dependent on awareness of the problem and knowledge of the actual and potential impact of the problem. This can be achieved through ongoing feedback of surveillance and epidemiological data to decision makers to facilitate comparison with competing priorities. This report is designed to provide feedback based on provincial data thereby facilitating decision-making at both the municipal and provincial level.

Benchmarking Pilot Project: Testing the Concept in Public Health

Introduction
The purpose of this article is to provide a brief overview of the achievements of the three initial benchmarking pilot projects. The discussion will include a brief summary of the pilot process and some key results of these pilots. One element of the Ontario Public Health Benchmarking Partnership's pilot projects was designed to test the concept of benchmarking in public health practice. The Ontario Public Health Benchmarking Partnership is a joint initiative of the Association of Local Public Health Agencies (alPHa), the Ontario Council on Community Health Accreditation (OCCHA), the Public Health Branch, Ontario Ministry of Health and Long-Term Care, and the Public Health Research, Education and Development (PHRED) Program. A basic outline for the pilot was proposed in the Stewart and Sales (1998) report1. Using the recommendations of the report, the Benchmarking Steering Committee developed a workplan to guide the implementation of the pilot projects in three areas of health protection. As the pilot projects progressed, the timelines and other aspects of the original workplan were adjusted to conform to the realities of managing the pilot.

Selection of Topics
An important consideration in designing the pilot projects was the need for early success. The Benchmarking Steering Committee chose three specific areas that had a reasonable chance of showing concrete results within a short time period (i.e., Food Premises Inspection, Immunization Record Process, and STD Contact Tracing). Another factor in the selection was that extensive background material about practices in these areas was available (e.g. CPHA Journal 1994 July/August Supplement). As well, some health units had begun work on performance measures in these areas.

Selection of Pilot Sites
Participation in the pilot projects required the designation of one or two participants from each health unit who would take the lead in benchmarking. There was a total of nine participating health units. Eastern Ontario, Elgin-St.Thomas, Timiskaming, Waterloo, and York Region were selected as exemplifying a good cross-section of Ontario's health units. In addition, four PHRED programs were pilot sites: Middlesex-London, Ottawa-Carleton, Sudbury and Toronto. Toronto was in the midst of amalgamation, however, since the process was not yet complete, they participated as six health units in the pilot. Consequently, a variation in the number of pilot health units represented in the graphs is evident. Furthermore, not all of the health units participated in each of the three pilot projects.

Selection of Comparators
The pilot projects demonstrated the importance of choice of comparators that could vary from issue to issue. One of the challenges of benchmarking in public health will be distinguishing differences in performance and practice from differences in context. Therefore, the interpretation of results must take into consideration the variations in context. Comparing health units across the province requires an understanding of the realities faced by each health unit. Demographics, geographic dispersion, and political realities, such as the amalgamation of metro Toronto, are examples of factors that can influence results.

Data Collection and Analysis
Data collection consisted of the development of program logic models, performance measurements for program components, and survey questionnaires. Each approach will be briefly described in the following section.

Program Logic Model
A necessary step in the benchmarking pilots was the preparation of a program logic model (using the Program Evaluation Tool Kit, developed by Porteous, Sheldrick, and Stewart, 19972) and the identification of program components and activities. Program components need to be few (i.e., 2-4) and have identifiable performance indicators. For instance, Food Premises Inspection activity is grouped into three areas: (1) evaluation of the food remises [surveillance and monitoring], (2) management consultation and education, and (3) enforcement. Immunization Record Process activity is grouped into two main areas: (1) obtaining information from new school enterers and transfers, and (2) assessing and reviewing existing records. Partner Notification for Chlamydia is grouped under two main headings: (1) contact with index case and (2) contact with partners.

Performance Measures
Performance measures for each program needed to be developed before survey tools could be prepared. The pilot health units developed performance measures for each component and then prepared questionnaires. This process required many reviews and revisions by benchmarking pilots (e.g., defining issues, collecting data in a clear and unambiguous way). Tables 1, 2, and 3 show the performance measurements identified for the program components of each pilot project: Food Premises Inspection, Immunization Record Process, and Partner Notification for Chlamydia, respectively.

Survey Tools
The design of the surveys was determined by the need to collect both the quantitative data for measures of effectiveness and efficiency, and the qualitative data that would allow the comparison of practices in each area. Another important factor in the design of survey tools is that data should be available without much effort. Also, clear and consistent definitions ensure questions allow for development of indicators. The complete survey was comprised of five components: a Health Unit Profile generic questionnaire that allowed comparison of health units on demographic characteristics, individual questionnaires for Food Premises Inspection and Immunization Record Process, and two questionnaires for Partner Notification for Chlamydia.

Benchmarking Pilot Project Results
The following section will present a selection of results from each pilot project to demonstrate the variability in the pilot results. The health units' identities in the following figures have been altered to protect the identification of the participating health units.

Food Premise Inspection
Figures 1and 2 depict the result for the effectiveness indicator related to surveillance and monitoring. It is important to note the variability between health units in each instance. Figure 1, in particular, shows one health unit with markedly different results than the other health units. A probable reason for the difference shown is that, for the past two years, this health unit has been tracking its critical food infractions in an attempt to standardize how Public Health Inspectors record infractions. Similar variation in results was seen in other key measures, for example, re-inspections by type of risk premise (not shown).

In Figure 3, the use of Medical Officer of Health (MOH)/Public Health Inspection (PHI) orders, Provincial Offences Acts (POAs), and the number of charges laid for repeat infractions of critical control points (CCP) are shown per 1000 food premises. Three health units use more enforcement interventions than do the other health units.

At the completion of the pilot there was agreement among the participating health units that the best overall effectiveness measure for Food Premise Inspection is the number of critical food infractions/10 routine inspections. The incidence of food-borne infections was not recommended as an effectiveness measure because most infections are not related to food premises. There was some discrepancies among health units in the way the number of critical infractions was recorded, however, standardization of this can be readily achieved.

Immunization Record Process.
The overall effectiveness indicator selected was the difference between the percentage of incomplete records in October and June as shown in Figure 4. Health units that had a low number of incomplete records in October had very little change in their incomplete rate in June. More variability in results was seen with health units that had a higher number of incomplete records in October. This indicator could be improved if enterers could be identified in IRIS in October and the status of incompletes tracked through the school year.

The result of the efficiency indicator pertaining to staffing FTEs/10,000 Immunization Records is shown in Figure 5. Staffing ratios also resulted in significant variability among pilot health units. A denominator, which better reflects workload, would be the sum of enterers and incomplete in October. This is not currently available in Immunization Record Information System (IRIS).

Figure 6 shows the efficiency indicator which examines the difference between the number of incomplete records in October and June with respect to the total number of records reviewed. What this graph does not show is the variations in the number of records health units reviewed. In the graph, it appears that health unit E was the most efficient, however in actual fact, they reviewed only 10% of their records in comparison to other health units that reviewed 100%.

Partner Notification for Chlamydia
This pilot was the most complex of the three pilots because the practices of health units in this area differ dramatically, with some health units doing most contact tracing while others leave this role primarily to clients and other health practitioners. This pilot decided to focus on comparing the different approaches to health unit follow-up. Since data to support performance measurement in partner notification is currently not routinely collected by health units, data forms were developed especially to follow individual cases and contacts for the pilot. Elapsed time indicators, instead of the measurement of staff time, were chosen as the measure of efficiency. Most health units have the same staff do partner follow-up for all STD's and have difficulty separating out time spent on any particular disease.

The first effectiveness indicator for activities related to Partner Notification and Follow-up is the proportion of named partners followed by the health unit who are successfully reached. The denominator for this indicator consists of all partners in the health unit's jurisdiction plus those for which the jurisdiction was unknown. The "Unknown Jurisdiction" partners were included because most health units attempted some follow-up. The numerator consists of the number of partners who are successfully reached. As shown in Figure 7, most health units attained 80% or better.

Figure 8 shows the average time from naming partner to initiation of contact attempts for pilot sites. There is considerable diversity of practice with some health units initiating follow-up the same day and others taking more than a week to initiate follow-up. Note that the number of cases and the proportion of cases followed up by the health unit should be considered in interpreting these comparisons. Despite the data problems and the dramatically different practices among health units, Figures 7 and 8 show the types of indicator which can be used to monitor performance.

Lessons Learned
Benchmarking in public health practice is new and uncharted territory. The pilot projects have proven that despite the complexity of public health practice and the constantly changing environment, it is possible to apply this methodology to our service delivery in public health. The pilot projects have made a good beginning in testing the concept, and in providing valuable lessons for future projects. As with any new venture, it is not possible to foresee all the challenges, therefore we learned to expect the unexpected. Summarized below are some of the lessons of the pilot project component of the benchmarking project:

• Benchmarking projects that use available sources of data are easier to implement than those that require the collection of primary data.
• Development of performance measures takes time.
• Standardization of data collection is crucial to the benchmarking process, otherwise, inter health unit comparisons cannot be made. The pilot process demonstrated that differences currently exist even for activities that are well established in public health practice. The process of agreeing on indicators and making comparisons is essential to achieve standardization.
• The reliability of the data is enhanced if clear definitions, collection procedures and guidelines are provided. In some instances province-wide training will be required. This would cut down on the variation in data collection due to different interpretations of the questions in a survey.
• Surveys are one method of collecting data for a benchmarking project, but other methods such as site visits may offer more in-depth information. A combination of a short survey followed by an on-site visit may be a more satisfactory means of identifying best practices than a survey alone.
• The determination of best practices is challenging because of the complexity of public health practice. It is also context dependent and needs to be empirically based, blending literature and evidence with actual practice. Synergistic collaboration between PHRED projects that approach the identification of best practices in different ways has the potential to enhance public health practice.
• The project has demonstrated that analyzing programs from the perspective of Performance Measurement does generate very useful new information about program performance. Identifying best practice is not simple. Even with reliable, valid and generally accepted indicators for comparison, best practice is highly context dependent. Health units must choose comparators carefully and be very selective in appropriating practices for their particular context.

The Public Health staff that was involved in the three pilot projects showed enthusiasm towards benchmarking and commitment to seeing the projects through to completion despite the uncertainty experienced when venturing into uncharted territory and the demands on their time and resources.

Benchmarking Pilot Outcomes

Food Premises Inspection
In this project we have identified a key outcome indicator of critical infractions per ten food premise inspected. The findings of this project will be presented to the Association of Supervisors of Public Health Inspectors of Ontario (ASPHIO) and the Ministry by the end of 1999. Ideally standardized data for benchmarking will begin to be collected across the province in 2000.

Immunization
This pilot has been able to generate a very useful benchmarking package which consists of a number of indicators which can readily be generated from IRIS as well as an inventory of practices which could influence those indicators. The immunization working group have identified some minor modifications to IRIS which they feel could enable the generation of greatly improved indicators for both the Ministry of Health and Long-Term Care and program management. We again hope to initiate province-wide data collection for the 2000-2001 school year.

Chlamydia Contact Tracing
This is the pilot where it is less clear how we move forward. There are enormous differences among health units in what proportion of cases they follow up themselves, and the degree to which they promote notification by the client. Because in most health units an information system is not in place, for benchmarking to move forward, health units would have to adopt a uniform recording system. Given the relatively low priority accorded to this activity in many health units, it is unclear that such an initiative would be well utilized. The issue in this area would seem to be the great diversity of practice across the province. This would need to be addressed at the level of all STD contact tracing. We are discussing the possibility of a provincial meeting to look at the policy issues that arise out of this pilot.

Future Initiatives
The Steering Committee of the Ontario Public Health Benchmarking Partnership recognizes the need to begin to address the challenge of benchmarking within the health promotion and disease prevention mandatory programs, such as Chronic Disease Prevention and Family Health. They have identified two new pilot projects based on the priorities identified by health units in the benchmarking survey, as well as, the practical lessons learned in the first three pilots. The first new pilot is heart health coalitions in which almost all health units are currently involved. The second is breastfeeding, Requirement 4 (a-e) of Child Health Mandatory Program. Several health units have volunteered to participate in the new pilots and are representative of the diversity across Ontario. Both projects, while challenging, will be of great value to public health units and should provide lessons and templates which can be readily generalized to other parts of the Mandatory Programs.

Acknowledgements
The success of the pilot projects would not have been possible without the ongoing support, committment and contribution by the following: Eastern Ontario Health Unit (Doreen Blais, Gisele Martin, Suzanne Ross), Elgin-St.Thomas Health Unit (Laura McLachlin), Region of Hamilton-Wentworth Social Services and Public Health Services Department (Dan McInnis), Middlesex-London Health Unit (Charlene Beynon), Region of Ottawa-Carleton Health Department (Mary McNamara), Sudbury and District Health Unit (Cheryl Dovigi, Louise Picard, Ido Vettoretti), Timiskaming Health Unit (Lynn Landriault, Esther Millar), Toronto Public Health ( John Dwyer, Marjolyn Pritchard, Pam Scharfe), Regional Municipality of Waterloo, Community Health Department (Bob Hart, Karen Verhoeve), York Region Health Services Department (Karim Kurji, Marie Muir), alPHa, and OCCHA. within the Ontario Public Health Benchmarking Partrnership

Contacts and Source
Dr. Geoff Dunkley, Deputy Medical Officer of Health
Verna Wilson, Benchmarking Pilot Project Manager
Monique Stewart, Program Planning & Evaluation Officer

PHRED Program
Region of Ottawa-Carleton Health Department

References

1. Stewart P, Sales P. A benchmarking process for public health programs in Ontario: A development plan. Ottawa: Paula J. Stewart & Associates Community Health Consulting, 1998.
2. Porteous N, Sheldrick B, and Stewart P. Program evaluation tool kit: A blueprint for public health management. Ottawa: PHRED Program, Ottawa-Carleton Health Department, 1997.
3. Balm, G. J. Benchmarking: A practitioner's guide for becoming and staying best of the best. Schaumburg, Illinois: QPMA Press, 1992; 16.
4. Sales, P. and Stewart, P. Benchmarking Tool Kit: A blueprint for public health practice. Middlesex-London Health Unit, 1998; 88-89.