Validation and enhancement of Australian Aboriginal and Torres Strait Islander psychiatric hospitalisation statistics through an Indigenous Mental Health Worker register

full article:

Introduction

Australia has two Indigenous groups, namely Aboriginal and Torres Strait Islander people, within and between which there is much diversity^1,2. An official and accepted working definition was developed in 1980¹:

An Aboriginal or Torres Strait Islander is a person of Aboriginal or Torres Strait Islander descent who identifies as an Aboriginal or Torres Strait Islander and is accepted as such by the community in which he (she) lives.

There is much disparity in most dimensions of health, including mental health, among Indigenous compared with non-Indigenous Australians. Problems in the reliability of data continue to impact on quantification of health gaps. Many Australian Government reports released in the mid-1990s identified severe problems in accurately reporting Aboriginal and Torres Strait Islander status on most record systems and urged improvement^3-5. This stimulated efforts to enhance ascertainment of Indigenous status on births, deaths, hospital records and social services data^6-8. Indigenous liaison officers and health workers have played a role in improving identification of Indigenous status, as have policy and procedural changes, incentives and feedback. Examining data collected in 2003/2004, the Australian Institute of Health and Welfare confirmed levels of Indigenous identification of over 80% in public hospital data in Western Australia, Northern Territory and South Australia, but lower levels in other jurisdictions⁸. Queensland Health estimated that 20% of Indigenous admissions in Queensland were not identified, based on the finding that 74.3% were correctly identified in two metropolitan hospitals, combined with an assumption that identification was likely to be higher in remote areas⁸.

The most recent national Indigenous identification study in Australia was conducted in 2007 and 2008⁹. While data on the Australian Capital Territory was based on data linkage, identification in the other states and territory was estimated through comparison of identification data collected through interviews with a selected sample of admitted patients and information reported on their records. This study estimated that 86% of Indigenous patients were correctly identified in the selected Queensland hospitals and wards, indicating that Queensland data was 'acceptable' for analysis. However the report also made numerous priority recommendations for continual monitoring and improvement of Indigenous status identification⁹.

National and jurisdictional statistics are essential, however they do not provide information at the individual hospital level or in different wards within hospitals. Indeed, considerable variation in correct identification is reported between hospitals, levels of remoteness and time points^9,10.

Studies within hospital settings have explored the reasons for, and the proportion of, under-recording of Indigenous status. In 1997 Brough et al found that less than 40% of patients in two metropolitan hospital wards (Brisbane, Queensland) were asked about their ethnicity on admission¹¹. The main barrier identified was reluctance among hospital clerks to ask about Indigenous status, and completion was often based on physical appearance alone. Jackson Pulver et al¹² observed an even lower rate of identification (10 out of 29 Aboriginal women identified) in the charts of an urban obstetric hospital ward in Sydney in 1999 despite a high acceptance of being asked about Indigenous status among both Aboriginal and non-Aboriginal women (≥95%). Comparing a births register with hospital records, Comino et al found between 70% and 80% consistency in identification between the two sources¹³.

Similar problems with under-identification of Indigenous status also occur in the USA^14,15, Canada^14,16 and New Zealand^14,17 and among other minority ethnic groups^18-20. Hence, underestimation of morbidity and mortality occurs with potential under-resourcing of services relative to needs. Inaccurate data also prevents clear recognition of the size of health and life-expectancy gaps between Indigenous and non-Indigenous residents of those countries, and hampers monitoring efforts to close these gaps^14,21,22.

Knowledge about consumers' ethnic and Indigenous status is essential at the clinical and management level because cultural background exerts a strong influence on many aspects of health^3,4,8,23-26. Understandings of, and influences on mental health assessment, mental illness and social avenues of support for recovery and maintaining wellness are highly culture-based. Experienced clinicians consider knowing Indigenous status fundamental to guiding communication and care^24-26.

Since 1999, the Cairns Base Hospital in the Far North Queensland region, Australia has used the Hospital Based Corporate Information System (HBCIS) database that captures demographic and service data on all hospital inpatients, including in-patients admitted to the 30 bed mental health unit. The Indigenous Mental Health Worker (IMHW) at the hospital has maintained an independent register of Indigenous consumers in the unit since 2003. The Register allows documentation of her work in accessing and supporting Indigenous consumers and families.

The purpose of this study was to estimate the proportion of Aboriginal and Torres Strait Islander admissions to the Cairns Base Hospital Mental Health Unit that were identified correctly in the hospital information system. This was done by comparing cases in the hospital information system with those identified in the IMHW Register (Queensland Health, unpubl. data, 2003-2006). An enhanced estimation of hospital usage by Indigenous consumers based on triangulation of these datasets is presented.

Methods

Data sources

The mental health information systems used throughout Queensland includes HBCIS and the Client Events Service Application (CESA), which capture data from in-patient and community services, respectively.

Hospital Based Corporate Information System data: The Clinical Benchmarking Unit, Cairns Base Hospital, analyses hospital separation data on a monthly basis from the HBCIS database. The HBCIS collects information about inpatients and their care, for example variables such as age, sex, Indigenous status, residence, diagnosis, admission and discharge date and length of stay. Indigenous status is coded as Aboriginal, Torres Strait Islander, both Aboriginal and Torres Strait Islander, not Aboriginal or Torres Strait Islander, or not specified.

Indigenous Mental Health Worker Register: The IMHW (currently an Advanced Health Worker [Mental Health]) is a highly respected Aboriginal person with 10 years of local experience in Indigenous mental health, including 6 years as the sole Indigenous Worker at the mental health unit. This provides extensive knowledge of individuals and families involved in mental health services. Ascertainment of Indigenous status is done via a routine networking process of the IMHW and includes continual cross-checking with Indigenous people in the unit about others they may know. Any informal notifications of non-Indigenous patients are corrected before patients are added to the Register. The Register includes name, age, identification of Aboriginal, Torres Strait Islander or dual heritage, residence, previous discharge date and reason for admission (Queensland Health, unpubl. data, 2003-2006).

Community Event Service Application: The CESA captures data on mental health care services provided at the community level and collects demographic information, including identification of Aboriginal and Torres Strait Islander status. The database is independent of HBCIS and assisted cross-checking where Indigenous status was mismatched or unspecified.

Validation and cleaning process

The data were exported from HBCIS into a Transition II database (Crystal), then downloaded to an MS Excel spreadsheet in November 2005 (for 2004/2005) and November 2006 (for 2005/2006). The spreadsheet was provided to the researchers who imported the data into SPSS v14 (www.spss.com) for analysis. Admissions of Far North Queensland health service area residents to the 30 bed mental health unit who were discharged between 1 July and 30 June were included in analysis. The total population of the area was estimated to be 229 289, of which an estimated 30 228 (13.2%) identified as Aboriginal and/or Torres Strait Islander. At the time the area consisted of five health districts which included a small urban area (Cairns District) with Indigenous people comprising 9.5% of its population, two rural districts with slightly lower representation (Innisfail and the Tablelands with 7.7% and 8.4% Indigenous, respectively), and the two remote regions Cape York (48.4% Indigenous) and the Torres Strait Islands (73.8%)²⁷.

Validation of Indigenous status identification in HBCIS involved comparisons of individual admission numbers captured by the IMHW Register and the extracted HBCIS data and confirmed using admission date. Mismatches and occasions where Indigenous status was unspecified were identified and manually cross-checked using CESA, the IMHW Register and the IMHW's knowledge. In 2004/2005, this was done retrospectively by full entry of IMHW data into a computer database for cross-checking both data sources for completeness. For 2005/2006, the process was streamlined to validate only the HBCIS data, whereby the IMHW received a monthly list of admissions of Indigenous patients identified in the HBCIS system, cross-checked the list with the Register and maintained a list of missed and incorrect identifications. Corrected estimates of Indigenous and non-Indigenous admissions on HBCIS based on this process were compared with uncorrected estimates.

Ethics approval

This research was approved by the Cairns Base Hospital and the University of Queensland Human Ethics Committee (#32:08:391) and performed in accordance with ethical standards laid down in the 1964 Declaration of Helsinki. Data were de-identified prior to receipt by the researchers.

Results

Assessment in 2004/2005

Between 1 July 2004 and 30 June 2005, HBCIS reported 162 discharged admissions of Aboriginal and Torres Strait Islander people at the Cairns Base Hospital Mental Health Unit. The IMHW Register included 159 discharged admissions over the same period. Details are provided (Table 1).

Table 1: Initial estimations of mental health unit usage by Far North Queensland health service
area residents in 2004/2005 and 2005/2006, before and after cross-checking

Although the estimated figures for IMHW and HBCIS data are almost identical, cross-checking of individual hospital numbers revealed matching on only 130 separations. Review of the 32 cases reported as Indigenous in HBCIS but missing from the IMHW Register revealed 25 who were later confirmed Indigenous and seven confirmed as non-Indigenous. Seven of the 24 occasions where status was not specified in HBCIS were confirmed to be Indigenous, and one of these was not identified in the IMHW Register.

Thus, adding the 26 missed cases to the 159 captured in the IMHW Register yielded an adjusted estimate of 185, and an estimated coverage of 85.9% of all Indigenous separations by the IMHW Register. Conversely, the HBCIS dataset in 2004/2005 captured 155 Indigenous admissions accurately out of 185 (83.8%) with seven non-Indigenous admissions incorrectly identified as Indigenous (of a total of 653 non-Indigenous admissions; 1.1%). Among the 30 missed cases, seven were among the 24 with an unspecified Indigenous status (29.1% Indigenous), five were missing from the database due to alternate admission pathways (not psychiatry), five were identified as Indigenous but incorrectly marked as resident outside of the regional health service area, and 13 were identified as non-Indigenous. Corrections led to a 14.2% increase in the number of counted Indigenous admissions and a 10% increase in occupied bed days.

Assessment in 2005/2006

The original download from the HBCIS database in 2005/2006 recorded 802 admissions from residents of Cairns District, of which 149 were identified as Indigenous and 27 were unspecified. Cross-checking with the IMHW Register identified 24 cases missing in the HBCIS download (including 6 due to admission to other wards and three with no fixed address) who were found to reside in the health service area: nine Indigenous patients were identified as non-Indigenous and six of the 27 (22.2%) had been designated 'unspecified'. Three of 24 admissions identified as Indigenous on HBCIS but not on the IMHW Register were confirmed non-Indigenous (3 of a total of 640 non-Indigenous admissions; 0.5%). Thus HBCIS correctly captured 85.9% of the total 170 Indigenous admissions in 2005/2006, and 79% of total bed days occupied by Indigenous patients (Table 1).

Examining unspecified entries and correcting status in previous years

Over the first years analysed (1999/2000 to 2003/2004), 174 separations had unspecified Indigenous status which involved 120 consumers. Examination revealed that among the 174 unspecified cases:

• 132 patients (75.9%) were identified as non-Indigenous through CESA; the IMHW had no knowledge that suggested any of these were Indigenous
• 30 patients (17.2%) were identified as Indigenous (28 Aboriginal and 2 Torres Strait Islander patients)
• 12 patients (6.9%) could not be identified through CESA, and the IMHW had no knowledge of their Indigenous status so they were included in the non-Indigenous group for analysis.

In a further 15 admissions, patients who were reported as non-Indigenous in earlier years were identified as Indigenous in later records and confirmed by the IMHW. The original and adjusted estimates of admissions and occupied bed days of Indigenous patients across all 7 years are provided (Table 2), including these additional episodes.

Table 2: Initial and adjusted estimates of Indigenous separations over entire 7 year
period before and after correcting for unspecified entries and later identifications

Overall estimated accuracy of identification in HBCIS

Based on 2 years of cross-validation with the IMHW Register, 301 Indigenous admissions were identified on the HBCIS database of an enhanced total of 355, with an overall correct identification rate of 84.8%. This estimate is similar to that achieved by the IMHW Register (85.9%). Corrections led to a 15.3% increase in the estimated occupied bed days for Indigenous admissions.

Of the 54 Indigenous admissions (15.2%) missed, identification as non-Indigenous occurred in 22 cases (6.2%) and as unspecified status in 13 (3.7%) cases. A further 19 (5.4%) were missed by incorrect or incomplete residence designation and admission to other wards.

Thirteen (25.4%) of the 51 unspecified admissions in the two study years, and 43 (19.1%) of the total 225 unspecified across the 7 years, were found to be Indigenous. Ten of the total 1293 non-Indigenous admissions (0.8%) had been incorrectly identified as Indigenous.

Discussion

This extensive validation exercise revealed that Indigenous hospitalisation data within the information system at the Cairns Base Hospital Mental Health Unit includes approximately 85% of the total admissions. This is similar to what was reported for Queensland hospitals overall^8-11. Importantly, the occupied bed days of Indigenous admissions which reflect resource allocation were underestimated by more than 500 days (21%) in 2005/2006.

The IMHW Register was considered the gold standard for quality of identification but both datasets missed some cases, indicating that cross-checking at the individual admission level using information from both sources would be optimal. For example, one finding was the need to consider admissions to other hospital units with part of their stay in the mental health unit - this revealed an additional 11 patients in the two validated years. Cases not included in the IMHW Register were mainly of short duration over weekends or when the IMHW was on leave.

Previous research has used other assessment methods to examine accuracy of identification, such as face-to-face interviews following admission, comparisons with population or survey data, and examining recording on multiple admissions; these generally involved an external process⁸. In contrast, the IMHW Register method is continuous, involves routine activities, and can feed directly into processes for improving identification and communication among staff¹³. An important benefit of this study has been the formal demonstration of the IMHW's contribution to the data enhancement process. Enhanced communication and provision of monthly lists of Indigenous patients from the IMHW Register to the HBCIS database managers since 2004/2005 has probably facilitated accuracy. This contributes to the implementation of recommendations of the Queensland Aboriginal and Torres Strait Islander Mental Health Policy Statement concerning the management of Indigenous mental health information²³.

An example is provided (Table 3) of data estimates that are analogous to procedures requiring Indigenous identification at every time point (before adjustment) versus applying the 'ever Indigenous' rule, that is adjusting all previous and subsequent records for a person when identification has occurred at least once^10,14,28. Major administrative datasets in New Zealand apply the 'ever Maori' rule as a preferred method of classification^14,29; however, the Australian Bureau of Statistics does not apply this rule. Research by Madden et al in 2012 showed that while some overestimation may occur due to amplifying mistakes, the 'ever Indigenous' method provides a more accurate and credible estimate of numbers of Indigenous deaths²⁸. In the present study, the experience of the Indigenous Mental Health Worker, who has worked side by side with Aboriginal and Torres Strait Islander patients and their families for over a decade, strongly confirms that the frequency of this group changing their Indigenous status was much lower than the incidence of consumers not being asked about their Indigenous status at the commencement of or during an admission.

Cairns Base Hospital Mental Health Unit, like many in Australia, has experienced increased bed occupancy rates in recent years which, as is shown (Table 2), doubled for Indigenous consumers between 2002/2003 and 2004/2005. This has led to escalating costs and pressure for beds at tertiary level. Analysis of the corrected HBCIS data has assisted in understanding the cause of this increase and facilitated targeted promotion and prevention efforts³⁰.

Full and accurate identification of Indigenous groups within health datasets in other developed countries continues to confound efforts towards improved data quality to guide policy and practice¹⁴. For example, the USA has supported local efforts for improvement through tribal government involvement and forming tribal epidemiological consortiums, but reported rates of identifying First Nations consumers fluctuates widely among states and regions¹⁴. In Canada, a lack of identifiers in many local, territory and provincial level data collection systems prevent accurate measurement and monitoring of the health of Meti, Inuit and First Nations peoples¹⁴. Australia and New Zealand, however, have made substantial progress in Indigenous identification through clear national commitment, but even here, more effort is needed to replace estimations and projections with full and accurate counts¹⁴.

Conclusions

Consistent Indigenous identification within administrative data collections is crucial in efforts to ensure valid and reliable health-service usage data. This study has demonstrated the value of an IMHW Register in monitoring accuracy of such reporting. Under-recording was observed with up to one-quarter of admissions with unspecified status found to be Indigenous, and a low frequency of false identification of non-Indigenous people was confirmed. A feedback loop from the IMHW back into the information system to correct errors could potentially enable 100% accuracy.

Acknowledgements

This study was part of the Australian Integrated Mental Health Initiative (AIMhi) funded by an National Health and Medical Research Council Partnership Grant in Mental Health (#219327), the Cairns and Hinterlands District Health Service of Queensland Health and a Priority Driven Research Grant of the Australian Health Ministers Advisory Council (AHMAC).

References

1. Department of Aboriginal Affairs. Report on a review of the administration of the working definition of Aboriginal and Torres Strait Islanders. Canberra, ACT: Commonwealth of Australia, 1981.

2. National Health and Medical Research Council. The NHMRC Road Map: A strategic framework for improving Aboriginal and Torres Strait Islander health through research. Canberra, ACT: Commonwealth of Australia, 2002.

3. Australian Institute of Health and Welfare. Indigenous identification in administrative data collections: best practice and quality assurance. Canberra, ACT: Commonwealth of Australia, 1996.

4. Australian Institute of Health and Welfare. The Aboriginal and Torres Strait Islander Health Information Plan... This time, let's make it happen. Canberra, ACT: Commonwealth of Australia, 1997.

5. Thomson N. An historical overview of the development of Indigenous health statistics. Canberra, ACT: Australian Institute of Health and Welfare, Commonwealth of Australia, 1996.

6. Epidemiology Services Unit, Queensland Health. Using health related data collections to monitor the health status of Aboriginal and Torres Strait Islander peoples in Queensland: current status. Technical Paper. Brisbane, QLD: Health Information Centre, Queensland Health, 2003.

7. Australian Institute of Health and Welfare. Data quality of Aboriginal and Torres Strait Islander identification: Seven community services data collections. Canberra, ACT: Commonwealth of Australia, 2004.

8. Australian Institute of Health and Welfare. Improving the quality of Indigenous identification in hospital separations data. Canberra, ACT: Commonwealth of Australia, 2005.

9. Australian Institute of Health and Welfare. Indigenous identification in hospital seperations data - quality report. Canberra, ACT: Commonwealth of Australia, 2010.

10. Kennedy B, Howell S, Breckell C. Indigenous identification in administrative data collections and the implications for reporting Indigenous health status. Technical Report #3. Brisbane, QLD: Health Statistics Centre, Queensland Health, 2009.

11. Brough M, Shannon C, Haswell-Elkins M. To be or not to be: recording Aboriginal identity on hospital records. Pacific Health Dialogue 2001; 8: 47-53.

12. Jackson Pulver LR, Bush A, Ward JE. An analysis to determine the number of Aboriginal and Torres Strait Islander women using an urban obstetric hospital. Australian Health Review 2001; 26: 19-25.

13. Comino EJ, Titmuss A, Harris E, Craig P, Pulver LJ. Identification of Aboriginal infants at an urban hospital. Journal of Paediatrics and Child Health 2007; 43: 623-626.

14. Jackson Pulver L, Haswell MR, Ring I, Waldon J, Clark W, Whetung V et al. Indigenous Health - Australia, Canada, Aotearoa New Zealand and the United States - Laying claim to a future that embraces health for us all. World Health Report (2010) Background Paper 33. Geneva: World Health Organisation, 2010.

15. Stehr-Green P, Bettles J, Robertson LD. Effect of racial/ethnic misclassification of American Indians and Alaskan Native on Washington State Death Certificates, 1989-1997. American Journal of Public Health 2002; 92(3): 443-444.

16. Smylie J, Anderson M. Understanding the health of Indigenous peoples in Canada: key methodological and conceptual challenges. Canadian Medical Association Journal 2006; 175(6): 602.

17. Bramley D, Latimer S. The accuracy of ethnicity data in primary care. Journal of the New Zealand Medical Association 2007; 120: 1264.

18. Gomez SL, Le GM, West DW, Satariano WA, O'Connor L. Hospital policy and practice regarding the collection of data on race, ethnicity and birthplace. American Journal of Public Health 2003; 93: 1685-1688.

19. Polednak AP. Collecting information on race, Hispanic ethnicity and birthplace of cancer patients: policies and practices in Connecticut hospitals. Ethnicity and Disease 2005; 15: 90-96.

20. Riddle BL. On the coding and reporting of race and ethnicity in New Hampshire for purposes of cancer reporting. Ethnicity and Disease 2005; 15: 324-331.

21. Bramley D, Hebert P, Jackson R, Chassin M. Indigenous disparities in disease-specific mortality, a cross-country comparison: New Zealand, Australia, Canada and the United States. New Zealand Medical Journal 2004; 117: 1207.

22. Ring I, Brown N. Indigenous health: chronically inadequate responses to damning statistics. Medical Journal of Australia 2002; 177: 629-631.

23. Queensland Health. Aboriginal and Torres Strait Islander People, Queensland Mental Health Policy Statement. Brisbane, QLD: Queensland Health, 1996.

24. Hunter E. Communication and Indigenous health. Australian Doctor 2004; 4 June: 35-37.

25. Vicary D, Westerman T.'That's just the way he is': some implications of Aboriginal mental health beliefs. Australian e-Journal for Advancement of Mental Health 2004; 3: 1-10.

26. Nagel T, Thompson C. Aboriginal mental health workers and improving Indigenous mental health service delivery model in the 'Top End'. Australasian Psychiatry 2006; 14: 291-294.

27. Queensland Health; Harper C, Cardona M, Bright M, Neill A, McClintock C, McCulloch B et al. Health Determinants Queensland 2004. Brisbane, QLD: Public Health Services Queensland Health, 2004.

28. Madden R, Tickle L, Jackson Pulver L, Ring I. Estimating Indigenous life expectancy: pitfalls with consequences. Journal of Population Research 2012; 29(3): 269-281.

29. Robson B, Harris R (Eds). Hauora: Maori Standards of Health IV, a study of the years 2000 - 2005. Wellington: Te Ropu Rangahau Hauora a Eru Pomare, 2007.

30. Haswell-Elkins M, Tulip F, Saunders V, Wargent R. Eight Year Review of hospitalisation patterns to the Mental Health Unit, Cairns Base Hospital 1999/2000 to 2006/2007. Cairns, QLD: University of Queensland, 2008.