download PDF
full article:
An effective primary health service is one that improves health outcomes and sustains the optimal health of its population1. Monitoring health service effectiveness requires a comprehensive data collection system that includes required personal and diagnostic data. These data are essential for two key reasons. First, knowing the health needs of a population enables health services to plan and target their services to maximise health outcomes. Second, collecting information about disease rates at regular intervals completes the continuous quality improvement cycle necessary to determine health service performance.
Aboriginal Australians living in remote areas have worse health outcomes compared to non-Aboriginal Australians, both nationally and in rural and remote areas2,3. It is therefore particularly important for health organisations serving remote Aboriginal Australians to determine and monitor the prevalence of important diseases in the population in order to ensure that health resources are being directed for greatest benefit. Such disease monitoring systems need to be easily accessible and not require large investments of staff time or money, particularly given the high turnover of staff in remote health centres, which requires frequent training to be provided. Data routinely collected by computerised clinical management systems are both accessible and available; however, as with all secondary data, there can be issues with data quality. The findings of the present study are similar to others: routinely collected electronic health is adequate for health service monitoring, auditing and evaluating service provision for continuous quality improvement, despite not being 100% accurate as required for research purposes4,5.
This study seeks to determine whether data in the existing clinical management system Communicare (Communicare Systems Pty Ltd; http://www.communicaresystems.com.au) is of sufficient quality for estimating disease burden, (to identify areas in which data entry and hence data quality can be improved and to calculate baseline rates of important diseases in a remote Aboriginal population, which can be used to inform health service program planning and evaluation.
Background
The Fitzroy Valley is a remote area of Western Australia's Kimberley region, located approximately 400 km from the nearest regional centre (Broome) and 2500 km from the state capital (Perth). The Fitzroy Valley population is estimated at about 4500 (although there are known challenges to determining an accurate resident population due to high population mobility), of whom the majority are Aboriginal6,7. Approximately 1500 people live in the main town of Fitzroy Crossing, with the remainder living in more than 45 small communities of no more than 250 residents located throughout the surrounding valley6,7. The majority of clinical services in the Fitzroy Valley are provided either by the Fitzroy Valley Health Service, which provides both primary and tertiary care from the hospital situated in Fitzroy Crossing, or the Kimberley Population Health Unit, which provides primary health care via outreach clinics in the larger communities. Nindilingarri Cultural Health Services, a community controlled organisation, provides non-clinical health services for the Fitzroy Valley, including cultural safety, environmental health service, aged care and health promotion.
Over the past 5 years a formal partnership between all health services reoriented the focus of health service provision towards an increasingly integrated and holistic primary health care model8. This transition included the introduction of the clinical management system Communicare, designed to improve patient management and the sharing of information between the health services in the Fitzroy Valley. From 2005, a gradual introduction of Communicare took place with transfer of patient health details from paper records to the electronic system. This included both administrative efforts to transfer previously diagnosed conditions and recent medical encounters (from 2000 onwards) as well as updates by clinical staff of medical records with relevant history when patients were reviewed. From the beginning of 2010, Communicare has been the primary patient record system used in the Fitzroy Valley. Its introduction has provided the potential to use Communicare to monitor the burden of disease and health outcomes as part of an ongoing health service evaluation framework.
Data were extracted using the report function of Communicare. Demographics, illness condition and the date of condition recorded between the 1 January 2000 and 19 June 2011 were obtained for all patients presenting at any of the clinical services with a current address in the Fitzroy Valley. Data quality was determined in two domains: completeness of demographic fields and accuracy of the 'conditions' field. (The conditions field is non-compulsory and allows clinicians to enter a diagnosis from an International Classification for Primary Care-2 plus 'picking list', for each consultation. It is used to generate population-based reports or audits of various conditions.)
For completeness, the demographic fields chosen for examination were Aboriginal status, date of birth and sex, with completeness defined as any value entered in the specified field other than 'not stated' or 'not applicable'. Accuracy of the conditions field was determined as follows: five conditions (two for those aged <15 years and three for those aged ≥15 years) were chosen based on their high prevalence and health implications for remote Aboriginal populations (Table 1). For each of the five conditions, a list of records was generated that did not have the chosen condition recorded in the conditions field. From each of these lists, 50 records were randomly selected and carefully examined to determine whether the condition was listed elsewhere in the patient record (eg in the free text). See Figure 1 for an example of how this method was used to determine accuracy of records for diabetes mellitus.
Table 1: Conditions of importance analysed, by age group
Figure 1: Method of determining accuracy of conditions field - example of diabetes mellitus.
Rates of disease were calculated for the five conditions examined for accuracy as well as for two additional conditions for those aged <15 years. Prevalence by age group was calculated for chronic conditions and age-group specific incidence for acute conditions. The calculation of disease rates was restricted to the Aboriginal population only, and given recognised difficulties with determining accurate population figures for the Fitzroy Valley7 the Communicare population list was used as the denominator for calculations of disease occurrence. The year in which chronic conditions were first recorded was calculated by extracting the year that the condition was first recorded in the conditions field. Data analysis was conducted using Stata v10 (StataCorp; http://wwwstata.com) and figures created using Microsoft Excel.
Ethics approval
Ethics approval (2011:26) was granted by the Kimberley Aboriginal Health Planning Forum Research Subcommittee, the Western Australian Aboriginal Ethics Council and the Western Australian Country Health Service Ethics Committee.
Data quality
Table 2 shows the quality of the data set. Of the 5227 individual Communicare records for the Fitzroy Valley, completeness of the demographics fields was high. Accuracy of the 'conditions' field by specific condition varied by diagnosis (Table 3).
Table 2: Completeness of key demographic fields
Table 3: Accuracy of conditions field (50 records where disease not included in 'conditions field' examined for each disease)
Disease prevalence and incidence
Disease prevalence and incidence were assessed from data for 3495 Aboriginal residents from 22 communities recorded as current Fitzroy Valley patients in Communicare. The population had a male-to-female ratio of 100:101, and was generally young with 33% aged <15 years, 53% aged 15-49 years and 13% aged ≥50 years. Fitzroy Crossing had the largest population (23%); all other communities had fewer than 15% of the total population (range 0.5-13%).
Figure 2 shows specific incidence, by age group, in children aged <15 years. Otitis media was the most commonly recorded condition for persons aged <5 years, while skin infections were the most commonly recorded conditions in the 5-9 and 10-14 years age groups.
Figure 3 shows prevalence of important conditions of adulthood. Diabetes was the most commonly recorded condition with an overall prevalence of 17% in those aged >15 years, while renal disease was the second most commonly reported condition with an overall prevalence of 14%. For all chronic conditions, prevalence increased with age, and for those aged ≥55 years, 66% had at least one of the three chronic conditions recorded. The year in which the condition was first recorded is shown in Figure 4, which indicates an increase in conditions recorded until 2006-2007 and then relatively steady numbers from 2008 onwards.
Figure 2: Age-group specific incidence of acute conditions in those aged <15 years.
Figure 3: Prevalence of important conditions in those aged ≥15 years.
Figure 4: Year of first recording for the conditions of hypertension, kidney disease and diabetes.
Discussion
This descriptive community study sought to examine the quality of routinely collected clinical data as the basis for calculating rates of important diseases in the Fitzroy Valley population for health service planning and evaluation, and for providing useful feedback to the community through the health service partnership.
Invariably, there are recognised issues associated with using routinely collected primary health care data for population health planning purposes, including lack of completeness and standardisation in clinical coding4,9,10. Different data fields in the Communicare system showed some variation in quality. Demographics were well recorded, as were chronic conditions in the conditions field. Acute conditions, however, were less well recorded. In reality, disease rates generated using these data reflect clinician diagnosis, and, in the absence of using validated case definitions, there was no standardised way of recording certain conditions (eg there were 12 different codes that could be used to record chronic kidney disease). Moreover, new diagnoses of chronic diseases (Fig3) are likely to reflect the increasing uptake of Communicare, so the first time a patient with such a condition was seen since the health provider had started using Communicare would be entered as date of onset, rather than the first date the condition was diagnosed. Additionally, this single database lacks power for research that larger sources of primary health care data (eg practice networks) can generate. These issues notwithstanding, routinely collected data entered in Communicare are free, easily obtainable and available to local staff, who can use them to understand the specific health needs of their population. A particular strength of these data is undoubtedly completeness of population coverage, since all healthcare providers in the Fitzroy Valley use Communicare thereby enabling a comprehensive account of the health of this remote community to be recorded.
Possibly, acute diseases were not as well recorded because this field is used to create a summary of patient health status. Therefore, while busy clinicians may see filling it in as important when a condition requires ongoing management (such as diabetes), this may not be the case for an acute and/or self-limiting conditions, which clinicians may find easier to record in free text. However, until the capacity to extract data from free text via routine reporting tools is available, it will be necessary to rely on clinicians to enter all coded data accurately for each consultation.
As a result of this study, the health service has initiated several quality improvement strategies to further improve these routinely collected data. For example, a list of standardised recording codes, including clear and easily understandable case definitions, has been finalised for a range of important conditions with regular auditing and data cleaning. A clinician engagement strategy has also been developed that emphasises the importance of the clinician's role in accurately entering data, which will ultimately be used to improve health outcomes. Implementation of these measures will undoubtedly serve to improve data quality for its use in evaluating health outcomes and the targeting of health services.
In terms of calculating baselines rates of diseases, the Communicare data show a very high prevalence of the chronic conditions considered, consistent with other studies of similar populations11,12. These rates highlight the importance of the current realignment of health services towards a comprehensive primary healthcare approach to prevent and treat chronic disease. It should be noted, however, that using prevalence rates of chronic diseases over time as the sole evaluation tool for health service improvement would not be sufficient, because of the long lag times associated with development of these diseases, the effect of other confounding factors that influence disease occurrence and possible improvements in screening for chronic disease under the new health service model13. Disease prevalence will therefore need to be used in concert with process indicators that are more sensitive to changes in health service provision and will therefore form part of a more comprehensive evaluation framework. Early evaluation using Communicare has already shown how health service provision can be improved14.
Although the rates of acute conditions in children are high, they are lower than those found by other studies of similar populations15,16. They are likely to be underestimates for two main reasons. The first is under-recording due to not filling in the conditions field. The second is that not all people with these diseases present for clinical treatment. Therefore, using changes in the rate of acute disease to evaluate the new model of health service will need to take these factors into account. A more useful and sensitive measure for monitoring purposes may be the number of infections per child within a specified time period. The possibility of having this as a 'routine' report is being discussed with Communicare developers.
Good medical information systems are essential for the planning and evaluation of the effectiveness of primary healthcare service delivery. This study found that Communicare is generally an effective tool for determining the burden of disease in a remote population, and has been integrated into an evaluation framework for monitoring health outcomes over time17. Although primary healthcare data routinely collected by health authorities invariably have some inherent limitations, they are nevertheless able to be used to establish population health indices with minimal cost and effort in a feasible way.
Acknowledgements
The authors gratefully acknowledge the input of Maureen Carter, who reviewed this manuscript, as well as the assistance of staff at the Kimberley Population Health Unit, who assisted with downloading the original data.
References
1. Starfield B, Shi L, Macinko J, Contribution of primary care to health systems and health. Milbank Quarterly 2005; 83(3): 457-502.
2. Vos T, Barker B, Stanley L Lopez A, The burden of disease and injury in Aboriginal and Torres Strait Islander Peoples 2003. Brisbane: University of Queensland, 2007.
3. Zhao Y, Guthridge S, Magnus A, Vos T. Burden of disease and injury in Aboriginal and non-Aboriginal populations in the Northern Territory. Medical Journal of Australia 2004; 180(10): 498-502.
4. Liaw S, Taggart J, Dennis S, Yeo A. Data quality and fitness for purpose of routinely collected data - a general practice case study from an electronic practice-based research network (ePBRN). AMIA Annual Symposium Proceedings 2011; 2011: 785-794.
5. Taggart J, Liaw S, Dennis S, Yu H, Rahimi A, Jalaludin B, Harris M. The University of NSW electronic practice based research network: disease registers, data quality and utility. Student Health Technology Information 2012; 178: 219-227.
6. Nindilingarri Cultural Health Services. Welcome to Nindilingarri Cultural Health Services. (Online). Available: http://www.nindilingarri.org.au (Accessed 30 June 2013).
7. Morphy F. Population, people and place: the Fitzroy Valley Population Project. Discussion paper no. 70. Canberra, ACT: Australian National University, Centre for Aboriginal Economic Policy Research, 2010.
8. Reeve C, Humphreys J, Wakerman J, Carroll V, Carter M, O'Brien, T, et al. Community participation in health service reform: the development of an innovative remote Aboriginal primary health-care service. Australian Journal of Primary Health 2015; 15(1).
9. de Lusignan S, Metsemakers JF, Houwink P, Gunnarsdottir V, van der Lei J. (Eds). Routinely collected data: goldmines for research? A report of the European Federation for Medical Informatics Primary Care Informations Working Group (EFMI PCIWG) from MIE2006, Maastricht, the Netherlands, 2006. Informatics in Primary Care 2006; 14(3): 203-209.
10. de Lusignan S, van Weel C. The use of routinely collected computer data for research in primary care: opportunities and challenges. Family Practice 2006; 23(2): 253-263.
11. Hoy WE, Davey RL, Sharma S, Hoy PW, Smith JM, Kondalsamy-Chennakesavan S. Chronic disease profiles in remote Aboriginal settings and implications for health services planning. Australian and New Zealand Journal of Public Health 2010; 34(1): 11-18.
12. Zhao Y, Connors C, Wright J, Guthridge S, Bailie R. Estimating chronic disease prevalence among the remote Aboriginal population of the Northern Territory using multiple data sources. Australian and New Zealand Journal of Public Health 2008; 32(4): 307-313.
13. Australian Institute of Health and Welfare. Risk factors contributing to chronic disease. cat. no. PHE 157. Canberra: AIHW, 2012.
14. Reeve C, Humphreys J, Wakerman J, Carter M, Carroll V, Reeve D. Strengthening primary health care: achieving health gains in a remote region of Australia. Medical Journal of Australia 2015; 202(9): 483-487.
15. Clucas D, Carville K, Connors C, Currie B, Carapetis J, Andrews R. Disease burden and health-care clinic attendances for young children in remote aboriginal communities of northern Australia. Bulletin of the World Health Organization 2008; 86(4): 275-281.
16. McMeniman E, Holden L, Kearns T, Clucas DB, Carapetis JR, Currie BJ, Connors C, Andrews RM. Skin disease in the first two years of life in Aboriginal children in East Arnhem Land. Australasian Journal of Dermatology 2011; 52(4): 270-273.
17. Reeve C, Humphreys J, Wakerman J. A comprehensive health service evaluation and monitoring framework. Evaluation and Program Planning 2015; 53: 91-98.