download PDF
full article:
Introduction
Quality of health care is a matter of increasing concern to governments, health authorities and consumers1,2, particularly in rural communities where health status is demonstrably worse than in metropolitan areas due to workforce shortages, impacts from service rationalisation, and difficulties accessing services3-9. Measures of consumer satisfaction can provide an important assessment of quality of health services not captured by other health service statistics, such as patient throughput, distance to nearest services or waiting times10,11. The extent to which rural residents are satisfied with their local health services is a key determinant underpinning their health behaviour and utilisation of health services12,13.
Timely, accessible, appropriate, safe, continuous and effective health services are essential elements to ensure quality health care14. In the absence of an agreed methodology for measuring consumer satisfaction, it is often difficult to gauge the extent to which these critical parameters are being met. Based on studies undertaken in several Australian rural communities, this article highlights a number of issues associated with studies measuring consumer satisfaction with rural health services, and provides some pointers to guide future studies.
The specific focus on rural health services stems from the risk that without some measure of consumer satisfaction, rural communities (many of which are already experiencing sub-optimal health services) are unlikely to see improvements in their health services. Moreover, particular logistical and resource difficulties often present when undertaking timely large-scale consumer surveys in rural areas, characterised by widely dispersed populations.
The glass is half full
Most studies of consumer satisfaction focus on those key aspects of quality of health care that are amenable to some form of policy influence or intervention. Such an approach was adopted by the authors in assessing consumer satisfaction with health services in three rural communities in western New South Wales (NSW), and also in a survey of eight rural communities in north-west Victoria15,16. Guided by the literature14,17,18, the study compared rural consumer satisfaction using five key sentinel indicators of the quality of health care, namely the extent to which health needs could be met locally (availability), the ability to get help at times of emergency (geographical access), the availability of bulk billing (economic access), choice of healthcare provider (choice), and seeing the same provider when required (continuity of health care).
Ethics approval for these studies was obtained from the Human Ethics Committee of Monash University, Victoria, Australia. In addition, ethics approval was granted by the NSW Far West and Macquarie Area Health Services for the New South Wales study.
The three NSW communities represented small rural inland towns with populations of less than 5000 people. These rural communities experience poor health status and face problems securing appropriate health services because of their small size and geographical isolation. Information was collected in September 2002 by delivery-and-collection of questionnaires to a 20% sample of all occupied private dwellings15.
In the Victorian study, Australia Post's 'unaddressed delivery service' (UDS) was used to deliver all the questionnaires in March-April 2005 to 5965 Australia Post private mail distribution points (private dwellings, postal boxes, postal counters and roadside delivery) in the eight communities16.
Householders in both studies were asked to rate their satisfaction with regard to the availability, accessibility, choice, continuity, and affordability of health services using a 5 point Likert scale from 1 = very satisfied to 5 = very dissatisfied19.
The results are presented in Table 1. Survey response rates varied between 59% and 78% for the three NSW communities, and of the 5965 surveys distributed in the Victorian study, 983 useable replies were received15,16. The results showed that consumer satisfaction was generally high with all five indicators. Levels of satisfaction were highest in both NSW and Victoria in relation to the extent to which day-to-day health needs can be met locally (over two-thirds of respondents in both study areas were satisfied or very satisfied), and accessing help at times of emergency (over 60% were satisfied or very satisfied in NSW and Victoria). In both studies satisfaction levels were slightly lower (less than 60% were satisfied or very satisfied) in relation to being able to see the same provider, choice of provider and the availability of bulk billing. Indeed, satisfaction with bulkbilling availability was particularly low in the Victorian study with only 50% of respondents satisfied or very satisfied with this aspect of health care.
Table 1: Percentage of respondents' levels of satisfaction with local health services in select rural communities in western New South Wales and north- west Victoria
In short, based on the generally high levels of consumer satisfaction with rural health services, it would be easy to conclude that all is well with health services in small rural communities, and that reports of serious consumer concerns with inadequate rural workforce supply7 and problems associated with the contraction of many procedural services in small rural communities20, fail to capture the extent to which the health needs of rural residents are adequately met. The issue remains of how to reconcile these differences, and the extent to which methodologies used to assess levels of consumer satisfaction may influence the results.
The glass is half empty
An alternative way of assessing levels of consumer satisfaction is to examine the nature of complaints made in relation to rural health services. The monitoring of consumer complaints about health service provision and quality plays an important role in health service improvement1,21,22. For this reason, a variety of consumer advisory committees and internal complaints-handling procedures (including independent health commissioners) have been established in each state and territory. Such agencies are responsible for considering how information obtained from complaint investigations may be used to improve health services.
A detailed analysis was undertaken of 23 866 records of complaints to the Victorian Health Services Commissioner from 1988 to 200123,24. Each record was classified into rural and urban by matching the postcode or suburb to the corresponding rural, remote and metropolitan (RRMA) classification25. The data were then analysed using SPSS vers. 11 (SPSS Inc; Chicago IL, USA) to identify differences in the number of complaints from urban and rural consumers in addition to the characteristics of type of provider, issue of complaint and outcome of complaint23,24. The significance of differences between observed and population-based expected frequencies of complaint was tested using the χ2 statistic.
The results showed that complaints from rural consumers were under-represented by 25% on a per capita basis compared with their metropolitan counterparts23,24. The nature of rural complaints also differed, with evidence that problems of accessing services when required are of much greater concern to rural than metropolitan consumers. There were also proportionally more complaints about rights and fewer about treatment from rural consumers compared with urban. There were slightly higher levels of rural complaints about GP services. Without follow-up research into reasons for this under-representation of rural complaints, it might appear that these findings validated the conclusion that rural consumers are generally satisfied with their health services.
In addition, our 'half-full' study found that a large number of consumer concerns about rural health care are not reported through complaints mechanisms16. While only 11% of respondents lodged some complaint in the previous year, another 34% wanted to complain but didn't. Access emerged as the most common issue, accounting for 72% of complaints not reported, with doctors (60%) and hospitals (31%) as the most commonly identified providers. Under-reporting of complaints was influenced by a sense of resignation among consumers that making a complaint does not really result in any difference to health services and may, in fact, impact negatively on the complainant16.
How can we determine how full the glass really is?
These contrasting studies show that the scope for demonstrating high levels of rural consumer satisfaction is great, when simultaneously there exists considerable consumer dissatisfaction with aspects of the quality of health services available to them. Part of the difficulty with measuring consumer satisfaction with health services is the multifaceted nature of the concept. Satisfaction levels do not always equate solely to quality of care. Numerous other factors influence satisfaction, including consumer perceptions, attitudes, expectations and experiences; their physical and psychological health; personal and societal values; and consumer knowledge of and exposure to health services26-28. Indeed the very act of defining satisfaction is problematic29,30.
Given the significance attributed to consumer input to healthcare planning by Australian governments2,28,31,32, how can we best gauge levels of rural consumer satisfaction with their healthcare services? In particular, what distinguishes the assessment of consumer satisfaction in rural areas compared with urban?
A number of important issues should be taken into consideration when measuring rural consumers' levels of satisfaction with their health services.
Logistical aspects of achieving representativeness of the target group
Achieving representative samples is often difficult to achieve in rural communities, due to the cost and logistics of operating in vast regions characterised by many small, dispersed communities. A representative sample is one that accurately represents the characteristics in the designated population33. Certain methods, such as focus groups and consumer representatives on advisory or reference groups, often under-represent key rural consumer groups, including non-users of services (because services are not locally available or easily accessible) or those who bypass local services in favour of more distant ones perceived to be superior. Moreover, in small rural settlements characterised by a strong sense of local community, soliciting consumer opinions about the quality of local services and their providers can be a particularly sensitive issue.
For this reason, it is essential to gather and use information in a way that ensures confidentiality and dissipates any fear of alienation or retribution34. In this regard, surveys using computer assisted telephone interviewing (CATI) or mailed self-completion questionnaires may be most appropriate because of their ease of transcending vast distances, and that they protect the identity of the respondent. At the same time, such techniques (while expedient, cost-effective and ensuring anonymity) may not ensure representativeness of the broad cross-section of users, especially marginalised groups, and can often yield low response rates. Extensive face-to-face interview surveys, however, may be more inclusive and maximise response rates and the richness of data, but are expensive, time-consuming and may raise issues of anonymity and confidentiality that can influence the nature of responses. Citizen juries, where consumers are randomly chosen and decisions are made to benefit the whole community rather than for the jury members as individuals, are worthy of consideration35.
The choice of methodology employed in satisfaction surveys is usually determined by the purpose of the study, its timing and the resources available. What is critical, however, is that the interpretation placed on the research findings is done with care and takes sufficient account of the limitations of the methodology used.
Minimising bias in satisfaction ratings
Many small rural communities in Australia have experienced significant reductions in the availability of local health services over recent years36. This context may result in participants recording what they think the researcher wants to hear. This may take the form, for example, of a fear of further service reduction or closure and may affect consumer responses, reflected in an excessively negative rating despite the high quality of those local health services remaining. Alternatively, participants may paint an excessively positive picture of existing services regardless of their quality or utilisation patterns in order to 'protect' their services from any further reduction37,38. In such circumstances, both the timing of the consumer satisfaction data collection and the nature of the survey instrument can influence the validity of the survey outcomes.
The specific nature of questions is particularly important. The use of satisfaction rating scales alone can be problematic because respondents usually report greater satisfaction than dissatisfaction with health care30,39-41. This skewing of response distribution may occur because consumers believe positive comments are more acceptable to survey researchers, they are reluctant to complain for fear of future unfavourable treatment, or they believe that more positive responses will contribute to continuation of services10. Such bias can be tempered by including both structured and unstructured questions to elicit different kinds of data. A Scottish study comparing consistency of satisfaction levels in three surveys found that levels of satisfaction or dissatisfaction depend very much on the wording of questions and the approach used26,40.
Communicating and sharing outcomes with rural consumers
The development of trust and respect between researchers and participants is always crucial to maximising participation from a wide segment of the study population, arguably nowhere more so than in health service satisfaction surveys in small rural communities42. Rural residents need to be clearly informed about the purpose of the study from the outset, in addition to researchers promoting consumer ownership via consumer input into planning and development processes.
In rural areas where services are particularly vulnerable to reduction or because services are limited, it can be difficult for residents of small communities to develop trust in researchers, given the potential for the research findings to be used by health authorities as the basis for changes and possibly further reduction of local services, in preference to investment required to improve the quality of existing local services.
Conclusion
The value of consumer input to the planning and provision of rural health services is not in dispute43. Without it, the quality of rural health services available remains predominantly a function of the viewpoints and agents of health authorities and providers, something that is increasingly determined by fiscal considerations rather than a focus on the health needs of consumers. Unfortunately the cost of assessing levels of consumer satisfaction with health services in rural areas remains high despite technological advances that expedite data collection. Moreover, the demands on rural consumers' time to participate in surveys perceived to be of limited value can lead to frustration and disillusionment at the community level about the role and value of consumer participation to the quality improvement process16. In examining rural and remote communities, researchers need to understand how their choice of method, satisfaction measure or instrument, and relationship with the community can influence the extent to which consumers actively participate in the research, as well as the potential for bias in their results.
In light of the increased recognition and importance being attached to the field of rural health research, as well as the greater use of this research by policy makers and health service funders, further analytical enquiry, debate and development is required into the best ways to apply tools and methods, (usually developed in other contexts) to the rural health agenda.
References
1. Lapsley H. Quality measures in Australian health care. In: Health Reform in Australia and New Zealand. A Bloom (Ed.). Melbourne, Vic: Oxford University Press, 2000; 282-292.
2. Taskforce on Quality in Australian Health Care. The final report of the taskforce on quality in Australian health care. Canberra: Australian Health Ministers Advisory Council, 1996.
3. Australian Institute of Health and Welfare. Health in rural and remote Australia, Canberra: AIHW, 1998.
4. Australian Institute of Health and Welfare. Health and Community Services Labour Force 2001, National health labour force series no. 27. AIHW cat. no. HWL 27. Canberra: AIHW, 2003.
5. Humphreys J. Health service models in rural and remote Australia. In: The new rural health. D Wilkinson and I Blue (Eds). Melbourne, Vic: Oxford University Press, 2002.
6. Humphreys J, Jones M, Jones J, Mara P. Workforce retention in rural and remote Australia: determining the factors that influence length of practice. Medical Journal of Australia 2002; 176: 472-476.
7. Productivity Commission. Australia's health workforce: research report. Canberra: Productivity Commission, 2005.
8. Sadkowsky K, Hagan P, Kelman C, Liu C. Health services in the city and the bush: measures of use derived from linked administrative data. Occasional papers: new series number 13. Canberra: Commonwealth Department of Health and Aged Care, 2001.
9. Australian Medical Workforce Advisory Committee. Annual report 2003-04. AMWAC report 2004.4. Sydney: AMWAC, 2004.
10. Sitzia J, Wood N. Patient satisfaction: a review of issues and concepts. Social Science & Medicine 1997; 45: 1829-1843.
11. Williams S, Calnan M. Convergence and divergence: assessing criteria of consumer satisfaction across general practice, dental and hospital care settings. Social Science & Medicine 1991; 33: 707-716.
12. Hadorn D. The role of public values in setting health care priorities. Social Science & Medicine 1991; 32: 773-781.
13. Rankin S, Hughes-Anderson W, House J, Aitken J, Health D, Mitchell A et al. Rural residents' utilisation of health and visiting specialist health services. Rural and Remote Health 2. (Online) 2002. Available: www.rrh.org.au (Accessed 16 November 2006).
14. Cambell S, Roland M, Buetow S. Defining quality of care. Social Science & Medicine 2000; 51: 1611-1625.
15. Smith K, Humphreys J, Lenard U, Jones J, Prince V, Han G-S. Still the doctor - by a country mile! Preferences for health services in two country towns in north-west New South Wales. Medical Journal of Australia 2004; 181: 91-95.
16. Jones J, Meehan-Andrews T, Smith K, Humphreys J, Griffin L, Wilson B. "There's no point complaining, nothing changes." Rural disaffection with complaints as a means to improve quality in health services. Australian Health Review 2006; 30: 322-332.
17. Donabedian A. The seven pillars of quality. Archives of Pathology & Laboratory Medicine 1990; 114: 1115-1118.
18. Maxwell R. Quality assessment in health. BMJ 1984; 288: 1470-1472.
19. Likert R. A technique for the measurement of attitudes. Archives of Psychology 1932; 140: 5-55.
20. Rural Doctors Association of Australia. Viable models of rural and remote practice. Canberra: RDAA, 2003.
21. Douglas D, Harrison R. Turning around patient complaints in a regional hospital. Australian Health Review 1996; 19: 126-137.
22. Preston-Shoot M. A triumph of hope over experience? Modernizing accountability: the case of complaints procedures in community care. Social Policy & Administration 2001; 35: 701-715.
23. Jones J, Wilson B, Humphreys J, Punshon P, Griffin L. Rural consumers' complaints about health services. In: Proceedings, 7th National Rural Health Conference. 1-4 March 2003; Hobart, Tasmania, Australia. National Rural Health Alliance, 2003.
24. Jones J, Humphreys J, Wilson B. Do health and medical workforce shortages explain the lower rate of rural consumers' complaints to Victoria's Health Services Commissioner? Australian Journal of Rural Health 2005; 13: 353-358.
25. Australian Institute of Health and Welfare. Rural, regional and remote health: a guide to remoteness classifications, AIHW cat. no. PHE 53. Canberra: AIHW, 2004.
26. Carr-Hill R. The measurement of patient satisfaction. Journal of Public Health Medicine 1992; 29: 439-446.
27. Hordacre A-L, Taylor A, Pirone C, Adams R. Assessing patient satisfaction: implications for South Australian public hospitals. Australian Health Review 2005; 29: 439-446.
28. NSW Health. NSW health surveys 1997 and 1998: health services utilisation - difficulties getting health care. Sydney: NSW Health, 2004. Available: www.health.nsw.gov.au (Accessed 11 April 2006).
29. Collins K, O'Cathain A. The continuum of patient satisfaction - from satisfied to very satisfied. Social Science & Medicine 2003; 57: 2465-2470.
30. Verbeek J, van Dijk F, Rasanen K, Piirainen H, Kankaanpaa E, Hulshog C. Consume satisfaction with occupational health services: should it be measured? Occupational and Environmental Medicine 2001; 58: 272-278.
31. Department of Health and Ageing. Consumer engagement in health care. Canberra: Department of Health and Ageing, 2002.
32. Department of Human Services. Participation in your health service system: Victorian consumers, carers, and the community working together with their health service and the Department of Human Services. Melbourne: Department of Human Services, 2005.
33. Fortune AE, Reid WJ. Research in Social Work (3rd edn). New York: Columbia University Press, 1999.
34. E-QUAL. Review of approaches to satisfaction surveys of clients of disability services. Perth: Australian Government Productivity Commission, 1998.
35. Mooney GH, Blackwell SH. Whose health service is it anyway? Community values in healthcare. Medical Journal of Australia 2004; 180: 76-78.
36. Humphreys J. Super clinics or a country practice? Contrasts in rural life and health provision in northern New South Wales. In: Change and adjustment in northern New South Wales. D Walmsley (Ed.). Armidale, NSW: University of New England, 1990.
37. Humphreys J, Mathews-Cowey S, Weinand H. Factors in accessibility of general practice in rural Australia. Medical Journal of Australia 1997; 166: 577-580.
38. O'Neal P. Methodological problems associated with measuring consumer satisfaction in the mental health field. Australian Social Work 1999; 52(3): 9-15.
39. Calnan M, Almond S, Smith N. Ageing and public satisfaction with the health service: An analysis of recent trends. Social Science & Medicine 2003; 57: 757-762.
40. Cohen G, Forbes J, Garraway M. Can different patient satisfaction survey methods yield consistent results? Comparison of three surveys. BMJ 1996; 313: 841-844.
41. Jamieson S. Likert scales and how to (ab)use them. Medical Education 2004; 38: 1217-1218.
42. Hicks N, Harford J. Summary report on consumer participation in resource allocation. Melbourne, Vic: National Resource Centre for Consumer Participation in Resource Allocation, 2000.
43. Australian Health Ministers' Advisory Council. Healthy horizons: outlook 2003-2007. A framework for improving the health of rural, regional and remote Australians. Canberra: Australian Government Publishing Service, 2003.