A multi-university evaluation of the rural clinical school experience of Australian medical students

full article:

Introduction

Australia has a relative shortage of medical practitioners in rural areas, in comparison with metropolitan areas, and although the supply is increasing slightly, changes in working practices means that the effective supply is diminishing¹. In addition, the health of rural people is worse than the health of their metropolitan counterparts in a number of areas², and it is postulated that the relative shortage of doctors is one of a number of contributing factors.

In a number of countries, programs have been instituted over the past 30 years to provide medical students with an early rural experience in the hope that this will encourage them to return to rural areas when they complete their training. Overall, the assessments of outcome have been modestly favourable. In Australia, a number of programs have been initiated at university, state and territory and national level. These primarily fund brief exposures in rural practice and scholarships³. However in 2000, the then Commonwealth government funded the rural clinical school (RCS) program, initially in 9 universities but subsequently expanding to its current total of 13 universities. This program requires 25% of students in each year to undertake at least 1 year of their clinical training in a rural setting⁴.

The program has been successful in meeting its short-term objectives, as assessed by a recent external evaluation⁵, although it will take a number of years to see if those students who experienced the program return to and practise in rural locations, but early signs are encouraging⁶.

In 2005, the heads of the RCSs agreed to develop a questionnaire about rural student intentions and experiences, to allow individual schools to assess their students' experiences, and to facilitate data analysis at a national level⁷.

In 2006, Medical Deans Australia and New Zealand (MDANZ) agreed to develop a longitudinal data base to include all medical students, and to research factors relating to student preferences and outcomes⁸. The rurally-developed questionnaire and the MDANZ survey instrument were co-ordinated to avoid student burnout through the requirement to complete multiple questionnaires.

The aim of this study was to evaluate the experiences of medical students who attended rural clinical schools during 2006, using the rural-specific questionnaire. This is the second part of a broader study conducted to explore medical students' views of rural clinical schools.

Methods

Questionnaires were distributed in 2006 to 166 RCS medical students who had completed 1 year at the RCS at 6 participating universities. Not all eligible universities participated in this initial study. Participation of both universities and of students was voluntary and in each participating university, the distribution of questionnaires and the collation of responses was left up to staff at the individual RCSs. Ethics approval was obtained for the study from each participating university human research ethics committee.

The questionnaire 'Rural Clinical School Evaluation 2006'⁷ was developed as an evaluation tool to determine baseline data about students studying in RCSs. Background characteristics assessed included age, gender, marital status, number of dependents, ethnicity, admission and entry into medical school, rural background and education. Using a six-point Likert scale (1 = disagree strongly, 2 = disagree moderately, 3 = disagree slightly, 4 = agree slightly, 5 = agree moderately, 6 = agree strongly) students were asked to rate their level of agreement on 29 items concerning their overall RCS experience, skills development and clinical supervision experience. The six-point Likert scale results showed a number of very small cell sizes. All authors individually reviewed individual cell responses for bias or 'outlying' concerns with no significant issues being identified. Thus, based on common agreement, 'agree' and 'disagree' cells were collapsed to allow a clearer picture of students' opinions and perspectives.

All data were pooled. The software SPSS v15 (SPSS; Chicago, IL, US: www.spss.com) was used to generate descriptive analyses and to evaluate responses to questionnaire items concerning students' experiences of the RCS. Qualitative responses to open-ended questions were analysed using standard quantitative methods.

Results

Responses were received from 125 of 166 students (75.3%) who received questionnaires. Respondents ranged in age from 21 to 43 years (mean 24.5 years, SD = 3.89, with nine students aged 30 years or older). Almost two-thirds of the sample was female (n = 77, 62%). The majority was single (n = 109, 87%), with the remaining 16 (13%) married or in a de facto relationship. Of the 125 participants, only six students (5%) had children under 16 years. Sixty-nine students (55%) held at least one scholarship.

The majority of students were born in Australia (n = 101, 83%), while the 21 students not born in Australia (17%) had been resident in Australia for an average of 19 years (mean 18.63 years, SD = 6.63). English was the second language for 15 students (12%).

The breakdown of participants by medical school is presented (Table 1).

Table 1: Participants according to medical school



The majority of respondents (n = 107, 86%) stated they would go to the RCS again if they had their time over and almost two-thirds (n = 77, 64%) stated they would spend more time at the RCS if they could.

Twelve questions were used to evaluate educational experiences at the RCS. To aid interpretation the original six-point response scale was collapsed into two categories: 'disagree' (disagree strongly, moderately, slightly) and 'agree' (agree strongly, moderately, slightly) (Table 2). All evaluation items recorded greater than 80% agreement (indicating very positive perceptions of the RCS experience). The highest level of agreement for general items was 'the RCS environment was conducive to learning' (n = 117, 94%). For the items concerning skills development, the highest level of agreement related to developing procedural skills (n = 121, 97%).

Table 2: Students' evaluation of their experience at rural clinical school



Using a six-point scale, students rated their clinical supervisors on 15 attributes, with responses also collapsed into 'disagree' or 'agree' (Table 3). For all items the agreement rate exceeded 80%. The majority of students found supervisors approachable (n = 121, 97%), enthusiastic (n = 120, 96%) and respectful (n = 119, 95%).

Table 3: Students' assessment of supervision at rural clinical schools



Students provided 59 open-ended comments concerning their experience at RCS. Forty-two percent (n = 25) were positive, 32% (n = 19) contained positive and negative comments and 25% (n = 15) were negative.

Students were positive about patient access and clinical teaching:

Very good access to patients. Good bedside teaching, I felt that I received excellent teaching and Educational experience has been excellent - lots of access to patients and the patients were very willing to see us. The academic staff have been very helpful and approachable.

Students reported concerns about lack of specialty training:

Too much input from allied health. Not enough from specialists and Missed out on some specialty area, eg. urology, plastic surgery, ear, nose and throat surgery and tertiary care.

Other students were concerned about teaching methodology and assessment:

A more didactic curriculum would be nice...

Needed more theoretical/lectures to promote and extend our learning in final year.

Problem lies in nebulous goals and unachievable expectations and lousy assessment techniques. Multiple choice questions and extended matching questions are terrible methods of assessment.

There were 41 comments provided about supervision at the RCS. Fifteen (37%) were positive, for example:

My clinical supervisors were amazing. I couldn't believe how keen they were to teach me. They had such a broad range of skills themselves and Our supervisors were excellent - couldn't ask for nicer, more enthusiastic people...Best teaching we have had in course to date.

However, 19 (46%) were mixed, and typically, these comments stated some supervision was excellent but some not to the same standard, for example:

Supervision was done well by some and not so well by others.

Very supervisor dependent, some great, some not so great.

Only 8 (19%) were negative, for example:

There just aren't enough clinicians to help the increasing number of students. Rural medical students suffer greatly.

Many students (n = 101, 81%) compared their RCS experience with their metropolitan-based peers. Of these, 53% (n = 54) stated the RCS experience was unequivocally positive in comparison, 30% (n = 30) provided mixed or neutral responses, 8% (n = 8) were totally negative and 9% (n = 9) stated they were not sure. Some students (n = 10, 10%) commented their experience was more 'hands on' than their metropolitan peers and 4 (3.9%) appreciated their increased patient access. Typical positive comments included:

...better, more patient contact, more teaching...

...clinical experience was fantastic. Was a great environment to learn.

Examples of a mixed/neutral response are:

Different. Not better, not worse, just different.

Is superior in academic and patient interaction but has negative aspects with respect to housing, schooling, social etc.

Negative comments included:

A little limited due to lack of exposure to clinical specialties...

Worse. The metro hospitals in my opinion had better patient access and education opportunities.

Eighty students (64%) responded to a question about the best aspect of their RCS experience. Nineteen (24%) commented on the quality of clinical teaching and the ability to learn clinical skills; for 16 (20%) patient exposure was the best aspect; while 8 (10%) commented on either increased autonomy or increased ability to work in teams. Ten students (13%) commented that the best thing was the lifestyle and friends.

The influence of medical staff as positive role models was addressed by 50 students (40%). Aspects that were considered most important in the role modelling were that the person was not arrogant but respectful, showed both exceptional clinical and other attributes as a doctor including motivation to teach, and demonstrated how one could have a balanced life between work and family in the country.

Discussion

The overall results about students' experiences in the RCS are unequivocally positive. Most importantly, the RCS environment was conducive to learning and the development of clinical skills, the students were able to see an adequate number of patients and were well prepared for examinations, and their supervisors were very good and acted as positive role models. In this respect, the results are not surprising because they reflect the results of previous reports^9-11. However, this is the first study combining data from several universities with distinct curricula, and including both undergraduate and graduate entry programs.

Prior to the initiation of the RCS program, there were concerns that the rural environments may not be conducive to learning, that students would feel inadequately prepared for examinations (as has been reported anecdotally by specialty trainees when asked to undertake rural terms before examinations) and that new supervisors without previous experience with medical students would not be able to provide the same level of academic teaching as their experienced metropolitan colleagues. This concern was expressed anecdotally by some faculty members and by potential rural supervisors; it is reassuring to see that the concern was unfounded.

However it is important that both the RCSs and their parent faculties, and the funder, do not become complacent. Negative comments were expressed, particularly in the opening comments, and more than 50% of those who responded to the open-ended questions had negative comments (or perhaps constructive criticism) about aspects of their RCS experience and supervision. It will probably be of more value to explore these in some detail and address specific aspects rather than believing that greater than 80% positive response to specific questions means that there is no further work to be done. In addition, some aspects that were positive for some students have the potential to be negative for others; for example, while some appreciate increased autonomy, others may need more support.

There is variation across the universities as to who can or is required to attend the RCS. Some programs are undergraduate entry (the number is decreasing) while others are graduate entry. The latter tend to attract students who are older and more likely to have family commitments that mean spending an extended period of time rurally is more problematic. Such social issues have been identified as being important and need continual attention¹², including data from the larger FRAME study¹³.

The RCSs have been breaking new ground in demonstrating that specialist curricula can be taught in rural environments, and not necessarily only by specialists¹⁴, and using new technologies to overcome distances for teaching¹⁵. Both of these developments are important for the universities, for the rural doctors who are supported by these measures, and most importantly, the rural patients who will benefit in the longer term.

Not all universities with RCSs chose to participate in the survey, which was a voluntary undertaking. However the approximately 75% response rate from those that did participate was remarkably good, although there was significant variation in response rate across the universities. It is clear that at some universities not all eligible students received questionnaires. This was partly due to fact that most RCSs had multiple campuses, this was the first time that multi-university questionnaires had been distributed, and there was some misunderstanding about who should receive the questionnaire. It is possible that the respondents were primarily those who had positive experiences, although it could have been expected that those who had significantly negative experiences would also respond, however this does not seem to have been the case. However the RCS program only commenced in 2002 and students in this study completed their training when the RCSs had no academic reputation or track record.

Fewer students are now 'conscripted' to RCSs. For example in 2006 one RCS had 64% of students conscripted while for 2009 the RCS was over-subscribed (D De Witt, pers. data, 2010). In addition one study has shown that RCS medical students perform at least as well as their metropolitan clinical school colleagues in formal assessments¹⁶, and the University of Western Australia RCS received an Australian Learning and Teaching Council award in 2007 for innovation in curricula, learning and teaching.

Several recent reports have shown that RCS students are more likely to chose a future rural career^17,18. In the future, original work by the FRAME investigators and the Medical Student Outcome Database will allow more complete data collection and an examination of longitudinal trends at a national level.

Kamien was one of the earliest advocates for rural medical training in Australia in 1975¹⁹. He suggested the concept of community medical training (at that time in a rural Aboriginal community) with the aim of balancing the bias of hospital-based training, and to broaden the student perspective, widen career choice, provide insight into the needs of socially disadvantaged groups and increase understanding of the wider role that the doctor can play in the community. It appears that the RCS program is on track to achieve these goals.

Conclusions

Students' experiences in the RCSs are unequivocally positive. Most importantly, the RCS environment was conducive to learning and the development of clinical skills, the students were able to see an adequate number of patients and were well-prepared for examinations, and their supervisors were very good and acted as positive role models. This augers well for the success of the RCS program and for their role in attracting future doctors to work in rural environments.

Acknowledgements

This study was funded by the Australian Government Department of Health and Ageing. It was authored on behalf of the FRAME Investigators and the authors acknowledge the support of the following FRAME members, and their staff at the RCSs, who agreed to allow, and facilitated, their students' involvement in this study: Professor Louis Pilotto (University of New South Wales), Professor Judi Walker (University of Tasmania), Associate Professor Joseph Canalese (University of Sydney), Associate Professor Jonathan Newbury (University of Adelaide), Professor Amanda Barnard (Australian National University).

References

1. Australian Institute of Health and Welfare. Medical labour force 2007, National Health Labourforce Series no 44. Canberra, ACT: AIHW, 2009.

2. Australian Institute of Health and Welfare. Rural, regional and remote health Indicators of health status and determinants of health, Rural Health Series no 9. Canberra, ACT: AIHW, 2008.

3. Department of Health and Ageing. Health workforce programs.(Online) 2010. Available: http://www.health.gov.au/internet/main/publishing.nsf/Content/work-prog (Accessed 17 January 2010).

4. Department of Health and Ageing. Rural clinical school program. (Online) 2010. Available: http://www.health.gov.au/internet/main/publishing.nsf/Content/work-st-rcs (Accessed 17 January 2010).

5. Department of Health and Ageing. Evaluation of the University Departments of Rural Health Program and the Rural Clinical Schools Program. Final Report. Canberra, ACT: Commonwealth of Australia, 2009.

6. Eley DS, Baker PG. Will Australian rural clinical schools be an effective workforce strategy? Early indications of their positive effect on intern choice and rural career interest. Medical Journal of Australia 2007; 187(3): 166-167.

7. DeWitt DE, McLean R, Newbury J, Shannon S, Critchley J. Development of a common national questionnaire to evaluate student perceptions about the Australian Rural Clinical Schools Program. Rural Remote Health 5:486. (Online) 2005. Available: http://www.rrh.org.au (Accessed 17 January 2010).

8. Medical Deans Australia and New Zealand. Medical Schools Outcome Database. (Online) 2010. Available: http://www.health.gov.au/internet/main/publishing.nsf/Content/work-prog (Accessed 17 January 2010).

9. Worley P, Silagy C, Prideaux D, Newble D, Jones A. The parallel rural community curriculum: an integrated clinical curriculum based in rural general practice. Medical Education 2000; 34(7): 558-565.

10. Lyon PM, McLean R, Hyde S, Hendry G. Students' perceptions of clinical attachments across rural and metropolitan settings. Assessment & Evaluation in Higher Education 2008; 33(1): 63-73.

11. Denz-Penhey H, Murdoch JC, Lockyer-Stevens VJ. 'What makes it really good, makes it really bad.' An exploration of early student experience in the first cohort of the Rural Clinical School in the University of Western Australia. Rural and Remote Health 4: 300. (Online) 2004. Available: http://www.rrh.org.au (Accessed 17 January 2010).

12. Jones GI, DeWitt DE, Elliott SL. Medical students' reported barriers to training at a Rural Clinical School. Australian Journal of Rural Health 2005; 13(5): 271-275.

13. Krahe LM, McColl AR, Pallant JF, Cunningham CE, DeWitt DE. A multi-university study of which factors medical students consider when deciding to attend a rural clinical school in Australia. Rural and Remote Health 10: 1477. (Online) 2010. (In production).

14. Worley P, Strasser R, Prideaux D. Can medical students learn specialist disciplines based in rural practice: lessons from students' self reported experience and competence. Rural Remote Health 4:338. (Online) 2004. Available: http://www.rrh.org.au (Accessed 17 January 2010).

15. Hays R, Veitch J, Lam A. Teaching clinical pathology by flexible delivery in rural sites. Australian Journal of Rural Health 2005; 13(4): 232-235.

16. Worley P, Esterman A, Prideaux D. Cohort study of examination performance of undergraduate medical students learning in community settings. BMJ 2004; 328(7433): 207-209.

17. Worley P, Martin A, Prideaux D, Woodman R, Worley E, Lowe M. Vocational career paths of graduate entry medical students at Flinders University: a comparison of rural, remote and tertiary tracks. Medical Journal of Australia 2008; 188(3): 177-178.

18. Eley D, Baker P, Chater B. The Rural Clinical School Tracking Project: more IS better--confirming factors that influence early career entry into the rural medical workforce. Medical Teaching 2009; 31(10): e454-459.

19. Kamien M. Education in community medicine with an emphasis on the health of an Aboriginal community: a pilot project. Medical Journal of Australia 1975; 2(13): 509-513.