Is small beautiful? Student performance and perceptions of their experience at larger and smaller sites in rural and remote longitudinal integrated clerkships in the Rural Clinical School of Western Australia

full article:

Introduction

The Commonwealth Government of Australia established a series of programs in the hope and expectation of improving the recruitment and retention of rural doctors, both GPs and rural specialists^1,2. Several models of rural medical education have been reported, including the Parallel Rural Community Curriculum² and various community based, patient-centred, longitudinal medical curricula^3,4. In Western Australia, where there are no large rural population centres or hospitals, an integrated Community Learning in Rural Communities program (CLERC)^5,6 has been developed, where students in the fifth year of a six-year undergraduate course spend a full academic year in a rural and remote setting.

Whenever an innovation is introduced into medical education it is incumbent on the faculty to evaluate the changes to ensure there are no unintended consequences and, if any are detected, to immediately rectify such problems⁷. Evaluation in medical education largely focuses on two aspects: providing assurance that the innovations do not impact negatively on the student learning as shown in the end of academic year marks; and assessing student perceptions of the climate of the educational environment^8-10, including the competence of teachers and teaching^11,12.

There is an increasing interest in sending students on rotation to rural environments¹³ to experience rural medicine. Rural settings for medical education have a complexity that leads to a unique curriculum delivery¹⁴. In the past, many of these rotations were very short, with as few as 2 weeks but a common time frame of 8 weeks, with fewer students undertaking 6 month to 1 year rotations^5,6,15-17. While each rotation is important, a single rotation of a few weeks is unlikely to be critical in a student's academic career. However, it is very different when, as in this case, the rural experience is for a continuous academic year.

Background

The Rural Clinical School of Western Australia (RCSWA) has been operating since 2003 with 25% of Western Australia's medical students having a year-long integrated clinical clerkship. Over that time a total of 245 students have completed the clerkship. Students live and work in 10 different sites in groups from 3 to 10 across the state of Western Australia, with some sent as far as 2400 km and others only 200 km. The host towns range from 4500 to approximately 36 000 population.

Whenever there is innovation in the medical curriculum students become anxious⁷. This was the case in the first year of the RCSWA, especially for students who knew that being sent to a small site meant they would have little contact with the specialists they believed necessary for a good learning experience^1,18. They were concerned that they would be disadvantaged in their learning.

Purpose of the evaluation

The data were collected for the overall evaluations of the School^18,19. However, it was important to know if those students at the smaller sites were disadvantaged by the smaller numbers of student colleagues, the smaller population catchment area and the potentially smaller number of different clinical presentations. It was also important to know if students were disadvantaged by being taught in an environment with few, and sometimes no specialists. For the purpose of comparison the natural division was between the 4 sites with a population greater than 30 000 and the 6 sites with a population of less than 20 000.

Methods

Mixed methods were used. Ethics approval was sought from the University Ethics Committee but was deemed unnecessary because this was part of ongoing evaluation of standard university courses.

End of year results were compared with student colleagues in the city (the state capital, Perth). Annual comparisons of end of year results for the years 2003-2006 were undertaken because all students undertook the same assessments and examinations. Means were compared annually and cumulatively for the 2003-2006 cohorts of fifth year undergraduate medical students, comparing metropolitan trained students, those students who undertook training at sites with populations of less than 20 000 and those at sites with populations of greater than 30 000. In 2007, the RCS WA students entered an assessment program that was different from their metropolitan trained counterparts; thus, a direct comparison between the rural and urban students could not be made for 2007-2009 cohorts.

Qualitative interviews were undertaken annually from 2003 to 2007 to identify subjective experiences. In 2007 the Dundee Ready Education Environment Measure (DREEM) survey was introduced to identify student perceptions of teaching and learning.

The qualitative interviews were taped, transcribed and analysed according to qualitative methods. Open-ended semi-structured questions were used and participants were given as little or as much time as they wished to speak. Questions each year related to curriculum content, curriculum delivery, site specific issues and personal concerns. Additional data was sought from staff by email and individual conversation after the sharing of the analysis annually, and this was added for ongoing analysis. Data were analysed using the constant comparative method of qualitative methods. Rigour was ensured by linking a range of validity and reliability checks suitable to qualitative methods.

The DREEM survey was used in 2007 and 2008. It contains 50 validated questions trialled across multiple cultures and countries and found to be valid^10,20,21. Questions are answered using a 5 point Likert scale and scored 4, 3, 2, 1, 0 - the higher the score the better. Roff quoted a range of studies which indicated that the average scores varied from 78 to 139 out of 200 with the ability to 'profile' a particular institution's strengths and weaknesses, and to make comparative analyses of students' perceptions between environments²¹. Results were compared between students at larger and smaller sites.

The populations and distance from the metropolitan medical faculty for each of the sites are given (Table 1).

Table 1: Population figures for Rural Clinical School of Western Australia sites, from Australian Bureau of Statistics 2007 estimates



Results

Results of the annual end of year marks for students showed a statistically significant increase of 1.3 and 1.4 marks in comparison with their city trained counterparts. There was no significant difference between students trained at sites with larger populations and those with smaller populations (Table 2).

Table 2: Comparison of student examination marks according to site size and location, 2003-2006



In the annual mid-year interviews (qualitatively analysed) with students training at the smaller sites reported a slightly stronger identification with their sites and indications that they were more enthusiastic about their teaching and learning opportunities, even when teaching conditions were less than ideal. These students were more self-directed and expressed a greater degree of self-efficacy. Some staff had difficulty with such qualitative findings and the DREEM was used so that a more quantitative statement could be made.

In 2008 the DREEM survey results were presented with the overall mean score for the RCSWA of 150 (Table 3). The findings from the DREEM subscale data supported the qualitative findings that the students perceived they had great opportunities for learning and had a stronger sense of personal efficacy.

Table 3: Comparison of large and small sites according to total Dundee Ready Education Environment Measurement (DREEM) scores and subscales



Discussion

Previous studies have shown that rural students do as well in their end of year examinations as those who attend metropolitan medical schools^5,22. However, there remains an ongoing concern to identify any subsets of the overall group who are disadvantaged. It is clear from the annual end of year examinations that students in the RCSWA were not disadvantaged, either by attendance at the RCS or by being rotated to smaller sites.

It is of interest that over the years, at mid-year evaluation interviews, students rotated to smaller sites have consistently reported satisfaction in their teaching and learning opportunities, despite the potential disadvantages of smaller populations, fewer cases passing through clinics and the hospital, and fewer or no specialists involved in teaching.

It has been established that issues and concerns identified in the annual interviews and the DREEM survey are similar²³. This is confirmed by the present study, where the DREEM results of positive feelings, already identified in the qualitative interviews, were also reflected in the numerical data.

The DREEM questions related to both clinical (including bedside) teaching and academic and tutorial teaching. Staff at the small sites (3-5 students each year) focused on getting their systems 'right' according to student evaluations; however, staff at the larger sites reported less flexibility to do this for various reasons.

The larger sites had a more rotational approach to students' clinical experience, similar to that of metropolitan rotations. There were more specialists available and, hence, more specialist input into teaching. At these sites there was a more structured approach to time (eg students having to be at a certain place at a particular time) and fewer opportunities for self-directed learning¹⁹.

Students at the smaller sites had to focus on whatever clinical cases presented, both in general practice and at the hospital. It was likely that the same doctor (and student) would manage the patient from the presenting clinic and through all the treatment. Both the academic and clinical education were largely taught by generalists. Due to a lack of competition for cases, students usually had access to all the odd, different and the technically challenging cases that presented in their area. This gave the students a strong sense of identification with their sites and a sense of belonging to the healthcare team. As a result students talked about flexibility, teamwork and contribution, rather than structure.

Learning at small sites has been shown to preserve and encourage patient-centered attitudes, compared with declining values for students undertaking the usual fragmented training²⁴. Thus, small-site learning may be the best model of integrated longitudinal clerkship experiences.

Conclusion

Learning was different in a rural environment, and even more so at smaller and more remote sites^25,26. However, with good infrastructure, clarity about learning objectives and a structured academic approach to the complexities of the first full clinical year's curriculum, this study shows that students need not be disadvantaged by being sent in small numbers to small and/or remote sites for their clinical education. This has been established academically with respect to students' end of year marks, and also from their reported subjective experiences.

References

1. Denz-Penhey H, Murdoch JC, Lockyer-Stevens V. 'What makes it really good, makes it really bad.' An exploration of early student experience in the first cohort of the Rural Clinical School in the University of Western Australia. Rural and Remote Health 4: 300. (Online) 2004. Available: www.rrh.org.au (Accessed 18 August 2010).

2. Worley P, Silagy C, Prideaux D, Newble D, Jones A. The parallel rural commmunity curriculum: an integrated clinical curriculum based in rural general practice. Medical Education 2000; 34: 558-65.

3. Sturmberg J, Reid A, Thacker J, Chamberlain C. A community based, patient-centred, longitudinal medical curriculum. Rural and Remote Health 3: 210. (Online) 2003. Available: www.rrh.org.au (Accessed 18 August 2010).

4. Margolis S, Davies L, Ypinazar V. Isolated rural general practice as the focus for teaching core clinical rotations to pre-registration medical students. BMC Medical Education 2005; 5(22): doi:10.1186/1472-6920-5-22.

5. Maley M, Denz-Penhey H, Lockyer-Stevens V, Murdoch JC. Tuning medical education for rural-ready practice: designing and resourcing optimally. Medical Teacher 2006; 28(4): 345-350.

6. Maley M, Worley P, Dent J. Using rural and remote settings in the undergraduate medical curriculum: AMEE guide no. 47. Medical Teacher 2009; 31: 969-983.

7. Till H. Climate studies: can students' perceptions of the ideal educational environment be of use for institutional planning and resource utilization? Medical Teacher 2005; 27(4): 332-337.

8. Roff S. The Dundee Ready Educational Environment Measure (DREEM) - a generic instrument for measuring students' perceptions of undergraduate health professions curricula. Medical Teacher2005; 27(4): 322-325.

9. Roff S, McAleer S. What is educational climate? Medical Teacher 2001; 23(4): 333-334.

10. Roff S, McAleer S, Harden R, Al-Qahtani M, Ahmed A, Deza H et al. Development and Validation of the Dundee Ready Education Environment Measurement (DREEM). Medical Teacher 1997; 19(4): 295-299.

11. McOwen K, Kogan J, Shea J. Elapsed time between teaching and evaluation: does it matter. Academic Medicine 2008; 83(10): S29-S32.

12. Gerbase M, Bermond M, Nendaz M, NV V. When the evaluated becomes evaluator: what can we learn from students' experiences duing clerkships? Academic Medicine 2009; 84(7): no pp.

13. Snadden D. From medicine's margins - Why all the interest in rural and remote education? Medical Teacher 2009; 31: 967-968.

14. Page S, Birden H. Twelve tips on rural medical placements: what has worked to make them successful. Medical Teacher 2008; 30: 592-596.

15. Voelker R. Medical students reach out to rural communities in ' WWAMI Land'. JAMA 2003; 290(4): 452-453.

16. Eley D, Young L, Wilkinson D, Chater A, Baker P. Coping with increasing numbers of medical students in rural clinical schools: options and opportunities. Medical Journal of Australia 2008; 188(11): 669-671.

17. Tesson G, Curran V, Pong R, Strasser R. Advances in rural medical education in three countries: Canada, the United States and Australia. Education for Health 2005; 18(3): 405-415.

18. Denz-Penhey H, Murdoch JC. 'It's really, really good, but it could be a lot better'. Qualitative evaluation of a Rural Clinical School, four years on. Medical Teacher 2009; 31(10): e443-e448.

19. Denz-Penhey H, Murdoch JC. Organising a clinical curriculum in rural settings: implementing quality control. Internet Journal of Medical Education. (Online) 2010. Available: http://www.ispub.com/journal/the_internet_journal_of_medical_education/volume_1_number_1_74/article_printable/organising-a-clinical-curriculum-in-rural-settings-implementing-quality-control-1.html (Accessed 18 August 2010).

20. Varma R, Tiyagi E, Gupta J. Determining the quality of educational climate across multiple undergraduate teaching sites using the DREEM inventory. BMC Medical Education 2005; 21: 8.

21. Roff S. Education environment: a bibliography. Medical Teacher 2005; 27(4): 353-357.

22. Worley P, Esterman A, Prideaux D. Cohort study of examination performance of undergraduate medical students learning in community settings. BMJ 2004; 328: 207-210.

23. Denz-Penhey H, Murdoch JC. A comparison between findings from the DREEM questionnaire and that from qualitative interviews. Medical Teacher 2009; 31(10): e449-e453.

24. Bell S, Krupat E, Fazio S, Roberts D, Schwartzstein R. Longitudinal pedagogy: a successful response to the fragmentation of the third-year medical student clerkship experience. Academic Medicine 2008; 83(5): 467-475.

25. Denz-Penhey H, Murdoch J. Rural learning is more than marks: sensitised to knowledge. Medical Teacher 2008; 30(8): 781-786.

26. Denz-Penhey H, Murdoch JC. A student view of the difference between general practice and rural and remote medicine. Rural and Remote Health 7: 641. (Online) 2007. Available: www.rrh.org.au (Accessed 18 August 2010).