Rural general practitioner preceptors - how can effective undergraduate teaching be supported or improved?

full article:

High quality teaching and assessment encourages reflective thinking and effective application of skills, knowledge and experience to novel problems¹. In addition, positive learning experiences based on good student-preceptor relationships^2-4 are a vital component of educational approaches to redress the well documented chronic and worldwide problem of recruiting and retaining appropriately prepared rural and remote GPs.

The majority of general practice teaching within the undergraduate medical program of the University of Tasmania, Australia, occurs in the final year. A significant proportion of this teaching is provided by rural General Practitioner (GP) preceptors who, as well as performing a significant teaching function, are responsible for apportioning 20% of the mark medical students receive in the General Practice examination at the end of the rotation.

Contractual requirements of The University of Tasmania ensure that all academic staff undertake a formal course of tertiary teaching training (provided through the Faculty of Education) within the first year of their appointment. However, despite their important educational responsibilities, this does not apply to GP preceptors, who are classified as honorary teaching staff.

To redress this omission, and in accordance with the stringent modern standards of Australian Medical School Accreditation⁵, a tripartite approach was taken by the Discipline of General Practice and the University Department of Rural Health to develop educational skills among rural GP-preceptors and promote effective undergraduate teaching. The program, POPPIES, had three distinct elements:

1. Undergraduate teaching evaluation.
   
2. Rural GP-preceptor needs assessment.
   
3. Interdisciplinary Workshops for undergraduate preceptors, based on the above research findings.

Funding assistance was provided through the Committee for University Training and Staff Development (CUTSD), a subgroup of the Commonwealth Department of Education, Training and Youth Affairs (DETYA), to conduct a pilot program over an eighteen-month period, commencing January 2000. Thereafter the UDRH was proposed to continue this initiative as core business.

Methods

Discipline of General Practice undergraduate teaching evaluation

Year 6 (final year) medical undergraduate feedback was obtained by the use of a questionnaire to evaluate the educational value and quality of GP-preceptor teaching during the 3-week rural, and 3-week urban (6 weeks in total) GP attachments within the Discipline of General Practice 'Community Health and General Practice' program.

Participants: All 64 final-year medical students who undertook a general practice attachment in one of four groups during the year 2000 were required to complete GP-preceptor evaluation questionnaires following the attachment. Sixty-four rural and 76 urban GP preceptors were assessed.

Survey tool: A horizontal visual analogue scale from 1 to 10 was used to obtain student perceptions of seven items. An area was provided for comments in each section (Appendix I).

Discipline of General Practice rural GP-preceptor needs assessment

A survey form and covering letter detailing the project was forwarded to 60 rural GP preceptors identified using past student placement lists.

Participants: The 60 GP-preceptor respondents had a representative geographical disbursement, were predominantly male (M : F = 3:1) and of mature age (36-55 years).

Survey tool: The survey consisted of three distinct sections:

1. The first section was designed to reflect the profile of rural GP preceptors. This section also provided data to determine the level, qualification and professional experience relevant to clinical teaching, and the level of staff-development activity that had been undertaken to prepare the teacher for the teaching role.


2. The second section consisted of 18 statements specifically related to rural clinical teaching. Respondents were required to rank their opinions in the form of "strongly agree/agree/ neutral/disagree/strongly disagree".


3. The third section presented three open-ended questions designed to reflect GP-preceptor educational and support needs. The final open-ended question provided the opportunity for participants to provide general comments related to the survey content.

University Department of Rural Health, Interdisciplinary Workshops for undergraduate preceptors, based on the above research findings

Workshops: Based on the findings of the GP-preceptor questionnaire survey, two CME-accredited education and support workshops were planned and implemented onsite at six rural health locations simultaneously throughout Tasmania (Smithton, Queenstown, Launceston, St Marys, Dover and Flinders Island in Bass Strait; Figure 1). The workshops were planned and facilitated by a multidisciplinary project-management team drawn from the Schools of Medicine, Nursing and Pharmacy, and supported by a project advisory committee of representatives from both the Faculty of Health Science and clinical professional bodies.



Figure 1: The six rural sites in Tasmania, Australia, for POPPIES education and support workshops.

The first workshop took place before students undertook their rural rotations. This one-day workshop provided an overview of health professional education in Tasmania and introduced the philosophical perspectives of the Schools. The pivotal educational focus was on the notion of students as active participants in their learning during their rural experience. The main topics discussed were:

• Supportive learning environments


• Learning domains


• Principles of adult learning


• Facilitating learning with poor performers


• Assessment and evaluation


• Performance feedback


• Rural preceptor roles

At this initial workshop, attending preceptors were encouraged to interact with each other and the facilitator throughout the sessions. Emphasis was placed on the sharing of ideas, experiences and individual specific concerns or views. The final session of this workshop was a 'brainstorming' discussion that focussed on the most suitable structure and content for the second workshop. Subsequent workshop content incorporated issues raised in this initial program.

The final workshop was designed to serve as the forum for academics from all three disciplines to discuss the many issues that arose during the rural Preceptor Onsite Preparation Program for Information Education and Support (POPPIES) program in order to address the needs of rural preceptors.

Results

Discipline of General Practice undergraduate teaching evaluation

The results of the student survey are shown (Table 1). While the median score for each section was comparatively high, ranging between 9 and 10, there was a wide distribution of scores. Teaching, feedback, accommodation and vocational encouragement scores varied between 1 and 10; consultations and variety of learning opportunities between 2 and 10; and skills acquisition between 4 and 10. The quality of surgery premises and attitudes of staff received more consistent scores, of between 5 and 10.

Table 1. Undergraduate teaching evaluation: medical students' feedback on rural GP-preceptors' performance



With regard to teaching quality, while 38% of preceptors received a perfect score of 10, 17% were rated between 1 and 7. Similarly, although 35% of preceptors scored 10 for quality of feedback provision, 11% were given a mark of 6 or below. Comparing best with worst scores for skills, consultations and variety of experiences, the pattern seen with teaching and feedback was repeated, with 30-35% of preceptors receiving a mark of 10, while a similar number were rated from 7 to as low as 2.

However, with regard to the important issue of the degree to which students were encouraged by their attachment experience to pursue a career in general practice, 70% of rural GP-preceptors scored 8 or above. On the basis of the very positive perceptions indicated by the majority of medical students regarding their rural GP attachments, it is to be hoped, however, that these may translate into increased future rural career choices.

Discipline of General Practice rural GP-preceptor needs assessment

Over half the respondents had over 20 years of clinical experience, with almost the same number having over 10 years teaching experience. Only 1 in 4 preceptors had completed professional development activities related to teaching. Further results are presented in Table 2. Despite the lack of formal preparation for teaching responsibilities, the majority of respondents indicated they felt confident when precepting, understood what students needed to learn, understood student evaluation, and felt able to set goals and provide appropriate performance feedback. In contrast most preceptors did not know how the teaching they provided fitted into the overall curriculum. Approximately half the preceptors also felt their role as a teacher and an assessor was undefined. Interestingly, nearly all respondents felt that clinicians could not automatically teach, and only half indicated they understood the principles of adult learning. The majority of respondents appeared comfortable with information technology, although a small minority appeared averse to email, videoconferencing, and the use of the internet for academic purposes.

Table 2. Rural health undergraduate preceptor support and training needs assessment



The seven distinct themes that emerged from the free text responses of the rural GP-preceptors follow.

1. Curriculum content: Preceptors expressed their need to be more familiar with specific clinical aims, and also emphasised the importance of having documentation which clearly defined learning objectives, goals and expected outcomes. The preceptors felt this was necessary for them to provide quality and concentrated experience and guidance for medical students in the appropriate areas.

2. Increased formal academic support: This was requested in the form of face to face contact, particularly during student rotations. Many preceptors voiced their concern at the apparent unavailability of academics for on-the-spot advice to assist in dealing with potentially very problematic situations that arise during student placements. The common scenario cited concerned the need for increased assistance and support in dealing with struggling or failing students.

The main recommendation for addressing this issue was the suggestion to improve existing information technology services for both clinicians and students. Preceptors expressed the need for academics to be easily accessible via email while students were in practice settings. Other preceptors felt it was important to have direct contact with academics, either in person, through videoconferencing or by telephone.

A consistent request from preceptors was for the provision of on-going annual staff development activities to enable quality teaching. Other GPs felt they needed additional support in terms of teaching resources, such as improved university library access, prescribed texts and videos.

3. Collaborative communication: Many preceptors felt the communication between the university, the students and themselves could be markedly improved. Some GPs suggested that predetermined student goals be set and documented prior to the commencement of clinical rotations. Information was also requested concerning students' prior academic performance. Failing and struggling students caused preceptors a good deal of stress and concern, and they felt they should be made aware of problematic students.

Concern was voiced that during past student rotations, curriculum changes had occurred of which preceptors had not been apprised. They were uncertain of what was expected of them by the university, and commented on the need for students to be better informed concerning the requirements of their placements, prior to arrival.

4. Increased rural health focus: A popular request by preceptors was for increased focus in the curriculum to accommodate specific rural health issues. Many GPs felt a large proportion of their time was spent highlighting differences between urban and rural general practice.

5. Formal Feedback: A large proportion of preceptors felt it was important to hear student feedback regarding their experiences with them. Performance feedback was regarded as necessary to identify strengths and weaknesses to facilitate their own professional development and improvement as teachers.

6. Minimum standards: Many preceptors voiced concern that they were not completely sure of the students' minimum standards. They also expressed concern about dealing with difficulties associated with poorly performing students, and the preceptors were unsure of what action was appropriate in such circumstances.

7. Staff development: In response to being asked what specific information should be included in staff development workshops, eight areas were flagged by preceptors:

• How to assess and evaluate students


• Teaching principles


• How to cope with struggling students


• How to motivate disinterested students


• How to impart experiential knowledge


• Principles of adult learning


• Politics of performance feedback


• Assessment and evaluation

One of the major changes that occurs in medical undergraduate training is the transition from pre-clinical to clinical training which, although variable in timing, is characterised by increasing proportions of experiential learning, with lessening dependence on written or audiovisual materials.

Individual or small group learning contacts with qualified senior colleagues form an integral part of practical clinical training, and in this environment the knowledge, skills, attitudes and personal qualities of those responsible for undergraduate teaching assume considerable importance^1,6.

Given that the four basic principles of assessment are validity, reliability, flexibility and fairness, the effect of poor teaching in a clinical setting has a significant potential to impact on student performance and subsequent evaluation. In this regard, the marked variability of students' assessment of their rural GP-preceptor's teaching performance is a cause for concern in relation to fairness of assessment⁷. Although direct evidence correlating GP-preceptor teaching quality with medical student performance outcomes was lacking, a need to improve and standardise the quality of undergraduate teaching and learning by rural GP-preceptors to avoid potentially unfair assessment processes was clearly apparent.

On the positive side, however, students generally felt encouraged by their GP attachments to pursue a career in General Practice. General practice currently comprises approximately 50% of the Australian medical practitioner workforce⁸, consequently requiring significant on-going high levels of graduate recruitment to maintain present levels of primary health-care delivery. Medical student perceptions in this section of the questionnaire were therefore of major relevance to the key issue of rural GP retention and recruitment and appeared reassuring.

With regard to rural preceptor educational needs, a requirement for major support in regard to improved direct contact and better communication between preceptors and university staff was clearly evident. Professional teaching development, particularly in regard to strategies for dealing with poorly performing students was also frequently requested, as was the provision of specific information concerning the overall curriculum structure and preceptor roles in teaching and assessment. A small minority of preceptors also indicated they were unfamiliar with computer technology.

On the basis of these combined findings and identified needs, the POPPIES was developed and implemented as a series of workshops, to be later supported by web-based information and resources.

Rural Health Undergraduate Preceptor Support and Training: POPPIES workshops

Each of the workshops was very well received, with attendances ranging from six to twelve participants. The workshops that followed the initial brainstorming session, incorporated a significant number of ideas and topics generated by participants of that initial program, in conjunction with issues raised by the rural preceptor needs assessment, not addressed in the first workshop. A half-day program was devised, which explored the following areas:

• The structure of the University of Tasmania Faculty of Health Science


• Rural health teaching sites


• Experiential curricula


• Clinicians as preceptors


• Faculty expectations of students in rural settings


• Challenging relationships


• Action protocol for students of concern


• Learning contracts


• Creating supportive learning environments


• Domains and principles of adult learning


• Some golden teaching rules and methods


• Types and principles of assessment


• Performance feedback

Specific emphasis was placed on the role rural preceptors play in student assessment and evaluation. Opportunities were provided throughout the program for discussion regarding avenues to provide rural preceptors with support during student placements, and, in response to a large number of preceptor requests, strategies to deal with student difficulties such as poor performance, personality clashes, inappropriate expectations or non-attendance.

One important area not specifically addressed in the program was the role of information technology in assisting rural health professionals to improve teaching and assessment skills and provide a high degree of ongoing academic contact and preceptor peer support.

All participants were asked to complete a questionnaire concerning the quality and format of the program. With regard to the first workshop, replies were obtained from 80% of those attending. There was general consensus that workshop facilitators presented the program effectively, and that the issues were relevant to undergraduate teaching and preceptor needs. The program, which ran from 9 am to 4.30 pm, was felt by the majority of respondents to be too long, and lacking in specific teaching strategies (particularly those to use when dealing with struggling or failing students). Based on this feedback, the second workshop was run as a half-day program, and focused strongly on the creation of supportive learning environments, teaching strategies, educational theory and assessment processes.

It was apparent, both from the needs assessment and workshop discussions, that there were wide variations in IT skills and literacy amongst rural health preceptors. Accordingly, the UDRH undertook the development of an educational resource for distribution in hard copy form to all preceptors requiring or requesting IT training. This resource consisted of a 23 page A4 book, entitled 'Booting Up - Computing Skills for Health Professionals' which covered the basic aspects of IT software and Hardware, designed to enable preceptors to access and navigate appropriate online resources, and use relevant applications such as word processing.

No data are yet available concerning participants' evaluations of the second or third workshops, or the response to the IT resource book. However, as a further initiative, the rural POPPIES program is being developed further to include print-based and web-based self-directed modules, supported by on-site workshops. A website is enhancing the academic and peer support infrastructure for rural and remote preceptors, through the provision of information and access to a discussion board. The information presented through the website is also available in print format. It is envisaged the development of a rural and remote preceptor database will result in a more streamlined approach to student placements in these areas, which will ultimately alleviate to easier preceptoring experiences.

Conclusion

A survey of 64 final-year medical students from the University of Tasmania, Australia, who undertook a general practice attachment in 2000, evaluated the performance of their rural GP-preceptor. Although the results of the survey showed a comparatively high median score for the seven items assessed there was a wide distribution of scores.

In a subsequent survey, 60 rural GP-preceptor respondents identified a number of areas in which many felt underprepared to supervise and/or assess students on rural placement. These areas included knowledge of the place of the student placement in the medical undergraduate curriculum; their role as teacher and assessor; the principles of adult learning; and managing information technology.

On the basis of the combined findings of the two surveys, POPPIES was developed and implemented as a series of workshops held throughout rural Tasmania for the support and training of rural GP-preceptors. Although at the time of writing there were no data available to establish improved teaching outcomes as a result of POPPIES, preliminary responses from rural GP-preceptors who attended the workshops indicated that the workshops met their educational needs by providing relevant professional teaching development in their own locality. It was already clear, however, that preceptors' perceptions of support from and communication with the Faculty of Health Science had improved as a result of the program.

References

1. Marsh HW. Students' evaluations of university teaching: research findings, methodological issues, and directions for future research. International Journal of Educational Research 1987; 11: 255-378.

2. Hays RB. Common international themes in rural medicine. Australian Journal of Rural Health 1999; 7: 191-194.

3. Mortimer D. Training model aims to build community links. Medical Observer Rural Report 00/4: 51.

4. Ramsden P. Learning to teach in higher education. London: Routledge, 1992; 217-247.

5. Australian Medical Council. Assessment and accreditation of medical schools: standards and procedures (Ref:AMC349). Canberra: Australian Medical Council, 2002.

6. Lines DR, Forsyth K, Worley PS. Teaching undergraduate paediatrics in a rural community. Focus on Health Professional Education 2000; 2: 25-30.

7. Rumsey D. Assessment: practical guide. Canberra: Australian Government Publishing Service, 1994; 12-27.

8. Madden R, Horvath J. Medical workforce supply and demand in Australia: a discussion paper. AMWAC report, 1998.10. Sydney: Australian Medical Workforce Advisory Committee, Australian Institute of Health and Welfare,1998; 15, 37.