The value of best-practice guidelines for OSCEs in a postgraduate program in an Australian remote area setting

full article:

Introduction

Nurses working in very remote areas of Australia are the mainstay of health services in these regions. Remote area nurses (RANs) provide many aspects of primary health care, including continuous, comprehensive and coordinated health care of individuals and their families across the lifespan within the context of their family and community¹. A rigorous form of practice-based education and assessment is important to support RANs and ensure the provision of quality client services in remote practice.

Remote area nursing is a specialised field of practice. It has been recognised that undergraduate nursing education is not sufficient to prepare nurses for the advanced practice role of working in remote communities². RANs are required to assess, diagnose and treat a range of acute and chronic conditions with limited health professional support².

Ideally, nurses undertake preparation for practice prior to, or shortly after, commencing employment in these areas as they require specialised knowledge, skills and abilities to deliver health services in these areas³. Key elements of practice that have been identified by rural and remote practitioners include advanced clinical skills, health history taking, comprehensive understanding of cultural and social principles, primary health care and community development knowledge and practice, teamwork and management skills^3,4.

The majority (78%) of nurses working in very remote primary health clinics work in Australian Indigenous communities⁵. It is vital that nurses are well prepared and able to engage with their clients in culturally appropriate communication and perform advanced assessment skills. Cultural awareness and sensitivity starts with the individual exploring their own understanding, attitudes and potential biases that underpin behaviours⁶. Further understanding of what is culturally relevant to one's clients requires in-depth knowledge and consideration to enable the demonstration of cultural sensitivity^7-9.

Mentoring, opportunities to learn, and guidance in the performance of nursing work are essential to adequately equip healthcare professionals¹⁰. Opportunities to practise and receive feedback in situations that mimic clinical environments are of paramount importance. In acute care sectors in metropolitan areas, there are generally many opportunities for this to occur. Unlike many health professionals who work in co-located teams, RANs rarely have this opportunity and generally work in isolation. Given the limited opportunity for practice, it is important that RANs have other options to practise and receive feedback of authentic situation.

Objective structured clinical examinations (OSCE) are a type of student learning experience that are widely used throughout health education in Australia and overseas for preparing students to practise and perform practical skills in a controlled environment where there is no threat to patient safety^11-13. An OSCE requires students to demonstrate specific skills and behaviours in a simulated clinical environment with standardised patients or mannequins¹².

Site

Alice Springs is the third largest town in the Northern Territory, with a population of approximately 25 000 people, and is situated in the geographic centre of Australia. The Centre for Remote Health, Alice Springs, is a joint centre between Flinders University, South Australia, and Charles Darwin University, Northern Territory. The Centre's key aim is to provide quality tertiary education, training and research focused on the health needs of people in remote communities of the Northern Territory.

Background

The OSCE is a key component of a core course or subject (Advanced Remote Nursing Practice and Pharmacotherapeutics), which is within a Graduate Certificate in Remote Health Practice as conducted by Flinders University in Alice Springs. This is a Graduate Certificate program designed to prepare health professionals in an advanced practice role in remote areas of Australia. It provides the foundation for advanced nursing practice in the unique environment and cultural context that characterises the inseparable work and living situation of remote health professionals worldwide. The RAN model of consultation is used as a framework for students to learn comprehensive health assessment, history taking, body systems examination, diagnostic investigations, cross-cultural communication, distance consultations and referrals for remote and Indigenous people across the lifespan. Students also develop knowledge and skills in pharmacotherapeutics, appropriate interventions and clinical procedures for management of people of all ages experiencing common, acute, urgent and unstable conditions in isolated, remote and Indigenous contexts.

The course requires the post-registration students to complete a two-week intensive teaching and learning block in Alice Springs or Darwin (Northern Territory), a two-week placement in a remote community and online learning over a 13 week semester. At the end of the two-week intensive period, which is in week 6 of the semester, students complete a formative assessment based on the RAN model of consultation using a simulated patient. This assessment can be successfully achieved using an OSCE if the OSCE is appropriately conducted. Therefore, the team drew on best practice guidelines (BPGs) in the literature to inform the delivery of the OSCE¹¹. This paper reports the results from the Alice Springs site as part of a larger project evaluating the implementation of OSCEs with four diverse student cohorts across four universities.

The aim of this study was to evaluate the effectiveness of the BPGs in providing quality OSCEs for student learning in a remote area postgraduate nursing program.

Modification of the OSCE

In previous years, OSCEs were conducted. These OSCEs were modified from previous iterations according to the BPGs¹¹. A pre-implementation visit was conducted by two of the project team, who worked with the site academic staff to consider how the current OSCEs performed against the BPGs. Table 1 illustrates the modifications that occurred in relation to the seven BPGs following this consultation period.

Table 1: Required modifications of original OSCE in relation to BPGs¹¹

Following the pre-site visit, two scenarios were developed based on the BPGs. Students were required to conduct an assessment of one of the two 'clients' using the RAN model of consultation. This was conducted on a simulated client who was an actor who was well briefed on the scenario and their role. Each scenario was developed with the requirement for clinical skills in assessment (for example, neurological assessment in Scenario 1 and respiratory assessment in Scenario 2), together with psychological and social assessments. Well-developed interviewing skills, good communication and cultural sensitivity were an expectation of both OSCEs. One hour was allotted for each student's OSCE and feedback was given immediately. This was a formative assessment and no actual mark was given but rather comments were made on what the student did well and areas for improvement. Students were asked to reflect (and share with their tutor) on their own assessment of their 'performance'. The five tutors were all experienced with this form of assessment and were fully briefed about the specific scenarios, new procedures and expectations of student attainment. The two scenarios are summarised in the Appendix.

Method

Members of the project team worked closely with course teaching staff to revise the existing course's teaching and learning materials and structure according to seven BPGs¹¹. Evaluation of the revised OSCE (Client Consultation) was conducted by a mixed methods approach with convergence of data from concurrent student surveys, student focus groups and semistructured interviews with staff. Descriptive statistics were used to describe the student sample. Thematic content analysis was used to code the focus group and semistructured interview data¹⁴. The quantitative and qualitative data were converged to better understand the aims of the research, with equal weighting given to both forms of data to inform the study results^15,16.

Sample

Study participants were students and tutors. The students were enrolled in the course (Advanced Remote Nursing Practice and Pharmacotherapeutics) and tutors taught and assessed within the course. The student cohort (n=15) were all registered nurses. They travelled from all around Australia for this study period. The tutors had previously organised and led OSCEs, which enabled reflection on the changes made to the current OSCEs.

Data collection

Student surveys were collected during the students' two-week intensive learning period following the conduct of the OSCE. Consent was implied through completion of the survey. Students deposited their surveys in a labelled box in a central room for collection. The survey consisted of 21 items - five demographic questions, 16 forced-answer questions and one open-ended question.

Students were also invited to participate in focus groups conducted by two of the project team, external to the course, one of whom took notes, Thirteen students formed two focus groups. The questions focused on particular aspects of the OSCE and BPGs and included: 'What did you think were key points of focus for you to display a good performance in the Client Consultation?', 'How did the Client Consultation prepare you for future clinical experience?' and 'What is the value of having the Client Consultation in your course?'.

The focus group notes were read back to the group at their completion to check and ensure that they were a valid summary of the group's thoughts and comments. Changes were made to ensure the accuracy of the representations.

A face-to-face interview was conducted with tutors (n=5) involved in the coordination, teaching and assessment of the OSCE. The interview questions related to processes of the revised OSCEs and tutors' observation of students' participation with the OSCEs. Transcripts of the interviews were sent to the academic for verification. Examples of the questions asked in the interviews are: 'How did the students demonstrate an integrated approach (ie focusing on the whole person, not just clinical skills) to the Client Consultation?', 'Did the student appear prepared for the final Client Consultation in regards to the key safety aspects required?' and 'How does the Client Consultation represent common/and or significant clinical events from practice?'.

Data analysis

Student survey data were entered into the Statistical Package for Social Sciences v20 (SPSS Inc; http://www.spps.com). Descriptive statistics were used to describe the student sample. The narrative data from the open-ended questions/items from the student surveys, student focus groups and the tutor interviews were transcribed verbatim and analysed using content analysis focusing on themes and patterns¹⁷. Codes were developed inductively using an iterative process. Inter-coder agreement was achieved by two members of the research team independently coding the transcripts, which were then discussed to determine whether the coders arrived at the same or similar codes and themes¹⁸. Triangulation was achieved with the convergence of the three separate data sources. Data were checked at the time in the focus groups and member checked a week later with the interviewees. An agreement on codes and themes was negotiated at team meetings in conjunction with the site team member.

Ethics approval

Ethical approval was obtained from the participating universities HREC NRS/12/11, 5295 SBREC prior to project commencement.

Results

Survey data

There were 15 students whose ages ranged from 23 to 60 years with a median age of 35 years. Of these, four were males and 11 were females. The students' previous clinical experience was broad with three students currently practising as RANs. Of these, two had worked in the role for between 2.5 and 7.25 years. The remaining students worked in a range of nursing roles around Australia, including emergency departments, acute care sector and community health.

Fifteen students (100%) completed the survey. All students agreed that an integrated approach to the OSCE was desirable. They focused on the clinical skills but also brought in all aspects of the RAN model of consultation in both their practised and the actual OSCEs, thus demonstrating a holistic client assessment. Students reported that prior to undertaking the OSCE, they had adequate time to practise generic components of the OSCE (but they did not know the actual scenario beforehand). During the practice sessions, students indicated that feedback from teaching staff and peers was both available and helpful. Students were asked to reflect and rate their overall performance and the majority awarded themselves a credit mark (65-74%). Three others gave themselves a distinction (75-84%) and the same number a pass mark (50-64%). One thought they would have failed if the assessment had been summative in nature.

Narrative data

Three themes emerged from the thematic analysis: (1) value of common and significant events in OSCE, (2) power of deliberate actions, and (3) learning cultural sensitivity.

The first theme, 'Value of common and significant events in OSCE', indicated that the students found the OSCE content extended their knowledge and practice beyond their previous levels. Of particular importance was that the OSCE challenged them to explore situations that would be regularly encountered by RANs in their practice. Students verbalised that the majority of skills they currently possessed through their years of experience as registered nurses were currently mismatched with the skills required of a RAN. The following example demonstrates this:

It is a (the Client Consultation/OSCE) holistic model, more holistic in your approach ... in ED you focus on the complaint; in Client Consultation (OSCE) you have to cover everything. (Participant 5)

The second theme was 'Power of deliberate actions'. The OSCE was seen as a valuable learning method and a well-performed OSCE was recognised by the students as the gold standard.

Good solid approach - today with Client Consultation (OSCE) - observer able to point out deficits (that I) need to re-visit to better myself. (Participant 2)

Client Consultation (OSCE) is the gold standard, then can go on from here, it provides a minimum standard. (Participant 8)

Tutors indicated at their interviews that the OSCEs provided important opportunities for students to familiarise themselves with the necessary skills to practise as RANs. They thought this was quite challenging. One said:

Very steep learning curve for people who aren't from RANing (Participant 1 - tutor)

The final theme, 'Learning cultural sensitivity', came through very strongly in the focus groups and interviews. The students had practised developing a rapport, which was very important in the RAN situation and one that was quite different from the situations where they had nursed. They identified that building rapport was important for better understanding clients in the longer term. Students learnt these skills as highlighted by one participant who said:

It starts with building rapport, find something in common to talk about first, then the history taking then hands-on clinical at the end. (Participant 11)

[Need to] draw out the real issue behind the interview. (Participant 3)

These responses that emerged during focus groups and interviews supported the benefits of the OSCE to the students who were grounded in the BPGs. The OSCE was instrumental in providing the opportunity to practise a core element of remote area nursing. As stated by one student (Participant 10):

... the OSCE gave the opportunity to practise and get feedback and reflect on practice in a safe environment that allowed us to attain the skills we needed.

Discussion

The overwhelming response from the surveys, focus groups and tutor interviews indicated that the OSCE modifications according to the BPGs were very worthwhile. Of particular impact in this setting was the value of BPG 2, 4 and 7. The first theme identified, 'Value of common and significant events', as a RAN aligns to BPG 2 'Practices which are most relevant to OSCE learning and assessment and likely to be commonly and/or significantly encountered in practice'. The students indicated that the client consultation framework when practised in common and significant situations was very useful in their approach to assessing the client. This guidance was instrumental in successively stepping them through the everyday challenges that they were presented with in the remote area nursing arena. In addition, the scenarios added complexity that was frequently missed by the student but they acknowledged it provided a trigger for their future learning.

The second theme that identified the power of deliberate actions aligns to BPG 4 'Require students to perform tasks in an integrated rather than piecemeal fashion by combining assessments of discrete skills in an authentic manner'. The specific acts that they rehearsed during the performance of the OSCE, such as opening a consultation (namely, introducing themselves), posing questions (and learning to be silent after the questions were posed), and an appropriate closure of the interaction, were crucial to deriving information from clients that assists towards the appropriate delivery of care. Students commented how important the rehearsal, namely the 'doing' was, as opposed to 'knowing how', which they had learnt prior to the two-week intensive period. They further had the opportunity to practise maintaining a comfortable silence when their prior experience was mostly performing tasks. Active listening can facilitate sharing of cultural needs^19,20 and can result in care being contextualised for individuals^9,21.

Following this process, students verbalised that they felt assured and confident in the practice area. Students felt able to provide high-quality individualised client care through this integrated approach to a client assessment.

The final theme of learning cultural sensitivity aligns to BPG 7 'Allow for ongoing practice of integrated clinical assessment and intervention skills in a safe supportive environment, thereby ensuring the appropriate and timely use of feedback to guide students' development and ongoing reflection'. Diverse student learning was largely attributable to students receiving feedback from their peers in practice, the actor 'client' in the OSCE and the expert tutor following the OSCE. Students commented they had more of a sense of how the client may have felt. Through the actor 'clients' sharing their experience of being examined as a body of interest, the students gleaned some sense of the discomfort of this position. They were better able to appreciate the importance of cultural sensitivity and identified some strategies about how to better understand 'another' perspective⁹. Improving cultural competence is congruent with the Northern Territory's health and community services sector policy of workforce development. The policy recognises that nursing in a cross-cultural environment requires specific applied knowledge and skill²².

This study has both strengths and limitations. The mixed method provided a broad approach to the project aim of evaluating the effectiveness of OSCEs utilising BPGs for student learning in a remote area postgraduate nursing program. The student cohort numbers were small but, importantly, all contributed to the data, thereby reducing selection bias. Only one site was used to examine the perceptions of RANs engaged with OSCEs; therefore, the results are considered to be illustrative and not definitive as the results are dependent on the very specific context and setting. This study contributes to the ongoing evaluation of BPGs for OSCEs, which are widely used in Australia and internationally¹¹.

Conclusions

Participants mostly had limited experience in working in remote area nursing prior to participation and therefore the opportunities that availed themselves were critical in adequately equipping them with the requisite knowledge, skills and abilities. While students were provided with relevant information prior to the two-week intensive course, there was a consistent belief in the value of making sense of this information through the opportunity to practise and receive feedback by way of the OSCEs. The OSCEs were positively evaluated by students and tutors, who valued that the BPGs provided an integrated approach with real-life scenarios with a strong cultural perspective - all important features to the RANs' future success in providing individualised care to clients in remote areas of Australia. Further use of BPGs is recommended with summative OSCE assessments incorporated into the course in the future.

Acknowledgement

This project acknowledges the financial support from the Office of Learning and Teaching Grant - PP10-1785.

References

1. CRANA. Council of remote area nurses Australia. Australian Journal of Rural Health 2003; 11: 107.

2. Lenthall S, Wakerman J, Knight S. The frontline and the ivory tower: a case study of service and professional-driven curriculum. Australian Journal of Rural Health 2009; 17(3): 129-133.

3. CRANA. Submission to the National Health and Hospitals Reform Commission. (Online) 2008 Available: www.health.gov.au/internet/nhhrc/publishing.nsf/Content/073-crana. Council of Remote Area Nurses of Australia (CRANA) Submission 2.pdf (Accessed 6 December 2012).

4. Czulowski L. Forum Consultation Report. CRANA & Flinders University, 1997.

5. Lenthall S, Wakerman J, Opie T, Dunn S, Macleod M, Dollard M, Rickard G, Knight S. The nursing workforce in very remote Australia, characteristics and key issues. Australian Journal of Rural Health 2011; 19(1): 32-37.

6. Campinha-Bacote J. The process of cultural competence in the delivery of healthcare services. (Online) 2002. Available: www.transculturalcare.net/Cultural_Competence_Model (Accessed 10 December 2012).

7. Ellis IK, Davey C, Bradford V. Cultural awareness nurses working with Indigenous Australian people. In: J Daly, D Jackson, S Speedy (Eds). Contexts of nursing. Australia: Elsevier, 2009; 301-313.

8. Leninger MM. Cultural care diversity and universality: a theory of nursing. New York: National League for Nursing Press, 2001.

9. Wilson D. The nurse's role in improving indigenous health. Contemporary Nurse 2003; 15(3): 232-240.

10. Lait J, Suter E, Arthur N, Deutschlander S. Interprofessional mentoring: enhancing students' clinical learning. Nurse Education in Practice 2011; 11(3): 211-215.

11. Nulty DD, Mitchell M, Jeffrey CA, Henderson A, Groves M. Best practice guidelines for use of OSCEs: maximising value for student learning. Nurse Education Today 2011; 31(2): 145-151.

12. Ward H, Barratt J. Assessment of nurse practitioner advanced clinical practice skills: using the objective structured clinical examination (OSCE). Primary Health Care 2005; 15(10): 37-41.

13. Wessel J, Williams R, Finch E, Gemus M. Reliability and validity of an objective structured clinical examination for physical therapy students. Journal of Allied Health 2003; 32(4): 266-269.

14. Creswell JW. Research design: qualitative, quantitative, and mixed methods approaches. California: Sage Publications, 2009.

15. Jones A, Bugge C. Improving understanding and rigour through triangulation: an exemplar based on patient participation in interaction. Journal of Advanced Nursing 2006; 55(5): 612-621.

16. Holloway I. Wheeler S. Qualitative research in nursing and healthcare (3rd ed.) Chichester, West Sussex: Wiley-Blackwell, 2010.

17. Annells M, Whitehead D. Analysis data in qualitative research. In: Z. Schneider (Ed.). Nursing & midwifery research: methods and appraisal for evidence-based practice. NSW, Australia: Elsevier, 2007.

18. Lincoln YS, Guba EG. Naturalistic inquiry. California: Sage, 1985.

19. Chambers-Evans J, Stelling J, Godin M. Learning to listen: serendipitous outcomes of a research training experience. Journal of Advanced Nursing 1999; 29(1): 79-87.

20. Conning AM, Rowland LA. Staff attitudes and the provision of individualised care: what determines what we do for long term psychiatric disabilities? Journal of Mental Health 1992; 1(1): 71-80.

21. Frost M, Green A, Gance-Cleveland B, Kersten R, Irby C. Improving family centered care through research. Journal of Paediatric Nursing 2010; 25: 144-147.

22. Department of Health and Community Services. Aboriginal cultural security: an outline of the policy and its implementation. Available: http://digitallibrary.health.nt.gov.au/prodjspui/bitstream/10137/149/1/Aboriginal%20cultural%20security.pdf (Accessed 9 December 2012).

________________________

Appendix: Client scenarios