download PDF
full article:
Introduction
In July 2000, the Australian College of Rural and Remote Medicine (ACRRM) was awarded a contract by the Commonwealth Department of Health and Ageing to develop a Quality Assurance (QA) and Continuing Medical Education (CME) radiology program for rural and remote non-specialist doctors. The program was in response to new Australian legislation that required doctors who had applied to the Health Insurance Commission (HIC) for a Remote Area Exemption (RAE) to participate in ongoing radiology professional development. The RAE enables rural and remote doctors who practice in isolation from specialist radiologists, and often radiographers, to read and report on more complex X-rays than urban and provincial GPs. The patients of RAE doctors are also able to access Medicare insurance for cost recovery of these complex X-rays. The program was developed in close collaboration with the target group of doctors and was based on research into their educational and quality assurance needs1,2. Based on feedback from the doctors, the program was developed to be flexible and accessible for rural and remote doctors who are often unable to leave their practices to attend educational events in distant cities.
The ACCRM QA/CME program required rural doctors to obtain 30 radiology QA/CME points over a 4 year period. At least 15-20 of these points had to be obtained by one of two mandatory options of the program: either: (i) film interpretation, report and review clinical audit activity; or (ii) a radiology clinical attachment. The first option required doctors to use a proforma which provided space for doctors to record the de-identified details of 15 X-ray films and then compare their findings with an expert radiologist. Recorded on the proforma was the date of review, type of film (eg chest, cervical spine), radiographic findings including history, relevant clinical information, impressions and conclusions/diagnosis/findings and radiologist comments.
The second option was to use a logbook which enabled doctors to record their own individual learning objectives for the attachment, describe their activities during the attachment, and provide details of three of their own clinical cases where they used education obtained during the attachment to enhance patient management.
Doctors submitted completed film review forms or logbooks to the program manager as confirmation of their educational activity to receive their QA/CME. This article describes the activities undertaken and provides preliminary self-reported indicators of the impact of this program in subsequent practice.
Method
Data from film review forms and clinical attachment logbooks were de-identified and entered into two Microsoft EXCEL spreadsheets. The data were grouped, categorised and analysed in an EXCEL spreadsheet. Only data from the period January 2001 to September 2004 were included in this study.
Results
Film interpretation, report and review results
During the study period, 419 doctors completed a radiology film interpretation, report and review activity. Eighty-three were female (19.8%).
The doctors reviewed their films at a variety of venues including private and public radiology clinics, hospitals, GP surgeries and radiology workshops. Seventy-five doctors performed an individual film review with a radiologist while 344 took part in a group film review with a radiologist and their generalist colleagues. Four education providers developed innovative ways for their more remote doctors to participate in a film review activity. Three divisions of general practice worked collaboratively with the ACRRM radiology program manager and radiologists to develop CD-ROMs of X-ray films which were posted to doctors to review on their own computers. The doctors then met via a live videoconference link-up with the specialist radiologists in capital cities to discuss the films they had reviewed. Forty-five doctors completed more than one film review, including 10 who completed three film reviews and 35 who completed two reviews during the program.
After completing the film review activity doctors recorded three changes they planned to make in their radiology practice. The main areas of planned change are shown (Table 1).
Table 1: Planned changes to doctors' radiology practice after a film review activity (n = 419)*
Radiology clinical attachment results
During the study period, 41 doctors completed a radiology clinical attachment. Six were women (14.6%). The radiology clinical attachments were performed in 31 radiology practices, which included public and private hospitals and private radiology clinics in seven Australian states. Doctors spent from 2 to 50 h on their radiology clinical attachment with a mean of 10.3 h. Each participant recorded specific learning objectives prior to their clinical attachment. The learning objectives were grouped and categorised and are listed (Table 2).
Table 2: Doctors' specific learning objectives for their clinical attachment (n = 41)*
The doctors were asked to describe the activities of their radiology clinical attachment. Forty doctors 'sat in' to review and discuss films a radiologist was reporting on. Five doctors also discussed films they took to the attachment. Five participants observed radiological procedures such as CT, MRI and ultrasound.
The doctors were also asked to describe the factors that assisted or hindered their learning in these attachments. Helpful factors included the approachability and friendly attitude of the radiologists, the personal 'one-on-one' teaching format, the wide variety of radiology cases, good equipment, the opportunity to ask 'silly' questions privately, obtaining an update on the latest radiological techniques and investigations, and the enthusiasm, patience and encouragement of radiology staff. One rural Queensland doctor wrote:
The staff at Royal Hobart Hospital are very experienced in teaching doctors in my situation, since they provide doctors doing attachments at remote Antarctic stations.
One of the rural doctors stated she found advance reading material on basic radiology interpretation and anatomy helpful, as was the attitude/teaching/knowledge of the specialist radiologists. The Picture Archive and Communication System (PACS) used at the Townsville hospital to digitally enhance plain films was mentioned, as well as the systematic approach and teaching skills of radiologists.
Factors that hindered the doctors' learning experience during the clinical attachment included the heavy workloads of the radiology departments and radiologists. Two doctors felt they 'got in the way' at times in the busy radiology clinics. There were interruptions when the radiologists were called away for procedures, and some activities were not relevant to the doctors' rural practice because they were too 'high tech'. One doctor wrote that he could 'only cram so much in my head in one day'.
Seven of the rural doctors' out of date anatomy was a hindrance. Ten doctors wanted more time for their attachment, with many planning to organise additional future attachments. One doctor arranged to return to the radiology department of a hospital on a Monday morning so he could sit in with the radiologist reading the football injury X-rays from the weekend. This doctor had a particular interest in sports injuries because he worked on ski slopes during the winter months. Six doctors stated they did not see enough radiology in their areas of interest (eg facial and skull X-rays, or spinal X-rays). Table 3 summarises what the doctors said they learned during their radiology clinical attachment.
Table 3: What doctors said they learned during their radiology clinical attachment (n = 41)*
Of the participants, 87.5% stated the clinical attachment improved their existing clinical and practical skills. In addition, 85% stated they had gained additional knowledge and 60% said they had learned new clinical and practical skills during the attachment. Table 4 outlines desired further learning doctors identified as a result of their radiology clinical attachment.
Table 4: Desired further learning identified as a result of the radiology clinical attachment (n = 41)*
The rural and remote doctors stated that their clinical attachment had helped them to gain an appreciation of the technology of a modern X-ray unit; it gave them more confidence. One doctor was disappointed at the obvious lack of clinical examination and inadequate clinical notes on request forms. This doctor also queried the number of requests with findings which made no difference to the clinical management of the patient. He felt this was understandable regarding litigation but that there must be a more sensible response that could be agreed by doctor and patient discussion. One doctor said he had gained a newfound interest in radiology following his attachment. The rural doctors were amazed and impressed with how quickly radiologists read and gave an opinion on films.
The attachment gave some of the doctors an appreciation of their own radiology strengths and weaknesses. One doctor found he was quite competent with trauma films, but not so competent with chest X-rays. Another doctor realised many of his films were poorly developed. One doctor said he obtained radiology education on the attachment by 'osmosis', and one said he learned to report on the whole X-ray and not just the primary pathology.
As part of the clinical attachment activity doctors were required to report on three cases from their own clinical practice within 3 months of the attachment. Some examples of these cases demonstrate how the knowledge and skills developed during the attachment were used to improve and enhance radiology practice and patient care.
Fourteen doctors reported improved confidence in reading and interpreting X-rays which helped them make more confident clinical diagnoses. As a result, three doctors elected to treat patients locally instead of transferring them to larger centres. Some examples included:
Case: Male involved in a fight presented some days later and hung over with a very swollen face. Because of the discussion with Dr L, I was confident he had no maxillary #'s, and could be treated locally.
Case: Male MCA presented with multiple injuries but no LOC or neck pain. He was however in a cervical collar. Lateral Cx spine X- ray showed good alignments and no soft tissue damage. Because of my discussion with Dr L, I was confident to remove the collar.
Case: Road traffic accident with cervical spine injury. I was confident in diagnosing a C5/C6 anterior dislocation.
Seventeen doctors noted improvements in distinguishing normal and abnormal variants, and there were 54 mentions of interpreting chest X-rays more easily in the cases reviewed and the lessons learnt from the attachment. The following examples are typical of cases described by the doctors in their clinical attachment logbooks.
Case: Neck X-ray of 8 yr old, C1 & C2 appeared subluxed but this is a normal variant in children. I learnt this in my attachment.
Case: Hip X-ray of patient with groin pain after a fall. Systematic examination detected degenerative changes and excluded fracture.
Sixteen doctors reported a general enhancement of their radiological skills, particularly in using a systematic approach to interpreting films and the use of pneumonics, schemas, signs and markings for different regions of the body, for example:
Case: Mr S C age 36 - pleuritic chest pain with SOB. Borderline increase in heart size. Attachment helped to clarify how exactly to determine if heart size is truly enlarged or not.
Case: Refreshed awareness and understanding of silhouette signs. Better understanding of looking at vascular markings in the lungs.
Case: Child 5 year old female fell off play equipment and presented with a swollen elbow. X-rays revealed normal alignment and the centres of ossification were where expected and consistent with Capitulum, Radial Internal (medial) epicondyle Trochlear, Olecranon, External (lateral) epicondyle (CRITOE), which Dr L had discussed. The child was therefore treated with a sling.
Two hundred and thirty doctors completed more than their required amount of radiology education and 165 still have to complete the requirements. The program enlisted 92 radiologists which shows the program has been successful in facilitating collaboration between rural doctors and their specialist colleagues, an important outcome of the program. Many doctors said how beneficial it was for them to get to know their radiologist colleagues personally.
Discussion
There were limitations to this educational evaluation research. Impact evaluation of an educational program can be demonstrated by measuring changes in knowledge, attitudes or practice which can be done by pre- and post-knowledge tests and pre- and post-competency tests3. It was not possible to use this type of evaluation in this program due to limited resources but it would be beneficial to do this in subsequent evaluations of the program.
Participating doctors did subjectively report and document changes to their levels of knowledge, confidence and practice after participating in one of the core program activities. The highest level of educational evaluation is the measurement of improved patient outcomes from an educational program which could be measured by an audit of actual radiology practice of the doctors4 in a future study.
This paper describes the evaluation of the radiology program after the first 3 years and 9 months and shows there are a large number of rural and remote Australian doctors undertaking professional development and quality assurance activities in radiology.
The main achievement of the ACRRM radiology program was the development of a flexible, accessible, rural friendly radiology program which has been accepted by rural and remote doctors. It addressed the requirements rural doctors asked for in the national radiology educational needs assessment1. Doctors asked for a program that was case-based, clinically relevant, practical, and had a variety of options. The program has strengthened the collaboration with radiology education providers and diagnostic imaging organisations. The national radiology program does have implications for other rural continuing medical education programs as it demonstrates a model of education which could be utilized in other rural education programs both within Australia and elsewhere.
The key lessons learned in the development and evaluation of this program were to base the educational program on research into the educational needs of the target group, then to work closely with rural and remote doctors and radiologists to develop a program which met those identified needs. It was important to use opinion leaders (respected rural and remote doctors) and to make programs accessible locally to enhance local ownership.
Conclusion
The evaluation of this program does demonstrate changes in the amount and variety of radiology education undertaken by rural and remote doctors since the program began, and some self-reported changes in radiology knowledge, confidence and practice. The ACRRM radiology program has been accepted by rural and remote doctors and radiologists. It has now been incorporated into the College's Professional Development Program. This will greatly assist its ongoing and sustainability.
Acknowledgement
The authors would like to acknowledge the financial contribution of the Commonwealth Department of Health and Ageing for the ACRRM Radiology Program from July 2000 to December 2004.
References
1. Glazebrook R, Chater B, Graham P. Rural and remote Australian general practitioners' educational needs in radiology. Journal Continuing Education in the Health Professions. 2001; 21: 140-149.
2. Glazebrook R, Smith T. Continuing education needs of rural and remote general practitioners in radiology and radiography. The Radiographer 2001; 48: 73-76.
3. Hays R, Veitch C (Eds). Continuing medical education for general practitioners. Townsville, QLD: School of Medicine, James Cook University, Australia, 1999.
4. Pitts J, Percy D, Coles C. Evaluating teaching. Education for General Practice, 1995; 6: 13-18.