Evaluation of nine pilot obstetric ultrasound education workshops for Australian rural and remote doctors

full article:

Ultrasound is becoming more popular with a variety of health professionals around the world. One of the main reasons for this is because ultrasound instruments have become smaller, less expensive and easier to use¹. In Australia, rural and remote doctors are using ultrasound to assist them with the diagnosis and management of their patients. Many of these doctors work in areas where there is no access to specialist radiologists, sonographers or obstetricians.

The Australian College of Rural and Remote Medicine (ACRRM) was funded by the Commonwealth Department of Health and Ageing during 2002-2003 to develop a national obstetric ultrasound education program for Australian rural and remote medical practitioners. The program was developed following a national educational needs assessment completed in 2001 that showed there was a large unmet need for basic obstetric ultrasound education within this target group². Ethics approval for the needs assessment on which the workshops were based was obtained from James Cook University Human Ethics Committee.

This article describes the implementation and evaluation of nine interactive pilot obstetric ultrasound workshops, which were developed by ACRRM to address some of the unmet educational needs. The CME literature shows that interactive workshops can result in changes to knowledge or skills, but that didactic lectures alone are unlikely to change professional practice³. Cantillon and Jones cite the most effective methods of CME are those linked to clinical practice and include, interactive education meetings, outreach events and strategies that involve multiple education interventions⁴. A review of the literature surrounding obstetric ultrasound education for non-specialist doctors shows that family physicians who are well trained can reliably perform obstetric ultrasound examinations for the majority of indications relevant to their practice^5-9. The authors of the present article were unable to find specific literature on education and training for rural doctors on obstetric ultrasound, although there is a large literature on rural continuing medical education. Ultrasound was not mentioned in two recent Australian reviews of rural medical education^10-11. This could be because only a small number of rural doctors are becoming interested in ultrasound. In the June quarter of 2001, ninety-five non-specialist doctors billed Medicare for obstetric ultrasound services (Australian Health Insurance Commission data, Department of Diagnostic Imaging, direct comm., September 2001).

Methods

Nine two-day obstetric ultrasound workshops were planned and held across the country between March 2002 and April 2003. Each workshop involved doctors learning and practising basic obstetric ultrasound skills on volunteer pregnant models in all stages of pregnancy, using lower-end ultrasound machines loaned by ultrasound machine companies. Thirty to thirty-five pregnant women were recruited locally by antenatal clinics, private obstetricians and general practitioners for each workshop.

ACRRM recruited two expert sonographers to be the main sonographic teachers at the workshops and contracted five to six extra local sonographers for each workshop. The nine workshops required the following resources: 45 sonographers, 10 obstetricians, 250 pregnant models and 44 portable ultrasound machines.

Most of the workshops were held in the day surgery areas of regional hospitals at the workshop sites on weekends. The only two capital-city based workshops were held in the ultrasound department of King Edward Memorial Hospital, a large tertiary teaching Women's hospital in Perth, Western Australia.

During the workshops, doctors were taught the physics of ultrasound and how to use the ultrasound machine controls to get the best image (knobology) and minimise artifacts. Topics covered included:

• Introduction to basic ultrasound.
  
• Practical scanning techniques in first, second and third trimester of pregnancy.
  
• Biometry, including: sac size, Crown-Rump Length (CRL), Biparietal Diameter (BPD), Femur Length (FL), Abdominal Circumference (AC), Head Circumference (HC) and Amniotic Fluid Index (AFI).

Each workshop also included the Royal Australian and New Zealand College of Obstetricians and Gynaecologists' (RANZCOG) Ultrasound - threatened miscarriage in the first trimester of pregnancy practice improvement project. This 2 h session was delivered by a RANZCOG obstetrician ultrasound sub-specialist. This part of the workshop covered early embryology, first trimester ultrasound anatomy, ectopic pregnancy, miscarriage, multiple pregnancy, nuchal translucency measurement, scanning techniques and abnormal pregnancies.

During the workshops, participants were required to complete two pre- and post-knowledge tests and two general evaluation forms. The first of these knowledge tests consisted of 17 true or false, and multiple choice questions covering physics and knobology (the term commonly used by sonographers to describe how to use the 'knobs/controls' on an ultrasound machine), and first, second and third trimester scanning. The second pre- and post-knowledge test covered the RANZCOG session and required the doctors to answer 24 multiple choice questions on early pregnancy scanning and to mark the correct place for placing the cursors with first trimester measurements and assessments. The questionnaires were matched and analysed in SPSS (SPSS Inc; Chicago, USA) using a paired sample t-test. Participants were also asked to rate the overall quality of the workshop they attended.

All doctors who attended an ultrasound workshop were also offered a follow-up assessment visit by an experienced sonographer within 2 months of the workshop where their obstetric ultrasound competency skills were assessed using their own ultrasound machines on their own patients in their own rooms or local hospitals. The competency assessment form was developed with the assistance of one of the expert sonographers and assessed the following:

• Overall system operation: patient data entry, pre-set selection, depth, focus, Time Gain Compensation (TGC), dynamic range, zoom/scroll, overall gain, annotation.
  
• First trimester scanning: transducer (tx) manipulation, access windows, sizing of display, image longitudinal uterus, curser placement, use of correct measurement planes and fetal CRL.
  
• Second Trimester Scanning: tx manipulation, access windows, sizing of display, display cervix/placental edge, assess fetal position, use of correct measurement planes, curser position, BPD, HC, AC, FL and fetal heart rate.
  
• Third Trimester Scanning: Tx manipulation, access windows, sizing of display, assess fetal position, use of correct measurement planes, curser position, BPD, HC, AC, FL, fetal heart rate, AFI, use of basic colour & Doppler controls (if available on machine) and cord Doppler measure (where available).

Competency standards were developed for each of these points and assessed by the visiting sonographer.

Results

The mean age of the 141 workshop participants was 43.8 years. Attendees included 24.1% females (n = 34) and 75.8% males (n = 107). The mean length of time the participants had been a medical practitioner was 17.8 years and the mean length of time in rural practice was 11.7 years. Only 25% of participants had received previous education or training in obstetric ultrasound even though 51% said they were already practising obstetric ultrasound. Of the doctors, 67% had never had any previous education in obstetric ultrasound and 7% did not respond to this question.

The doctors who attended one of the workshops included 10.7% urban/regional doctors, 55.3% rural doctors and 34% remote doctors. Of participants, 96% were primary care doctors and 4% were specialist obstetricians.

Doctors reported an increase in their levels of confidence during the workshops (Fig 1).




Figure 1: Participants' self-rated levels of confidence before and after the obstetric ultrasound workshop.

When these data were broken down into specific skills, most doctors reported confidence in all areas except for AFI (Table 1). The doctors indicated they still needed a lot more practice, supervision and experience and that not all patients were ideal (eg due to body size).

Table 1: Participants' self-reported confidence in performing fetal measurements after a workshop



The doctors were also asked to list three things they did not learn which they would still like to learn. The replies included transvaginal scanning (15 doctors), color Doppler (6 doctors), early pregnancy scans (6 doctors), how to recognise an abnormal fetus (such as abdominal wall defects, neural tube defects; 4 doctors), miscarriage (3 doctors), how to find the cervix (2 doctors), tips for buying an ultrasound machine (2 doctors), ectopic pregnancies (2 doctors) and cardiac anomalies (2 doctors).

A small number of doctors also wanted to learn about other types of ultrasound including gynaecological scanning, Deep Vein Thrombosis (DVT) evaluation, endometrial thickness for menorrhagia and HRT, postpartum use, general ultrasound, biopsy/trauma parts, locating and examining the ovaries, evaluation of renal calculi/hydronephrosis (non-obstetric), and musculoskeletal ultrasound.

Many of the doctors reported that they planned to use their new skills on a day-to-day basis in their medical practice. Examples included using ultrasound to perform early dating scans, assessing their pregnant patients presenting with pain and bleeding to determine the cause, assessing the viability of the fetus, and assisting in antenatal clinic and intrapartum decision making. Seven doctors specifically mentioned using ultrasound in remote Aboriginal communities to assist with dating pregnancies. They reported that their new skills would prevent women having to be transferred out of the remote communities which should save health-system costs. They also indicated that it would save patients the inconvenience of having to leave home and travel long distances.

Some doctors suggested that they would now be able to appreciate ultrasound reports better and help train medical and nursing staff in their own organizations. They would also use their new knowledge to improve the quality of their documentation and to more intelligently order and interpret ultrasound scans.

Thirteen doctors planned to look at buying their own ultrasound machine or to try and persuade their local rural hospitals to buy one and use it in maternity, labour ward and accident and emergency areas.

Results for pre- and post-knowledge tests

There were complete matched results for 114 doctors in the first pre- and post-knowledge test. The reason for this was because the general workshop pre-knowledge test was only introduced after the first two workshops. Results for these 114 doctors showed an increase in knowledge from a mean pre-score of 13.9 to a mean post-score of 15.5 (p = 0.001 Fig 2).



Figure 2: ACRRM obstetric ultrasound pre- and post-knowledge test results - overall workshops: the numbers of doctors who obtained a specific score out of the possible total 17 correct answers.

Assessment of the RANZCOG section

The RANZCOG section of the workshops on early first trimester threatened miscarriage also included a pre- and post-knowledge test, and 136 doctors completed both pre- and post-tests. The paired sample t-test results showed a mean pre-test score of 9.56 and a mean post-test score of 15.12 (p = <0.001 Fig 3) for this part of the workshops.



Figure 3: RANZCOG pre- and post-knowledge test results: the numbers of doctors who obtained a specific score out of the possible total 24 correct answers.


Typical comments on the general evaluation forms included, 'I learned that basic ultrasonography doesn't take months to learn', (I learned) 'to find things logically rather than serendipitously', 'I realised that the use of ultrasound in rural clinical practice has enormous potential and feel confident in going ahead to obtain an ultrasound machine'.

Follow-up assessment

All the doctors who attended a workshop were offered a follow-up competency assessment visit in their own communities using their own ultrasound machine with their own patients by a qualified sonographer within several months of the workshop. Up to December 2003, 40 doctors had had their follow-up visit and the visiting sonographer had returned a completed competency assessment form to ACRRM. The reasons given by some doctors who declined a visit included they didn't have access to an ultrasound machine or they were not doing obstetrics anymore. Some of the doctors said they would arrange to have their visit at a later date after they had obtained an ultrasound machine or upgraded their current ultrasound machine.

The majority of the doctors were assessed by the sonographers as being competent in first trimester scanning. Of the doctors, 90% were competent in imaging the longitudinal uterus, 87.5% were competent in curser placement and sizing of the display, 85% in using the correct measurement planes, 80% of the doctors were competent in Tx manipulation and assessing windows and 77.5% were competent in measuring the fetal CRL. The clinicians had limited themselves to their scope of practice. The results from second trimester assessment showed the doctors required further ongoing skills development. Only between 30% and 60% were competent in areas of second trimester measurements.

Thirty-one of the 40 doctors (77.5%) who had a competency assessment visit could recognise the fetal position in third trimester competently and could measure the BPD, 70% could measure the HC accurately, 77.5% the FL and 72.5% the AC. Of the doctors, 72.5% could size the display competently and use the correct measurement planes (75%). Only 35% of the follow-up doctors were competent in using the cursor position, 50% were competent in measuring the fetal heart rate and 60% in measuring the AFI. At the time of writing, the full evaluation of the follow-up assessment has not been completed.

Discussion

Australian rural and remote doctors who attended a two-day practical obstetric ultrasound workshop did show a statistically significant improvement in knowledge. They also reported an increase in confidence in using obstetric ultrasound techniques. The workshop participants planned to use their new knowledge and skill in their own practices and rural hospitals with pregnant women in antenatal clinics and in emergency situations in labour ward and emergency departments.

This study did have some limitations because there was no control group, and the change in knowledge could be attributed to memory rather than learning. The educational literature suggests that changes in knowledge do not always lead to changes in practice¹².

The follow-up competency assessment visit did measure the actual ultrasound skills of these doctors. The authors understand that the best way to measure a change in practice is to measure the practice before and after the educational intervention or use a matched control group, but due to the small numbers of doctors in rural areas and a lack of funding for pre-workshop assessment or control group, this was not feasible.

The follow-up competency assessments documented the need for ongoing education for participants. Since the workshops, ACRRM has developed a teaching obstetric ultrasound CD ROM, DVD, two training videos and a practical obstetric ultrasound manual. It has also produced two national obstetric ultrasound satellite broadcasts which covered some of the topics identified in the national educational needs assessment and the needs identified in the evaluation reports from the nine pilot obstetric ultrasound workshops. The College has also produced an online obstetric ultrasound module which includes a 'case of the month' section with a new case each month. Doctors can join in an online discussion forum and have obstetric ultrasound questions answered by a qualified sonographer moderator. A description and evaluation of these enduring educational resources will be the subject of a future article.

Conclusion

This study has shown that an intensive educational intervention can successfully address pre-determined educational needs of Australian rural and remote doctors. The preliminary follow-up competency data suggests that such an intervention is only the first step, and further practice is required to achieve competency. The intervention has encouraged participants to pursue ongoing education in this important clinical area.

Although it is expensive, due to distance and the logistics of visiting remote areas, the authors recommend observation of the skills of health professionals in their actual practice as being the best way to evaluate skills-based workshops for rural and remote health professionals.

Acknowledgements

The authors would like to thank the Diagnostics and Technology Branch of the Australian Commonwealth Department of Health and Ageing for funding to support the ACRRM obstetric ultrasound program. Also thanked are the district hospitals for allowing ACRRM to hold the workshops in their hospitals, the ultrasound industry for loaning lower-end machines, the sonographers and obstetricians who took part in the workshops and the ACRRM obstetric ultrasound steering committee. The authors would especially like to thank the 250 pregnant women who volunteered to be models to assist rural and remote doctors to enhance their skills in obstetric ultrasound.

References

1. WHO Study Group. Training in diagnostic ultrasound: essentials, principles and standards. Geneva: WHO, 1998.

2. Glazebrook R, Manahan D, Chater B et al. Educational needs of rural and remote Australian non-specialist medical practitioners for obstetric ultrasound. Australian Journal of Rural Health 2004; 12: 73-80.

3. Mazmanian PE, Davis DA. Continuing medical education and the physician as a learner. Guide to the evidence. JAMA 2002; 288: 1057.

4. Cantillon P, Jones R. Does continuing medical education in general practice make a difference? BMJ 1999; 318: 1276-1279.

5. Ornstein SM, Smith MA, Peggs J, Garr D, Gonzales J. Obstetric ultrasound by family physicians. Adequacy as assessed by pregnancy outcome. Journal of Family Practice 1990; 30: 403-8.

6. Hahn RG, Roe LD, Ornstein SM et al. Obstetric ultrasound training for family physicians. Results from a multi-site study. Journal of Family Practice 1988; 26: 553-558.

7. Macmillan R, Prislin MD, Orientale E, McConnell M, Hahn RG. Family practice obstetric ultrasound in an urban community health centre. Birth outcomes and examination accuracy of the initial 227 cases. Journal of Family Practice 1990; 30: 163-168.

8. Keith R, Frisch L. Fetal biometry: a comparison of family physicians and radiologists. Family Medicine 2001; 33: 111-114.

9. Rodney WM, Deutchman ME, Hartman KJ, Hahn RG. Obstetric ultrasound by family physicians. Journal of Family Practice 1992; 32: 186-194, 197-200.

10. Hoyal FM. Skills and topics in continuing medical education for rural doctors. Journal of Continuing Education in the Health Professions 2000; 20: 13-19.

11. Booth B, Lawrance R. Quality assurance and continuing education needs of rural and remote general practitioners: how are they changing? Australian Journal of Rural Health 2001; 9: 265-274.

12. Davis D, Thomson MA, Oxman AD, Haynes RB. Changing physician performance. A systematic review of the effect of continuing medical education strategies. JAMA 1995; 274: 700-705.