Pilot project and evaluation of delivering diabetes work-based education using video conferencing

full article:

Context

The incidence of diabetes, both type 1 and type 2, within Scotland is continuing to rise and it is evident that there is a pressing need for healthcare professionals to have the necessary skills and knowledge to provide patients with adequate care, treatment and advice¹. The Diabetes Framework Action Plan presented by the Scottish Executive in 2006 outlined the key delivery targets required in order to provide a cohesive and comprehensive service for those with diabetes, with concurrent requirements for the development of skills and knowledge for the workforce². In response to the Diabetes Framework Action Plan, the Highland Diabetes Managed Clinical Network (HDMCN) established the skills and knowledge gap that existed in relation to the care and treatment of people living with diabetes and the existing skills and knowledge of healthcare and allied healthcare workers involved in the management of diabetes.

The geographical spread of NHS Highland is approximately 33 000 km². Critical to the Diabetes Framework Action plan is the objective of establishing services within local areas to increase access to services and ensure that effective care is delivered as locally as possible. Within NHS Highland, this presents additional challenges in terms of ensuring a skilled workforce. Attending relevant training to develop currency of practice can be costly in several ways, including the financial outlay required to provide adequate cover of service (backfill), the time required for travel and the personal costs in terms of childcare and other domestic arrangements. The HDMCN confirmed that these issues were significant barriers for National Health Service (Scotland) Highland Health Board professionals.

The use of videoconferencing (VC) can overcome barriers imposed by geographic dispersal. Videoconferencing has been used in education to good effect, although the curriculum covered has largely been knowledge-based^3,4. More recently, there has been some limited use of VC to deliver skills-based training with good results^5-7. Based on these positive previous findings and in order to circumvent the barriers of access, the use of teleconferencing was identified by the HDMCN as a useful format for delivering a course of diabetes skills and knowledge training.

The specific aim of this pilot project was to deliver diabetes training to healthcare professionals across NHS Highland who had a role in providing basic level care for, and management of, people with diabetes. In order to address the geographical barriers to training, it was decided to pilot the use of VC to deliver the training after establishing the format initially by face to face (F2F) training.

Issue

Educational package and delivery

A trainer was employed to develop the materials, match the materials to a knowledge and skills framework (NHS KSF⁸) and deliver the training through VC. The trainer was provided with training on both the technical aspects of using a VC suite and the skills of lecturing using VC⁸.

The content of the educational package was constructed in accordance with the demands of the Diabetes Framework Action Plan 2006 and informed by steering group members consisting of clinical specialists in diabetes care, the the Consultant Diabetologist and an educational specialist from the Health Service nursing education and teaching department. It was compartmentalized into four categories:

1. Blood glucose monitoring
2. Foot screening
3. Diet, weight and exercise
4. Cardiovascular risk reduction.

The four categories were delivered over two half-days. This was in response to the difficulties in releasing staff for a full day. Further, good practice in delivering training or lecturing through VC also takes into account the problem of VC fatigue. Each training element included didactic delivery of knowledge and skills, practical demonstrations, opportunities for discussion and a question-and-answer session.

Posters to advertise the workshop were sent to managers who were responsible for implementing policy for the management of people with long-term conditions; as well as to practice settings within the Highland Health Board area. Packs of learning materials were provided that gave background information about each of the key areas covered. Students were informed that this was a pilot project that was being evaluated and they were provided with participant information sheets about the nature of the evaluation. All participants were invited to complete the questionnaires but could opt out.

The trainer delivered the training to the first two cohorts F2F. Training to other sites was delivered through VC point-to-point, with one site connecting to the central training delivery site in any given delivery. The showing of DVDs, electronic presentations (Microsoft PowerPoint), discussions with the course tutor and demonstrations all took place through the medium of the VC suite. Two sessions were affected by technical difficulties, resulting in delays to delivery of training. In all cases, assessment of learning was undertaken one week following training delivery. The trainer travelled to the site to deliver the assessment in a F2F environment. The assessment used a simulated patient. Evaluation questionnaires were distributed at the end of the training.

Evaluation

Semi-structured questionnaires were used to gain feedback on attitudes to VC delivery as well as training content in addition to the application of training in practice settings. The locations of participants, delivery style of education and the number of questionnaires received from participants are provided (Table 1).

Table 1: Participant outcomes related to location and delivery style

Feedback

Participants attended training in order to:

• upgrade and modernize skills and knowledge
• gain confidence in working with clients who have diabetes.

The content was seen as relevant and useful by all participants; the video on foot screening was noted as being particularly useful. All participants noted that the use of VC provided accessibility to training where distance, cost and other issues would have made this difficult.

It was quite locally available; plenty of VC facilities; having it locally makes it easier; I wouldn't have gone to it if it had been further out.

Some respondents had experienced dropped VC connections and commented on the waste of time that this entails, the unpredictability of VC and the lack of expertise available to fix VC problems. Other disadvantages noted were 'stilted conversations', 'inability to interact', and 'feeling a bit distant'. However, two participants commented that VC was a welcome surprise.

Better than I thought it would be.

Most rated the quality as only moderate. Regardless of mode of delivery, participants unanimously felt that the circulated educational material had adequately prepared them for the training, and that the training itself enabled more confident practitioners, was useful to practice and easy to apply. The percentage of assessments passed (81% F2F vs 80% VC) illustrates that there was no difference in the delivery and retention of skills and knowledge between those who received the training F2F and those who received the training by VC.

Lessons learned

Feedback from participants indicates that the VC facilitated access for geographically peripheral participants who would otherwise have had difficulties attending the training. Overall, the content of the training was well received. Some students commented on the restricted interactivity within a VC environment and felt that this compromised their enjoyment of the training and their feeling of engagement. This is not an unusual response to VC delivery. Many people who participate in education through VC note that it has capacity to handle message generation but that it deters normal participation in discussions^3,6,9,10. While students recognized the value of training through VC, some noted that they would have preferred F2F training. A previous study has noted that the convenience of minimizing travel and loss of work time will compensate for the potential for communication disruption¹¹.

Some students commented that there was only moderate picture and sound quality. These views are not unique and a previous study using VC to teach clinical skills within the regional hospital also illustrated that while students may have been aware of the potential value of VC delivery, they felt that sound and picture quality needed to be much more reliable¹². While VC may have restricted the enjoyment of the training, the educational impact was not adversely affected and equitable results on assessments were recorded for those who attended the VC training and those who attended the F2F training.

There is considerable previous evidence that delivery of training through VC requires a subset of specific skills on the part of the trainer^7,13. Earlier evaluations have concluded that trainers require development sessions to enable them to transfer their skills to a VC environment⁶. Interactivity can be more difficult to manage through VC, although it is still feasible; as noted, it requires that the trainer undertake training in order to adapt delivery methods. Different techniques can help to enhance interactivity through VC, such as preparing students prior to a session so that they take turns in presenting aspects of the session themselves⁶; establishing eye contact between participants and trainers through suitable room arrangements¹¹; and ensuring the trainer provides handouts prior to the session⁶. Training through VC also requires that students and the trainer take regular breaks in order to avoid eye strain and fatigue¹⁴.

The evaluation of this pilot study was limited by the small numbers of deliveries of the project and hence the limited number of students and educational participants who supplied information. It would have been advantageous to have received feedback from practitioners who chose not to undertake the training in order to illustrate whether negative preconceptions of VC were acting to dissuade professionals from attendance. The use of VC has at times been dogged by the use of inadequate VC equipment and users and trainers not suitably trained in the specific skills of teaching/meeting through this environment¹². Inevitably, through real as well as through hearsay from others' experiences of VC, there is a perception that it is necessary but not as desirable or effective as F2F communication¹⁵. This perception may inhibit some from participating in this platform of learning.

This project was a pilot to test the viability of delivering diabetes education and skills training through VC. Given that participants performed well in assessments, the project illustrates the capacity of disseminating clinical skills training through this medium. The Highland Health Board are interested in a wider expansion of the delivery, while necessarily paying attention to the limitations already mentioned in this report.

Conclusion

Lessons from this evaluation can be added to the overall portfolio of clinical skills training evaluations. The current project illustrates that VC can alleviate the difficulties of accessing training for clinical professionals who are geographically peripheral and there is potential to capitalize on the knowledge and expertise of specialists^14,16. Practical skills training is feasible through VC, although feedback indicates that the use of this medium reduces the interactive nature of the learning and teaching experience. Further, the cost of travel and of the administration needed to set up central training venues is reduced, although this needs to be offset against the cost of the VC facility and the administration and training required to undertake this type of teaching and learning³. Trainers need to be suitably trained in order to deliver content in a medium-specific format. Importantly, while enjoyment of the education may be modified by the use of VC, it does not disrupt student learning. Given the value of accessing education remotely, the use of VC demands further attention to ensure robust educational delivery that can also enable interactivity.

Acknowledgements

Astra Zeneca provided a sponsorship grant for this independent program and the education facilitator post was funded with a grant from NHS Education Scotland.

References

1. Scottish Diabetes Survey Monitoring Group. Scottish Diabetes Survey 2010. (Online) 2010. Available: http://www.diabetesinscotland.org.uk (Accessed 20 June 2012).

2. Scottish Executive. The Scottish Diabetes Framework Action Plan 2006. (Online) 2006. Available: http://www.scotland.gov.uk/Publications/2006/06/12111211/0 (Accessed 28 June 2012).

3. Ferrell M and McGrath I. The use of desktop video conferencing in enhancing the quality of teaching and learning for undergraduate nursing students in rural Victoria. In: Proceedings, HERDSA Conference; 7-10 July 2002; Perth, WA; 2002.

4. Kerns A, McDonough J, Groom J, Kalynych N, Hogan G. Televideoconferencing: is it as effective as 'In person' lectures for nurse anesthesia education? American Association of Nurse Anesthetics Journal 2006; 74(1): 1-3.

5. Moorman S. Prof-in-a-box: using internet-videoconferencing to assist students in the gross anatomy laboratory. BMC Medical Education 6:55. (Online) 2006. Available: http://www.biomedcentral.com/1472-6920/6/55 (Accessed 20 July 2011).

6. Spitzer F, Kives S, Ornstein M, Caccia N, Stephens D, Flood C et al. Videoconferencing for resident teaching of subspecialty topics: the paediatric and adolescent gynaecology experience at the Hospital for Sick Children. Journal of Pediatric and Adolescent Gynecology 2008; 21: 343-346.

7. Augestad K, Lindestmo R. Overcoming distance: video-conferencing as a clinical and educational tool among surgeons. World Journal of Surgery 2009; 33: 1356-1365.

8. Department of Health. National Health Service Knowledge and Skills Framework and the Development Review Process (October 2004). (Online) 2004. Available: http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_4090843 (Accessed 20 June 2010).

9. Ho D, Hu P, Carmack D, Hayda R, Pohl A, Dunbar R et al. Design and evaluation of international video teleconference for orthopaedic trauma education poster presentation. In: Proceedings, AMIA Symposium; 11-15 November; Washington, DC; 2006.

10. Allen M, Sergeant J, Mann K, Fleming M, Premi J. Videoconferencing for practice-based small group continuing medical education: feasibility, acceptability, effectiveness and cost. Journal for Continuing Education in the Health Professions 2003; 23: 38-47.

11. Harris V, Smith A, Armfield N. Education for regional health professionals using mobile videoconferencing. Telemed Telecare 2007; 13: 44-47.

12. Barry N, Reid M, Ibbotson T, Bower DJ. Telemedicine diffusion in Scotland: training and technical support issues. Journal of Health Care Computing 2000; 17(5): 20-23.

13. Bertsch T, Callas P, Rubin A, Caputo M, Ricci M. Effectiveness of lectures attended via interactive video conferencing versus in-person in preparing third internal medicine clerkship students for clinical practice examinations (CPX). Teaching and Learning in Medicine 2007; 9(1): 4-8.

14. Miller P, Huijbregts M, French E, Taylor D, Reinikka K, Berezny L et al. Videoconferencing a stroke assessment training workshop: effectiveness, acceptability, and cost. Journal of Continuing Education in the Health Professions 2008; 28(4): 256-269.

15. Richards H, Kinga G, Reid M, Selvaraja S, McNicol I, Brebnerd E et al. Remote working: survey of attitudes to eHealth of doctors and nurses in rural general practices in the United Kingdom. Family Practice 2005; 22: 2-7.

16. Talbert J-A. Distance education: one solution to the nursing shortage? Clinical Journal of Oncology Nursing 2009; 13(3): 269-270.