Telehealth use in rural and remote health practitioner education: an integrative review

Pauline Calleja; Susan Wilkes; Melinda Spencer; Steven Woodbridge

doi:10.22605/RRH6467

Review Article

Telehealth use in rural and remote health practitioner education: an integrative review

AUTHORS

name here
Pauline Calleja¹ PhD, Head of College Nursing & Midwifery *

name here
Susan Wilkes² GDip Midwifery, Nurse Educator

name here
Melinda Spencer³ BSc (Hons), Research Assistant

name here
Steven Woodbridge⁴ GC IntensiveCareNurs, Clinical Nurse

CORRESPONDENCE

*Prof Pauline Calleja

AFFILIATIONS

¹ School of Nursing, Midwifery & Social Sciences, CQUniversity, Cairns, Qld 4870, Australia; Retrieval Services Queensland, Kedron, Qld 4031, Australia; and School of Nursing & Midwifery, Griffith University, Nathan, Qld 4111, Australia

^{2, 4} Retrieval Services Queensland, Kedron, Qld 4031, Australia

³ School of Nursing & Midwifery, Griffith University, Nathan, Qld 4111, Australia; and Menzies Health Institute Queensland, Griffith University, Southport, Gold Coast, Qld 4222, Australia

PUBLISHED

17 January 2022 Volume 22 Issue 1

HISTORY

RECEIVED: 7 October 2020

REVISED: 25 August 2021

ACCEPTED: 16 September 2021

CITATION

Calleja P, Wilkes S, Spencer M, Woodbridge S. Telehealth use in rural and remote health practitioner education: an integrative review. Rural and Remote Health 2022; 22: 6467. https://doi.org/10.22605/RRH6467

AUTHOR CONTRIBUTIONS

ETHICS APPROVAL

Nil ethics required for review study

This work is licensed under a Creative Commons Attribution 4.0 International Licence

go to url Cited by

download icon download PDF

Tweet this article

abstract:

Introduction:  For rural and remote clinicians, quality education is often difficult to access because of geographic isolation, travel, time, expense constraints and lack of an onsite educator. The aims of this integrative review were to examine what telehealth education is available to rural practitioners, evaluate the existence and characteristics of telehealth education for rural staff, evaluate current telehealth education models, establish the quality of education provided through telehealth along with the facilitators or enablers of a successful service and develop recommendations for supporting and developing an education model for rural and remote health practitioners through telehealth.
Methods:  An integrative review was conducted following the five-stage integrative review process. Searches were conducted in the electronic databases CINAHL, Medline, Nursing & Allied Health (Proquest), PubMed, Johanna Briggs Institute Evidence Based Practice (JBI EBP) and Embase.
Results:  Initial searches revealed more than 7000 articles; final inclusion and exclusion criteria refined results to 60 articles to be included in this review. Included articles were original research, case studies, reviews or randomised controlled studies. Countries of origin were countries in North and Central America, the UK, Europe, and Africa, and Australia and India. One issue noted with this review was classifying rural and remote; contexts used included rural, remote, regional, isolated, peripheral, native communities and outer regional or inner regional.
Sample sizes in the studies ranged from 20 to more than 1000 participants, covering a broad range of health education topics. Delivery was mostly by a didactic approach and case presentations. Some included a mix of videoconferencing with face-to-face sessions. Overall, telehealth education was well received, with participants reporting mostly positive outcomes as signified by feeling less isolated and more supported.
One interesting result was that quality in telehealth education is poorly established as there appears to be no definitions or consensus on what constitutes quality in the delivery of telehealth education. Very few studies formally tested increase in skill or knowledge, which is usual with professional development programs that do not result in further qualifications. For those that did assess these, formal knowledge and skills assessment indicated that telehealth using videoconferencing is comparable to face-to-face training with significant benefits related to travel reduction and therefore cost. Recommendations were difficult to synthesise because of the broad issues uncovered and lack of quality in many of the studies.
Conclusion:  The applications for telehealth are still evolving, with some applications having poor evidence to support use. Overall, telehealth education is well received and supported, with positives far outweighing negatives. Anything that can improve connection with a community and decrease isolation experienced by rural clinicians can only be beneficial. However, further planning and evaluation of the quality of delivery of telehealth education and addressing how education outcomes can be measured needs to be addressed in this widely growing area of telehealth.

Keywords:

education, in-service training, remote health services, rural health services, staff development, telemedicine, telehealth, videoconferencing.

full article:

Introduction

Accessing education in rural and remote settings can be challenging for health practitioners. Telehealth education with videoconferencing (VC) as a main delivery mode is widely researched. Research is well established for telehealth to provide patient education and follow-up appointments, but little is known about the impact of telehealth education using VC for rural and remote health practitioners, and several recommendations note the need for further exploration¹^,2.

Telehealth as defined by the International Organization for Standardization is the ‘use of telecommunication techniques for the purpose of providing telemedicine, medical education, and health education over a distance’³. For clarity, telemedicine has a narrower focus of clinical services that are remotely delivered, with the International Organization for Standardization definition as the ‘use of advanced telecommunication technologies to exchange health information and provide healthcare services across geographic, time, social and cultural barriers’³. VC is described as ‘electronic form of communications that permits people in different locations to engage in face-to-face audio and visual communication. Also, a collection of technologies that integrate video with audio, data, or both to convey in real time over distance for meeting between dispersed sites’³.

VC is one of the most common models of audio and visual technology used to provide telehealth and telemedicine. This article talks about telehealth predominantly and VC specifically as a form of providing health professionals from rural and remote locations an opportunity to connect with colleagues and gain support for patient care, refer patients and many other aspects that historically were a ‘refer and treat’ process, without connection to specialists in real time.

VC is becoming more widely used in telehealth, including to provide education to health practitioners in rural or remote settings. Quality education opportunities in rural and remote settings have traditionally been difficult to access because of geographic isolation, travel, time and expense constraints and often the lack of an onsite educator. Within telehealth, VC is promoted as a practicable alternative for educational opportunities^4-7, but like all new technology approaches, uptake has overtaken evidence for use. When establishing their own telehealth education approach, the authors found gaps in the literature related to how telehealth has been used for specific education in rural and remote settings, along with little evidence about education strategies and their modes of delivery strengths and weaknesses. Therefore, this review was undertaken to identify, evaluate and synthesise the available evidence for using telehealth for education of rural and remote healthcare providers.

The objectives of this integrative review were to: (1) determine the existence and characteristics of telehealth education in rural and remote setting; (2) evaluate current telehealth education models and resources; (3) establish the quality of education provided through telehealth along with the facilitators or enablers of a successful service; and (4) develop recommendations for supporting and developing an education model for rural and remote health practitioners through telehealth.

Methods

Research design

An integrative review was undertaken following the five-stage integrative review process described by Whittemore and Knafl⁸ consisting of problem identification, literature search, data evaluation, data analysis, and presentation.

Data sources and search strategy

Searches were conducted in the electronic databases CINAHL, Medline, Nursing & Allied Health (Proquest), PubMed, Johanna Briggs Institute Evidence Based Practice (JBI EBP) and Embase, using the strategy outlined in Table 1. Studies that were published in English post-2007, to reflect current telehealth education models, were included in the review. Reference lists of included studies were then hand searched for potentially relevant articles and included in the search results.

Table 1: Search strategy applied to databases table image

Inclusion and exclusion criteria

Studies were included if they were published in a peer-reviewed journal and met the following inclusion criteria: (1) telehealth to provide targeted clinical education, training or supervision; (2) health practitioners including doctors, nurses and allied health; (3) rural or remote setting (including underserved). Studies were excluded if they (1) were written in a language other than English; (2) editorials, commentaries, news items, grey literature, could not be sourced, poorly constructed or written papers that made understanding difficult, and if their content did not meet inclusion criteria. In relation to inclusion criterion 1, targeted clinical education was related to education about clinical presentation types rather than health service process or mandatory education that is not focused on clinical practice. The authors chose to limit the inclusion of mandatory or process-driven education owing to the prescriptive nature of this type of education. The main area of interest was for patient outcomes, practice improvement, or clinician professional development.

Data extraction

One reviewer (MS) screened all records for relevance to the topic and excluded articles based on title or abstract that clearly did not meet the inclusion criteria. Two independent reviewers (PC, MS) assessed full-text articles for eligibility. Data were extracted by six reviewers (PC, MS, LT, SW, SWO, COB) and underwent consensus review until agreement was reached for final inclusion in the review.

Restrictions were not placed on study/methodology quality before information was extracted from articles and summarised into a table for review. The following data were collected from the included studies: first author, country of publication, aim, sample size, study design/methodology/intervention, relevant results/findings and recommendations, limitation/bias. This enabled a thematic summary of the studies’ findings (see Table 2).

Table 2: Thematic summary of the included studies table image

Results

Study selection

The initial search using the search terms within the six designated databases yielded 7122 articles. To improve relevance of the search, a further screening of title and abstract only was applied to three databases – Nursing and Allied Health (Proquest), PubMed and Embase databases – as they had identified 4661, 1422 and 487 articles respectively. This screening reduced the total number to 914 articles. Duplicates were removed and then records were screened for relevance to topics. Then two reviewers (PC, MS) independently assessed 117 full-text articles for eligibility. An additional 79 studies were identified after reference list searching of all included articles and were added to the review. These studies were not identified in the initial search because of the variability of keywords that authors had attributed to their articles. A further 21 articles were excluded during data extraction and a final total of 60 articles were included in the study (Fig1).

A detailed summary of the 60 included studies is given in Supplementary table 1.

table image Figure 1: Modified PRISMA flow diagram of included studies.

Study characteristics

Studies were original research^8-57, conference abstracts^58-60, case studies^61-63, reviews⁶⁴^,65, reviews and original research⁴^,66, or a randomised controlled trial⁶⁷. (See Supplementary table 1 for details about study methodology.)

Supplementary table 1: Characteristics of the included studies

Countries of study intervention

Over half (60%) of the included studies were conducted in the USA only^9-12^,14-16^,20^,21^,23^,25-27^,30-32^,34-36^,40^,41^,44^,45^,48-51^,54^,56-62^,67, followed by a smaller number of studies in Australia²⁴^,29^,42^,43^,47^,52^,63^,64, Canada¹⁶^,19^,52^,53^,55^,65^,66, the UK³⁷^,55, the USA and Europe³³, US-affiliated Pacific Islands¹⁷, Africa⁴^,13^,18, Guatemala³⁸, India³⁹, Norway²² and Madagascar⁴⁶.

Context

There was a range of contexts used to describe the healthcare setting in the included studies, including rural¹⁵^,18^,19^,21^,23^,27^,28^,30^,32^,33^,38^,40-46^,48^,50^,51^,55^,58^,60-64^,68, remote¹⁴^,18^,35^,52^,54^,63^,67, rural and remote¹⁷^,24^,53^,56^,64^,66, rural and underserved^10-12^,31^,39^,49^,57^,59^,67, underserved⁹^,25^,33^,36, rural, remote or underserved¹⁶, isolated¹³, regional²⁹, rural and regional⁶³, peripheral³⁷, remote military army and navy medical facilities³²^,34, native communities²⁰^,45 and US-affiliated Pacific Islands¹⁷. More specifically, one study⁴⁷ used outer regional, inner regional, remote and very remote locations. One study²¹ classified ‘rural’ as defined by the Health Resources and Services Administration Shortage Designation Branch, US Government.

Population

Populations were divided into two specific groups, including the healthcare professional cohort who was receiving the education and the patient population the clinical topic was focused on. For example, studies focusing on the type of healthcare professional targeted included primary healthcare practitioners only¹¹^,16^,25^,36^,53^,58^,61^,68, nurses only³⁸^,51^,52^,68, physiotherapists⁵⁵, and psychiatrists and psychiatrist trainees²⁷. Some studies targeted the clinical topic to a particular patient cohort, including veterans¹⁴^,15^,26^,40^,42^,54^,56, Indigenous groups²⁰^,21^,28^,32^,45, current military personnel³³^,34 and those in prison environments¹²^,59. Within these studies, the majority (12 of the 16 studies) targeted multiple health practitioner disciplines servicing those particular patient populations.

Sample size

Of the included studies, participant numbers ranged from fewer than 20 participants¹⁸^,39^,55^,56^,58^,62^,66, 21–50 participants^9-11^,13^,14^,16^,20^,26^,30^,37^,38^,40^,41^,43^,45^,46^,51^,52^,67, 51–100 participants¹⁷^,38^,42^,49^,54^,61, 101–500 participants¹⁵^,19^,21^,23-29^,33^,35^,47^,48^,53^,59^,68, 500–999 participants³¹^,36^,50, and one study with more than 1000 participants³². The smallest number of participants was 1⁶², and the largest number of participants was 1079³². In four studies, the sample size of participants was not specified; however, the number of clinics and patients treated was provided instead¹²^,22^,34^,63. In one study, the sample size of participants, number of clinics and patients treated was not given⁶⁰.

Interventions

Forty-nine studies applied a single combined intervention. This usually included didactic VC, along with case study discussions and/or interactive questions^9-16^,18-24^,26^,27^,29-41^,44^,45^,47^,49-54^,56-63^,67^,68. One study applied a combination of e-mentoring and group-based case study discussions⁵⁵. Two studies compared two individual interventions – VC and DVD⁴⁶ and VC versus standard lecture⁴⁸. One cross-sectional study compared two intervention arms – educational sessions with case study discussions against two control groups; another study compared four intervention arms – VC, audioconferencing and ‘live’ lecture including interactive questioning, and recorded presentation slides (inability to ask questions)¹⁷.

Topics covered by telehealth education

Pain⁹^,14^,16^,26^,28^,31-34^,50^,54^,61^,67 was the most frequent health topic delivered by telehealth interventions, followed by psychiatry, mental and behavioural health¹⁸^,19^,25^,27^,39^,42^,50^,53^,61, hepatitis C virus^10-12^,15^,45^,48^,50^,65, addiction or substance abuse⁹^,16^,32^,36^,39^,50^,53, diabetes¹⁰^,17^,20^,37^,54^,56, oncology/cancer care¹⁸^,22^,23^,28^,52, paediatrics²⁹^,38^,55^,59^,63, HIV³⁵^,40^,41^,50^,57, unclear/unknown¹³^,42^,65^,66, opioid prescribing⁹^,16^,34, variety of topics²¹^,64, palliative care⁴⁷^,60, perioperative⁵¹^,52, oral health¹⁷, metabolic syndrome¹⁷, dementia²⁴, multiple sclerosis³⁰, antenatal/child health³⁸, medicine⁴⁴, sleep⁴⁴, liver disease⁴⁹, cardiology⁵⁴, physiotherapy⁵⁵, asthma⁵⁹, dermatology⁶², midwifery⁶⁸ and antiretroviral therapy⁶⁸.

Current telehealth education models and resources

Within current telehealth education modules and resources, there were 10 themes identified: (1) improved practice change; (2) improved clinician confidence; (3) increased clinician knowledge; (4) improved self-efficacy; (5) increased clinical competency; (6) sense of community and interaction decreases isolation; (7) improved patient outcomes; (8) satisfaction, activity and reach equals acceptability and feasibility; (9) VC education is acceptable and feasible for rural and remote clinicians; and (10) weak measures used to identify success (see Table 2 for a summary of themes and subthemes).

Most VC education provided used a combination of didactic and case presentations¹⁰^,11^,14^,16^,18^,20^,23^,25^,26^,28-32^,36-41^,43-45^,47^,49-51^,53^,54^,57^,59^,60^,62^,67, while others used a combination of VC and face-to-face interactions¹⁹^,35^,62. The sessions varied in length, often from 30 minutes to 1 hour²⁸^,31^,36^,38^,56, and up to 2 hours duration⁹^,10^,12^,46^,53^,60. The frequency of sessions delivered varied from twice weekly³⁴, weekly¹⁴^,22^,30^,31^,53^,57^,61^,67, to monthly²³^,28^,45^,47. A discrete education model was identified in the literature called Project ECHO (Extension for Community Healthcare Outcomes). Project ECHO accounted for over half of the retrieved literature (33/60 or 55%), with only three of these occurring outside of the USA. The most common topic for Project ECHO was pain (n=9) followed by chronic disease management not specified (n=5), then mental health (n=4). The usual model for Project ECHO was a short didactic lecture (15–30 min) followed by two or three de-identified patient case studies with peer-led review of case presentations, and management recommendations all via telehealth or VC, with a focus on creating a community of practice to narrow the health disparity gaps. Project ECHO’s focus is to support clinicians to access collaborative learning in a knowledge-sharing network model⁶⁹^,70. Most studies reported on followed closely to the Project ECHO model, with only some variations in frequency and duration of sessions, and all were focused on case study presentation. One study had added in a two-day onsite training session before using the usual Project ECHO model²⁰. One study also had their didactic session pre-recorded, and the debriefing as a live session.

Telehealth sessions were regarded as being as beneficial as face-to-face sessions³⁸^,68, while the technology was often challenging to some participants¹³^,24. Educators’ knowledge on how best to use telehealth or their technical ability also affected (either positively or negatively) the delivery and provision of education using this technology, and therefore the development of those providing education in this way should be ensured to improve delivery and outcomes¹⁷^,37^,42^,44^,48^,52^,54.

Most of the education provided was delivered by specialists in the area or by multidisciplined teams¹⁰^,12^,18^,19^,22-25^,29^,30^,32^,33^,35-38^,50^,51^,56^,62^,63 and this appeared to have a positive effect by increasing and promoting the development of professional networks¹¹^,23^,25^,43^,45^,47^,63. Use of telehealth modalities for education also had other beneficial outcomes, such as feelings of decreased isolation for participants¹⁶^,27^,39^,45^,49^,57^,60 as well as an increase in the availability and access to education opportunities¹³^,20^,63^,64.

Participants who had undertaken telehealth education also had increased levels of self-efficacy and felt that they were able to better apply what was learned to clinical practice⁹^,23^,26^,28^,31^,43^,60. There appeared to be an increased sense of support being provided²⁰^,22 by using a telehealth education model, as well as increases in the levels of confidence of education recipients¹⁰^,19^,20^,22^,23^,25^,30^,39^,47^,57. Recipients also reported that they gained knowledge as a result of attending telehealth education sessions⁹^,10^,23^,25^,31^,41^,48^,54^,64 and satisfaction levels with education provided using this technology were high¹⁷^,19^,23^,24^,28^,53^,60.

Quality of education provided through telehealth along with the facilitators of a successful service

Related to quality of education provided and facilitators of a successful service, there were three themes identified: (1) successful VC education has specific characteristics; (2) technology can support or hinder education in rural and remote communities; and (3) barriers to uptake include resourcing, scheduling and ease of use (see Table 2 for themes and subthemes).

Quality is not well established as far as agreed measures of what quality is. Satisfaction, level of self-reported confidence, knowledge gains, self-efficacy and engagement are all presented as elements of good education. Very few studies formally tested increase in skill or knowledge as is usual with professional development programs that do not result in further qualifications. For those that do (perioperative nurses, asthma educators and diabetes educators), formal knowledge and skills assessment indicates VC is comparable to face-to-face training with significant benefits related to travel reduction and therefore cost. Most of the Project ECHO publications that described clinical supervision relate education to patient outcomes such as measurable change in practice. Reach, extension of services through topic areas and new clinical environments, was presented as proof of acceptability. Although Sevean et al⁵² and Chipps et al⁴ touch on teaching styles, teacher comfort, expertise and ability to adapt to VC teaching, what enables or is deemed as successful education approaches is not presented in detail. Other studies relate advice on length of and time of day of the sessions both affect the ability of participants to attend. Related to this is ensuring repetition of sessions to allow local uptake, and advocating the use of local champions to support local uptake, which all relate to success along with technology quality, ease of use and access to support.

The quality of education provided through telehealth has so far not been mentioned or addressed in the literature included in this study. Most studies report using case-based presentations and didactic teaching as a chosen method of delivery, but don’t refer to the quality of material that was presented²^,4^,10^,12^,14^,16^,17^,20^,21^,23-32^,34^,36^,38-41^,43^,44^,47-49^,53-57^,59-63^,67.

Some studies¹⁴^,25^,34 mentioned the use of evidence-based or best practice guidelines in their sessions. While these studies stated that clinicians self-reported increased confidence and knowledge, no formal testing was done to substantiate this. Other studies highlighted clinicians’ feelings of increased knowledge or comfort, but no testing or quantifiable measures were demonstrated or tested to prove these perceptions⁹^,16^,19^,20^,28^,38-40^,47^,48^,57. Knowledge scores improved in testing post VC education, showing improved learning; however, limitations are noted as to whether this increased knowledge is transferred into clinical practice⁹^,38^,48.

There appear to be no definitions or agreed consensus as to what constitutes quality when using telehealth to deliver education. Feedback from participants indicated presenter approachability/connection¹⁹ and presenter comfort with equipment/technology⁴ were issues that affected engagement. Educators felt that there was a need to alter or adapt teaching format or style³⁷^,42^,52, but the details of what should be altered or how were not clearly presented.

Specific recommendations were difficult to synthesise as the focus of education strategies used was extremely broad. This was complicated by limited evaluation of education strategies used and perhaps this is an indication that we are still in the phase of proliferation of publications that describe rather than evaluate a new intervention/approach. The few papers that do evaluate quality or outcomes of using a telehealth strategy for education may be the start of the professional conversation of questioning how to use a new approach like VC for telehealth education, most effectively for health professional education in rural and remote settings.

The 13 themes and subthemes distilled from the selected are presented in detail in Table 2. These included the following (theme names in italics).

Improved practice change, which focused on the changed practices of the clinicians as a result of engaging in the education provided. This was closely linked to patient outcomes as a result, and to improved clinician confidence and increased clinician knowledge and improved self-efficacy.

Increased clinical competency related to tested clinical competence, whether this was tested or self-reported, and in some cases extended the scope of practice of the clinician.

Sense of community and interaction decreases isolation related to being able to collaborate with peers, access support and as a result feel more connected and reduced stress related to being a healthcare practitioner in an isolated environment.

Improved patient outcomes related to measurable patient clinical outcomes and being able to access health care in a rural or remote environment for a broader scope of care.

Satisfaction, activity and reach equals acceptability and feasibility was closely related to weak measures used to identify success; however, this would benefit from more robust outcomes-based research.

Successful VC education has specific characteristics focused on relevance of the education to the target audience, being interactive in nature, providing a mixture of educational approaches, and tailorable to what is needed locally. Specific educator skills and characteristics also made up a large portion of this theme; for example, being knowledgeable, approachable, comfortable with equipment, being able to create a confidential, safe and comfortable learning environment. Training for educators also features aspects such as being able to adapt teaching style to VC sessions, using a specific format, technological skills, and accessibility for questions and consistency of session information presented across multiple groups.

VC education is acceptable and feasible for rural and remote clinicians related to reported satisfaction of participants, being able to attend education, decreased time and money required to access education, and the additional benefits of having VC education frequently versus face-to-face education sporadically.

Technology can support or hinder education in rural and remote communities is related to how much support and infrastructure is available to troubleshoot issues, quality of original set-up of easy-to-use systems. Barriers related to poor audio quality and video capability, overloaded servers that were not set up for the increased traffic, and ability to access on mobile devices.

Barriers to uptake including resourcing, scheduling and ease of use included complexity for end users to complete the education, availability of times that sessions are scheduled, infrastructure of equipment and training for clinicians to use the equipment, being released from clinical work to attend education, session length (some are too long to be able to work with clinical needs locally), lack of time to use resources that are identified in the education, being able to attend sessions when scheduled (therefore if they are recorded, or run multiple times) and scheduling clashes.

Discussion

Telehealth continues to grow into novel application areas faster than evidence exists to support uptake. Using telehealth for education delivery is clearly well established in the literature, but less well measured. This gap is particularly noticeable in this review, specifically in relation to the third objective – establish the quality of education provided through telehealth along with the facilitators or enablers of a successful service. The authors are unable to make specific recommendations about the quality of education that should serve as a guide for future educators and researchers when they are designing telehealth education approaches and evaluation of their services. VC was the most used strategy in the telehealth suite of education; however, it was used variably and perhaps not to its full potential. For example, assessment of clinical skills after the attendance of education could be undertaken by videoconference, enhancing support for rural and remote nurses and midwives in their endeavour to maintain clinical currency. This has been demonstrated in specific care areas such as the use of tele-ICU to support rural nurses caring for critically ill patients, providing teaching, assistance and further education in real time⁵.

This study found improving practice change was a theme related to telehealth education provided to health professionals in rural and remote areas. However, the measures supporting this outcome were not rigorous. Future research related to quality measures of tangible outcomes is needed. This is not specific to VC education; measuring practice change as a result of education provided is more complex and needs careful planning to be able to creditably achieve clear outcomes because often training is not the only answer⁶⁸. While patient education can be measured directly by observing behaviour change and asking their intention to change behaviour, this is more complex to do with clinician education outcomes. There are many more factors involved in measuring outcomes of clinician education in addition to increased knowledge, intention to change practice and observed practice change. This would be an interesting and valid area of research to develop more effective approaches to measuring education outcomes within the health professional cohort.

In one integrative and thematic synthesis⁷¹ that investigated education and professional opportunities in rural and remote environments when using telehealth programs, engagement and participation was found to be further enhanced by the personalisation to the program, contributing value and meaning. This is closely linked to several of the current themes. For example, social isolation was reduced, and networking opportunities provided with the interactive learning were found to be most effective as an education technique for influencing patient outcomes⁷¹. Regardless of geographical location, the use of telehealth to provide education to health professionals in rural and remote areas is seen as feasible, cost-effective and beneficial⁶⁹^,72. While this may not have clear economic measures outlined in the studies included in this review, the unequivocal potential for economic savings simply due to decreased travel is clear; and this becomes more attractive the more geographically distant clinicians work from their preferred education sources. An interesting example of this was a study on shared care models trialled in the Kimberley region of Western Australia, where telehealth provided patients with specialist care and health professionals with education sessions. This significantly reduced time normally taken to travel to specialised care for patients and education for staff⁷³.

One of the most widely applied models for telehealth education for clinical staff is the ECHO model. The ECHO model has been used worldwide, incorporating the use of telehealth to impact a lack of resources in rural and remote areas, applying case-based education to manage intricacy and the sharing of best practices⁶⁹^,70. The ECHO model applies four principles: use technology to influence limited resources, impart evidence-based practices, individual patient-based education, and an internet database to monitor outcomes⁶⁹^,70^,74^,75. Of all the education models reported on in the literature, the ECHO model had the most replicable approach, although this may only be because other education models have not been presented in a way that makes them easy to replicate and implement in different environments. Project ECHO presents a scalable, low-stakes, relatively low resource-intensive approach (apart from the telehealth infrastructure) to providing education to health professionals that does not require specific educational skills or resources on behalf of the presenters. Didactic presentations are short and therefore should not take a lot of resources to develop, and case study presentations are chosen from recent practice, thereby improving relevance. There was little discussion in articles about the disadvantages of implementing Project ECHO; however, it is likely that while having a specific structure can assist scalability, it can also be a barrier and lack flexibility. Additionally, as it is peer led, there is little training about ‘how’ to undertake education for your peers that would help support and sustain this model. Staff also undertake this model most often in their own time, which can have financial implications.

To support retention, the World Health Organization⁷² advocates for continuing education together with professional development programs to be made available for rural and remote health professionals from their home locations. It has been identified that VC is a viable, cost-effective means of providing education to rural and remote staff⁶⁹^,70. However, VC education is not without its challenges, including internet connection issues in remote areas and the ability of staff to work equipment¹^,6^,18^,76.

There are no hard measures for success; however, success has been indicated by the overall accomplishment of the reportative qualitative feedback and the proliferation of different programs using telehealth. There needs to be higher level evaluation research planned and undertaken to provide further evidence of the effectiveness of telehealth education and its impacts on patient care.

Another interesting finding was the wide variety in clinical education topic areas covered by the included studies. Pain was the most prevalent topic area. It could be argued that since pain is a common reason for seeking health care⁷⁷ and managing pain effectively is complex, there is a significant need for clinicians to access education about pain management. An alternative reasoning is that perhaps those who provide pain education have had to provide professional development to others via alternative approaches for a long time, and therefore have better established services in this regard.

Recommendations for supporting and developing an education model for rural and remote health practitioners through telehealth

Positive factors for the use of telehealth include the ability to have real-time interaction and active participation, facilitating questions, receiving clarification and discussing care presentations²³^,38^,42. This has been demonstrated in studies undertaken with the ECHO model, with participants indicating they had increased knowledge, liked group discussions and felt a reduction in professional isolation; however, the impact on patient outcomes warrants further exploration⁹^,19^,20^,25^,28^,39-41^,47^,65. Duplantie et al⁶⁶ discuss the use of the Delphi study involving 12 participants and how education via VC could impact positively on recruitment and retention, providing the ability to work in collaboration to help recruit and retain staff.

Program evaluation and curriculum development, training for educators and staff accessing VC education would need further consideration for program success. This includes development of educators to enable content transfer to participants, and being able to adapt to the needs of participants, format and style for program success¹⁸^,29^,44^,52^,54^,60^,62.

The identification of participant barriers for education via VC needs to be explored, including education provided, time and length of sessions, internet availability, equipment instruction and facility-level barriers²⁴^,26^,32^,37^,43^,45. Further measures beyond participant satisfaction need to be considered for evaluation of program success, including practice change and impact on patient outcomes⁹^,19^,20^,25^,28^,40^,41^,44^,47^,65^,66.

In future studies, recommendations to improve the quality of studies reporting on using VC to deliver education in rural and remote environments include:

1. Use multiple success factors to test for within the study, including education model, teaching approaches, skills of facilitators, technology components and infrastructure, to publish a set of principles, skills and abilities. These measures should also consider the following aspects.

2. Report or measure the alignment of education aims with results being reported:

Should satisfaction of participants be the only measure of success?
Does reach equals acceptability or is it reported as a measure because that is all that is available to the researchers?
What is the feasibility related to local infrastructure and resourcing (which will vary and may not be transferrable)?
What does good teaching in a VC format look like and why do participants and teachers feel this way?
In relation to interactive education, is all interactivity equal? Does the interactivity serve application or key educational goals?
Are there specific teaching approaches that influence learning and retention of learning within the VC format?
What level of participant input makes a program successful? What are the measures of success? Why have these measures been chosen? How do they align with educational goals?
What measures make most sense for the level of education that best fits the aim of education (eg access to education vs practice change will have very different measures)?
What level of training or what training is needed for educators/facilitators to adapt to teaching via VC? What resources are developed, and can these be made more widely available?
ECHO is a model that has been widely adopted by practitioners for practitioners and clinical supervision. Are there other versions/models that can be replicated (recommend Delphi study to gain consensus of acceptable model components)?

Limitations

The limitations in this study include variations in what defines a rural or remote area and the variety of settings where studies were undertaken. Another issue was the quality of the studies included. Outcome measures were generally weak, and therefore recommendations cannot be made on this basis; however, if all studies were excluded for this reason, there would be very few to include in this study, and the authors feel that this is a finding in itself. The nature of the emerging work in this area means there is some way to go in improving the research design of education program evaluations in telehealth use for health practitioner education in rural and remote areas.

Conclusion

Telehealth education, and in particular VC, is seen as a viable, cost-effective means of providing education to rural and remote staff⁷⁸^,79. With technological advances, a variety of education resources can be made available, impacting on knowledge and skills in the rural and remote arena, and assessment of knowledge and skills can be assessed by VC methods.

However, VC education is not without its challenges in rural and remote areas, including poor internet connection and the clinicians’ ability to operate equipment¹^,6^,76. It is highlighted that equipment familiarity is essential when using VC for education, and Chipps et al² discusses the importance of well-defined guidelines and operating instructions with videoconference use to alleviate concerns of participants and educators.

Notwithstanding the methodological limitations of the studies, there appears to be adequate evidence for VC education to have an impact on cultivating knowledge for rural and remote nurses and midwives. The paucity of high-quality research warrants further investigation focusing on scrupulously planned studies. Larger studies evaluating effectiveness of videoconference educational interventions pertaining to the impact on knowledge and skills is needed to quantify the scholastic gains.

Acknowledgements

The authors acknowledge Cherie O’Brien for involvement in data extraction, and Leeanne Trenning for data extraction and validation of data analysis.

references:

1 McConnell KA, Krisher LK, Lenssen M, Bunik M, Montes SB, Domek GJ. Telehealth to expand community health nurse education in rural Guatemala: a pilot feasibility and acceptability evaluation. Frontiers in Public Health 2017; 5: 60. DOI link, PMid:28405582

2 Chipps J, Brysiewicz P, Mars M. A systematic review of the effectiveness of videoconference‐based tele‐education for medical and nursing education. Worldviews on Evidence‐Based Nursing 2012; 9(2): 78-87. DOI link, PMid:22409341

3 International Organization for Standardization. ISO/TR 16056-1:2004(en). Health informatics - interoperability of telehealth systems and networks. Part 1: introductions and definitions. Available: web link (Accessed 9 January 2021).

4 Chipps J. The use of synchronous videoconferencing teaching to increase access to specialist nurse education in rural Kwazulu-Natal, South Africa. Knowledge Management and E-Learning 2010; 2(2): 154-168. DOI link

5 Hendrickx L, Winters C. Access to continuing education for critical care nurses in rural or remote settings. Critical Care Nurse 2017; 37(2): 66-71. DOI link, PMid:28365651

6 Jerome-D'Emilia B, Merwin E, Stern S. Feasibility of using technology to disseminate evidence to rural nurses and improve patient outcomes. Journal of Continuing Education in Nursing 2010; 41(1): 25-32. DOI link, PMid:20102140

7 Page R, Hynes F, Reed J. Distance is not a barrier: the use of videoconferencing to develop a community of practice. Journal of Mental Health Training, Education and Practice 2019; 14(1): 12-19. DOI link

8 Whittemore R, Knafl K. The integrative review: updated methodology. Journal of Advanced Nursing 2005; 52(5): 546-553. DOI link, PMid:16268861

9 Anderson D, Zlateva I, Davis B, Bifulco L, Tierney G, Coman E, et al. Improving pain care with Project ECHO in community health centers. Pain Medicine 2017; 18(10): 1882-1889. DOI link, PMid:29044409

10 Arora S, Geppert CMA, Kalishman S, Dion D, Pullara F, Bjeletich B, et al. Academic health center management of chronic diseases through knowledge networks: Project ECHO. Academic Medicine 2007; 82(2): 154-160. DOI link, PMid:17264693

11 Arora S, Kalishman S, Dion D, Som D, Thornton K, Bankhurst A, et al. Partnering urban academic medical centers and rural primary care clinicians to provide complex chronic disease care. Health Affairs 2011; 30(6): 1176-1184. DOI link, PMid:21596757

12 Arora S, Kalishman S, Thornton K, Dion D, Murata G, Deming P, et al. Expanding access to hepatitis C virus treatment - Extension for Community Healthcare Outcomes (ECHO) project: disruptive innovation in specialty care. Hepatology 2010; 52(3): 1124-1133. DOI link, PMid:20607688

13 Bagayoko CO, Perrin C, Gagnon M-P, Geissbuhler A. Continuing distance education: a capacity-building tool for the de-isolation of care professionals and researchers. Journal of General Internal Medicine 2013; 28(S3): S666-S670. DOI link, PMid:23780652

14 Ball S, Wilson B, Ober S, McHaourab A. SCAN-ECHO for pain management: implementing a regional telementoring training for primary care providers. Pain Medicine 2018; 19(2): 262-268. DOI link, PMid:28525633

15 Beste LA, Glorioso TJ, Ho PM, Au DH, Kirsh SR, Todd-Stenberg J, et al. Telemedicine specialty support promotes Hepatitis C treatment by primary care providers in the Department of Veterans Affairs. American Journal of Medicine 2017; 130(4): 432-438 e3. DOI link, PMid:27998682

16 Carlin L, Zhao J, Dubin R, Taenzer P, Sidrak H, Furlan A. Project ECHO telementoring intervention for managing chronic pain in primary care: insights from a qualitative study. Pain Medicine 2018; 19(6): 1140-1146. DOI link, PMid:29036381

17 Chen TH, Buenconsejo-Lum LE, Braun KL, Higa C, Maskarinec GG. A pilot evaluation of distance education modalities for health workers in the US-affiliated Pacific Islands. Pacific Health Dialog 2007; 14(1): 22-30.

18 Chipps J, Ramlall S, Madigoe T, King H, Mars M. Developing telepsychiatry services in KwaZulu-Natal - an action research study. African Journal of Psychiatry 2012; 15(4): 255-263. DOI link, PMid:22829228

19 Church EA, Heath OJ, Curran VR, Bethune C, Callanan TS, Cornish PA. Rural professionals' perceptions of interprofessional continuing education in mental health. Health and Social Care in the Community 2010; 18(4): 433-443. DOI link, PMid:20522117

20 Colleran K, Harding E, Kipp BJ, Zurawski A, MacMillan B, Jelinkova L, et al. Building capacity to reduce disparities in diabetes:training community health workers using an integrated distance learning model. The Diabetes Educator 2012; 38(3): 386-396. DOI link, PMid:22491397

21 Cunningham BJ, Vande Merwe R. Virtual grand rounds: a new educational approach in social work that benefits long-term care providers and patients in rural Idaho. Rural and Remote Health 2009; 9(1): 1073. DOI link, PMid:19215188

22 Donnem T, Ervik B, Magnussen K, Andersen S, Pastow D, Andreassen S, et al. Bridging the distance: a prospective tele-oncology study in northern Norway. Supportive Care in Cancer 2012; 20(9): 2097-2103. DOI link, PMid:22076621

23 Doorenbos AZ, Kundu A, Eaton LH, Demiris G, Haozous EA, Towle C, et al. Enhancing access to cancer education for rural healthcare providers via telehealth. Journal of Cancer Education 2011; 26(4): 682-686. DOI link, PMid:21336979

24 Doyle C, Jackson D, Loi S, Malta S, Moore K. Videoconferencing and telementoring about dementia care: evaluation of a pilot model for sharing scarce old age psychiatry resources. International Psychogeriatrics 2016; 28(9): 1567-1574. DOI link, PMid:27189501

25 Fisher E, Hasselberg M, Conwell Y, Weiss L, Padron NA, Tiernan E, et al. Telementoring primary care clinicians to improve geriatric mental health care. Population Health Management 2017; 20(5): 342-347. DOI link, PMid:28106511

26 Frank JW, Carey EP, Fagan KM, Aron DC, Todd-Stenberg J, Moore BA, et al. Evaluation of a telementoring intervention for pain management in the Veterans Health Administration. Pain Medicine 2015; 16(6): 1090-1100. DOI link, PMid:25716075

27 Greenwood J, Williams R. Continuing professional development for Australian rural psychiatrists by videoconference. Australasian Psychiatry 2008; 16(4): 273-276. DOI link, PMid:18608161

28 Haozous E, Doorenbos AZ, Demiris G, Eaton LH, Towle C, Kundu A, et al. Role of telehealth/videoconferencing in managing cancer pain in rural American Indian communities. Psychooncology 2012; 21(2): 219-223. DOI link, PMid:22271543

29 Harris V, Smith AC, Armfield NR. Education for regional health professionals using mobile videoconferencing. Journal of Telemedicine and Telecare 2007; 13(S3): 44-47. DOI link

30 Johnson KL, Hertz D, Stobbe G, Alschuler K, Kalb R, Alexander KS, et al. Project Extension for Community Healthcare Outcomes (ECHO) in multiple sclerosis: increasing clinician capacity. International Journal of MS Care 2017; 19(6): 283-289. DOI link, PMid:29270085

31 Katzman JG, Comerci G, Jr, Boyle JF, Duhigg D, Shelley B, Olivas C, et al. Innovative telementoring for pain management: project ECHO pain. Journal of Continuing Education in the Health Professions 2014; 34(1): 68-75. DOI link, PMid:24648365

32 Katzman JG, Fore C, Bhatt S, Greenberg N, Griffin Salvador J, Comerci GC, et al. Evaluation of American Indian health service training in pain management and opioid substance use disorder. American Journal of Public Health 2016; 106(8): 1427-1429. ps://doi.org/10.2105/AJPH.2016.303193, PMid:27196642

33 Katzman JG, Galloway K, Olivas C, McCoy-Stafford K, Duhigg D, Comerci G, et al. Expanding health care access through education: dissemination and implementation of the ECHO model. Military Medicine 2016; 181(3): 227-235. DOI link, PMid:26926747

34 Katzman JG, Qualls CR, Satterfield WA, Kistin M, Hofmann K, Greenberg N, et al. Army and Navy ECHO pain telementoring improves clinician opioid prescribing for military patients: an observational cohort study. Journal of General Internal Medicine 2019; 34(3): 387-395. DOI link, PMid:30382471

35 Knapp H, Fletcher M, Taylor A, Chan K, Goetz MB. No clinic left behind: providing cost-effective in-services via distance learning. Journal for Healthcare Quality 2011; 33(5): 17-24. DOI link, PMid:23845129

36 Komaromy M, Duhigg D, Metcalf A, Carlson C, Kalishman S, Hayes L, et al. Project ECHO (Extension for Community Healthcare Outcomes): a new model for educating primary care providers about treatment of substance use disorders. Substance Abuse 2016; 37(1): 20-24. DOI link, PMid:26848803

37 Maltinsky W, Hall S, Grant L, Simpson K, MacRury S. Pilot project and evaluation of delivering diabetes work-based education using video conferencing. Rural and Remote Health 2013; 13: 2053. DOI link, PMid:23414080

38 McConnell KA, Krisher LK, Lenssen M, Bunik M, Bunge Montes S, Domek GJ. Telehealth to expand community health nurse education in rural Guatemala: a pilot feasibility and acceptability evaluation. Frontiers in Public Health 2017; 5: 60. DOI link, PMid:28405582

39 Mehrotra K, Chand P, Bandawar M, Rao Sagi M, Kaur S, Aurobind G, et al. Effectiveness of NIMHANS ECHO blended tele-mentoring model on integrated mental health and addiction for counsellors in rural and underserved districts of Chhattisgarh, India. Asian Journal of Psychiatry 2018; 36: 123-127. DOI link, PMid:30086513

40 Moeckli J, Stewart KR, Ono S, Alexander B, Goss T, Maier M, et al. Mixed-methods study of uptake of the Extension for Community Health Outcomes (ECHO) Telemedicine model for rural veterans with HIV. Journal of Rural Health 2017; 33(3): 323-331. DOI link, PMid:27557039

41 Ness TE, Annese MF, Martinez-Paz N, Unruh KT, Scott JD, Wood BR. Using an innovative telehealth model to support community providers who deliver perinatal HIV care. AIDS Education and Prevention 2017; 29(6): 516-526. DOI link, PMid:29283274

42 Newman C, Martin E, McGarry DE, Cashin A. Survey of a videoconference community of professional development for rural and urban nurses. Rural and Remote Health 2009; 9(2): 1134. DOI link, PMid:19382828

43 Newman L, Bidargaddi N, Schrader G. Service providers' experiences of using a telehealth network 12 months after digitisation of a large Australian rural mental health service. International Journal of Medical Informatics 2016; 94: 8-20. DOI link, PMid:27573307

44 Parsons EC, Mattox EA, Beste LA, Au DH, Young BA, Chang MF, et al. Development of a sleep telementorship program for rural Department of Veterans Affairs primary care providers: Sleep Veterans Affairs Extension for Community Healthcare Outcomes. Annals of the American Thoracic Society 2017; 14(2): 267-274.

45 Pindyck T, Kalishman S, Flatow-Trujillo L, Thornton K. Treating hepatitis C in American Indians/Alaskan Natives: a survey of Project ECHO(Ò) (Extension for Community Healthcare Outcomes) utilization by Indian health service providers. SAGE Open Medicine 2015; 3: 2050312115612805. DOI link, PMid:26770809

46 Rakototiana LB, Rajabo, Gottot S. Internet or dvd for distance learning to isolated rural health professionals, what is the best approach? BMC Medical Education 2017; 17: 152. DOI link, PMid:28874191

47 Ray RA, Fried O, Lindsay D. Palliative care professional education via video conference builds confidence to deliver palliative care in rural and remote locations. BMC Health Services Research 2014; 14: 272. DOI link, PMid:24947941

48 Rossaro L, Tran TP, Ransibrahmanakul K, Rainwater JA, Csik G, Cole SL, et al. Hepatitis C videoconferencing: the impact on continuing medical education for rural healthcare providers. Telemedicine Journal and e-Health 2007; 13(3): 269-277. DOI link, PMid:17603829

49 Salgia RJ, Mullan PB, McCurdy H, Sales A, Moseley RH, Su GL. The educational impact of the Specialty Care Access Network-Extension of Community Healthcare Outcomes program. Telemedicine Journal and e-Health 2014; 20(11): 1004-1008. DOI link, PMid:25226452

50 Scott JD, Unruh KT, Catlin MC, Merrill JO, Tauben DJ, Rosenblatt R, et al. Project ECHO: a model for complex, chronic care in the Pacific Northwest region of the United States. Journal of Telemedicine and Telecare 2012; 18(8): 481-484. DOI link, PMid:23209269

51 Seibert PS, Reddy T, Whitmore T, Valerio J, Dehaas C. The use of telemedicine to train perioperative nurses in rural settings. Journal of Telemedicine and Telecare 2013; 19(6): 311-314. DOI link, PMid:24163294

52 Sevean P, Dampier S, Spadoni M, Strickland S, Pilatzke S. Bridging the distance: educating nurses for telehealth practice. Journal of Continuing Education in Nursing 2008; 39(9): 413-418. DOI link, PMid:18792607

53 Sockalingam S, Arena A, Serhal E, Mohri L, Alloo J, Crawford A. Building provincial mental health capacity in primary care: an evaluation of a Project ECHO mental health program. Academic Psychiatry 2018; 42(4): 451-457. DOI link, PMid:28593537

54 Stevenson L, Ball S, Haverhals LM, Aron DC, Lowery J. Evaluation of a national telemedicine initiative in the Veterans Health Administration: factors associated with successful implementation. Journal of Telemedicine and Telecare 2018; 24(3): 168-178. DOI link, PMid:27909208

55 Stewart S, Carpenter C. Electronic mentoring: an innovative approach to providing clinical support. International Journal of Therapy and Rehabilitation 2009; 16(4): 199-206. DOI link

56 Watts SA, Roush L, Julius M, Sood A. Improved glycemic control in veterans with poorly controlled diabetes mellitus using a Specialty Care Access Network-Extension for Community Healthcare Outcomes model at primary care clinics. Journal of Telemedicine and Telecare 2016; 22(4): 221-224. DOI link, PMid:26253745

57 Wood BR, Unruh KT, Martinez-Paz N, Annese M, Ramers CB, Harrington RD, et al. Impact of a telehealth program that delivers remote consultation and longitudinal mentorship to community HIV providers. Open Forum Infectious Diseases 2016; 3(3): ofw12. DOI link, PMid:27703991

58 Cheng KMD, Jetmalani ANMD, Sedighi B, McDonnell MM, Pettersen T. Using the Project Extension for Community Healthcare Outcomes model to train rural primary care clinicians to deliver child mental health services in Oregon. Journal of the American Academy of Child & Adolescent Psychiatry 2017; 56(10): S186-S187. DOI link

59 Harkins M, Raissy H, Moseley K, Luttecke K, Arora S. Project ECHO: improving asthma care in New Mexico with telehealth technology. Chest 2011; 140(4). DOI link

60 Marr LMD, Neale DMD. Project ECHO: bringing palliative care consultation to rural New Mexico through a novel telemedicine format. Journal of Pain and Symptom Management 2012; 43(2): 448-449. DOI link

61 Katzman JG. Making connections: using telehealth to improve the diagnosis and treatment of complex regional pain syndrome, an underrecognized neuroinflammatory disorder. Journal of Neuroimmune Pharmacology 2013; 8(3): 489-493. DOI link, PMid:23054372

62 Lewis H, Becevic M, Myers D, Helming D, Mutrux R, Fleming D, et al. Dermatology ECHO - an innovative solution to address limited access to dermatology expertise. Rural and Remote Health 2018; 18(1): 4415. DOI link, PMid:29409325

63 Smith AC, Armfield NR, White MM, Williams ML, Koh TH, Hurley T, et al. Clinical services and professional support: a review of mobile telepaediatric services in Queensland. Studies in Health Technology and Informatics 2010; 161: 149-158.

64 Moffatt JJ, Eley DS. The reported benefits of telehealth for rural Australians. Australian Health Review 2010; 34(3): 276-281. DOI link, PMid:20797357

65 Zhou C, Crawford A, Serhal E, Kurdyak P, Sockalingam S. The impact of Project ECHO on participant and patient outcomes: a systematic review. Academic Medicine 2016; 91(10): 1439-1461. DOI link, PMid:27489018

66 Duplantie J, Gagnon MP, Fortin JP, Landry R. Telehealth and the recruitment and retention of physicians in rural and remote regions: a Delphi study. Canadian Journal of Rural Medicine 2007; 12(1): 30-36.

67 Eaton LH, Godfrey DS, Langford DJ, Rue T, Tauben DJ, Doorenbos AZ. Telementoring for improving primary care provider knowledge and competence in managing chronic pain: a randomised controlled trial. Journal of Telemedicine and Telecare 2020; 26(1-2): 21-27.

68 Kirkpatrick J, Kirkpatrick W. Training on trial. Journal for Quality and Participation 2010; 33(3): 26-31.

69 Western Victoria Primary Health Network. Project Echo overview 2019. Available: web link (Accessed 3 April 2021).

70 Children's Health Queensland Hospital and Health Service. Project ECHO 2019. Available: web link (Accessed 25 April 2021).

71 Moran AM, Coyle J, Pope R, Boxall D, Nancarrow SA, Young J. Supervision, support and mentoring interventions for health practitioners in rural and remote contexts: an integrative review and thematic synthesis of the literature to identify mechanisms for successful outcomes. Human Resources for Health 2014; 12(1): 10. DOI link, PMid:24521004

72 World Health Organization. Increasing access to health workers in remote and rural areas through improved retention: global policy recommendations. Geneva: WHO, 2010.

73 Vale S, Roche I, Noble V, Markus T, Loh R. P44: a shared care model for allergy - Kimberley region pilot. Internal Medicine Journal 2018; 48(S6): 17. DOI link

74 Victorian Opioid Management ECHO. Opioid management for primary care practitioners. 2019. Available: web link (Accessed 25 February 2021).

75 Mitruka K, Thornton K, Cusick S, Orme C, Moore A, Manch RA, et al. Expanding primary care capacity to treat hepatitis C virus infection through an evidence-based care model - Arizona and Utah, 2012-2014. Morbidity and Mortality Weekly Report 2014; 63(18): 393-398.

76 Macedo CR, Macedo EC, Torloni MR, Atallah ÁN. Evaluation of a national evidence-based health care course via teleconference in a developing country. Journal of Evaluation in Clinical Practice 2013; 19(4): 713-719. DOI link, PMid:23654346

77 St Saver JL, Warner DO, Yawn BP, Jacobson DJ, McGree ME, Pankratz JJ, et al. Why patients visit their doctors: assessing the most prevalent conditions in a defined American population. Mayo Clinic Proceedings 2013; 88(1): 56-67. DOI link, PMid:23274019

78 Newman C, Martin E, McGarry DE, Cashin A. Survey of a videoconference community of professional development for rural and urban nurses. Rural and Remote Health 2009; 9(2): 1134. DOI link, PMid:19382828

79 Doorenbos AZ, Kundu A, Eaton LH, Demiris G, Haozous EA, Towle C, et al. Enhancing access to cancer education for rural healthcare providers via telehealth. Journal of Cancer Education 2011; 26(4): 682-686. DOI link, PMid:21336979

Supplementary material is available on the live site https://www.rrh.org.au/journal/article/6467/#supplementary

Characteristics of the included studies

Article Title

First Author
(year),
country

Population and context

Aim of study

Sample size

Study design/
methodology/
Intervention

Relevant results/findings and recommendations

Limitations/bias

Army and Navy ECHO Pain telementoring improves clinician opioid prescribing for military patients: an observational cohort study

Katzman

(2019)[34]

USA

Primary care clinicians (PCCs), including physicians, advanced practice clinicians and care teams at remote military, army and navy, medical treatment facilities (MTFs)

To assess whether Chronic Pain and Opioid Management TeleECHO (Extension for Community Healthcare Outcomes) clinic (ECHO Pain) telementoring improved pain management and safe opioid prescribing skills

99 ECHO Pain clinics

1283 control clinics

52,431 ECHO patients (annualised total)

1,187,945 control patients (annualised total)

Oct 2013 - Sep 2016

Observational

Cohort study

Pre-/post-test time series

Statistical analysis

Intervention group and comparison group

Primary outcome: decreased prescriptions

Secondary outcome: evaluation of dosage and co-prescribing

Onsite and virtual training of expert teams was provided to the hub and spoke clinicians taking part in the ECHO model

Military MTFs, 5 US Army and 2 US navy sites were chosen as hubs based on geographical location and availability of integrative and interdisciplinary pain specialists to serve as facilitators

Intervention: 2hr weekly, videoconferencing sessions including short evidence-based pain and addiction didactics, case-based learning and evidence-based recommendations

52% of PCCs participated in four or more training sessions

PCCs in the intervention group had greater declines in

annual opioid related prescriptions per patient (-23% vs -9%, p < 0.001)

days of co-prescribed opioids and benzodiazepines per user (-28% vs -73%, p < 0.02)

number of opioid users (-202% vs -08%, p < 0.001)

Future prospective clinical trials may provide information regarding benefits of the ECHO model at the patient level

The study could not randomise the assignment of PCCs and patients into matched groups therefore the baseline demographics for the comparison group are skewed towards male and active-duty patients

PCCs volunteered to attend ECHO Pain clinics

Data was provided on individual clinics and not individual providers or patients

It was not possible to quantify how much each individual dose changed or address specific patient/level causes of reductions in prescriptions

SCAN-ECHO for pain management: implementing a regional telementoring training for PCPs

Ball

(2018)[14]

USA

Primary care providers (PCPs) including MD/DOs, PharmDs, and nurse practitioner's community-based outpatient clinics (CBOCs)

Cleveland, South Texas or Wisconsin Veterans Health Administration (VHA) medical centres

To describe and provide an effectiveness assessment of Specialty Care Access Network (SCAN)-ECHO for pain management (PM) training program

21 MD/Dos

2 PharmDs

2 nurse practitioners

2011-2014

Brief research report

Pilot

Mixed methods

Pre- and post-training questionnaires

Qualitative, virtual group interviews immediately following SCAN-ECHO PM sessions

Statistical analysis

Multidisciplinary team of pain management specialists at the Cleveland Veterans Affairs Medical Center delivered the ECHO program

Intervention: Weekly VC sessions including pain condition didactics and real-time discussion on patient cases

Statistically significant increases in confidence ratings and knowledge scores on questionnaires pre- to post- training

Group interviews:

Gains in knowledge; self-efficacy - increase in confidence in providing treatment; program format - conducive to learning; improved patient care; improvements in networks and communication between PM specialists and PCPs

Future studies should examine whether and how this program has improved access to specialty care, changes in provider behaviour and cost savings

A sustainability analysis will be critical to ensure that reported gains can be and are maintained

Small sample size and non-randomised sampling

Data are limited to the participants who were interviewed only (2 sessions each with 7 PCP's)

Potential site-specific cultural differences and large variations in veteran demographics across VHA facilities

Project ECHO telementoring intervention for managing chronic pain in primary care: insight for a qualitative study

Carlin

(2018)[16]

Canada

PCPs including physicians, nurses, nurse practitioners, occupational or physical therapists, pharmacists and social workers from rural, remote or underserved areas of Ontario, Canada

To report participants' experiences and assessment of Project ECHO

17 physicians

20 allied health professionals

2014-2015

Qualitative, descriptive

Sandelowski's qualitative-descriptive method of analysis

Six 1 hr focus group discussions

Interdisciplinary team of pain management experts including practitioners of psychiatry, neurology, nursing, pharmacy, psychology, social work, physical therapy, occupational therapy, chiropractic, addiction medicine and family medicine delivered Project ECHO

Intervention: Weekly ECHO sessions including didactic lectures and three de-identified case discussions

Focus groups findings:

Challenges of managing chronic pain in primary care - feeling helpless before ECHO

Gained confidence and knowledge -chronic pain clinical encounters were less stressful; responsible opioid prescribing; changes in patient management of cases "ripple effect" not presented at ECHO; valued acquisition of knowledge about treating chronic pain

Sharing of newly acquired knowledge with patients and colleagues

Developed a sense of community and feeling less isolated

Disadvantages -

Time constraints, less time spent on didactic vs case presentations, connection issues, viewing ECHO session highlighted PCP isolation

Further qualitative research is underway to evaluate patient outcomes

Future research will focus on the role and experience of the ECHO "hub" of experts

The use of a relatively public, open nature of discourse might cause hesitation in the participants to reveal information

Participation was on a voluntary basis

Knowledge acquisition was self-reported rather than assessed independently

Data is cross-sectional in nature and does not reflect change over participants' time in ECHO

Effectiveness of NIMHANS ECHO blended tele-mentoring on integrated mental health and addiction counsellors in rural and undeserved districts of Chhattisgarh, India

Mehrotra

(2018)[39]

USA

Counsellors including clinical psychologists and psychiatric social workers from 8 rural and underserved districts of Chhattisgarh, India

To evaluate the feasibility and effectiveness of National Institute of Mental Health and Neurosciences (NIMHANS) ECHO in training counsellors

12 counsellors

41 patient cases discussed

Sep 2017 - Feb 2018

Pilot

Pre-/post-test (3, 6 months) online surveys

Moore's evaluation framework (first five levels) - Level 1: participation, Level 2: satisfaction, Level 3: learning, Level 4: competence and Level 5: performance; based on a 5-point Likert scale

Statistical analysis

The specialist hub team consisted of a clinical psychologist, psychiatrist and a psychiatrist social worker with expertise in mental health and substance use related issues.

Needs assessment tele-ECHO clinic initially held to formulate the curriculum

Intervention: 12 fortnightly tele-echo clinics - brief didactic session, patient case presentations and peer-led discussion

6 participants attended over 80% and all participants attended over 60% of clinics There were no dropouts indicating acceptability of the model

Statistically significant increase in mean scores for perceived knowledge, self-confidence and self-efficacy after 6 months

Participants liked "relevance to the courses to clinical practice"; "group-based discussions" and "a reduction in professionals' isolation"

Future studies: explore the long-term effects of the ECHO program on both learner and patient outcomes; interview patients to enable an evaluation of direct patient satisfaction

The small numbers limit the generalisation of the findings

The long-term effectiveness of the training was not monitored

Building provincial mental health capacity in primary care: an evaluation of Project ECHO mental health program

Sockalingam

(2018)[53]

Canada

PCPs including family physicians, nurse practitioners, nurses, social workers, counsellors, and other allied health professionals from rural health integration networks

To evaluate a mental health and addictions-focussed ECHO Ontario Mental Health (ECHO-ONMH) program on improving primary care knowledge and competency, specifically provider self-efficacy related to mental health and addictions care

131 participants

26 sites

Oct 2015 - June 2016

Pre-/post-test knowledge and self-efficacy surveys

Weekly satisfaction surveys

5-point Likert scale

Moore's evaluation framework (first 4 levels) - Level 1: participation, Level 2: satisfaction, Level 3: learning, Level 4: competence

Bandura's theory of self-efficacy and guidelines for self-efficacy scale development

The ECHO-ONHM hub consisted of a team of specialised mental health care at the Centre for Addictions and Mental Health (CAMH) and the University of Toronto including psychiatrists with expertise in child and youth, psychological trauma and medical psychiatry, family medicine, addictions medicine, pharmacist, social worker and health librarian

Intervention: 32 weekly, 2 hr ECHO-ONMH clinics, didactic lecture and case reviews, written summary of recommendations

First reported outcomes of ECHO model for mental health

Primary outcome: PCP knowledge and perceived self-efficacy improved post-ECHO (knowledge change was significant, p < 0.001; self-efficacy approached significance p = 0.056)

Attrition was low (77%)

Average of 26% weekly attendance

Satisfaction ratings were high - change in patient management; increased diligence screening; increased knowledge

Future studies will focus on determining how knowledge acquisition translates to PCP practice patterns and patient health outcomes

Data was obtained from a single province in Canada

Only group mean scores were used so unable to compare individual participant changes pre- to post-

Long-term data is unavailable to determine sustained effects of ECHO-ONMH

Did not capture data on primary care sites' practice change related to knowledge increases in the management of mental health and substance use disorders

Evaluation of a national telemedicine initiative in the Veterans Health Administration: factors associated with successful implementation

Stevenson

(2018)[54]

USA

PCPs and clinical leaders, specialty care providers (SCPs) in pain management, diabetes or cardiology, other providers (nurse practitioners, pharmacists), support staff (medical support assistants) from remote locations

To provide guidance and support for the implementation and spread of SCAN-ECHO

52 initial SCAN-ECHO surveys sent to 37 sites prior to implementation

June - July 2012

55 interviews

2^nd survey 198/450 providers

77 sites

Sept 2012 - Jan 2013

Convergent, mixed methods

2 quantitative surveys

Semi-structured telephone interviews approximately 1-year post-implementation of the initiative

Consolidated framework for implementing research (CFIR)

Specialist teams deliver SCAN-ECHO

Intervention: SCAN-ECHO 12-18 month curriculum of didactic presentation and case studies

22 SCAN-ECHO leaders responded to initial survey viewing 14/39 CFIR constructs as important

86% agreed that SCAN-ECHO increased PCPs knowledge and competencies

All sites expressed positive attitudes to SCAN-ECHO

Design quality and packaging, compatibility and reflecting and evaluating were 3/14 CFIR constructs that distinguished high and low implementation sites

Complexity of programme was a challenge for participants

Leadership engagement - failure of leadership to provide protected time was a barrier to participation

Recommendations including: more effort given to developing and distributing educational materials; restructure primary care staffing and team composition such as engaging more nurse practitioners; establish an audit and feedback mechanism for monitoring and improving the programme, developing a formal implementation blueprint to reduce complexity; obtain local leadership support for PCPs with dedicated time for participation

Use of only 9 sites for qualitative interviews limits the generalisability of findings

Survey data sample although larger, was still insufficient to achieve significance

The project was conducted in a large, integrated health system and it is not known whether similar findings would be obtained in smaller, community healthcare systems

Improving pain care with Project ECHO in community health centres

Anderson

(2017)[9]

USA

PCPs including internists, family doctors and family nurse practitioners in medically underserved areas of Connecticut and Tucson, Arizona

To evaluate the impact of Project ECHO Pain on knowledge and quality of pain

10 intervention group/10 comparison group

12 clinic sites at Community Health Center Inc, Connecticut

4 clinic sites at El Rio Community Health Center, Tucson, Arizona

48 ECHO Pain sessions

107 unique patients

Jan - Dec 2013

Quasi-experimental

Pre-/post-test with comparison/control group

Mixed methods

Surveys

KnowPain-50 (KP50) pain care knowledge assessment tool

6-point Likert scale

Focus groups

Statistical analysis

A multidisciplinary team of pain specialists in anesthesiology/pain medicine, behavioural health, occupational medicine, addiction medicine, Chinese medicine/acupuncture and primary care from the Integrative Pain Center of Arizona (IPCA) delivered Project ECHO

Intervention: 48, weekly, 2-hour Project ECHO Pain sessions included 20-30 min didactics and 3-4 case presentations

Compared with the control group, PCPs have a statistically significant increase in pain-related knowledge and self-efficacy (p < 0.001)

PCPs suggested that they had acquired knowledge that they were able to apply in their practice

Increased referrals to behavioural health and physical therapy

Significant reduction in prescribing of opioids

Further research is needed to assess whether the knowledge leads to lasting improvements in adherence to guidelines and improvement in patient outcomes

Studies are underway to assess the impact of Project ECHO Pain with a more limited time commitment

Absence of data on dosage of opioids prescribed limits conclusions that can be drawn

Changes in knowledge following the intervention were relatively small to account for the change in behaviours of the intervention group

Telemedicine specialty support promotes Hepatitis C treatment by primary care providers in the department of veterans' affairs

Beste

(2017)[15]

USA

PCPs in 21 regions of USA nationwide (including rural)

To assess whether primary care provider participation in Veterans Affairs Extension for Community Health Outcomes (VA-ECHO) was associated with hepatitis C treatment and sustained virological response

152 sites, nationally

376 VA-ECHO PCPs

6431 VA-ECHO patients

559 case reviews

3797 non-VA-ECHO PCPs

32,322 control patients

April 2011 - June 2015

Quantitative

Statistical analysis

VA-ECHO program was run by hepatitis C specialists from a regional hub at 1 of 7 tertiary facilities

Intervention: 1-2 weekly, 60-90 min sessions, brief didactics and case presentations

VA-ECHO exposed patients were more likely to be rural (25%) vs unexposed patients (20%)

The rate of PCP-initiated antiviral medication was 214% among referred patients for case review on VA-ECHO compared with 25% for control patients

VA-ECHO patients without individualised case review had similar treatment rates compared to unexposed patients

No difference in sustained virological response for patients with exposed primary care providers

Cost-effectiveness and sustainability remain to be demonstrated

Future randomised controlled trials to clarify effects of the ECHO model

Participation in VA-ECHO was voluntary

Each VA region had the authority to tailor its didactic curriculum therefore the curriculum was heterogeneous

Telementoring primary care clinicians to improve geriatric mental health care

Fisher

(2017)[25]

USA

PCCs including physicians, nurse practitioners, physician assistants, care managers, social workers, pharmacy/medical students and program managers in rural and underserved communities in

10 New York counties

To evaluate whether Project ECHO GEMH (geriatric mental health) is an effective strategy to address geriatric mental health challenges in rural and underserved communities

154 participants

54 sites

33 ECHO sessions

Sep 2014 - Feb 2016

Mixed-methods

Claims data analysis

Semi-structured interviews

Specialists in geriatric psychiatry, medicine, nursing, social work, psychology and pharmacy from the University of Rochester Medical Center (URMC) deliver Project ECHO GEMH

Intervention: Didactic sessions and case presentations on best practices related to geriatric mental health care

26 participants were interviewed

Participants reported improvements in clinician GEMH knowledge, confidence and treatment practices, improved health outcomes for patients, and increase in sense of professional support

Some participants reported sharing information with colleagues had changed treatment practices while other colleagues were not open to new information

Health insurance claims data suggest that emergency room costs decreased for patients with mental health diagnoses

Additional research is needed to understand the potential impact of program on patient health outcomes, as well as long term impacts on health care costs

Future research should focus on opportunities to optimise and sustain clinical engagement, effects of program dosage and long-term impacts

The study team was only able to interview a small portion of program participants and interviews were voluntary

The team only had access to utilisation and cost data from a single heath plan

The study focussed on select mental health disorders; other conditions may influence outcomes of interest

Project extension for community healthcare outcomes (ECHO) in multiple sclerosis - in increasing clinician capacity

Johnson

(2017)[30]

USA

PCPs including neurologists, psychiatrists, internists, family medicine specialists, physician assistants, naturopathic physician in the Pacific Northwest, including rural areas of Alaska, Idaho, Montana and Washington State

To determine the feasibility of using the Project ECHO telehealth model to increase capacity and capability of clinicians in rural areas to treat people with multiple sclerosis (MS)

24 clinicians

12 neurologists

3 psychiatrists

3 internists

2 family medicine specialists

3 physician assistants

1 naturopathic physician

13 sites

38 unique case presentations

12-week series

Pilot

Interviews

The MS Project ECHO program included 3 neurologists, 2 psychiatrists, a rehabilitation psychologist and a rehabilitation counsellor from the University of Washington (UW) Medicine Multiple Sclerosis Center and staff from the National MS Society

Intervention: Weekly, 20-30 min didactic education sessions and 30-40 min case presentations

The primary barrier to participation was the time of day or time constraints (9/15; 60%); participants were spread across 3 time zones

67% of participants (10/15) involved in follow-up interviews indicated an increase in confidence in treating MS

60% indicated that the program met their expectations and showed interest in future participation

87% appreciated having a mix of specialities represented

The Pilot warrants further investigation regarding its potential effects on access to MS delivery for underserved populations

It was a challenge to get participants to come forward with cases due to the small numbers of patients with MS

Telehealth to

expand community health nurse education in rural Guatemala: A pilot feasibility and acceptability evaluation

McConnell

(2017)[38]

USA

Community health nurses (CHNs) who conduct home and group visits for pregnant women and children up to 3 years of age

in rural Guatemala (population - 30,000 people)

To evaluate the interactive education via telehealth to CHNs in rural Guatemala via telehealth

Average of 55 learners

2 instructors

10 lectures

Pilot

Pre-/post test

Surveys

Likert scale (1-4) to rate statements regarding lecture content, technology & personal connection

Statistical analysis

Instructors were a lead paediatrician, 2 other Paediatricians, a Paediatric nurse practitioner and a recent medical school graduate from Center for Global Health at University of Colorado

Child health lectures organised into two blocks of 5 topics - selected based on CHNs preferences and instructor availability

Intervention: 30 min of didactic lectures and 15 min for questions, case presentations and discussions of current cases and experiences in the community

Pilot results from survey responses from instructors and learners were combined across 10 lectures

Survey questions assessed knowledge gain as measured by percent improvement per subject & overall pre to poste scores overall was 109%

Learners evaluation was overall positive especially regarding lecture content, strongly positive for technology questions regarding ease of use and convenience Learners were better able to hear the instructor at 94.5% than see them at 87.3% 94.5% agreed that lecture via telehealth was as good as in person

3 lectures rescheduled due to lack of internet connection at the site in Guatemala - Internet connection most frequently took 5-10 min Instructors evaluation - audio quality was better than video quality 100% agreed upon ease and convenience

Strengths were low start-up costs and minimal resources needed to develop and implement the program

Major advantage using telehealth technology over web-based e-learning is the ability to have real time interaction and active participation that facilitates question, receiving clarification and discussing case presentations

Specific cost effectiveness analysis was not performed as the set-up of equipment, bandwidth, licensing of software Faculty & CHN salaries were all including free of charge

Paucity of data overall - especially in the educational (non-direct patient care) usage of telehealth - likely due to equipment, time & software used are shared among projects in an academic setting therefore no tracking of costs spent or saved with specific telehealth program

Instructor & learner time commitments were not calculated in the evaluation

CHNs completed surveys anonymously, preventing pairing of pre and post test scores or the ability to adjust for lecture attendance - this was done to encourage full participation and remove concerns the CHNs may have to relation to job performance

Small data set

Unable to define if the knowledge gain was translated to better work performance

Mixed-method study of uptake of the extension for community health outcomes (ECHO) telemedicine model of rural veterans with HIV

Moeckli

(2017)[40]

USA

Human immunodeficiency virus (HIV) specialists, PCPs and administrators in West Coast and Midwest metropolitan and rural areas

To evaluate 3 HIV ECHO programs in the VHA, focussing on uptake of primary care clinics and veterans

31 participants

21 primary care clinics

Mixed methods

Data analysis

Semi-structured interviews

HIV ECHO was run by physicians, pharmacists, psychologist and /or nurse case managers with expertise in HIV from the University of Iowa

Intervention: Didactics and discussion of cases presented

43% of primary care clinics adopted HIV ECHO

HIV ECHO had limited uptake due to sense that HIV care should not be integrated into the wider primary care system, a reluctance to share ownership of care between specialty and primary care clinics and a perception that HIV care is too infrequent to drive the program.

Incentives to adopt HIV ECHO included a perception of improving access to care for patients, professional development and trust in the HIV ECHO team

Did not interview PCPs who refused to participate in HIV ECHO, who may have had unique perspectives

Did not examine process or outcome measures of HIV care quality

Using an innovative telehealth model to support community providers who deliver perinatal HIV care

Ness

(2017)[41]

USA

PCPs in low resource and rural settings in Washington, Alaska, Montana, Idaho, Oregon and Colorado

Evaluate the impact of an ECHO telementoring program on the management of perinatal HIV

53 community clinicians

13 patient cases

Longitudinal

Embedded mixed methods

Surveys

Mountain West (MW) AETC (AIDS Education and Training Centre) and University of Washington

Intervention: 15 min didactic update on clinically relevant topics, followed by brief question and answer period; and 1 hour of case consultations and discussion using de-identified cases; written final recommendations

77.4% response rate for survey completion

100% of providers reported that ECHO had "very much" impacted management of the case

All cases had the successful outcome of prevention of mother-to-child transmission of HIV

88% of providers reported increased knowledge

93% of providers reported that ECHO had increased awareness of perinatal guidelines

30% had acted as a resource to colleagues and 42% used knowledge gained to help colleagues manage patients

Future studies should evaluate the cost-effectiveness, assess the community level impact and impact on provider turnover

Survey responder bias may exist

Surveyed only active ECHO participants (selection bias)

Patient case sample size is small

Limited information provided to the ECHO network

Development of a sleep telementorship program for rural Department of Veterans Affairs PCPs: sleep veterans' affairs extension for community healthcare outcomes

Parsons

(2017)[44]

USA

PCPs including licensed independent practitioners (medical doctors and advanced registered nurse practitioners), registered nurses, pharmacists and occupational and respiratory therapists in

Northwest VA Health Network of Alaska, Washington, Idaho, Oregon, Montana and California

To describe the feasibility of the VA-ECHO program for sleep medicine - Sleep VA-ECHO

39 participants

26 primary care

7 mental health

6 other specialties

25 unique sites

14 states

August 2014 - initial needs assessment sent to 63 participants

July 15, 2015 - October 1, 2015

Pilot

Descriptive

Survey - individual session evaluation

Survey - summative program evaluation (participants who attended at least one session)

Survey - quality improvement evaluation (participants who attended 2 or less sessions)

5-point Likert scale

Curriculum development based on a need's assessment survey

Sleep, pharmacy and mental health specialists from regional VA and Department of Defence facilities and affiliated academic institutions delivered Sleep VA-ECHO

Intervention: 10 stand-alone, 1-hr sessions combining didactics with integrated case reviews

38% of participants worked in rural healthcare

44% completed the summative program evaluation

93% of respondents anticipated practice change

80% reported increased comfort managing common sleep complaints especially sleep-disordered breathing, insomnia and post-traumatic stress disorder

Follow-up survey of invitees who attended 2 or less sessions reporting scheduling conflicts (62%) and lack of protected time for attendance (52%)

Future work is needed to identify objective measures of practice change and address participation barriers

Program was conducted within an integrated, single-payer healthcare network

Project focussed on a small group of voluntary participants who may be highly motivated

Project focussed on participant perception of value and not objective measures of practice change

Internet or DVD for distance learning to isolated rural health professionals, what is the best approach?

Rakototiana

(2017)[46]

Madagascar

Heads of Health Based Centers (HBC) including doctors and paramedics (nurse or midwife) in three rural regions of Miarinarivo, Moramanga and Manjakandriana

To compare the knowledge acquisition of the heads of HBC for the management of hypertension using 2 training modalities-via internet - videoconferencing (VS) and videoconferencing (VD) (interactive), and via DVD (non-interactive)

56 doctors and paramedics via VS or VD

36 doctors and paramedics via DVD

Empirical quasi-experimental study

Mixed methods

Pre-/post-test

Questionnaires

Focus groups

Statistical analysis

VS was delivered by Medical Institute of Madagascar, VC and DVD was delivered by Faculty of Medicine Antananarivo

Intervention: 2-hours videoconferencing (VS) and videoconferencing (VD) (interactive), and via DVD (non-interactive)

By modality

Retention of knowledge using both methods of delivery demonstrated no difference in the mean score of knowledge obtained (p = 0.076)

By profession

In the internet group the average score after training for both the doctors and paramedics was the same (not significant)

Doctors using internet vs DVD was significant (p = 0.008) DVD group showed greater retention

The paramedic group showed an identical gain in knowledge in both the internet and the DVD group

Acceptance:

All participants had issues with both methods, but DVD was able to be re-watched which meant that could do so if did not understand anything-with internet could not do this. Could not ask questions in either modality

DVD cheaper to implement, and better accessibility as internet coverage not equal across whole of country

Recommendations:

If internet access available choice should be made between internet delivery or DVD

If internet access patchy then DVD would be best choice

No randomisation used to group participants

Limited bias as overall score for basic knowledge was same for both groups prior to training

Videoconferencing and telementoring about dementia care: evaluation of a pilot model for sharing scarce old age psychiatry resources

Doyle

(2016)[24] Australia

Healthcare professionals in regional, rural and remote areas of Victoria and Northern Territory (NT) managing clients with dementia

To evaluate a pilot expansion of mentoring and consultation services to test a model for improving medical supervision and clinical governance for staff within regional and remote areas using remote information technology

18 dementia service staff, including staff from linked services and psychiatrists in regional Victoria and NT

58 interviews were conducted

18 education sessions were conducted remotely

352 staff attended the sessions

186 evaluation forms returned

Pilot

Mixed methods

Using before, mid-point and post-implementation semi-structured interviews and questionnaires to examine orientation, acceptance & impact underpinned by theoretical approaches to evaluation

Likert scale from strongly agree to strongly disagree - 3 open ended questions

Old age psychiatrists located at Victorian and NT Dementia Behaviour Management Advisory Services (DBMAS) sites delivered the model

Intervention: Weekly (reduced to monthly), 1-hour teleconference and web-based team case presentations and education sessions

Education sessions

Participants strongly agreed or agreed to the five-statement indicating acceptance and satisfaction with the program

High satisfaction received for relevant topics to learning needs

Training techniques rated the lowest - this was mainly related to dissatisfaction with the quality of audio and VC facilities

Case conference outcomes

Closed questions form case conference interviews - 59% clinicians felt the process was acceptable and improved impacts for clients through their increased confidence in managing clients with BPSD

Overall, clinicians reported that case conference process contributed to improved outcomes for clients, family and staff

Almost all staff considered the pilot increased their access to professional development opportunities

Particularly beneficial for sole clinicians in the NT who are professionally isolated as they could access professional development, multidisciplinary team, peer support, debrief opportunities

Future education topics were identified

Reduction in travel time across large geographical area utilising VC for education and case conferences

Use of technology proved to be a big challenge

IT difficulties were audio feedback and sound quality issues - problems stemming from overloaded hospital-based servers resulting in intermittent outages and delays

Time and frequency of session was found to be a critical factor in attendance

Lack of direct input from family carers or people living with dementia

Inability to analyse full recording of interviews - leading to crucial pieces of information being missed

Expanding health care access through education: dissemination and implementation of the ECHO model

Katzman

(2016)[33]

USA

PCPs and specialists in rural (and urban) underserved areas

To create a replication and training tool of the ECHO model for the Army's Pain Management ECHO Pain collaboration - ECHO Hub Readiness Replication Model to ensure fidelity, create consistency in the replication process and be adaptable for use by all ECHO programs

150 participants

25% physicians

18% mid-level providers (nurse practitioners, physician assistants, pharmacists or psychologists)

10% licensed rehabilitators

12% nurses or nursing assistants

35% other specialists (integrative medicine providers, health technicians etc)

Mixed methods

Surveys of ECHO Pain boot camp

Focus groups for hub clinicians

Hub readiness ECHO (4-phase) replication model to create consistency

Specialty care teams delivers the program from 4 hubs of Northern Regional USA, Southern Regional USA, Pacific Regional and Europe Regional Medical Command sites

Intervention: cased based learning and evidence-based didactics

Four phases of the model are deliberate and add fidelity to the original four-point model

Surveys demonstrated that both on-site and virtual participants had an increase in knowledge of pain management

During the roll-out participants were eligible to receive over 9,689 hours

The tool can be easily adopted for all chronic and complex diseases and conditions across a large enterprise

Provides flexibility to account for personnel and geographical variation

ECHO Pain has helped to replicate the ECHO programs for many other healthcare-related entities and has the potential to improve the fidelity of the ECHO replication efforts around the world

Limitation of dedicated time and multiple competing priorities are constraints for hub clinicians

Evaluation of American Indian Health Service training in pain management and opioid substance use disorder

Katzman

(2016)[32]

USA

Indian health service (IHS) clinicians from large rural areas in the West, Southwest and Midwest regions including Arizona, New Mexico, Minnesota and Oklahoma

To examine the benefits of IHS training in pain management and opioid substance use disorder to address the high rates of unintentional drug overdose in American Indians/Alaskan Natives

1079 clinicians

92% from IHS, tribal or urban AI/AN programs

28 states

January 2015

Descriptive

Pre-/post-test

Surveys

Statistical analysis

Facilitators at the IHS Pain and Addiction TeleECHO clinic and University of New Mexico Pain Center and ECHO Pain

Intervention: 7, 5-hour training courses in pain and opioid substance use disorder didactics

Significant improvement (p < 0.001) in pre-/post- course knowledge, self-efficacy and attitudes

Thematic responses showed the trainings to be comprehensive, interactive and convenient

Participants suggested that training sessions be shorter, more frequent and adaptable, include more details on pain medications and resolve some technical issues

Project ECHO (Extension for Community Healthcare Outcomes): a new model for educating PCPs about treatment of substance use disorders

Komaromy

(2016)[36]

USA

PCPs in underserved areas

To describe a teleECHO clinic that is focussed on treatment of substance use disorders (SUDs) and behavioural health disorders (BHDs)

654 unique participants

900 unique patient cases

Brief Report

Descriptive

Recruitment for buprenorphine training (DATA-2000 waiver)

Specialists in treatment of SUDs and BHDs at University of New Mexico Health Sciences Center (UNMHSC) delivered the program

Intervention: 30 min didactic session followed by de-identified case presentations, recommendations are summarised and sent to participants

An average of 147 participants joined each year since 2008.

285 participants have attended more than one session since 2010.

Opioids are the most commonly discussed substances (31%), followed by alcohol (21%) and cannabis (12%)

ECHO model provides an opportunity to promote rapid expansion of information on new research and epidemiological trends

Service providers' experiences of using a telehealth network 12 months after digitisation of a large Australian rural mental health service

Newman

(2016)[43]

Australia

Telepsychiatry service providers including rural mental health teams, directors of nursing at rural hospitals, metropolitan-based psychiatrists and registrars, the metropolitan-based mental health team dedicated to rural provider support, rural GPs, administrative staff, and the executive group of the state rural health department in rural and remote South Australia

To study service providers experience of an existing regional telehealth network for mental health care practice 12 months after digitisation in order to identify the benefits of digital telehealth over an analog system for mental health care purposes in rural Australia

Over 40 service providers in South Australia ranging from metro central operations to health providers up to 600km away in same state

70 sites

June - September 2013

Mixed methods

Qualitative interviews

Focus groups

Fieldwork was conducted 12 months after digitisation was introduced

Thematic analyses, focusing on 3 key areas on innovation diffusion theory: relative advantage, technical complexity and technical compatibility

Intervention: Enhanced telehealth network (Digital Telehealth Network - DTN)

Staff support

Remote clinical supervision of social workers and mental health staff

Education/training

Therapy training

Peer support

Clinical case review meetings - clinical learning opportunity and building team rapport

Improved clinical practice

Analysis identified 5 themes and 11 sub-themes

Digitised telehealth network results in a range of relative advantages in access and quality of mental healthcare patients in rural and remote areas

Service providers need support to overcome challenges of achieving compatibility of telehealth with clinical practice

The development of staff training and sharing of innovative uses could help overcome challenges

Greater uptake could be encouraged by service providers being given time, cost and staff support for use (especially GPs)

Paper provided the experiences of providers who had prior experience with an analog system and who were reflecting in the interviews

Study did not include the views of the patients or their families

Improved glycemic control on veterans with poorly controlled diabetes mellitus using a specialty care access network-extension for community healthcare outcomes model at primary care clinics

Watts

(2016)[56]

USA

PCPs at 2 remote rural sites within the Cleveland VA hospital network

To determine the efficacy of the Cleveland Veteran Affairs SCAN-ECHO program for diabetes mellitus

2 PCPs

2 mini-clinics/outpatient sites

39 patients

19 Dec 2012 - 28 Mar 2014

Retrospective chart review

Intervention group: 2 "SCAN-ECHO clinics"/Control group: 2 "usual care clinic"

Statistical analysis

An endocrinologist, health psychologist, nurse practitioner a registered dietician (who are certified diabetes educators) deliver the ECHO program from Cleveland VA Medical Center

Intervention: 24 sessions per year, 20 min didactic discussion, 2-3 case studies and management recommendations

Mean glycolated haemoglobin (HbAIc) improved over the follow-up period, which was statistically significant (p < 0001)

Statistically significant chance that the HbAlc percentages increased over time in the comparison clinics

Future research: randomised-controlled study of usual care vs mini-clinics

Retrospective analysis of patients who agreed to partake

Non-randomised study

Aggregate clinical data was used due to lack of individualised patient data limits the analysis of variance

Impact of telehealth program that delivers remote consultation and longitudinal mentorship to community HIV providers

Wood

(2016)[57]

USA

HIV practitioners in rural and underserved areas

To describe key components of the program, report types of clinical problems requested, and evaluate changes in participants' self-assessed HIV care confidence and knowledge

45 clinicians

172 ECHO sessions

553 case presentations

1051 clinical questions

Feb 2012 - Aug 2015

Prospective

Pre-test/post-test

Surveys

Provider self-efficacy assessments

Statistical analysis

A multidisciplinary team of academic specialists experienced in infectious disease, psychiatry, addiction medicine, pharmacy and social work at MW AETC and University of Washington

Intervention: Weekly, interactive video session, 15 min didactic update on clinically relevant topics, followed by brief question and answer period; and

1 hour of case consultations and discussion using de-identified cases;

written summaries of the panel's recommendations and key points from the discussion

Providers most frequently sought consultation for changing antiretroviral therapy, evaluating acute symptomatology and managing mental health issues

Significant increase (p < 0.05) in participants' self-reported confidence to provide essential HIV care

Significant increase in feeling part of an HIV community of practice

Significant increase in feeling professionally connected to academic faculty, correlating with level of program engagement

Varying length of time between first and last self-assessment

Variations in level of engagement of those surveyed

Reliance on provider self-reported outcomes

Evaluation of a telementoring intervention for pain management in the Veterans Health Administration

Frank

(2015)[26]

USA

PCPs from regional VHA networks - VA medical centres, urban community-based outpatient clinics (CBOCs) and rural CBOCs

To evaluate the impact of the pilot, VHA's SCAN-ECHO pain management program (SCAN-ECHO-PM) on primary care delivery of multidisciplinary pain care

322,435 patients with chronic, noncancer pain (CNCP)

159 PCPs with 22,454 exposed patients; 257 patient cases were presented at SCAN-ECHO-PM

8643 non-participating PCPs with 299,981 patients

7 VHA regional networks

July 2011 - December 2013

Longitudinal, observational

Pre/post comparison

Pain care specialists in physical medicine, mental health, substance use disorder and specialty pain services, nonopioid pharmacological management delivered the program

Intervention: 60 to 90 min sessions, 1-2 weekly, didactic and case presentations on images submitted for consultation

12.9% (exposed and 9.1% of unexposed patients were from rural CBOC

SCAN-ECHO-PM was associated with an increase in use of physical medicine services and initiation of nonopioid medications among patients with chronic non-cancer pain, but not mental health, substance use disorder or specialty pain clinics

Further studies are needed to address facility-level barriers to uptake of SCAN-ECHO-PM program

Provider participation was voluntary

Exposure measure did not indicate the quality or intensity of the training delivered to participating providers

Delivery of curricula may vary across sites or over time

Utilisation of outpatient care or medications has not been captured outside the VHA networks

Not able to assess the impact on patient-centred outcomes eg functional status and quality of life or appropriateness of pain management decisions on individual patients

Treating hepatitis C in American Indians/Alaskan natives: a survey of Project ECHO (extension for community healthcare outcomes) utilisation by Indian health service providers

Pindyck

(2015)[45]

USA

Navajo Area Indian health service (NAIHS) providers including physicians, pharmacists, physician assistants and nurse practitioners for American Indians/Alaskan natives (AI/ANs) in Arizona, New Mexico and Utah

To delineate IHS-specific barriers to utilising the University of New Mexico (UNM) HCV TeleECHO clinic by surveying NAIHS providers interested in treating HCV

13 NAIHS providers mentored to treat at least one patient with HCV ECHO

25 NAIHS providers with an interest in HCV ECHO

15 facilities

March 2013 - 2014

Surveys

Statistical analysis

Active participant survey (APS) IHS providers mentored to treat at least one patient with HCV ECHO

Non-participant survey (NPS) sent to IHS providers with an interest in HCV ECHO

UNM academic medical centre specialists provide ECHO services

Intervention: Monthly IHS exclusive HCV TeleECHO clinics with VC technology

100% of respondents rated being well-informed, self-efficacy, access to expertise, collegiality and decreased professional isolation as moderate to major benefits

50% of APS respondents stated they would not treat HCV without regular participation in the clinic

67% of NPS respondents reported lack of administrative time as the major barrier to utilising this resource

Due to the limited scope of the study a power analysis was not conducted

Survey results are limited by the inherent biases associated with self-reporting and the limited number of respondents

Innovative telementoring for pain management: Project ECHO Pain

Katzman

(2014)[31]

USA

PCPs including physicians, physician assistants, dentists and nurse practitioners in rural and underserved communities

Assess over a 3-year period how the Chronic Pain and Headache Tele-ECHO Clinic (ECHO Pain) met the objectives, and quantify attendance and case presentation data

763 participants

42% physicians/dentists

19% mid-level clinicians

9% other clinicians

30% nurses/other

3835 instances of participation

191 sites

29 states

Jan 2010 - Dec 2012

Mixed methods:

Surveys

5-point Likert scale

Focus groups

Statistical analysis

Annual clinical data

ECHO pain specialists at UNM facilitate the clinics

Intervention: Weekly, 30 min didactics, 2 or 3 de-identified case-based presentations, demonstrations and written summaries

Statistically significant improvements in participant self-reported knowledge, skills and practice were demonstrated

Focus group (14 participants) analyses of 5 topics detailed specific practice improvements through learning and applying concepts and new knowledge from case presentations and didactics to their patients

Future studies will investigate this model within certain replicating sites and health care systems

Palliative care professional education via video conference builds confidence to deliver palliative care in rural and remote locations

Ray

(2014)[47]

Australia

Rural health practitioners including medical students, doctors, nurses and allied health professionals (occupational therapists, physiotherapists, speech pathologists and social workers) across North QLD

To evaluate the educational impact of video conferencing to increase confidence among doctors and other health professionals to provide quality palliative care in rural and remote areas

174 rural health practitioners

101 health professionals completed the study

10 Medical doctors/students

71 Nurses

5 Occupational therapists

4 physiotherapists

2 speech pathologists

5 social workers

4 Other

34 worked in outer regional areas

35 worked in inner regional areas

28 worked in remote location

4 worked in very remote location

Mixed methods approach with a qualitative component that utilised the RATS guidelines on qualitative research

Quantitative/qualitative

Surveys

Educational needs - pre-intervention assessment

4 monthly, evaluation surveys - post intervention

Statistical analysis

Accessibility/Remoteness Index of Australia (ARIA) scores were used to categorise locations

Members of the tertiary level multidisciplinary palliative care team and a palliative care researcher delivered the video conferences

Intervention: Monthly, education lecture-style sessions on palliative care followed by case-based discussion and sharing of resources

Participants rated the content of the video education useful

No significant effect on profession was found (p = 0.088) but there were significant differences in rating of effect between allied Health professionals and both medical doctors (p = 0.033) and nurses (p= 0.018) suggesting that allied health professionals found it less useful that nurses and doctors

No significant difference in content usefulness based on location of work

Medical students/doctors had the highest confidence in palliative care topics pre-intervention, followed by nurses, then allied health professionals

Allied health professionals had significant lower confidence than nurses (p = 0.018) and medical doctors/students (p = 0.013)

Participants in inner and regional rural areas had lower confidence in palliative care that those in rural areas

There was an increase in confidence in the VC palliative care education across all participants with no difference between location

Doctors reported the highest level of confidence but there was no significant difference between the professions

The number of patients with palliative care issues affected had a significant impact on the confidence change with education-those caring for less than 5 pts had a greater confidence change

Results indicate that locally generated content and the ability to deliver education repeatedly enhanced confidence across all participant groups

The VC education was found to be valuable with staff with greater than 20 years finding it beneficial

The discussion that occurred between rural and remote practitioners with tertiary sites assisted to enhance professional networks

The VC education sessions that were provided demonstrated that they met a significant need for rural practitioners by providing regular opportunities for education, professional support and maintaining confidence in their ability to provide palliative care in their context

Further research required to explore other aspects of palliative care in rural and remote areas

Finding of study cannot be generalised as participants self-selected to be involved were limited to those with work email addresses, access to computers and those who were willing to take time to complete surveys

The educational impact of the Specialty Care Access Network-Extension of Community Healthcare Outcomes program

Salgia

(2014)[49]

USA

PCPs including physicians, physician assistants, and nurse practitioners in rural and underserved areas of the VHA network in Michigan, Indiana, North-western Ohio and east-central Illinois

To determine factors that led to participation in the SCAN-ECHO liver program and the educational impact of the program

55 participants

203 cases

183 unique patients

24 survey participants

13 (54%) physicians

4 (17%) physician assistants

7 (29%) nurse practitioners

14 sites

June 2011 - Sep 2012

Descriptive

Pilot

Surveys

Specialists in liver disease deliver the SCAN-ECHO program from the VA Ann Arbor (tertiary care) Medical Center

Intervention: Twice monthly, pre-recorded didactic session and "live" case presentations

More than 75% participated in more than one SCAN-ECHO clinic

76% indicated that the motivation for participation was to learn more about liver disease

75% applied the knowledge they gained to future patients

75% desired to increase collaboration with specialists

71% reported that the didactic component and case-based discussion were equally important

75% had personally discussed information that they had learned from the case presentations with colleagues

42% had helped a colleague care for their patient with their learned knowledge

42% indicated that preventing professional isolation was important

77% indicated the main barrier to participation was their time schedule

Survey design - the potential for recall bias depending on the time between provider participation in SCAN-ECHO and survey completion

Majority of questions were choice questions - to overcome contextual interpretation of answers providers were given the option of free-text responses

Survey studies are limited by response rates

Potential for social desirability bias, as providers who like the SCAN-ECHO program are more likely to respond favourably

Continuing Distance Education: A capacity-building tool for the de-isolation of care professionals and researchers

Bagayoko

(2013)[13]

Switzerland

Healthcare professionals (HCPs) including general practitioners, nurses, medical assistants, midwives and others, using RAFT (Telemedicine network in French Speaking Africa) in isolated care facilities in African countries

To evaluate the impact of distance continuing education to build capacity, increase satisfaction and enhance the performance of care professionals in isolated health care facilities

39 HCPs completed surveys

33% general practitioners

28% nurses

13% medical assistants

8% midwives

18% other professionals

17 African countries with more than 100 connected sites

Descriptive

Survey

62 questions

Likert scale 1-5 designed to probe 11 domains

Statistical analysis

An educational committee consisting of 6 RAFT local leaders (including a recognised medical authority and a medical co-ordinator who is a junior physician) coordinates the educational program

Intervention: weekly webcast followed by 30 min discussion

87% completed surveys

Results suggest that the availability of eHealth access for continuous education, improves retention and facilitates the recruitment of young health care professionals to remote locations

Access to continuing medical education is consistently considered the most important with a preference for live, interactive courses

The possibility to interact with other HCPs is particularly important

Intermittent connectively and mobility are important key drivers of accessing information - the system has been upgraded to allow courses to be downloaded when the user has connectivity and played on mobile devices later

Focusses on reporting education platform

No context or focus on clinicians

Pilot project and evaluation of delivering diabetes work-based education using video conferencing

Maltinsky

(2013)[37]

Health care professionals in peripheral settings within National Health Service (NHS) in the Scottish Highlands regions of Fort William, Golspie, South East Community Health Partnership, Lorne and Islands

To deliver teleconferenced diabetes training to healthcare and allied healthcare professionals who provide base level care and management of people with diabetes, and to evaluate this training

12 participants

face to face group

2 sites

15 participants

videoconference group

3 sites

Pilot

Short communication

Descriptive

Semi structured questionnaires

A control group was established at two sites and content was delivered in the traditional face to face method with the other cohort using video conference in three sites

A trainer from the Highland Diabetes Managed Clinical Network delivered the educational package

Intervention: Didactics, practical demonstrations, discussion and question-and-answer session

Pass rates in both cohorts were similar 9/11 in face to face and 8/10 in video conference

Participants attended training to upgrade their skills and gain confidence in treating diabetes

Concerns raised focused on moderate sound and picture quality, and the lack of interactivity in VC compared to traditional face to face education

Some respondents noted that the decrease in travel/commute time increased their willingness to enrol in training

Recommendations include trainers require development sessions to enable them to transfer their skills to a videoconference environment

Training through videoconference should include regular breaks in order to avoid eye strain and fatigue

Some technical issues highlighted with video conferencing dropping out

Small pilot

No feedback sought from practitioners that chose not to undertake the course and their reasons for not participating

Methodology and what have been evaluated are unclear

Outcomes are unclear

The use of telemedicine to train perioperative nurses in rural settings

Seibert

(2013)[51]

USA

Perioperative, registered (RN) nurses in rural areas of Idaho and Oregon

To investigate whether a preceptor programme delivered by telemedicine could be as effective as one delivered in person for nurses working in the OR

35 RN working within a perioperative area

22 RN (telemedicine group) at 6 rural hospitals in Idaho and Oregon

13 RN (person group) at 1 regional medical centre in Idaho

2007 - 2013

Comparative study between a rural RN group and an urban RN group undertaking an OR training program over a 6-year period

Rural group - telemedicine preceptorship model

Urban group - in situ preceptorship model

Both groups undertook the AORN examination at the end of the course

Post course completion satisfaction survey

10 RN preceptors at an urban hospital delivered the programme

Intervention: Self-paced preceptor 26-module curriculum and weekly sessions of questions and answers and setting weekly goals

No significant difference between both groups in final exam scores-all participants passed

No significant difference in the time taken to complete the course between both groups

No significant difference in the clinical skills assessment scores between both groups

Both the telemedicine and urban RN groups thought the program was successful (92% urban vs 91% telemedicine)

The preceptors considered the program to be successful

86% telemedicine group believed the course increased their knowledge and would strongly recommend it to others

Both telemedicine group and urban group education delivery models were equally effective in training perioperative nurses

Variables (sex, age, computer knowledge, prior hospital exp) did not affect performance in the training

Research suggests that telemedicine should be considered for training of perioperative nursing and that it may be able to be generalised across nursing in general

?? small group/numbers

Due to the lengthy time of research the program may have been tweaked to improve it over time which may have biased the outcome?

? based in the US do not know if it compares across other countries OR training with educational content etc

Developing telepsychiatry services in KwaZulu-Natal - an action research study

Chipps

(2012)[18]

South Africa

Psychiatrists, medical offers and other staff in the rural province of KwaZulu-Natal (UKZN)

To develop telepsychiatry services in the province and to facilitate change to the current psychiatric outreach services

Note: Paper structure was hard to follow

12 participants

8 completed surveys

4 medical officers

1 medical student

1 manager

2 Other

3 hospitals

2009 - 2011

Mixed methods

Action research study

Cycle 1 & 2: Participant satisfaction survey

Cycle 2: Qualitative Interviews

Pre-/post-test surveys

Statistical analysis

Teaching staff from Department of Psychiatry at the University of UKZN and a consultant psychiatrist and registrar from two hospitals

Intervention: Telepsychiatry service including specialist and non-specialist education in psychiatry and the provision of clinical videoconference consultation as part of the routine clinical outreach service.

Cycle 1: Telepsychiatry -online remote interactive system of multiple PowerPoint presentations

Cycle 2: DVD sessions

5 mental health professionals rated satisfaction with different bandwidths and reported that 128Kbsp is suitable for education and 384Kbps is preferred for clinical consultation

Videoconference and DVD sessions were beneficial to health staff without post-qualification psychiatry training

Building capacity to reduce disparities in diabetes: training community health workers using an integrated distance learning model

Colleran

(2012)[20]

USA

Community health workers (CHWs) including paraprofessionals, promoters, lay health workers and community health representatives in native communities across New Mexico with direct involvement with diabetes patients

To determine whether the ECHO integrated distance training program will increase CHW competence related to diabetes care

23 participants

21 completed

All female

61% Native American

35% Hispanic

Mixed methods

Pre-/post-test surveys

Focus groups

Statistical analysis

Project ECHO infrastructure delivered the teleECHO program

Intervention: Two, 2-day, on-site training session and weekly teleECHO sessions of didactics on program syllabus, case presentations and discussion, informal discussions, question and answer, and resource sharing

6-month program

Significant improvement in participants' diabetes knowledge (p = 0.002), diabetes attitude (p = 0.04), confidence in clinical and nonclinical skills, respectively (p < 0.001 and p = 0.04)

Focus group discussion reported that participant organisations and systems currently do not provide the needed support and resources

Focus group discussions indicated that participants gained competency, confidence and resources for their work and communities

Studies are ongoing to determine how participation affects the advancement of role of CHWs and whether the training has a positive impact on patient outcomes

Many participants had trouble connecting to the weekly TeleECHO sessions due to computer or internet access and they connected by telephone instead

6-month format created the potential for participant dropout - 2/23 dropped out due to retirement or relocation

Bridging the distance: a prospective tele-oncology study in Northern Norway

Donnem

(2012)[22]

Norway

Cancer health care providers including physicians, nurses, and others at 1 local hospital and in 5 remote communities in Alta (distance from University Hospital of Northern Norway (UNN), 309 km), Vadso (748 km), Honningsvaag (517 km), Karasjok (502 km), and Porsanger (481 km) in Northern Norway

To evaluate the feasibility and benefit of the use of videoconferences (VCs) as a tool to support cancer health care providers

106 VCs

101 patients

167 patient cases discussed

1 tertiary level hospital providing cancer services

1 smaller primary local hospital as well as

5 rural area communities of health services

18-month period from Spring 2009

Prospective registration study

Descriptive data, individual patient discussion via VC

Survey

Clinicians at UNN deliver the teleconferences

Intervention: Weekly, VCs where clinicians discuss patient cases with specialist cancer care service providers from a major tertiary facility

Days waiting for a VC consultation was significantly shorter (p = 0.001) than the estimated waiting time for an alternative consultation

Patients stayed at their home facility in 82% of cases after VC

Transfer and admission to tertiary facility reduced 13% to 6% with primary health care providers expressing that VC improved patient care in 85% of cases

There were minimal technical issues (77% of cases)

VC appears to be a useful supplemental tool to support health care providers in rural communities and primary local hospitals

Clinicians felt that VC improved quality of patient care, made them feel more confident that the care that they provide is adequate

Due to significant difference in waiting times (VC versus alternate consultation) it was felt that VC provided a more efficient service for the patient

Use of VC made local clinicians feel more confident in the delivery of adequate care

Small scale study that needs to be replicated before being able generalise to greater population

Role of telehealth/videoconferencing in managing cancer pain in rural American Indian communities

Haozous

(2012)[28]

USA

Rural health care providers including MDs, PAs, NPs, RNs, pharmacists and health assistants caring for American Indian /Alaskan Native (AI/AN) cancer patients at rural tribal clinics and hospitals in Washington State and Alaska

To determine the feasibility and effect of using telehealth/VC to deliver cancer-related pain management education and case consultation to health care providers in rural AI/AN communities

52 providers

11 sites (62-922 miles from the nearest pain management specialist)

4 educational sessions

Average 17 providers/session

Average 5 sites/ session

93 providers from 16 non-duplicated sites (74-1127 miles from the nearest pain management specialist)

9 case conferences

Average 10 providers/session

Average 4 sites/ session

32 providers who did not attend the case conferences (comparison group)

Cross-sectional

Descriptive

Surveys

Telehealth satisfaction survey

Perceived competence scale - 7-point Likert scale

Statistical analysis

Consulting pain management specialist at the University of Washington delivered the educations sessions

Intervention 1: Monthly video-conferenced educational sessions in cancer -related pain issues

Intervention 2: Case conferences - 15 min expert presentation; brief Q & A; case presentations; management recommendations

46% educational session attendees completed surveys

34% case conference attendees completed surveys

Educational and case conference participants both reported a high level of satisfaction with the telehealth system

Case conference participants scored a higher self-perceived competence in treating pain compared with the comparison group

Future studies with focus groups or interviews may provide a greater analysis of program impact on participants, and strengths and areas for improvement

Pre-tests and post-tests were not implemented

No randomisation of providers into telehealth and comparison groups

Project ECHO: a model for complex, chronic care in the Pacific Northwest region of the United States

Scott

(2012)[50]

USA

Clinicians from (sparsely populated, rural) Pacific Northwest

To teach rural providers how to evaluate and treat complex, chronic health conditions including hepatitis C, chronic pain, integrated addictions and psychiatry, and HIV/AIDS by using telehealth and case presentations

900 clinicians

700 patient cases

2009 - June 2012

Pilot

UW clinicians presented the videoconference clinics

Intervention: 1-hour, weekly videoconference - 15 min didactic session on a topic relevant to the disease being discussed

23 videoconference clinics for Hepatitis C - 263 clinicians, representing 399 patient cases

167 patients started antiviral therapy for Hepatitis C

Project expanded to include:

16 clinics in addiction and psychiatry

97 clinics in chronic pain

13 clinics in HIV/AIDS

Plan to provide clinics in other chronic, complex conditions, including rheumatology, cardiovascular disease, chronic pulmonary diseases and child psychiatry

Reluctance of some participating clinicians to present cases

Unknown methodology

Partnering urban academic medical centres and rural primary care clinicians to provide complex chronic disease care

Arora

(2011)[11]

USA

PCCs including physicians, nurses, pharmacists, nurse practitioners, physician assistants, health educators and community health workers in underserved, rural areas

To describe the Project ECHO model and discuss early results

34 participants in 12-month survey

94 participants in 2010 Project ECHO annual meeting surveys

Descriptive

2 survey types

12-month participation survey

Annual meeting survey

Likert scale

Statistical analysis

Specialist HCV team at UNMHSC, including a hepatologist, pharmacologist, mental health specialist a nurse delivers the telemedicine clinics

Two-day, in-person orientation in HCV treatment protocol as well as ECHO model format

Intervention: Weekly 2-hour telemedicine clinics - case-based learning, case management, brief didactic presentations

12-month survey:

88% of PCCs agree with the importance of becoming well-trained in hepatitis C care

58% agree with the importance of becoming the local hepatitis C expert

91% agree with importance of support of teleconference network

73% agree with the importance of providing consultation to other PCCs in the care of hepatitis C patients

1-day, ECHO annual meeting survey:

94 PCCs provided mean ratings of 3.66 - 4.73 associated with best practice care, connecting with peers and enhancing professional development and

60-72 PCCs strongly agreed with benefits to patients

The model needs to be incorporated into the health care system so that reimbursement mechanisms are created to pay for the various services it provides

Enhancing access to cancer education for rural healthcare providers via telehealth

Doorenbos

(2011)[23]

USA

Healthcare providers including physicians, nurse practitioners, nurses, physician assistants and community outreach workers for AI/AN people in rural Washington and Alaska

To demonstrate that telehealth technology is a feasible method for delivering real time, interactive cancer education to multiple rural sites

368 participants (some participants attended multiple sessions

131 sessions

28 tribal clinic sites in Alaska and Washington State, mean of 5 sites joined the VC presentation

Open ended survey

Written satisfaction survey after each of the first 10 monthly sessions

Survey used Likert scale questions to gather information about use of telehealth for professional education sessions

Responses were anonymous

Healthcare providers were engaged in selecting topics

Experts from Native People for Cancer Control Telehealth Network (NPCCTN, including physicians, a nurse scientist, a nurse practitioner, a clinical psychologist, a legal expert, and an Alaskan native healer delivered the presentations

Intervention: Monthly, 45 min cancer education series and 15 min Q&A

54% returned the survey

Health care providers in rural areas face barriers to obtaining continuing education including geographic isolation, distance from tertiary care, lack of financial resources for travel, inability to take time away from work due to lack of coverage

Overall satisfaction with telehealth was high, mean rating 3.6 on a 4-point scale

Participating providers rated their feeling about telehealth prior to and after their first educational session. Scores increased from 3.0 - 3.48, indicating use of telehealth improved in satisfaction ratings

Usefulness of information provided by educational sessions rated high with a mean of 3.59

Open ended responses supported the use of VC technology in making continuing professional education a reality for rural healthcare providers

The cancer education series demonstrated that telehealth technology is a feasible method for delivering real-time, interactive cancer education to multiple rural sites

Telehealth is a viable solution to the barriers faced by rural health providers in obtaining continuing education

Incorporating participant input during program planning enhances content relevance for providers, and ultimately may enhance the delivery of quality cancer care

Results depend of the technology used

Technical difficulties with VC delivery of education are typically minor

Lag in audio transmission can hinder discussion when learners are from two sites respond simultaneously

No clinic left behind: providing cost-effective in-services via distance learning

Knapp

(2011)[35]

USA

MD, PhD and BA staff based at remote-located clinics (RLC) and full-service medical centres (FSMC)

To implement and examine the cost-effectiveness of a distance-learning model to offer provider education in HIV to geographically remote sub(facilities)

91 online participants

30 RLC sites

129 in-person participants (control)

10 FSMC sites

Descriptive

Comparative study - Online training package vs in-person training for HIV point of care testing

Surveys

Likert scale

An RLC clinical manager delivered the presentations

Intervention: 20-minute power point presentation with 5 min Q&A and discussion

Participants rated in-person presentations higher than online, however mean scores for both methods were > 80%

Online presentations were found to be considerably more affordable than in-person

A significant increase in HIV testing with a 200-300% increase in testing among at-risk patients compared with the control site which showed no improvement

Cost of initiating in-person presentation in one state was five-fold compared with the remote approach

A potential limitation is that all states were already doing HIV testing, so study was directed at tools and techniques to help increase HIV testing

It is unclear if remote learning would work for introducing a completely novel intervention

Expanding access to Hepatitis C virus treatment - extension for community healthcare outcomes (ECHO) project: disruptive innovation in specialty care

Arora

(2010)[12]

USA

PCPs in federally qualified health centres (FQHCs) in rural and underserved areas of New Mexico state and prisons

To describe the ECHO model and its application in Hepatitis C virus (HCV) care in detail and present data from initial surveys of participants

415 HCV telehealth clinics

21 participating HCV agencies

More than 5000 patient consultations for HCV

69% of patients are from rural areas and prisons

Descriptive

2 survey types

6-month participation survey

Annual meeting surveys for 2006, 2007 and 2008

Likert scale

Statistical analysis

Multidisciplinary specialists from UNMHSC in medical specialties, mental health and substance abuse deliver the ECHO model

Intervention: Weekly 2-hour telemedicine clinics - case-based learning, case management, brief didactic presentations

HCV providers reported after 6-month participation a significant improvement in providers' knowledge (n=52); a source of learning (n = 38), and professional satisfaction

Annual meeting survey (2006) (n=17) reports a reduction in sense of professional isolation, expanded access to specialists and providers' professional enhancement

Annual meeting survey (2006/2007) (n=25) reported an increase in competence and ability to serve as local consultants about HCV to other providers

Annual meeting survey (2007) (n=25) reports a large degree of transfer of knowledge to benefit care of other patients (with diseases not represented at ECHO) and to clinical staff

Annual meeting survey (2008) (n=23) reports a transfer of knowledge to clinical care, confidence in patient safety and quality of care and support in their clinical environment

Model can be adapted to a broad range of chronic health issues

ECHO model has expanded to cover 12 additional diseases

255 partner teams

More than 10,000 patient consultations

Potential of self-selection bias of respondents

The number of ECHO HCV providers is small

Rural professionals' perceptions of interprofessional continuing education in mental health

Church

(2010)[19]

Canada

Mental health professionals including community development workers, doctors, nurses, nurse practitioners, occupational therapists, recreational therapists, paramedics, dietitians, pharmacists, police, school counsellors, social workers, youth workers and clergy in 6 rural Newfoundland and Labrador (Canadian) communities

To design and assess the impact of the Rural Mental Health Interprofessional Training Program (RMHITP)

125 mental health professionals

37 nurses

35 social workers

8 community development workers

7 nurse practitioners

7 police/justice

7 physicians

4 clergy/pastoral care

4 clerical staff

3 ambulance attendants/paramedics

3 school counsellors/psychologists

3 youth regional co-ordinators

2 dieticians

2 occupational therapists

2 recreational therapists

1 pharmacist

4 did not indicate profession

45 completed a confidence questionnaire

15 professions

Sep 2006 - Dec 2007

Mixed methods

Pre, during, post

questionnaires

5-point Likert scale

Written feedback

Focus groups

Interviews

Field notes

Statistical analysis

A psychologist from the primary health care team in each rural community delivered the sessions except for the one on crisis intervention which was led by an outside expert

Intervention:10-session program primarily via videoconference; 8 domains of mental health practice

First 2 domains were presented on-site in 1 day

Program had a beneficial effect on professionals with mental health training and they attended more regularly

High levels of satisfaction for all topics and all aspects of the presentations, and opportunity to interact with other professionals

Participants reported developing a more reflective mental health practice, integrating new knowledge and skills into their work

Increased confidence using a range of mental health interventions

Interprofessional referrals, inter-agency linkages and collaborations had increased

Least satisfaction with videoconference technology was mitigated by the strong connection between the facilitator and participants

Future research will measure the impact of such a program on professionals'

practice and patient care

Not possible to have a control group as study was one group, pre-test/post-test

Virtual grand rounds: A new educational approach in social work that benefits long-term care providers and patients in rural Idaho

Cunningham

(2009)[21]

USA

Social workers and residential care co-ordinators in rural Idaho

To examine participant satisfaction of the Telehealth Idaho virtual grand rounds (VGRs) education delivered via interactive videoconferences and to promote retention in their workforce of social workers and residential care co-ordinators

359 participants

July 2002 - Dec 2006

Quantitative

Questionnaire

Five-point Likert scale

Specialist presenters (often located outside Idaho delivered the VGRs

Intervention: Telehealth Idaho Program - Nine, 2.06 hours (per participant) sessions of VGRs

79% evaluation forms returned

Participants approved of the delivery method and the overall satisfaction rating was 4.1 out of 5

Whether the information presented would result in a change in practice' scored lower at 3.25

Overall a positive score

Model of education could easily be duplicated throughout regional centres in the USA

This model provided education to a cohort that may not have received or been able to access previously; therefore, increasing knowledge and decreasing cost for travel

The program was expanded to include other facility staff and other topics

Unclear how many episodes of education was provided as individual episodes

Numbers presented as a total of 359 attendees x 2.06 hours

The familiarity of using the technology (set up and dialling in) and the interaction would increase with the frequency of telehealth education episodes and therefore improve the level of satisfaction and potentially impact 'change of practice'

Content was discussed but not how it was transformed into content that was meaningful and relevant - as such a slightly lower 'change in practice' score

Survey of a videoconference community of professional development for rural and urban nurses

Newman

(2009)[42]

Australia

Healthcare professionals including registered nurses, clinical nurse specialists, clinical nurse consultants, clinical nurse educators, nurse managers, students, allied health professionals, mental health workers and aboriginal health workers in rural and urban regions of New South Wales (NSW)

To present findings of a survey of a videoconference community in NSW in 2008

75 participants

55% completed the survey

4 hospital sites in NSW

Macquarie - 46 participants

Wyong - 15 participants

Gosford - 3 participants

Broken Hill - 11 participants

Quantitative

Salmon's principles of building online communities

access and motivation, socialisation, information exchange and knowledge construction

Surveys

Four-point Likert scale

Intervention: Videoconference symposium (focus on videoconferencing) to build a learning community among leading academic, researchers and clinicians in a more accessible ways than by attendance at traditional conferences

56 attendees (55%) completed the survey

VC technology has the potential to enable rural and remote healthcare professionals to participate in a community for professional development

Thorough preparation, rehearsals and the presence of technicians minimised possible technical difficulties that could be faced as cited in literature

There was overall success with respondents across the 4 areas indicating positive results

The attempted move toward knowledge construction was not successfully achieved, despite encouragement of an interactive environment

It was identified that more research needs to understand how to engage an interactive environment, suggesting that a naturalistic approach to communication that avoids a mediator at each site repeating questions and channelling information may be productive

The convenience nature of the sample and the response rate of 55% must be factored into the interpretation of survey results

Temporary loss of visual link with Broken Hill and Wyong which impacted on timing of the symposium

Limitations to the ability for those on screen to see the full audience at the larger site of Macquarie

Questions form the audience must be restated by the speaker or site convenor for all community members to hear

Electronic mentoring: an innovative approach to providing clinical support

Stewart

(2009)[55]

Physiotherapists in paediatric clinical practice in rural British Columbia, Canada

To implement and evaluate a pilot e-mentoring programme to provide support of rural physical therapists

1 experienced physiotherapist mentors

2 sole charge physical therapists with no paediatric experience

3 months in 2007

Pilot

Mixed methods

Pre-test/post-test questionnaires

Field notes

Final group meeting

7-point adjectival scale

Action research design

The primary researcher, a physical therapist with 24 years of paediatric experience assumed the role of mentor and delivered the program

Intervention:

iChat - 2 times/week

Case study -monthly videoconference meeting involving all 3 participants

Mentees reported feelings of isolation dissipated with mentoring

Videoconferences were helpful in developing clinical reasoning

It was more difficult to type complex clinical questions than articulating the questions verbally

Enabling factors: technology (lack of technological problems), logistics of communication flexibility and timing, attitude and commitment of the mentor and mentee, the colleague-supporting-colleague relationship

Mentor/mentee interaction facilitated outcomes - communication through group learning, improved clinical reasoning, confidence in clinical decision-making and knowledge translation through skill building and access to resources

Further research is needed to evaluate long-term benefits and effectiveness of e-mentoring

Sample size and program length were chosen for pragmatic reasons - conducted as part of a master's program

Continuing professional development for Australian rural psychiatrists by videoconference

Greenwood

(2008)[27]

Australia

Rural psychiatrists and trainees in rural and remote areas of Australia

To decrease professional isolation and facilitate opportunities to obtain credit in ongoing professional registration and to provide peer review for psychiatrists working in isolation

Overall aim: to improve retention and recruitment of psychiatrists to rural and remote areas

106 participants

67% consultant psychiatrists

17% trainees

16% allied health professionals

6 seminars

June 2006 - June 2007

Mixed methods

Online surveys - evaluations were completed following each seminar

Qualitative data was also obtained by telephone interviews with randomly selected psychiatrists from each state and territory

Royal Australian and New Zealand College of Psychiatrists (RANZCP) members were surveyed for topic and speaker preferences

Intervention:

6 video-conferenced seminars - interactive participation between rural psychiatrists and specialists

West Australian Pilot peer review group - 6 participants/6 meetings: case presentations and discussions of supervision issues

83% of respondents found all six events they attended via VC beneficial and preferred to attend seminars from remote sites than to travel significant distances

84% agreed that participating via VC reduced their feelings of professional isolation

Concluded that VC is now a proven, effective and accessible training mode for the delivery of education to rural health practitioners

VC enables live and synchronous exchange

Focus on psychiatrists

Two pieces of work included in the one article The National Program and the WA Program

Bridging the distance: Educating nurses for Telehealth Practice

Sevean

(2008)[52]

Canada

Registered Nurses

(preoperative and oncology) in remote communities of North-western Ontario

To explore the impact of telehealth technology on health assessments performed by nurses delivering health services to isolated populations

5 nurse educators

37 registered nurses

13 communities

Mixed methods

Pre- post- test

Pre- workshop survey and a 42- item questionnaire to identify demographics and learning needs was completed

Post survey consisting of open- ended questions

Likert scale

Statistical analysis

Nurse educators from the regional teaching hospital delivered the presentations

Intervention: 1-day, telehealth (videoconference) workshops included telehealth protocols and policies, physical assessment skills, perioperative skills and oncology assessment skills

Outlined the advantages of using telehealth not only for nurses working in remote locations but the benefits for families and community members that reside in geographically isolated areas

Example: Oncology - 2-year study - tele-hospice model

Highlighted the need for a confidential learning environment, to create a sense of space for both parties - a head and shoulder view of participants and to provide a feeling of being 'up close and personal'

Location, room, background, audio, lights and power included as part of the recipe for success

Approx. 50% of nurses indicated a need for orientation and hands on practice with the telehealth equipment

All nurses indicated that the education session exceeded their expectations in terms of being interesting, informative, practical and well organised

The Nurse educators had difficulty adapting their teaching style to the VC classroom More refinement required to cover high priority topics for future workshops

Highlighted that there is 'format' required and style' to deliver successful content via telehealth

Lacked depth in design. Could have gained more insight if conducted face to face interviews for participants and nurse educators

Unsure of the level of expertise of nurse educators; highlighted a need that there is a 'format' needed to deliver successful education via telehealth

No details are given about the format of the videoconference

Academic health centre management of chronic diseases through knowledge networks: Project ECHO

Arora

(2007)[10]

USA

PCPs including nurse practitioners, physicians, physician assistants and pharmacists in rural community clinics in underserved areas

To demonstrate how a partnership of academic medicine, public health, corrections and community health centres can foster the capacity of rural physician partners to provide safe and effective treatment of HCV infection in any population, consistent with the accepted standard of care.

29 PCPs

Aug 2004 - June 2005

Descriptive

Annual survey (2006)

5-point Likert scale

Intervention: Weekly, 2-hour, short didactic sessions and case presentations

96% reported enhanced knowledge about management and treatment of HCV patients

84% cited access to expertise in behavioural and mental health resources as helpful when treating HCV patients

92% believed they had obtained competence in treating HCV patients

From July 2016 Project ECHO has been expanded to substance abuse disorders, rheumatology, gestational diabetes, and management of health disorders

The methodology of Project ECHO can be generalised to many common, complex and chronic conditions

A pilot evaluation of distance education modalities for health workers in the US - affiliated pacific islands

Chen

(2007)[17]

USA

Healthcare workers including doctors, nurses, dentists and

others in US affiliated pacific islands (USAPI) participating in the 2004 American Pacific Nursing Leaders Council (APNLC) and the Pacific Basin Medical Association (PBMA) annual conferences in Pohnpei, Federated States of Micronesia (FSM)

To assess the effectiveness of various distance education modalities for health workers in the USAPI

59 participants

13 hospital nurses

1 public health nurses

2 nurse midwives

2 nurse practitioners

15 doctors

3 dental (other)

3 health assistants

3 teachers

2 dentists

4 others

(1 missing)

Pilot

Pre-test/post-test questionnaires

Statistical analysis

A physician from University of Hawaii presented the modules from the main library at FSM national campus

Intervention: Training modules in diabetes/oral health and metabolic syndrome

3 distance education modalities:

live videoconference

live audioconference

Recorded computer-based slide presentation

Control live lecture - 45 min educational presentation

All "live" participants were able to ask questions

Comparison of pre-test and post-test scores for tested knowledge gain showed statistically significant score increases among live lecture and videoconference groups for diabetes/oral health modules and for all 3 modalities for the metabolic syndrome module

Computer based training was well-accepted

Results are used to guide ongoing development and evaluation of distance education resources

Lack of power in determining differences between modalities due to the relatively small numbers in each group

Limited ability to measure differences in knowledge gain between the groups due to small numbers of test questions and participants

Limited ability to generalise the findings to allied health workers in the region

Anonymous feedback was not provided so opinions may not be representative of the group comment session in the presence of the facilitator

The study did not attempt to address technical limitations

Education for regional health professional using mobile videoconferencing

Harris

(2007)[29]

Australia

Clinicians, mostly nurses in Emerald and Mount Isa

To determine the feasibility of this form of distance professional education.

To encourage use of telehealth / VC to augment health practice

To support remote / regional hospital staff

To provide opportunities for case discussions and facilitate inter professional dialogue

To promote closer working relationships between physicians, nurses and allied health practitioners in remote regions and colleagues at tertiary hospitals

201 participants

2 hospitals

23 clinical forums

March 2007 - June 2007

Short article

Quantitative

Surveys

5-point Likert Scale

Statistical analysis

Paediatric consultants, clinical nurse consultants and allied health professionals with expertise in paediatrics presented the forums from Royal Children's Hospital, Brisbane

Intervention: Weekly, telePaediatric clinical forums involving formal and informal presentations, clinical case discussions, demonstrations and questions and answers.

166 participants completed surveys

88% participants agreed or strongly agreed that the sessions were relevant, of adequate depth (86%) and provided new content (100%)

Participants consistently indicated that they had opportunity for questions and discussions

Participants also appreciated the opportunity to access specialists located in the tertiary hospital

These sessions were delivered in the ward which was viewed as an advantage for engagement of staff

97% agreed or strongly agreed that the sessions should be continued routinely

Limited to two hospitals

Hepatitis C videoconferencing: The impact on continuing medical education for rural healthcare providers

Rossaro

(2007)[48]

USA

PCPs including doctors, nurse practitioners, physician assistants and registered nurses in rural clinics in Northern Carolina

To compare the impact of VC(VC) versus standard lecture (ST) on education regarding natural history, diagnosis and management of HCV

175 participants

68 physicians

27 nurse practitioners/physician assistants

80 registered nurses

62 VC

113 ST

14 sites

Prospective study

Quantitative

Pre- post- test

Surveys

Statistical analysis

A gastroenterologist and educator at University of California delivered the VC and ST lectures

Intervention: VC and ST lecture

All types of learners improved their knowledge scores following intervention via VC

Registered nurses showed the greatest improvements

Results demonstrate that VC is equivalent, if not better than standard continuing medical education

VC can potentially improve clinical education regarding the history, diagnosis and management of HCV VC has the potential to eliminate the financial and geographic barriers to professional education for rural practitioners

To ensure consistency the standard lectures and VC content were delivered by the same person

A robust quantitative method employed which demonstrated improved learning

However, more depth would have been gained if face to face interviews were conducted to evaluate the perception and true impact of VC and how the knowledge gained is transferred into clinical practice

Using the project extension for community healthcare outcomes model to train rural primary care clinicians to deliver child mental health services in Oregon

Cheng

(2017)[58]

USA

PCPs in rural areas of Oregon

To demonstrate that the Project ECHO model is an effective means for providing child psychiatry education to rural primary care providers

31 participants

Aug 2016 - Mar 2017

Conference abstract

Pre-test/post-test

Surveys

No description of intervention - ECHO consultation and education model

23/31 completed the pre-program survey

19/31 completed the post-program survey

27/31 participants (87%) completed the program

Post training 26% provided mental health psychoeducation in their daily practice compared to 9% prior to ECHO

Post training 76% of participants felt comfortable treating patients with mental health disorders compared to 43% prior to ECHO

Improved confidence in using psychotropic medications increased from 47% to 89%

Project ECHO: Bringing palliative care consultation to rural New Mexico through a novel telemedicine format

Marr

(2012)[60]

USA

Multidisciplinary PCPs in rural areas of New Mexico

To determine the feasibility of a telemedicine curriculum, Palliative care ECHO clinic, to educate rural PCPs about palliative care

Number of participants is not provided

Number of sites is not provided

Initiation in April 2011

Conference Abstract

An interdisciplinary team delivers the ECHO program

Intervention: Weekly, 2-hour short didactic, case presentations, and case discussion and recommendations by an interdisciplinary team at UNM

Feasibility was demonstrated

Provider satisfaction was high

Self-efficacy improved

Sense of isolation decreased

Further expansion to other regions of the country

Not enough detail in this abstract to gauge strengths or limitations

Project ECHO: improving asthma care in New Mexico with telehealth technology

Harkins

(2011)[59]

USA

PCPs in FQHCs in rural and underserved areas of the state, prisons, schools and NM Dept of Health

To engage providers and allied health personnel as partners to follow the national guidelines for asthma treatment and education

200 participants

50 sites

Conference abstract

Pilot

Specialists at UNM deliver the clinics

Intervention: Bi-weekly clinics, didactic sessions and case presentations and hands on training at Adult and Paediatric Asthma Clinics at UNM

Increased number of nationally certified asthma educators from 22 to 40

Not enough detail in this abstract to gauge strengths or limitations

Dermatology ECHO - an innovative solution to address limited access to dermatology expertise

Lewis

(2018)[62]

USA

PCPs in rural Missouri

To identify practical applications and utilization of telemedicine VCtechnology for treatment and management of various skin conditions

1 participant

Patient case study from rural Missouri

Observational

Clinical case report

The Dermatology ECHO hub team at the University of Missouri consisted of general dermatologists, Paediatric dermatologists, a dermatopathologist, a clinical psychiatrist and an advanced practice nurse

Intervention: Weekly, 1 hr long sessions including brief didactics and 3-5 de-identified case discussions

Dramatic improvement in specific patient's condition to PCP participation to guide diagnosis and treatment of bacterial skin infection

Further investigation will assess pre- and post- Dermatology ECHO treatment outcomes as well as participation referral rates to dermatology speciality clinics

1 patient case study

Making connections: using telehealth to improve the diagnosis and treatment of complex regional pain syndrome, an underrecognized neuroinflammatory disorder

Katzman

(2013)[61]

USA

PCPs from varying clinical backgrounds in rural areas of 10 states

To partner the UNM Project ECHO Chronic Pain telehealth clinic with the Reflex Sympathetic Dystrophy Association (RSDSA) and UNM Clinical Pain Center to develop a month-long curriculum dedicated to chronic regional pain syndrome (CRPS) to increase the knowledge and self-efficacy of primary care clinicians in treating CRPS

62 unique attendees

153 participants

August 2012

Case study

Descriptive

Surveys

Clinician specialists from the UNM interdisciplinary Pain Center deliver ECHO Pain

Intervention: CRPS themed month of weekly sessions of didactics and de-identified patient cases

95% of the items rated by participants in each weekly session were 4.0 or higher on a 5-point rating scale implying strong agreement that the sessions were effective, relevant to their practice and that they were likely to apply information in the future with their patients

Clinical services and professional support: A review of mobile telepaediatric services in Queensland

Smith

(2010)[63]

Australia

Clinicians in rural and regional hospitals in Gladstone, Mt Isa, Emerald, Gympie, Nambour, Hervey Bay and Townsville

Providers across the multidisciplinary team were able to access services for general paediatric support, sub-specialist support and education and training

To describe clinician experiences using mobile (robot) VC systems for the delivery of paediatric support in the clinical setting

7 sites

966 consultations

465 sessions

228 hours of VC

Jan 2005 - July 2010

44 remote education sessions

3 sites - Mt Isa, Emerald and Gympie

Average 7 participants

Mar 2007 - Jun 2008

Descriptive

Retrospective review and case studies

3 case studies - consultation only

Specialists at Royal Children's Hospital (RCH), Brisbane delivered the education sessions

Intervention: 35-40 min robot education sessions involving informal case conferences providing support in paediatrics

The telehealth mobile systems served three key functions, provision of general paediatric support, sub specialist paediatric support, and professional education

Some outcomes supported by case studies showed increased communication and inter facility teamwork

Decreased travel times and increased efficiency for senior clinicians

Potential decreased incidences in transferal of patients to tertiary centres

The mobile telehealth units provided a convenient mechanism to deliver staff education and to promote networking opportunities

The system had a single point of contact allowing for easier access and greater uptake of the resource

This also allowed for deceased training time to become familiar with the resource

Technical difficulties with VC delivery of education are typically minor

No indication of user satisfaction of the resource and what feedback the end user was required to provide

The impact of Project ECHO on participant and patient outcomes: a systematic review

Zhou

(2016)[65]

Canada

PCPs from remote areas

A review of the evidence of the impact of all Project ECHO programs on participant and patient outcomes

39 Project ECHO studies addressing 17 medical conditions

Jan 2000 - Aug 2015

Systematic review

Quality assessment of outcomes based on Moore's evaluation framework Level of evidence: Level 1: participation, Level 2: satisfaction, Level 3: learning, Level 4: competence, Level 5: performance, Level 6: patient health, Level 7: community health

Specialists at academic health centres deliver Project ECHO

Intervention: All Project ECHO programs

30 studies addressing populations in remote areas

2 studies met criteria for high fidelity

28 studies reported at least 1 outcome from Moore's evaluation framework

13 studies used surveys and/or semi-structured interviews

4 studies assessed PCPs knowledge with pre-/post-test comparison and thy reported a high level of satisfaction with the intervention

7 studies reported an increase in PCP competence

It is unclear whether self-reported confidence is a good measure of clinical competence

1 study tested participants' performance however there was no control group

6 studies suggested changed patient outcomes

2 studies suggested it was cost effective

Further research examining efficacy is needed and identifying and addressing potential barriers

Inclusive approach for studies regardless of quality

Studies comparing outcomes with subspecialist care alone have been limited to Hepatitis C Virus and few have looked beyond level 5 outcomes

Little verification has been done for studies stating they implement "evidence-based" treatment methods

No studies included used of a randomised controlled trial methodology

Database search limited to: PubMed, MEDLINE, EMBASE, PsycINFO and ProQuest

The reported benefits of telehealth for rural Australians

Moffatt

(2010)[64]

Australia

Primary studies of health professionals in rural areas

Literature review to identify the reported benefits attributed to telehealth for people living and professionals working in rural and remote areas of Australia

147 studies

1998 - 2008

Narrative review

Health professionals have benefited from: access to continuing education and professional development, provision of enhanced local service, experiential learning and networking and collaboration

The use of telehealth appears to be a path to up-skilling for rural and remote practitioners

Review of Qld government telehealth VC network showed over 2 years 60% was for education and professional support services, in WA this was 40%, and for mental health nationally (except Vic) 83% of telemedicine programs provided education programs via telehealth

Benefits include increased access with reduced travel, increased access to variety of education

Limited information about systematic review process and analysis

Database search limited to Scopus, websites and government documents

The use of synchronous videoconferencing teaching to increase access to specialist nurse education in rural KwaZulu-Natal, South Africa

Chipps

(2010)[4]

South Africa

Registered nurses in rural South Africa

Review the literature and evaluate the current use of VC education for nurses in KwaZulu-Natal

81 papers in literature review

2 nurse VC education programs

Clinical Antiretroviral therapy (ARV) - 186 participants at 3 sites

Advanced Midwifery (ADM) - 53 participants at 5 sites

Literature review

Program evaluation of 2 nurse education programs delivered via VC

Questionnaires

Audit log

Enhancing Care Initiative (ECI) staff at the Medical School at UKZN deliver the first education program and School of Nursing staff deliver the second education program

Intervention 1: 2-day ARV - Two, 8-hour sessions

Intervention 2: One, 3-hour session/week over 6 months

80% of ARV participants were from remote non-Durban sites

Participants reported that VC training is comparable to face-to-face training, once technology is established and staff are comfortable there is no downside to VC training and potential cost savings are significant but mostly increased access was the greatest positive

Comfort of presenter was commented on and their ability to engage participants within the training affected the training outcomes

Therefore, orientation to teaching using this method is essential and increasing interaction with remote sites during training and using PowerPoint slides to support that Clear protocols for technical support are needed

Not enough research for outcomes of VC being equal to face-to-face teaching

Reporting only on what was found in the literature, not of how articles were included or excluded or quality etc

Not a lot of information about rigour or ability to replicate

Telehealth and the recruitment and retention of physicians in rural and remote regions: a Delphi study

Duplantie

(2007)[66]

Canada

Telehealth experts including managers, users and researchers/evaluators

To identify a reliable list of recruitment and retention factors on which telehealth could impact

2 literature reviews

Delphi study - 12 participants

Questionnaires

Delphi study - compared the degree of written agreement among experts, who were not in contact at any time

A minimum of 2 questionnaires were used to get consensus from the experts

Good review of the literature identifying 7 factors (including educational) of telehealth that could be related to medical workforce recruitment and retention

The experts agreed that telehealth could favour recruitment and retention by facilitating contact with peers, enforced with a system that is easy to use with human faces and connectivity for all

Identified traits and characteristics of remote physicians that were deems important

The highest consensus was: liking challenges, the capacity to work in collaboration, being helpful for the community and having a facility of adaptation

The consensus reached by the experts through the Delphi study shows the potential that telehealth could have on a set of individual, professional, organisational and educational factors related to recruitment and retention of physicians in rural and remote areas

Brief summary only of literature review findings

Decreased response to the second round of questionnaires sent out to experts which could challenge the robustness of the method used

Telementoring for improving primary care provider knowledge and competence in managing chronic pain: A randomised controlled trial

Eaton

(2018)[67]

USA

PCPs including physicians, physician assistants, nurse practitioners and registered nurses in University of Washington (UW) Medicine Telehealth network communities in rural and underserved areas of Washington, Wyoming, Montana, Oregon and New Mexico

To evaluate the effect of a telementoring intervention, TelePain, on knowledge and perceived competence related to chronic pain management

41 PCPs

23 intervention group

18 control group

22 physicians

13 physician assistants

3 nurse practitioners

3 registered nurses

12-week study

Cluster randomised clinical trial (RCT)

Questionnaires

Knowledge and Attitudes (regarding pain)

Know-Pain-12

Perceived competency scale

Statistical analysis

Specialists with expertise including pain medicine, internal medicine, anaesthesiology, rehabilitation medicine, psychiatry, addiction medicine, nursing and complementary and integrative pain management provided the telementoring consultations

Intervention: 90-min TelePain: Weekly, 30-min didactic presentation and 60-min patient case consultations, discussion and symptom management recommendations

First study to use RCT to evaluate pain management telementoring

78% of the intervention group presented at least one case

89% attended at least one TelePain session

4 control group PCPs attended an average of 6 sessions (see limitations)

Knowledge and perceived competence did not increase for the intervention group compared with providers in the control group

Future research is recommended to test the effectiveness of the intervention

Not able to prevent control group from attending TelePain sessions, therefore findings may not be a true reflection of intervention's effect

Considerable variability in the number of TelePain sessions attended by intervention group

You might also be interested in:

2021 - Researching ‘others’

2016 - The Cumbria Rural Health Forum: initiating change and moving forward with technology

2007 - Education, training and support needs of Australian trained doctors and international medical graduates in rural Australia: a case of special needs?

This PDF has been produced for your convenience. Always refer to the live site https://www.rrh.org.au/journal/article/6467 for the Version of Record.

Review Article

Telehealth use in rural and remote health practitioner education: an integrative review

AUTHORS

CORRESPONDENCE

AFFILIATIONS

PUBLISHED

HISTORY

CITATION

AUTHOR CONTRIBUTIONS

ETHICS APPROVAL

abstract:

Keywords:

full article:

references:

show: SUPPLEMENTARY CONTENT

You might also be interested in: