Review Article

Telehealth use in rural and remote health practitioner education: an integrative review

AUTHORS

name here
Pauline Calleja
1 PhD, Head of College Nursing & Midwifery * ORCID logo

name here
Susan Wilkes
2 GDip Midwifery, Nurse Educator

name here
Melinda Spencer
3 BSc (Hons), Research Assistant

name here
Steven Woodbridge
4 GC IntensiveCareNurs, Clinical Nurse

CORRESPONDENCE

*Prof Pauline Calleja

AFFILIATIONS

1 School of Nursing, Midwifery & Social Sciences, CQUniversity, Cairns, Qld 4870, Australia; Retrieval Services Queensland, Kedron, Qld 4031, Australia; and School of Nursing & Midwifery, Griffith University, Nathan, Qld 4111, Australia

2, 4 Retrieval Services Queensland, Kedron, Qld 4031, Australia

3 School of Nursing & Midwifery, Griffith University, Nathan, Qld 4111, Australia; and Menzies Health Institute Queensland, Griffith University, Southport, Gold Coast, Qld 4222, Australia

PUBLISHED

17 January 2022 Volume 22 Issue 1

HISTORY

RECEIVED: 7 October 2020

REVISED: 25 August 2021

ACCEPTED: 16 September 2021

CITATION

Calleja P, Wilkes S, Spencer M, Woodbridge S.  Telehealth use in rural and remote health practitioner education: an integrative review. Rural and Remote Health 2022; 22: 6467. https://doi.org/10.22605/RRH6467

AUTHOR CONTRIBUTIONSgo to url

ETHICS APPROVAL

Nil ethics required for review study

This work is licensed under a Creative Commons Attribution 4.0 International Licence


abstract:

Introduction:  For rural and remote clinicians, quality education is often difficult to access because of geographic isolation, travel, time, expense constraints and lack of an onsite educator. The aims of this integrative review were to examine what telehealth education is available to rural practitioners, evaluate the existence and characteristics of telehealth education for rural staff, evaluate current telehealth education models, establish the quality of education provided through telehealth along with the facilitators or enablers of a successful service and develop recommendations for supporting and developing an education model for rural and remote health practitioners through telehealth.
Methods:  An integrative review was conducted following the five-stage integrative review process. Searches were conducted in the electronic databases CINAHL, Medline, Nursing & Allied Health (Proquest), PubMed, Johanna Briggs Institute Evidence Based Practice (JBI EBP) and Embase.
Results:  Initial searches revealed more than 7000 articles; final inclusion and exclusion criteria refined results to 60 articles to be included in this review. Included articles were original research, case studies, reviews or randomised controlled studies. Countries of origin were countries in North and Central America, the UK, Europe, and Africa, and Australia and India. One issue noted with this review was classifying rural and remote; contexts used included rural, remote, regional, isolated, peripheral, native communities and outer regional or inner regional.
Sample sizes in the studies ranged from 20 to more than 1000 participants, covering a broad range of health education topics. Delivery was mostly by a didactic approach and case presentations. Some included a mix of videoconferencing with face-to-face sessions. Overall, telehealth education was well received, with participants reporting mostly positive outcomes as signified by feeling less isolated and more supported.
One interesting result was that quality in telehealth education is poorly established as there appears to be no definitions or consensus on what constitutes quality in the delivery of telehealth education. Very few studies formally tested increase in skill or knowledge, which is usual with professional development programs that do not result in further qualifications. For those that did assess these, formal knowledge and skills assessment indicated that telehealth using videoconferencing is comparable to face-to-face training with significant benefits related to travel reduction and therefore cost. Recommendations were difficult to synthesise because of the broad issues uncovered and lack of quality in many of the studies.
Conclusion:  The applications for telehealth are still evolving, with some applications having poor evidence to support use. Overall, telehealth education is well received and supported, with positives far outweighing negatives. Anything that can improve connection with a community and decrease isolation experienced by rural clinicians can only be beneficial. However, further planning and evaluation of the quality of delivery of telehealth education and addressing how education outcomes can be measured needs to be addressed in this widely growing area of telehealth.

Keywords:

education, in-service training, remote health services, rural health services, staff development, telemedicine, telehealth, videoconferencing.

full article:

Introduction

Accessing education in rural and remote settings can be challenging for health practitioners. Telehealth education with videoconferencing (VC) as a main delivery mode is widely researched. Research is well established for telehealth to provide patient education and follow-up appointments, but little is known about the impact of telehealth education using VC for rural and remote health practitioners, and several recommendations note the need for further exploration1,2.

Telehealth as defined by the International Organization for Standardization is the ‘use of telecommunication techniques for the purpose of providing telemedicine, medical education, and health education over a distance’3. For clarity, telemedicine has a narrower focus of clinical services that are remotely delivered, with the International Organization for Standardization definition as the ‘use of advanced telecommunication technologies to exchange health information and provide healthcare services across geographic, time, social and cultural barriers’3. VC is described as ‘electronic form of communications that permits people in different locations to engage in face-to-face audio and visual communication. Also, a collection of technologies that integrate video with audio, data, or both to convey in real time over distance for meeting between dispersed sites’3.

VC is one of the most common models of audio and visual technology used to provide telehealth and telemedicine. This article talks about telehealth predominantly and VC specifically as a form of providing health professionals from rural and remote locations an opportunity to connect with colleagues and gain support for patient care, refer patients and many other aspects that historically were a ‘refer and treat’ process, without connection to specialists in real time.

VC is becoming more widely used in telehealth, including to provide education to health practitioners in rural or remote settings. Quality education opportunities in rural and remote settings have traditionally been difficult to access because of geographic isolation, travel, time and expense constraints and often the lack of an onsite educator. Within telehealth, VC is promoted as a practicable alternative for educational opportunities4-7, but like all new technology approaches, uptake has overtaken evidence for use. When establishing their own telehealth education approach, the authors found gaps in the literature related to how telehealth has been used for specific education in rural and remote settings, along with little evidence about education strategies and their modes of delivery strengths and weaknesses. Therefore, this review was undertaken to identify, evaluate and synthesise the available evidence for using telehealth for education of rural and remote healthcare providers.

The objectives of this integrative review were to: (1) determine the existence and characteristics of telehealth education in rural and remote setting; (2) evaluate current telehealth education models and resources; (3) establish the quality of education provided through telehealth along with the facilitators or enablers of a successful service; and (4) develop recommendations for supporting and developing an education model for rural and remote health practitioners through telehealth. 

Methods

Research design

An integrative review was undertaken following the five-stage integrative review process described by Whittemore and Knafl8 consisting of problem identification, literature search, data evaluation, data analysis, and presentation.

Data sources and search strategy

Searches were conducted in the electronic databases CINAHL, Medline, Nursing & Allied Health (Proquest), PubMed, Johanna Briggs Institute Evidence Based Practice (JBI EBP) and Embase, using the strategy outlined in Table 1. Studies that were published in English post-2007, to reflect current telehealth education models, were included in the review. Reference lists of included studies were then hand searched for potentially relevant articles and included in the search results.

Table 1: Search strategy applied to databasestable image

Inclusion and exclusion criteria

Studies were included if they were published in a peer-reviewed journal and met the following inclusion criteria: (1) telehealth to provide targeted clinical education, training or supervision; (2) health practitioners including doctors, nurses and allied health; (3) rural or remote setting (including underserved). Studies were excluded if they (1) were written in a language other than English; (2) editorials, commentaries, news items, grey literature, could not be sourced, poorly constructed or written papers that made understanding difficult, and if their content did not meet inclusion criteria. In relation to inclusion criterion 1, targeted clinical education was related to education about clinical presentation types rather than health service process or mandatory education that is not focused on clinical practice. The authors chose to limit the inclusion of mandatory or process-driven education owing to the prescriptive nature of this type of education. The main area of interest was for patient outcomes, practice improvement, or clinician professional development.

Data extraction

One reviewer (MS) screened all records for relevance to the topic and excluded articles based on title or abstract that clearly did not meet the inclusion criteria. Two independent reviewers (PC, MS) assessed full-text articles for eligibility. Data were extracted by six reviewers (PC, MS, LT, SW, SWO, COB) and underwent consensus review until agreement was reached for final inclusion in the review.

Restrictions were not placed on study/methodology quality before information was extracted from articles and summarised into a table for review. The following data were collected from the included studies: first author, country of publication, aim, sample size, study design/methodology/intervention, relevant results/findings and recommendations, limitation/bias. This enabled a thematic summary of the studies’ findings (see Table 2).

Table 2: Thematic summary of the included studiestable imagetable image

Results

Study selection

The initial search using the search terms within the six designated databases yielded 7122 articles. To improve relevance of the search, a further screening of title and abstract only was applied to three databases – Nursing and Allied Health (Proquest), PubMed and Embase databases – as they had identified 4661, 1422 and 487 articles respectively. This screening reduced the total number to 914 articles. Duplicates were removed and then records were screened for relevance to topics. Then two reviewers (PC, MS) independently assessed 117 full-text articles for eligibility. An additional 79 studies were identified after reference list searching of all included articles and were added to the review. These studies were not identified in the initial search because of the variability of keywords that authors had attributed to their articles. A further 21 articles were excluded during data extraction and a final total of 60 articles were included in the study (Fig1).

A detailed summary of the 60 included studies is given in Supplementary table 1.

table image Figure 1:  Modified PRISMA flow diagram of included studies.

Study characteristics

Studies were original research8-57, conference abstracts58-60, case studies61-63, reviews64,65, reviews and original research4,66, or a randomised controlled trial67. (See Supplementary table 1 for details about study methodology.)

Supplementary table 1:  Characteristics of the included studies

Countries of study intervention

Over half (60%) of the included studies were conducted in the USA only9-12,14-16,20,21,23,25-27,30-32,34-36,40,41,44,45,48-51,54,56-62,67, followed by a smaller number of studies in Australia24,29,42,43,47,52,63,64, Canada16,19,52,53,55,65,66, the UK37,55, the USA and Europe33, US-affiliated Pacific Islands17, Africa4,13,18, Guatemala38, India39, Norway22 and Madagascar46.

Context

There was a range of contexts used to describe the healthcare setting in the included studies, including rural15,18,19,21,23,27,28,30,32,33,38,40-46,48,50,51,55,58,60-64,68, remote14,18,35,52,54,63,67, rural and remote17,24,53,56,64,66, rural and underserved10-12,31,39,49,57,59,67, underserved9,25,33,36, rural, remote or underserved16, isolated13, regional29, rural and regional63, peripheral37, remote military army and navy medical facilities32,34, native communities20,45 and US-affiliated Pacific Islands17. More specifically, one study47 used outer regional, inner regional, remote and very remote locations. One study21 classified ‘rural’ as defined by the Health Resources and Services Administration Shortage Designation Branch, US Government.

Population

Populations were divided into two specific groups, including the healthcare professional cohort who was receiving the education and the patient population the clinical topic was focused on. For example, studies focusing on the type of healthcare professional targeted included primary healthcare practitioners only11,16,25,36,53,58,61,68, nurses only38,51,52,68, physiotherapists55, and psychiatrists and psychiatrist trainees27. Some studies targeted the clinical topic to a particular patient cohort, including veterans14,15,26,40,42,54,56, Indigenous groups20,21,28,32,45, current military personnel33,34 and those in prison environments12,59. Within these studies, the majority (12 of the 16 studies) targeted multiple health practitioner disciplines servicing those particular patient populations.

Sample size

Of the included studies, participant numbers ranged from fewer than 20 participants18,39,55,56,58,62,66, 21–50 participants9-11,13,14,16,20,26,30,37,38,40,41,43,45,46,51,52,67, 51–100 participants17,38,42,49,54,61, 101–500 participants15,19,21,23-29,33,35,47,48,53,59,68, 500–999 participants31,36,50, and one study with more than 1000 participants32. The smallest number of participants was 162, and the largest number of participants was 107932. In four studies, the sample size of participants was not specified; however, the number of clinics and patients treated was provided instead12,22,34,63. In one study, the sample size of participants, number of clinics and patients treated was not given60.

Interventions

Forty-nine studies applied a single combined intervention. This usually included didactic VC, along with case study discussions and/or interactive questions9-16,18-24,26,27,29-41,44,45,47,49-54,56-63,67,68. One study applied a combination of e-mentoring and group-based case study discussions55. Two studies compared two individual interventions – VC and DVD46 and VC versus standard lecture48. One cross-sectional study compared two intervention arms – educational sessions with case study discussions against two control groups; another study compared four intervention arms – VC, audioconferencing and ‘live’ lecture including interactive questioning, and recorded presentation slides (inability to ask questions)17.

Topics covered by telehealth education

Pain9,14,16,26,28,31-34,50,54,61,67 was the most frequent health topic delivered by telehealth interventions, followed by psychiatry, mental and behavioural health18,19,25,27,39,42,50,53,61, hepatitis C virus10-12,15,45,48,50,65, addiction or substance abuse9,16,32,36,39,50,53, diabetes10,17,20,37,54,56, oncology/cancer care18,22,23,28,52, paediatrics29,38,55,59,63, HIV35,40,41,50,57, unclear/unknown13,42,65,66, opioid prescribing9,16,34, variety of topics21,64, palliative care47,60, perioperative51,52, oral health17, metabolic syndrome17, dementia24, multiple sclerosis30, antenatal/child health38, medicine44, sleep44, liver disease49, cardiology54, physiotherapy55, asthma59, dermatology62, midwifery68 and antiretroviral therapy68.

Current telehealth education models and resources

Within current telehealth education modules and resources, there were 10 themes identified: (1) improved practice change; (2) improved clinician confidence; (3) increased clinician knowledge; (4) improved self-efficacy; (5) increased clinical competency; (6) sense of community and interaction decreases isolation; (7) improved patient outcomes; (8) satisfaction, activity and reach equals acceptability and feasibility; (9) VC education is acceptable and feasible for rural and remote clinicians; and (10) weak measures used to identify success (see Table 2 for a summary of themes and subthemes).

Most VC education provided used a combination of didactic and case presentations10,11,14,16,18,20,23,25,26,28-32,36-41,43-45,47,49-51,53,54,57,59,60,62,67, while others used a combination of VC and face-to-face interactions19,35,62. The sessions varied in length, often from 30 minutes to 1 hour28,31,36,38,56, and up to 2 hours duration9,10,12,46,53,60. The frequency of sessions delivered varied from twice weekly34, weekly14,22,30,31,53,57,61,67, to monthly23,28,45,47. A discrete education model was identified in the literature called Project ECHO (Extension for Community Healthcare Outcomes). Project ECHO accounted for over half of the retrieved literature (33/60 or 55%), with only three of these occurring outside of the USA. The most common topic for Project ECHO was pain (n=9) followed by chronic disease management not specified (n=5), then mental health (n=4). The usual model for Project ECHO was a short didactic lecture (15–30 min) followed by two or three de-identified patient case studies with peer-led review of case presentations, and management recommendations all via telehealth or VC, with a focus on creating a community of practice to narrow the health disparity gaps. Project ECHO’s focus is to support clinicians to access collaborative learning in a knowledge-sharing network model69,70. Most studies reported on followed closely to the Project ECHO model, with only some variations in frequency and duration of sessions, and all were focused on case study presentation. One study had added in a two-day onsite training session before using the usual Project ECHO model20. One study also had their didactic session pre-recorded, and the debriefing as a live session.

Telehealth sessions were regarded as being as beneficial as face-to-face sessions38,68, while the technology was often challenging to some participants13,24. Educators’ knowledge on how best to use telehealth or their technical ability also affected (either positively or negatively) the delivery and provision of education using this technology, and therefore the development of those providing education in this way should be ensured to improve delivery and outcomes17,37,42,44,48,52,54.

Most of the education provided was delivered by specialists in the area or by multidisciplined teams10,12,18,19,22-25,29,30,32,33,35-38,50,51,56,62,63 and this appeared to have a positive effect by increasing and promoting the development of professional networks11,23,25,43,45,47,63. Use of telehealth modalities for education also had other beneficial outcomes, such as feelings of decreased isolation for participants16,27,39,45,49,57,60 as well as an increase in the availability and access to education opportunities13,20,63,64.

Participants who had undertaken telehealth education also had increased levels of self-efficacy and felt that they were able to better apply what was learned to clinical practice9,23,26,28,31,43,60. There appeared to be an increased sense of support being provided20,22 by using a telehealth education model, as well as increases in the levels of confidence of education recipients10,19,20,22,23,25,30,39,47,57. Recipients also reported that they gained knowledge as a result of attending telehealth education sessions9,10,23,25,31,41,48,54,64 and satisfaction levels with education provided using this technology were high17,19,23,24,28,53,60.

Quality of education provided through telehealth along with the facilitators of a successful service

Related to quality of education provided and facilitators of a successful service, there were three themes identified: (1) successful VC education has specific characteristics; (2) technology can support or hinder education in rural and remote communities; and (3) barriers to uptake include resourcing, scheduling and ease of use (see Table 2 for themes and subthemes).

Quality is not well established as far as agreed measures of what quality is. Satisfaction, level of self-reported confidence, knowledge gains, self-efficacy and engagement are all presented as elements of good education. Very few studies formally tested increase in skill or knowledge as is usual with professional development programs that do not result in further qualifications. For those that do (perioperative nurses, asthma educators and diabetes educators), formal knowledge and skills assessment indicates VC is comparable to face-to-face training with significant benefits related to travel reduction and therefore cost. Most of the Project ECHO publications that described clinical supervision relate education to patient outcomes such as measurable change in practice. Reach, extension of services through topic areas and new clinical environments, was presented as proof of acceptability. Although Sevean et al52 and Chipps et al4 touch on teaching styles, teacher comfort, expertise and ability to adapt to VC teaching, what enables or is deemed as successful education approaches is not presented in detail. Other studies relate advice on length of and time of day of the sessions both affect the ability of participants to attend. Related to this is ensuring repetition of sessions to allow local uptake, and advocating the use of local champions to support local uptake, which all relate to success along with technology quality, ease of use and access to support.

The quality of education provided through telehealth has so far not been mentioned or addressed in the literature included in this study. Most studies report using case-based presentations and didactic teaching as a chosen method of delivery, but don’t refer to the quality of material that was presented2,4,10,12,14,16,17,20,21,23-32,34,36,38-41,43,44,47-49,53-57,59-63,67.

Some studies14,25,34 mentioned the use of evidence-based or best practice guidelines in their sessions. While these studies stated that clinicians self-reported increased confidence and knowledge, no formal testing was done to substantiate this. Other studies highlighted clinicians’ feelings of increased knowledge or comfort, but no testing or quantifiable measures were demonstrated or tested to prove these perceptions9,16,19,20,28,38-40,47,48,57. Knowledge scores improved in testing post VC education, showing improved learning; however, limitations are noted as to whether this increased knowledge is transferred into clinical practice9,38,48.

There appear to be no definitions or agreed consensus as to what constitutes quality when using telehealth to deliver education. Feedback from participants indicated presenter approachability/connection19 and presenter comfort with equipment/technology4 were issues that affected engagement. Educators felt that there was a need to alter or adapt teaching format or style37,42,52, but the details of what should be altered or how were not clearly presented.

Specific recommendations were difficult to synthesise as the focus of education strategies used was extremely broad. This was complicated by limited evaluation of education strategies used and perhaps this is an indication that we are still in the phase of proliferation of publications that describe rather than evaluate a new intervention/approach. The few papers that do evaluate quality or outcomes of using a telehealth strategy for education may be the start of the professional conversation of questioning how to use a new approach like VC for telehealth education, most effectively for health professional education in rural and remote settings.

The 13 themes and subthemes distilled from the selected are presented in detail in Table 2. These included the following (theme names in italics).

Improved practice change, which focused on the changed practices of the clinicians as a result of engaging in the education provided. This was closely linked to patient outcomes as a result, and to improved clinician confidence and increased clinician knowledge and improved self-efficacy.

Increased clinical competency related to tested clinical competence, whether this was tested or self-reported, and in some cases extended the scope of practice of the clinician.

Sense of community and interaction decreases isolation related to being able to collaborate with peers, access support and as a result feel more connected and reduced stress related to being a healthcare practitioner in an isolated environment.

Improved patient outcomes related to measurable patient clinical outcomes and being able to access health care in a rural or remote environment for a broader scope of care.

Satisfaction, activity and reach equals acceptability and feasibility was closely related to weak measures used to identify success; however, this would benefit from more robust outcomes-based research.

Successful VC education has specific characteristics focused on relevance of the education to the target audience, being interactive in nature, providing a mixture of educational approaches, and tailorable to what is needed locally. Specific educator skills and characteristics also made up a large portion of this theme; for example, being knowledgeable, approachable, comfortable with equipment, being able to create a confidential, safe and comfortable learning environment. Training for educators also features aspects such as being able to adapt teaching style to VC sessions, using a specific format, technological skills, and accessibility for questions and consistency of session information presented across multiple groups.

VC education is acceptable and feasible for rural and remote clinicians related to reported satisfaction of participants, being able to attend education, decreased time and money required to access education, and the additional benefits of having VC education frequently versus face-to-face education sporadically.

Technology can support or hinder education in rural and remote communities is related to how much support and infrastructure is available to troubleshoot issues, quality of original set-up of easy-to-use systems. Barriers related to poor audio quality and video capability, overloaded servers that were not set up for the increased traffic, and ability to access on mobile devices.

Barriers to uptake including resourcing, scheduling and ease of use included complexity for end users to complete the education, availability of times that sessions are scheduled, infrastructure of equipment and training for clinicians to use the equipment, being released from clinical work to attend education, session length (some are too long to be able to work with clinical needs locally), lack of time to use resources that are identified in the education, being able to attend sessions when scheduled (therefore if they are recorded, or run multiple times) and scheduling clashes.

Discussion

Telehealth continues to grow into novel application areas faster than evidence exists to support uptake. Using telehealth for education delivery is clearly well established in the literature, but less well measured. This gap is particularly noticeable in this review, specifically in relation to the third objective – establish the quality of education provided through telehealth along with the facilitators or enablers of a successful service. The authors are unable to make specific recommendations about the quality of education that should serve as a guide for future educators and researchers when they are designing telehealth education approaches and evaluation of their services. VC was the most used strategy in the telehealth suite of education; however, it was used variably and perhaps not to its full potential. For example, assessment of clinical skills after the attendance of education could be undertaken by videoconference, enhancing support for rural and remote nurses and midwives in their endeavour to maintain clinical currency. This has been demonstrated in specific care areas such as the use of tele-ICU to support rural nurses caring for critically ill patients, providing teaching, assistance and further education in real time5.

This study found improving practice change was a theme related to telehealth education provided to health professionals in rural and remote areas. However, the measures supporting this outcome were not rigorous. Future research related to quality measures of tangible outcomes is needed. This is not specific to VC education; measuring practice change as a result of education provided is more complex and needs careful planning to be able to creditably achieve clear outcomes because often training is not the only answer68. While patient education can be measured directly by observing behaviour change and asking their intention to change behaviour, this is more complex to do with clinician education outcomes. There are many more factors involved in measuring outcomes of clinician education in addition to increased knowledge, intention to change practice and observed practice change. This would be an interesting and valid area of research to develop more effective approaches to measuring education outcomes within the health professional cohort.

In one integrative and thematic synthesis71 that investigated education and professional opportunities in rural and remote environments when using telehealth programs, engagement and participation was found to be further enhanced by the personalisation to the program, contributing value and meaning. This is closely linked to several of the current themes. For example, social isolation was reduced, and networking opportunities provided with the interactive learning were found to be most effective as an education technique for influencing patient outcomes71. Regardless of geographical location, the use of telehealth to provide education to health professionals in rural and remote areas is seen as feasible, cost-effective and beneficial69,72. While this may not have clear economic measures outlined in the studies included in this review, the unequivocal potential for economic savings simply due to decreased travel is clear; and this becomes more attractive the more geographically distant clinicians work from their preferred education sources. An interesting example of this was a study on shared care models trialled in the Kimberley region of Western Australia, where telehealth provided patients with specialist care and health professionals with education sessions. This significantly reduced time normally taken to travel to specialised care for patients and education for staff73.

One of the most widely applied models for telehealth education for clinical staff is the ECHO model. The ECHO model has been used worldwide, incorporating the use of telehealth to impact a lack of resources in rural and remote areas, applying case-based education to manage intricacy and the sharing of best practices69,70. The ECHO model applies four principles: use technology to influence limited resources, impart evidence-based practices, individual patient-based education, and an internet database to monitor outcomes69,70,74,75. Of all the education models reported on in the literature, the ECHO model had the most replicable approach, although this may only be because other education models have not been presented in a way that makes them easy to replicate and implement in different environments. Project ECHO presents a scalable, low-stakes, relatively low resource-intensive approach (apart from the telehealth infrastructure) to providing education to health professionals that does not require specific educational skills or resources on behalf of the presenters. Didactic presentations are short and therefore should not take a lot of resources to develop, and case study presentations are chosen from recent practice, thereby improving relevance. There was little discussion in articles about the disadvantages of implementing Project ECHO; however, it is likely that while having a specific structure can assist scalability, it can also be a barrier and lack flexibility. Additionally, as it is peer led, there is little training about ‘how’ to undertake education for your peers that would help support and sustain this model. Staff also undertake this model most often in their own time, which can have financial implications.

To support retention, the World Health Organization72 advocates for continuing education together with professional development programs to be made available for rural and remote health professionals from their home locations. It has been identified that VC is a viable, cost-effective means of providing education to rural and remote staff69,70. However, VC education is not without its challenges, including internet connection issues in remote areas and the ability of staff to work equipment1,6,18,76.

There are no hard measures for success; however, success has been indicated by the overall accomplishment of the reportative qualitative feedback and the proliferation of different programs using telehealth. There needs to be higher level evaluation research planned and undertaken to provide further evidence of the effectiveness of telehealth education and its impacts on patient care.

Another interesting finding was the wide variety in clinical education topic areas covered by the included studies. Pain was the most prevalent topic area. It could be argued that since pain is a common reason for seeking health care77 and managing pain effectively is complex, there is a significant need for clinicians to access education about pain management. An alternative reasoning is that perhaps those who provide pain education have had to provide professional development to others via alternative approaches for a long time, and therefore have better established services in this regard.

Recommendations for supporting and developing an education model for rural and remote health practitioners through telehealth

Positive factors for the use of telehealth include the ability to have real-time interaction and active participation, facilitating questions, receiving clarification and discussing care presentations23,38,42. This has been demonstrated in studies undertaken with the ECHO model, with participants indicating they had increased knowledge, liked group discussions and felt a reduction in professional isolation; however, the impact on patient outcomes warrants further exploration9,19,20,25,28,39-41,47,65. Duplantie et al66 discuss the use of the Delphi study involving 12 participants and how education via VC could impact positively on recruitment and retention, providing the ability to work in collaboration to help recruit and retain staff.

Program evaluation and curriculum development, training for educators and staff accessing VC education would need further consideration for program success. This includes development of educators to enable content transfer to participants, and being able to adapt to the needs of participants, format and style for program success18,29,44,52,54,60,62.

The identification of participant barriers for education via VC needs to be explored, including education provided, time and length of sessions, internet availability, equipment instruction and facility-level barriers24,26,32,37,43,45. Further measures beyond participant satisfaction need to be considered for evaluation of program success, including practice change and impact on patient outcomes9,19,20,25,28,40,41,44,47,65,66.

In future studies, recommendations to improve the quality of studies reporting on using VC to deliver education in rural and remote environments include:

1.  Use multiple success factors to test for within the study, including education model, teaching approaches, skills of facilitators, technology components and infrastructure, to publish a set of principles, skills and abilities. These measures should also consider the following aspects.

2.  Report or measure the alignment of education aims with results being reported:

  • Should satisfaction of participants be the only measure of success?
  • Does reach equals acceptability or is it reported as a measure because that is all that is available to the researchers?
  • What is the feasibility related to local infrastructure and resourcing (which will vary and may not be transferrable)?
  • What does good teaching in a VC format look like and why do participants and teachers feel this way?
  • In relation to interactive education, is all interactivity equal? Does the interactivity serve application or key educational goals?
  • Are there specific teaching approaches that influence learning and retention of learning within the VC format?
  • What level of participant input makes a program successful? What are the measures of success? Why have these measures been chosen? How do they align with educational goals?
  • What measures make most sense for the level of education that best fits the aim of education (eg access to education vs practice change will have very different measures)?
  • What level of training or what training is needed for educators/facilitators to adapt to teaching via VC? What resources are developed, and can these be made more widely available?
  • ECHO is a model that has been widely adopted by practitioners for practitioners and clinical supervision. Are there other versions/models that can be replicated (recommend Delphi study to gain consensus of acceptable model components)?

Limitations

The limitations in this study include variations in what defines a rural or remote area and the variety of settings where studies were undertaken. Another issue was the quality of the studies included. Outcome measures were generally weak, and therefore recommendations cannot be made on this basis; however, if all studies were excluded for this reason, there would be very few to include in this study, and the authors feel that this is a finding in itself. The nature of the emerging work in this area means there is some way to go in improving the research design of education program evaluations in telehealth use for health practitioner education in rural and remote areas.

Conclusion

Telehealth education, and in particular VC, is seen as a viable, cost-effective means of providing education to rural and remote staff78,79. With technological advances, a variety of education resources can be made available, impacting on knowledge and skills in the rural and remote arena, and assessment of knowledge and skills can be assessed by VC methods.

However, VC education is not without its challenges in rural and remote areas, including poor internet connection and the clinicians’ ability to operate equipment1,6,76. It is highlighted that equipment familiarity is essential when using VC for education, and Chipps et al2 discusses the importance of well-defined guidelines and operating instructions with videoconference use to alleviate concerns of participants and educators.

Notwithstanding the methodological limitations of the studies, there appears to be adequate evidence for VC education to have an impact on cultivating knowledge for rural and remote nurses and midwives. The paucity of high-quality research warrants further investigation focusing on scrupulously planned studies. Larger studies evaluating effectiveness of videoconference educational interventions pertaining to the impact on knowledge and skills is needed to quantify the scholastic gains.

Acknowledgements

The authors acknowledge Cherie O’Brien for involvement in data extraction, and Leeanne Trenning for data extraction and validation of data analysis.

references:

1 McConnell KA, Krisher LK, Lenssen M, Bunik M, Montes SB, Domek GJ. Telehealth to expand community health nurse education in rural Guatemala: a pilot feasibility and acceptability evaluation. Frontiers in Public Health 2017; 5: 60. DOI link, PMid:28405582
2 Chipps J, Brysiewicz P, Mars M. A systematic review of the effectiveness of videoconference‐based tele‐education for medical and nursing education. Worldviews on Evidence‐Based Nursing 2012; 9(2): 78-87. DOI link, PMid:22409341
3 International Organization for Standardization. ISO/TR 16056-1:2004(en). Health informatics - interoperability of telehealth systems and networks. Part 1: introductions and definitions. Available: web link (Accessed 9 January 2021).
4 Chipps J. The use of synchronous videoconferencing teaching to increase access to specialist nurse education in rural Kwazulu-Natal, South Africa. Knowledge Management and E-Learning 2010; 2(2): 154-168. DOI link
5 Hendrickx L, Winters C. Access to continuing education for critical care nurses in rural or remote settings. Critical Care Nurse 2017; 37(2): 66-71. DOI link, PMid:28365651
6 Jerome-D'Emilia B, Merwin E, Stern S. Feasibility of using technology to disseminate evidence to rural nurses and improve patient outcomes. Journal of Continuing Education in Nursing 2010; 41(1): 25-32. DOI link, PMid:20102140
7 Page R, Hynes F, Reed J. Distance is not a barrier: the use of videoconferencing to develop a community of practice. Journal of Mental Health Training, Education and Practice 2019; 14(1): 12-19. DOI link
8 Whittemore R, Knafl K. The integrative review: updated methodology. Journal of Advanced Nursing 2005; 52(5): 546-553. DOI link, PMid:16268861
9 Anderson D, Zlateva I, Davis B, Bifulco L, Tierney G, Coman E, et al. Improving pain care with Project ECHO in community health centers. Pain Medicine 2017; 18(10): 1882-1889. DOI link, PMid:29044409
10 Arora S, Geppert CMA, Kalishman S, Dion D, Pullara F, Bjeletich B, et al. Academic health center management of chronic diseases through knowledge networks: Project ECHO. Academic Medicine 2007; 82(2): 154-160. DOI link, PMid:17264693
11 Arora S, Kalishman S, Dion D, Som D, Thornton K, Bankhurst A, et al. Partnering urban academic medical centers and rural primary care clinicians to provide complex chronic disease care. Health Affairs 2011; 30(6): 1176-1184. DOI link, PMid:21596757
12 Arora S, Kalishman S, Thornton K, Dion D, Murata G, Deming P, et al. Expanding access to hepatitis C virus treatment - Extension for Community Healthcare Outcomes (ECHO) project: disruptive innovation in specialty care. Hepatology 2010; 52(3): 1124-1133. DOI link, PMid:20607688
13 Bagayoko CO, Perrin C, Gagnon M-P, Geissbuhler A. Continuing distance education: a capacity-building tool for the de-isolation of care professionals and researchers. Journal of General Internal Medicine 2013; 28(S3): S666-S670. DOI link, PMid:23780652
14 Ball S, Wilson B, Ober S, McHaourab A. SCAN-ECHO for pain management: implementing a regional telementoring training for primary care providers. Pain Medicine 2018; 19(2): 262-268. DOI link, PMid:28525633
15 Beste LA, Glorioso TJ, Ho PM, Au DH, Kirsh SR, Todd-Stenberg J, et al. Telemedicine specialty support promotes Hepatitis C treatment by primary care providers in the Department of Veterans Affairs. American Journal of Medicine 2017; 130(4): 432-438 e3. DOI link, PMid:27998682
16 Carlin L, Zhao J, Dubin R, Taenzer P, Sidrak H, Furlan A. Project ECHO telementoring intervention for managing chronic pain in primary care: insights from a qualitative study. Pain Medicine 2018; 19(6): 1140-1146. DOI link, PMid:29036381
17 Chen TH, Buenconsejo-Lum LE, Braun KL, Higa C, Maskarinec GG. A pilot evaluation of distance education modalities for health workers in the US-affiliated Pacific Islands. Pacific Health Dialog 2007; 14(1): 22-30.
18 Chipps J, Ramlall S, Madigoe T, King H, Mars M. Developing telepsychiatry services in KwaZulu-Natal - an action research study. African Journal of Psychiatry 2012; 15(4): 255-263. DOI link, PMid:22829228
19 Church EA, Heath OJ, Curran VR, Bethune C, Callanan TS, Cornish PA. Rural professionals' perceptions of interprofessional continuing education in mental health. Health and Social Care in the Community 2010; 18(4): 433-443. DOI link, PMid:20522117
20 Colleran K, Harding E, Kipp BJ, Zurawski A, MacMillan B, Jelinkova L, et al. Building capacity to reduce disparities in diabetes:training community health workers using an integrated distance learning model. The Diabetes Educator 2012; 38(3): 386-396. DOI link, PMid:22491397
21 Cunningham BJ, Vande Merwe R. Virtual grand rounds: a new educational approach in social work that benefits long-term care providers and patients in rural Idaho. Rural and Remote Health 2009; 9(1): 1073. DOI link, PMid:19215188
22 Donnem T, Ervik B, Magnussen K, Andersen S, Pastow D, Andreassen S, et al. Bridging the distance: a prospective tele-oncology study in northern Norway. Supportive Care in Cancer 2012; 20(9): 2097-2103. DOI link, PMid:22076621
23 Doorenbos AZ, Kundu A, Eaton LH, Demiris G, Haozous EA, Towle C, et al. Enhancing access to cancer education for rural healthcare providers via telehealth. Journal of Cancer Education 2011; 26(4): 682-686. DOI link, PMid:21336979
24 Doyle C, Jackson D, Loi S, Malta S, Moore K. Videoconferencing and telementoring about dementia care: evaluation of a pilot model for sharing scarce old age psychiatry resources. International Psychogeriatrics 2016; 28(9): 1567-1574. DOI link, PMid:27189501
25 Fisher E, Hasselberg M, Conwell Y, Weiss L, Padron NA, Tiernan E, et al. Telementoring primary care clinicians to improve geriatric mental health care. Population Health Management 2017; 20(5): 342-347. DOI link, PMid:28106511
26 Frank JW, Carey EP, Fagan KM, Aron DC, Todd-Stenberg J, Moore BA, et al. Evaluation of a telementoring intervention for pain management in the Veterans Health Administration. Pain Medicine 2015; 16(6): 1090-1100. DOI link, PMid:25716075
27 Greenwood J, Williams R. Continuing professional development for Australian rural psychiatrists by videoconference. Australasian Psychiatry 2008; 16(4): 273-276. DOI link, PMid:18608161
28 Haozous E, Doorenbos AZ, Demiris G, Eaton LH, Towle C, Kundu A, et al. Role of telehealth/videoconferencing in managing cancer pain in rural American Indian communities. Psychooncology 2012; 21(2): 219-223. DOI link, PMid:22271543
29 Harris V, Smith AC, Armfield NR. Education for regional health professionals using mobile videoconferencing. Journal of Telemedicine and Telecare 2007; 13(S3): 44-47. DOI link
30 Johnson KL, Hertz D, Stobbe G, Alschuler K, Kalb R, Alexander KS, et al. Project Extension for Community Healthcare Outcomes (ECHO) in multiple sclerosis: increasing clinician capacity. International Journal of MS Care 2017; 19(6): 283-289. DOI link, PMid:29270085
31 Katzman JG, Comerci G, Jr, Boyle JF, Duhigg D, Shelley B, Olivas C, et al. Innovative telementoring for pain management: project ECHO pain. Journal of Continuing Education in the Health Professions 2014; 34(1): 68-75. DOI link, PMid:24648365
32 Katzman JG, Fore C, Bhatt S, Greenberg N, Griffin Salvador J, Comerci GC, et al. Evaluation of American Indian health service training in pain management and opioid substance use disorder. American Journal of Public Health 2016; 106(8): 1427-1429. ps://doi.org/10.2105/AJPH.2016.303193, PMid:27196642
33 Katzman JG, Galloway K, Olivas C, McCoy-Stafford K, Duhigg D, Comerci G, et al. Expanding health care access through education: dissemination and implementation of the ECHO model. Military Medicine 2016; 181(3): 227-235. DOI link, PMid:26926747
34 Katzman JG, Qualls CR, Satterfield WA, Kistin M, Hofmann K, Greenberg N, et al. Army and Navy ECHO pain telementoring improves clinician opioid prescribing for military patients: an observational cohort study. Journal of General Internal Medicine 2019; 34(3): 387-395. DOI link, PMid:30382471
35 Knapp H, Fletcher M, Taylor A, Chan K, Goetz MB. No clinic left behind: providing cost-effective in-services via distance learning. Journal for Healthcare Quality 2011; 33(5): 17-24. DOI link, PMid:23845129
36 Komaromy M, Duhigg D, Metcalf A, Carlson C, Kalishman S, Hayes L, et al. Project ECHO (Extension for Community Healthcare Outcomes): a new model for educating primary care providers about treatment of substance use disorders. Substance Abuse 2016; 37(1): 20-24. DOI link, PMid:26848803
37 Maltinsky W, Hall S, Grant L, Simpson K, MacRury S. Pilot project and evaluation of delivering diabetes work-based education using video conferencing. Rural and Remote Health 2013; 13: 2053. DOI link, PMid:23414080
38 McConnell KA, Krisher LK, Lenssen M, Bunik M, Bunge Montes S, Domek GJ. Telehealth to expand community health nurse education in rural Guatemala: a pilot feasibility and acceptability evaluation. Frontiers in Public Health 2017; 5: 60. DOI link, PMid:28405582
39 Mehrotra K, Chand P, Bandawar M, Rao Sagi M, Kaur S, Aurobind G, et al. Effectiveness of NIMHANS ECHO blended tele-mentoring model on integrated mental health and addiction for counsellors in rural and underserved districts of Chhattisgarh, India. Asian Journal of Psychiatry 2018; 36: 123-127. DOI link, PMid:30086513
40 Moeckli J, Stewart KR, Ono S, Alexander B, Goss T, Maier M, et al. Mixed-methods study of uptake of the Extension for Community Health Outcomes (ECHO) Telemedicine model for rural veterans with HIV. Journal of Rural Health 2017; 33(3): 323-331. DOI link, PMid:27557039
41 Ness TE, Annese MF, Martinez-Paz N, Unruh KT, Scott JD, Wood BR. Using an innovative telehealth model to support community providers who deliver perinatal HIV care. AIDS Education and Prevention 2017; 29(6): 516-526. DOI link, PMid:29283274
42 Newman C, Martin E, McGarry DE, Cashin A. Survey of a videoconference community of professional development for rural and urban nurses. Rural and Remote Health 2009; 9(2): 1134. DOI link, PMid:19382828
43 Newman L, Bidargaddi N, Schrader G. Service providers' experiences of using a telehealth network 12 months after digitisation of a large Australian rural mental health service. International Journal of Medical Informatics 2016; 94: 8-20. DOI link, PMid:27573307
44 Parsons EC, Mattox EA, Beste LA, Au DH, Young BA, Chang MF, et al. Development of a sleep telementorship program for rural Department of Veterans Affairs primary care providers: Sleep Veterans Affairs Extension for Community Healthcare Outcomes. Annals of the American Thoracic Society 2017; 14(2): 267-274.
45 Pindyck T, Kalishman S, Flatow-Trujillo L, Thornton K. Treating hepatitis C in American Indians/Alaskan Natives: a survey of Project ECHO(Ò) (Extension for Community Healthcare Outcomes) utilization by Indian health service providers. SAGE Open Medicine 2015; 3: 2050312115612805. DOI link, PMid:26770809
46 Rakototiana LB, Rajabo, Gottot S. Internet or dvd for distance learning to isolated rural health professionals, what is the best approach? BMC Medical Education 2017; 17: 152. DOI link, PMid:28874191
47 Ray RA, Fried O, Lindsay D. Palliative care professional education via video conference builds confidence to deliver palliative care in rural and remote locations. BMC Health Services Research 2014; 14: 272. DOI link, PMid:24947941
48 Rossaro L, Tran TP, Ransibrahmanakul K, Rainwater JA, Csik G, Cole SL, et al. Hepatitis C videoconferencing: the impact on continuing medical education for rural healthcare providers. Telemedicine Journal and e-Health 2007; 13(3): 269-277. DOI link, PMid:17603829
49 Salgia RJ, Mullan PB, McCurdy H, Sales A, Moseley RH, Su GL. The educational impact of the Specialty Care Access Network-Extension of Community Healthcare Outcomes program. Telemedicine Journal and e-Health 2014; 20(11): 1004-1008. DOI link, PMid:25226452
50 Scott JD, Unruh KT, Catlin MC, Merrill JO, Tauben DJ, Rosenblatt R, et al. Project ECHO: a model for complex, chronic care in the Pacific Northwest region of the United States. Journal of Telemedicine and Telecare 2012; 18(8): 481-484. DOI link, PMid:23209269
51 Seibert PS, Reddy T, Whitmore T, Valerio J, Dehaas C. The use of telemedicine to train perioperative nurses in rural settings. Journal of Telemedicine and Telecare 2013; 19(6): 311-314. DOI link, PMid:24163294
52 Sevean P, Dampier S, Spadoni M, Strickland S, Pilatzke S. Bridging the distance: educating nurses for telehealth practice. Journal of Continuing Education in Nursing 2008; 39(9): 413-418. DOI link, PMid:18792607
53 Sockalingam S, Arena A, Serhal E, Mohri L, Alloo J, Crawford A. Building provincial mental health capacity in primary care: an evaluation of a Project ECHO mental health program. Academic Psychiatry 2018; 42(4): 451-457. DOI link, PMid:28593537
54 Stevenson L, Ball S, Haverhals LM, Aron DC, Lowery J. Evaluation of a national telemedicine initiative in the Veterans Health Administration: factors associated with successful implementation. Journal of Telemedicine and Telecare 2018; 24(3): 168-178. DOI link, PMid:27909208
55 Stewart S, Carpenter C. Electronic mentoring: an innovative approach to providing clinical support. International Journal of Therapy and Rehabilitation 2009; 16(4): 199-206. DOI link
56 Watts SA, Roush L, Julius M, Sood A. Improved glycemic control in veterans with poorly controlled diabetes mellitus using a Specialty Care Access Network-Extension for Community Healthcare Outcomes model at primary care clinics. Journal of Telemedicine and Telecare 2016; 22(4): 221-224. DOI link, PMid:26253745
57 Wood BR, Unruh KT, Martinez-Paz N, Annese M, Ramers CB, Harrington RD, et al. Impact of a telehealth program that delivers remote consultation and longitudinal mentorship to community HIV providers. Open Forum Infectious Diseases 2016; 3(3): ofw12. DOI link, PMid:27703991
58 Cheng KMD, Jetmalani ANMD, Sedighi B, McDonnell MM, Pettersen T. Using the Project Extension for Community Healthcare Outcomes model to train rural primary care clinicians to deliver child mental health services in Oregon. Journal of the American Academy of Child & Adolescent Psychiatry 2017; 56(10): S186-S187. DOI link
59 Harkins M, Raissy H, Moseley K, Luttecke K, Arora S. Project ECHO: improving asthma care in New Mexico with telehealth technology. Chest 2011; 140(4). DOI link
60 Marr LMD, Neale DMD. Project ECHO: bringing palliative care consultation to rural New Mexico through a novel telemedicine format. Journal of Pain and Symptom Management 2012; 43(2): 448-449. DOI link
61 Katzman JG. Making connections: using telehealth to improve the diagnosis and treatment of complex regional pain syndrome, an underrecognized neuroinflammatory disorder. Journal of Neuroimmune Pharmacology 2013; 8(3): 489-493. DOI link, PMid:23054372
62 Lewis H, Becevic M, Myers D, Helming D, Mutrux R, Fleming D, et al. Dermatology ECHO - an innovative solution to address limited access to dermatology expertise. Rural and Remote Health 2018; 18(1): 4415. DOI link, PMid:29409325
63 Smith AC, Armfield NR, White MM, Williams ML, Koh TH, Hurley T, et al. Clinical services and professional support: a review of mobile telepaediatric services in Queensland. Studies in Health Technology and Informatics 2010; 161: 149-158.
64 Moffatt JJ, Eley DS. The reported benefits of telehealth for rural Australians. Australian Health Review 2010; 34(3): 276-281. DOI link, PMid:20797357
65 Zhou C, Crawford A, Serhal E, Kurdyak P, Sockalingam S. The impact of Project ECHO on participant and patient outcomes: a systematic review. Academic Medicine 2016; 91(10): 1439-1461. DOI link, PMid:27489018
66 Duplantie J, Gagnon MP, Fortin JP, Landry R. Telehealth and the recruitment and retention of physicians in rural and remote regions: a Delphi study. Canadian Journal of Rural Medicine 2007; 12(1): 30-36.
67 Eaton LH, Godfrey DS, Langford DJ, Rue T, Tauben DJ, Doorenbos AZ. Telementoring for improving primary care provider knowledge and competence in managing chronic pain: a randomised controlled trial. Journal of Telemedicine and Telecare 2020; 26(1-2): 21-27.
68 Kirkpatrick J, Kirkpatrick W. Training on trial. Journal for Quality and Participation 2010; 33(3): 26-31.
69 Western Victoria Primary Health Network. Project Echo overview 2019. Available: web link (Accessed 3 April 2021).
70 Children's Health Queensland Hospital and Health Service. Project ECHO 2019. Available: web link (Accessed 25 April 2021).
71 Moran AM, Coyle J, Pope R, Boxall D, Nancarrow SA, Young J. Supervision, support and mentoring interventions for health practitioners in rural and remote contexts: an integrative review and thematic synthesis of the literature to identify mechanisms for successful outcomes. Human Resources for Health 2014; 12(1): 10. DOI link, PMid:24521004
72 World Health Organization. Increasing access to health workers in remote and rural areas through improved retention: global policy recommendations. Geneva: WHO, 2010.
73 Vale S, Roche I, Noble V, Markus T, Loh R. P44: a shared care model for allergy - Kimberley region pilot. Internal Medicine Journal 2018; 48(S6): 17. DOI link
74 Victorian Opioid Management ECHO. Opioid management for primary care practitioners. 2019. Available: web link (Accessed 25 February 2021).
75 Mitruka K, Thornton K, Cusick S, Orme C, Moore A, Manch RA, et al. Expanding primary care capacity to treat hepatitis C virus infection through an evidence-based care model - Arizona and Utah, 2012-2014. Morbidity and Mortality Weekly Report 2014; 63(18): 393-398.
76 Macedo CR, Macedo EC, Torloni MR, Atallah ÁN. Evaluation of a national evidence-based health care course via teleconference in a developing country. Journal of Evaluation in Clinical Practice 2013; 19(4): 713-719. DOI link, PMid:23654346
77 St Saver JL, Warner DO, Yawn BP, Jacobson DJ, McGree ME, Pankratz JJ, et al. Why patients visit their doctors: assessing the most prevalent conditions in a defined American population. Mayo Clinic Proceedings 2013; 88(1): 56-67. DOI link, PMid:23274019
78 Newman C, Martin E, McGarry DE, Cashin A. Survey of a videoconference community of professional development for rural and urban nurses. Rural and Remote Health 2009; 9(2): 1134. DOI link, PMid:19382828
79 Doorenbos AZ, Kundu A, Eaton LH, Demiris G, Haozous EA, Towle C, et al. Enhancing access to cancer education for rural healthcare providers via telehealth. Journal of Cancer Education 2011; 26(4): 682-686. DOI link, PMid:21336979

You might also be interested in:

2021 - Researching ‘others’

2016 - The Cumbria Rural Health Forum: initiating change and moving forward with technology

2007 - Education, training and support needs of Australian trained doctors and international medical graduates in rural Australia: a case of special needs?