Continuing education training focused on the development of behavioral telehealth competencies in behavioral healthcare providers

full article:

Introduction

Telehealth

The use of telecommunications technology in the healthcare field offers a tool for extending services to rural residents with fewer services and providers available to them than urban residents^1-5. Left without accessible mental health care, residents of rural communities are at risk for problems associated with untreated mental health concerns⁶. Interventions offered through telehealth provide a remedy allowing rural residents access to care⁷. Behavioral telehealth, a term used to describe behavioral health services delivered through telehealth technology, not only extends care to geographically isolated individuals but also provides a means for reaching individuals who are homebound as well as other special populations^8,9. In addition, behavioral telehealth provides a means for extending specialty care, which tends to be very expensive, to residents of underserved areas at a low cost⁹. The amount and type of behavioral telehealth services continues to grow and includes online treatment programs, self-help chat rooms, assessments, adherence-enhancing strategies and consultation services¹⁰. This study describes one model for delivering behavioral telehealth training and evaluates its effectiveness.

Rural Alaska

Alaska is the largest state in the USA, consisting of 1.52 million km² (586 412 miles²)¹¹. One road system connects Southcentral Alaska with interior Alaska and Alaska's North Slope. Diverse groups of people, including both non-native and Alaska Native people, inhabit the communities linked together by this road system¹². Many small rural communities exist away from the road system and are only accessible by plane, boat or snow machine. Alaska Native people tend to inhabit the majority of these communities¹³. Alaska's healthcare system is tasked with extending healthcare services to geographically isolated, ethnically diverse and underserved populations.

In conjunction with local village health clinics, larger 'regional hubs' provide most behavioral health services for rural Alaska¹⁴. Regional hubs are staffed with healthcare professionals who may travel to the villages on a rotating basis^15-18. These professionals frequently leave and positions remain vacant for extended periods^19-21. To address provider shortages, specialists from urban areas also provide itinerant services in the state's regional hubs, and Alaskan healthcare organizations are developing telehealth for extending health care (Gifford V; unpubl data; 2007}^14,20,22,23. Currently, the urban centers, regional hubs and village clinics have the capability to connect with one another via telehealth technology. In fact, professional medical providers in regional hubs routinely connect via telehealth with paraprofessional health aides in rural villages to diagnose and treat patients. Telehealth technology is not only prevalent in physical health care but is also extending into the field of behavioral health.

Telehealth as an acceptable tool for service delivery

The literature indicates an overall openness to telehealth services even though doubts about its overall effectiveness compared to traditional face-to-face interventions exist. A study of rural and urban primary care patients suggested that the majority of such patients held moderately positive attitudes, perceptions and expectations of telehealth; are open to the possibility of accessing medical and psychiatric telehealth services; and would be willing to use these services if it saves them from traveling to access care⁷. These findings are further supported by rural participants describing the most beneficial aspect of behavioral telehealth services as alleviating financial costs of taking time off of work to travel many miles to a larger community to access care²⁴. Rural physicians likewise noted reduced travel time as a primary benefit of telehealth²⁵. Overall, the literature indicates positive perceptions about behavioral telehealth services offered locally and its ability to provide a cost effective and time saving alternative to traditional face-to-face services available in larger urban centers. However, some appear to find behavioral telehealth acceptable only if other options are unavailable or if behavioral telehealth prevents the need for extensive travel²⁶.

Access to technology and computer literacy have been noted as hindrances to a patient's ability to access telehealth care and continue to remain worthy considerations^10,27. As technology continues to advance, rural areas and communities with underserved populations may have growing opportunities to develop technological infrastructure facilitating access to video conferencing in local primary care clinics and other settings. Grubaugh and colleagues found that rural and urban participants had equal access to technology, and that participants' age and educational level did not influence their openness to telehealth services⁷.

Readiness, buy-in and staffing

Careful development and planning of a telehealth program coupled with community interest, provider training, and healthcare organization support for service delivery influence program success²⁸. The literature identifies the importance of assessing patient, healthcare provider, healthcare organization and community readiness for the development and implementation of telehealth services²⁹. Readiness includes the community's sense of a genuine need for telehealth services; the process of weighing its benefits and risks; understanding of telehealth service delivery; and considerations regarding available infrastructure such as funding, technology, technical support, consultation and training²⁹.

Establishing buy-in from stakeholders is necessary for successful development and implementation. Buy-in can best be established when contexts and cultures associated with populations accessing telehealth services are recognized and programs are tailored to meet the needs of the population³⁰. Other literature supports the use of a stepwise process to facilitate organizational and community buy-in and implementation²⁸. By involving various members of stakeholder groups throughout the process, customization and tailoring of the telehealth program to context and culture can occur and facilitate the ability of the program to meet community needs.

Healthcare organizations implementing telehealth programs must consider the staff needed to support the program. The literature supports the importance of a professional telehealth coordinator possessing the extensive knowledge, skills and expertise necessary for working on multidisciplinary teams as well as in leadership, educational and administrative roles³¹. However, many rural communities may not have access to a telehealth coordinator possessing such skills and expertise. Instead, behavioral telehealth programs in rural communities may rely on the local expertise of paraprofessional staff to coordinate the services. Maximizing the effectiveness of the telehealth coordinator role requires organizational support through funding, adequate technology, administrative support and training³¹.

Telehealth training

Patient and staff training is an important consideration when implementing a telehealth program as some individuals may not be technically adept at using the technology³². Therefore, training is essential for implementing a successful program³³. It is important for users to be comfortable, competent and ready to use the telehealth equipment³⁴. Maintaining a well-trained telehealth staff is particularly challenging due to high turnover in remote areas³². Moreover, staff may experience frustration associated with troubleshooting technical difficulties and increases in workloads associated with implementing new telehealth services^32,35.

The use of telehealth technology is a growing resource in the healthcare field. Training providers to competently utilize this resource is critical for successful implementation of telehealth programs and more accessible healthcare services. In fact, it has been suggested that healthcare training programs incorporate telehealth into the educational experiences of their students^27,36. Yet, the literature is lacking in the area of provider telehealth competency training, particularly in behavioral telehealth service delivery. This gap in knowledge is being addressed by the efforts of the Alaska Rural Behavioral Health Training Academy (ARBHTA) through a continuing education training program offered to behavioral health providers and evaluation of this training. The training is based on an identified set of core behavioral telehealth competencies that assume a lack of provider familiarity with videoconferencing equipment and the nuances of providing behavioral health services via videoconferencing.

Alaska Rural Behavioral Health Training Academy

The Alaska Rural Behavioral Health Training Academy is a continuing education provider for behavioral healthcare providers that targets a variety of training needs including behavioral telehealth^37,38. In October 2007, ARBHTA delivered behavioral telehealth training with the intention of increasing access to behavioral health care for residents of rural Alaska. In addition to behavioral telehealth skill development, the training addressed provider skill development in other areas vital to successful service delivery in rural Alaska, including cultural competency, ethics in rural practice, and identifying and preventing vicarious trauma. A team of instructors with substantial experience in the content areas provided the training. University faculty members, Alaska Native Elders, seasoned professionals utilizing telehealth service delivery and two graduate students comprised the team of content experts. The aim of the current study was to assess the effects of the Behavioral Telehealth Ethical Competencies Training program on behavioral health providers' development of behavioral telehealth competencies.

Methods

Procedures

Participants completed behavioral telehealth competency training. Assessment of the effects of the training utilized triangulation by drawing on multiple methodologies. Pre- and post-test assessments examined participants' behavioral telehealth competencies through their responses to video vignettes. The post-test assessments were followed by participant self-reports of their perceived competencies. Between 4 and 5 months following completion of the training program, participants were sent a follow-up survey querying them about their progress on the behavioral telehealth service delivery goals that they set during the training program.

Participants

Of the 21 participants who completed the training, 14 completed the video vignette portion of the study, 17 completed self-reported competency assessments and nine completed the follow-up report on their goal progress. Participants included Alaska Native and non-Native professional and paraprofessional behavioral health providers from both rural and urban areas of Alaska. Professions represented included counselors, social workers and psychologists. Demographic information was available for 16 participants. Eight of the participants were male, and eight were female. Ten of the participants were from urban centers of Alaska and six were from rural areas.

Intervention

The program offered 3 days of face-to-face expert instruction in behavioral telehealth competency. During the training, ethical concerns and cultural considerations pertaining to service delivery in Alaska were also addressed. Participants practiced using video-conferencing equipment, role played clinical scenarios over behavioral telehealth equipment, and engaged in small and large group discussions. Content experts developed the training based on a specific set of competencies believed to be important for behavioral telehealth providers to possess. Trainers integrated the rural ethics and cultural competency portions of the training into the behavioral telehealth competencies. For example, the training addressed the common ethical dilemmas faced by rural behavioral healthcare providers practicing in rural areas. Topics discussed included navigating dual relationships along with client concerns regarding stigmatization and confidentiality, which are heightened by the interconnected nature of small rural communities. Further, the instructor team addressed the importance of cultural competency when delivering behavioral telehealth services in rural communities with unique cultural values, worldviews and resources. Strategies for practicing behavioral telehealth in rural communities in a culturally competent manner were discussed. Participants also received training strategies to detect and prevent vicarious trauma, which occurs when a behavioral health provider experiences secondary trauma through empathic engagement with the patients' trauma³⁹. Some of the strategies that were addressed in an effort to prevent vicarious trauma included diversifying client caseload, accessing routine clinical supervision and adhering to a well-balanced self-care plan.

Roles and responsibilities of the behavioral telehealth coordinator: The first set of six behavioral telehealth competencies required participants to understand the roles, responsibilities and duties of behavioral telehealth coordinators working at the distal site. The behavioral telehealth coordinator position may be held by a paraprofessional and it requires an individual to be knowledgeable about the video conferencing equipment. The coordinator serves as a liaison between the client and therapist.

• To achieve proficiency on Competency 1, the participants must have identified the coordinator's responsibility for conducting equipment checks to ensure that connectivity and equipment are operational prior to the session.
• Competency 2 required knowledge about the coordinator's duty to ensure that appropriate paperwork is collected and returned to the therapist, such as a completed informed consent form.
• Competency 3 addressed awareness of the coordinator's responsibility for fostering a sense of privacy and confidentiality with the environment and videoconferencing equipment. For instance, the coordinator may place a privacy sign on the door at the distal site and ensure that the videoconference session remains uninterrupted by others.
• Competency 4 involved identifying the importance of the coordinator training the client on the basic use of the equipment.
• Competency 5 involved recognizing that the coordinator is responsible for providing the client with a plan for handling equipment failure.
• Competency 6 required knowledge of the coordinator's role in introducing the client to the therapist over the videoconferencing equipment.

Roles and responsibilities of the therapist: The second set of eight behavioral telehealth competencies involved roles and responsibilities of the therapist.

• Competency 7 addressed understanding the importance of the therapist effectively managing technology. This competency requires successfully utilizing the remote control to correctly frame themselves and their clients on the screen, utilizing picture-in-picture features to monitor their appearances on their client's screen, and instructing their clients on the use of the remote control.
• Competency 8 involved confirming and managing relevant paperwork. This competency requires recognizing the importance of the therapist working with the coordinator to ensure that the client has a copy of the informed consent and other documents, which are to be reviewed during the session.
• Competency 9 addressed understanding of the therapist's responsibility for fostering a sense of privacy and confidentiality with behavioral telehealth services. This competency includes the therapist spanning the office with the camera allowing the client to see that no one else is present in the office at the distal site. It also involves the therapist addressing the client's concerns regarding privacy and confidentiality.
• Competency 10 required attention to the therapist demonstrating appropriate video 'presence'. Attending to video 'presence' includes an awareness of the influence of lighting and wall color on the quality of the video. It also includes ways to convey empathy across the screen by remaining visually attentive during the session and exaggerating facial expression and verbal responses at times. Further, participants were instructed on how to correctly frame themselves on screen from the waist up, allowing the client to see the therapist's face and body language. Finally, participants were encouraged to consider ways to set up their workspace to not distract the client or block the client's view of the therapist.
• Competency 11 required understanding the importance of the therapist acknowledging technical difficulties and responding to problems appropriately. This competency involved developing comfort with openly discussing technical problems with the client as they arise.
• Competencies 12 and 13 addressed understanding of the coordinator and therapist working together to maintain client confidentiality and establishing an emergency plan in advance of providing client care. Participants were encouraged to consider ways to handle paperwork, case notes and scheduling to protect client confidentiality. They were also encouraged to think about safety planning and crisis intervention procedures. Competency included acknowledging the importance of knowing the community resources and emergency contact numbers at the distal site.
• Competency 14 required understanding the importance of a therapist demonstrating sensitivity to specific aspects of the client's culture or unique location.

Pre- and post-test assessment

In order to assess the competency of the participants before and after the training program, the researchers, along with other members of the instructor team, developed two sets of video vignettes consisting of various mock therapy scenarios. The vignettes utilized picture-in-picture technology depicting a therapist and a client engaging in behavioral telehealth counseling services. Each vignette targeted key competencies that participants were expected to develop during the training. Participants were asked to view one set of vignettes in which mistakes were made such as the therapist failing to secure client confidentiality, exemplified by a staff person knocking on the door of the client's therapy room at the distal site and walking into the room. As participants viewed the vignettes, they were allotted time to provide written responses to a series of written prompts assessing their understanding of competencies relevant to each scenario. The written prompts were developed by researchers and members of the instructor team and included open-ended questions such as 'What should the coordinator and therapist do to ensure client privacy/confidentiality?', 'How would you handle this segment of the therapy session?' and 'How would you handle the concerns presented in this segment?'. During the training, participants reviewed and discussed the video vignettes from the pre-test assessment as an instructional tool for teaching the various competencies. To control for testing effects, a different set of vignettes (based on the same competencies) was used at post-test. The participants viewed the second set of video vignettes and provided written responses to prompts after completing the training.

Participant responses were independently coded by two raters and each response was assigned a code, as follows: 0 = competency not identified; 1 = competency partially identified; and 2 = competency fully identified. Coders reached an 86% interrater reliability. All disagreements in codes between the two raters were discussed to consensus. Competency scores for each participant at pre- and post-test consisted of the sum of ratings for the 14 competencies.

Self-report evaluation and follow-up survey

At the end of the training program and following the post-test assessment, participants completed anonymous evaluations of the training program. Evaluation prompts sought to measure participants' perceived competency in behavioral telehealth, rural ethics and cultural considerations, and vicarious trauma. Participants responded to prompts on a Likert scale ranging from 1 to 5, where 1 = low (do not agree) and 5 = high (completely agree).

Follow-up surveys were sent to participants 4 to 5 months after the program. Participants rated progress on the goals they set during the training program using a 5-point Likert scale, where 1 = no progress and 5 = substantial progress. Participants described factors hindering and promoting progress and how they planned to sustain their progress or overcome barriers to their progress. The researchers along with other members of the instructor team developed the self-report evaluation and follow-up survey.

Ethics approval

This study was approved by the University of Alaska Fairbanks Institutional Review Board as a program evaluation.

Results

Pre- and post-test assessment

Fourteen participants completed both a pre-test and post-test video vignette assessment. A paired-samples t-test was conducted to evaluate whether behavioral telehealth competency increased after the 14 participants completed the training program. The mean competency score after the training program (M = 12.07, SD = 4.2) was significantly greater than prior to the training program (M = 7.57, SD = 4.2; t(13) = 3.38, p<0.005). Results suggest that participants increased their overall skills in the competency areas addressed in the training program.

Participants demonstrated the most growth on competencies 2, 10, 11 and 14 (Table 1). Competency 2 involved identifying the importance of the coordinator ensuring appropriate paperwork is completed. Competency 10 involved identifying the need for the therapist to demonstrate appropriate video 'presence'. Competency 11 involved identifying the importance of the therapist acknowledging technical difficulties and responding to problems appropriately. Finally, competency 14 required understanding the importance of a therapist demonstrating sensitivity to specific aspects of the client's culture or unique location.

Table 1: Growth of participant competencies from pre-test to post-test

Self-report evaluation and follow-up survey

Responses from 17 participants who participated in the self-report portion of the study were analyzed. Participants completed the evaluative surveys at the end of the training program. Several participants had missing data on items assessing a particular construct and thus were not included in analyses of that construct. Results indicated excellent internal consistency reliability on the three constructs assessed: perceived telebehavioral health competency, perceived rural ethics and cultural competency, and perceived vicarious trauma competency α = 0.96, 0.91 and 0.92; n = 16, 17 and 15; SE = 0.015, 0.033 and 0.028; 95% CIs [0.93, 0.99], [0.84, 0.97] and [0.87, 0.98], respectively). Larger coefficient α scores suggest that items are more likely to contribute to a reliable scale. According to Nunnally and Bernstein, 0.70 and above is an acceptable reliability coefficient⁴⁰. The lower limits of the confidence intervals, assessed through SPSS programming codes supplied by Iacobucci and Duhacheck, were all considerably above this criteria, giving greater confidence that the items contribute to a reliable scale⁴¹. It is reasonable to combine these related items into three separate scales: Perceived Telebehavioral Health Competency Scale (PTHCS), Perceived Rural Ethics and Cultural Competency Scale (PRECCS), and Perceived Vicarious Trauma Competency Scale (PVTCS) (Fig1).

Figure 1: Items composing the perceived competency scales.

Based on a 5-point Likert scale, analysis indicated a mean response of 4.3 (SD = 0.72) on the PTHCS, indicating that participants perceived themselves as competent in behavioral telehealth service delivery. The mean response for the PRECCS was 4.15 (SD = 0.75), suggesting that participants perceived themselves as competent in rural ethics and culturally appropriate service delivery. Responses on the PVTCS produced a mean of 4.38 (SD = 0.61), which suggests that participants perceived themselves as competent in identifying and preventing vicarious trauma.

Nine participants responded to the follow up survey. Seven of the nine participants reported on one goal whereas two participants reported on three goals. A total of 13 goals were reported on by the nine participants. Analysis of participant goals indicated three main categories of goals: (i) develop and coordinate a behavioral telehealth program; (ii) provide behavioral telehealth information, education and training to others; and (iii) improve current behavioral telehealth services. Substantial progress was noted on 15% of the goals participants reported on in the survey. Forty-six percent of the goals achieved some progress, 23% of the goals made minimal progress, 8% achieved no progress and 8% were described with uncertainty regarding the progress made. Participants reported a lack of resources and interagency collaboration as primary hindrances to their progress. Lack of resources included difficulty acquiring funds for the equipment and personnel needed to develop, implement and routinely staff a behavioral telehealth program. Some participants alleviated these hindrances by interagency collaboration. Other participants found it necessary to build or repair relationships with other organizations and provide them with information about behavioral telehealth prior to soliciting their support.

In sum, results indicated that participants' behavioral telehealth competencies increased after completing the training. This finding was strengthened by participants' self-reports indicating positive perceptions regarding their own behavioral telehealth competency. Follow-up surveys indicated that participants made progress on the majority of the behavioral telehealth service delivery goals set during the training program even though obstacles existed.

Discussion

This study evaluated a behavioral telehealth training model tailored to meet the needs of providers working in rural Alaska with culturally diverse clientele. It utilized an innovative method of pre- and post-test assessment, measured participant self-perceptions of telehealth competency following the training program, and queried them about progress made on goals set during the training program. Findings converged and supported the benefits of the training as evidenced by participants improving their overall behavioral telehealth competency scores by over 50%. Further, participants reported positive self-perceptions of competencies in all three domains (behavioral telehealth, rural ethics and cultural competency, and vicarious trauma) and noted progress on over 60% of the behavioral telehealth goals set during the training.

This training allowed participants to develop behavioral telehealth competencies, a unique set of skills not typically part of training programs for helping professionals. Participants demonstrated the most growth in understanding the need for the therapist to demonstrate appropriate video 'presence'. Other literature supports the need for developing specific skills to foster a therapeutic relationship over video conferencing equipment. Behavioral telehealth providers must accommodate for the loss of non-verbal stimuli innate to videoconferencing and learn to use the technology to maximize observations of non-verbals²⁴. Behavioral telehealth therapists report being more explicit and deliberate in their non-verbal communication, asking more questions about their clients' nonverbal behavior, and find it important to meet the client face-to-face in the client's rural community during the first phase of therapy²⁴. All of these findings are important considerations for behavioral telehealth providers and highlight the need for establishing appropriate video 'presence' and developing a therapeutic relationship, both of which were major emphases of ARBHTA's training.

As professionals, many providers expect to find ways to adapt tools to fit their practice, and telehealth tools require both patients and providers to adapt. Comprehensive telehealth training for providers influences their attitudes toward and perceptions of telehealth service delivery. During the ARBHTA training, participants were encouraged to acknowledge discomfort associated with learning to use technology and developing a new set of skills. Further, they were encouraged to be transparent with their clients about their initial discomfort and the potential for technical problems. This training focus likely explains the substantial growth in understanding the importance of the therapist acknowledging technical difficulties and responding to problems appropriately. During the training, trainers normalized participant concerns and encouraged them to find creative solutions to problems that presented, such as muting the videoconferencing equipment and continuing to talk by telephone if the sound was choppy or pausing for a few seconds to avoid talking over the client. In a recent study, the quality of audio and video teleconferencing influenced the content of telehealth sessions, including the time spent on patient education and psychosocial issues⁴². Factors such as delays in signals causing disruptions to normal communication patterns and technical difficulties may also affect the therapeutic relationship. However, research suggests that clients and providers accommodate to the signal delays and make adjustments for technical difficulties by holding shorter sessions or sessions over the telephone²⁴.

Although participant competency appeared to increase overall, findings also revealed that participant scores predominately remained the same on some competencies. This finding can best be explained by a number of participants having prior experience working with paraprofessionals located in remote areas of Alaska. Consequently, most participants would likely have been familiar with the importance of the client knowing how to contact the paraprofessional if needed (competency 5) and the role of the paraprofessional acting as a liaison by introducing the client to the therapist (competency 6). Further, participants ranged from limited to substantial behavioral telehealth knowledge, experience and skills. Consequently, some competencies may have remained unchanged for those participants with prior knowledge, experience and skill.

The Alaska Rural Behavioral Health Training Academy's training intentionally built-in opportunities for participants to become familiar with videoconferencing equipment to reduce feelings of intimidation regarding the technology. Hands-on familiarization with equipment was essential to the training, which sought to address findings in the literature indicating that healthcare providers can find telehealth technology intimidating, react critically to it, and doubt their ability to foster a positive relationship and attend to patient needs in a comprehensive manner with reliable connectivity via telehealth⁴³. In addition to developing comfort and familiarization with the technology, organizational support for the behavioral telehealth program is also important for provider success. Results from this study indicate that participants struggled to progress on some goals as a result of a lack of funding and support for equipment and personnel to staff a behavioral telehealth program. Other barriers noted in the literature include professionals reacting negatively to telehealth due to increased time demands for initial equipment set up, maintenance and upgrades along with the requirement to develop a new set of skills in order to install, maintain and utilize the equipment⁴³. Taken together, it is important to consider the influence of organizational support of a telehealth program on the providers' attitudes, perceptions and experiences of telehealth service delivery.

An important feature of ARBHTA's training is the perspective of behavioral telehealth as a tool for service delivery. Trainers challenged participants to consider the strengths and limitations of the technology while helping them brainstorm ways to adapt their services to integrate telehealth as a tool. In particular, participants were encouraged to consider creative ways to manage record keeping across the distance. This training focus may explain the substantial growth in understanding the importance of the coordinator ensuring appropriate paperwork is collected and returned to the therapist.

Although interactions through telecommunications differ from face-to-face interactions, examples in the literature of behavioral telehealth as a supported modality are increasing, and it is considered reasonable care⁴⁴. Telepsychiatry assessments produce reliable results, positive clinical outcomes, and overall patient and provider satisfaction^45-47. In addition, a number of studies suggest that behavioral telehealth services are comparable to face-to-face services with many populations, including older individuals⁴⁸, Native veterans^44,47,49,50, veterans^45,51, prisoners⁵², children and adolescents⁵³, and adults residing in geographically remote areas⁵⁴. The literature supports efforts to continue training Alaskan providers to deliver services via telehealth in an effort to diminish the inequities in access to care for culturally diverse clientele. Findings from this study revealed substantial growth in understanding the need for the therapist to demonstrate sensitivity to specific aspects of the client's culture or unique location. This finding suggests that this model of continuing education training positively affected participants' awareness of the importance of cultural competency in telehealth service delivery, an essential training component that was interwoven throughout the entire training.

Conclusion

This study utilized multiple methods to measure participant behavioral telehealth competency. By combining responses to innovative video vignettes with self-report methods, the overall strength of the study was increased. To control for testing effects, a different set of vignettes was used in the pre-test than in the post-test assessment. Even with a multiple method design, limitations exist in this study. The training was limited to 21 participants to allow enough time for each participant to become familiar with the videoconferencing equipment. Therefore, the sample size was small and a control group was not available to compare against the intervention group. Further, researchers discovered that the pre- and post-test video vignettes contained too many competencies per vignette. Hence, participants may have struggled with thoroughly attending to all of the competencies being depicted and assessed in each vignette. In addition, many participants focused on the clinical skills depicted in the vignette, which may have interfered with their ability to focus on the behavioral telehealth competencies being assessed. This distraction may explain why some participants achieved lower scores on the post-test than on the pre-test assessment for some competencies. Future development of video vignettes for assessing competencies are advised to test only one or two competencies per scene, use written prompts specific to the scene, and develop the scenes in a way that does not confuse suboptimal clinical skills with inadequate behavioral telehealth skills. Finally, participants reported on self-perceptions of behavioral telehealth competency following completion of the training. Data on their perceptions of competency was not gathered prior to the training; therefore, baseline self-perceptions could not be compared with post-training self-perceptions of competency.

Future directions could include developing and assessing a model for best practice and competency in behavioral telehealth service delivery. Essential elements of the model may include utilization of a paraprofessional site coordinator; cultural awareness and sensitivity specific to the area and clients being served at the distal site; videoconferencing equipment expertise; ability to process client concerns about behavioral telehealth services and technical difficulties; competence with adapting clinical skills to fit behavioral telehealth service delivery; managing documentation appropriately; and safety planning for clients located miles away. Future directions could also include refining an assessment tool that utilizes video vignettes to assess provider competency.

This study contributes to the knowledge base on behavioral telehealth service delivery. It identifies specific competencies necessary to deliver effective behavioral telehealth services. Moreover, the identified competencies provide a baseline for the development of a best practice model for behavioral telehealth service delivery. This study offers a unique training model that incorporated behavioral telehealth, rural ethics, cultural competency and vicarious trauma training into a 3 day, face-to-face intensive training program led by an instructor team consisting of content experts including university professors and Alaska Native Elders. Lastly, this study describes the use of an innovative video vignette assessment instrument for evaluating the effectiveness of continuing education training. Such evaluation can influence future training efforts to develop a competent workforce, reduce health disparities and improve access to care for residents of rural Alaska.

References

1. Ciarlo JA, Wackwitz JH, Wagenfeld MO, Mohatt DF, Zelarney PT. Focusing on 'frontier': isolated rural America - Letter to the field no. 2. (Online) 1996. Available: http://www.wiche.edu/MentalHealth/Frontier/letter2.asp (Accessed 29 April 2011).

2. Johnson M, Brems C, Warner T, Roberts L. Rural-urban health care provider disparities in Alaska and New Mexico. Administration and policy. Mental Health 2006; 33(4): 504-507. doi:10.1007/s10488-005-0001-7.

3. Mohatt DF. (1997). Access to mental health services in frontier America - Letter to the field no. 4. (Online) 1997. Available: http://www.wiche.edu/MentalHealth/Frontier/letter4.asp (Accessed 29 April 2011).

4. Williams A, Cutchin M. The rural context of health care provision. Journal of Interprofessional Care 2002; 16(2): 107-115. doi:10.1080/13561820220124120.

5. Human J, Wasem C. Rural mental health in America. American Psychologist 1991; 46(3): 232-239. doi:10.1037/0003-066X.46.3.232.

6. Roberts L, Battaglia J, Epstein R. Frontier ethics: mental health care needs and ethical dilemmas in rural communities. Psychiatric Services 1999; 50(4): 497-503.

7. Grubaugh A, Cain G, Elhai J, Patrick S, Frueh B. Attitudes toward medical and mental health care delivered via telehealth applications among rural and urban primary care patients. Journal of Nervous and Mental Disease 2008, 196(2): 166-170. doi:10.1097/NMD.0b013e318162aa2d.

8. Nickelson D. Telehealth and the evolving health care system: strategic opportunities for professional psychology. Professional Psychology: Research and Practice 1998; 29(6): 527-535. doi:10.1037/0735-7028.29.6.527.

9. Glueckauf R, Nickelson D, Whitton J, Loomis J. Telehealth and health care psychology: current developments in practice and research. In: RG Frank, A Baum, JL Wallander (Eds). Handbook of clinical health psychology: Volume 3. Models and perspectives in health psychology. Washington, DC: American Psychological Association, 2004; 377-411.

10. Saab P, McCalla J, Coons H, Christensen A, Kaplan R, Melamed B et al. Technological and medical advances: implications for health psychology. Health Psychology 2004; 23(2): 142-146. doi:10.1037/0278-6133.23.2.142.

11. Alaska Center for Rural Health, University of Alaska Anchorage. Rural/frontier. (Online) 2011. Available: http://acrh-ahec.uaa.alaska.edu/frontier/alaska.html (Accessed 29 April 2011).

12. University of Alaska Anchorage Institute of Social and Economic Research. Alaskool central: Alaska Native languages. (Online) 2004. Available: http://www.alaskool.org/language/languageindex.htm (Accessed 29 May 2011).

13. Alaska Department of Labor and Workforce Development. Borough/census area maps and demographic profiles. (Online) 2002. Available: http://labor.alaska.gov/research/cgin/cenmaps/statemap.htm (Accessed 29 May 2011).

14. Sherry P. Health care delivery for Alaska Natives: a brief overview. International Journal of Circumpolar Health 2004; 63(2Suppl) 254-262.

15. Bristol Bay Area Health Corporation. Welcome. (Online) 2006. Available: http://www.bbahc.org/index.html (Accessed 29 April 2011).

16. Council of Athabascan Tribal Governments. Behavioral Health. (Online) 2012. Available: http://www.catg.org/ index.html (Accessed 27 November 2012).

17. Norton Sound Health Corporation. Our Services. (Online) 2012. Available: http://nortonsoundhealth.org/index.html (Accessed 27 November 2012).

18. Yukon Kuskokwim Health Corporation. Services. (Online) 2012. Available: http://www.ykhc.org (Accessed 26 November 2012).

19. Cunningham P, Madigan R, Mann C, Ward K. The 2002 social services job survey: final report to the University of Alaska President Mark Hamilton. (Online) 2003. Available: http://www.uaa.alaska.edu/psych/upload/2002JobSurvey.pdf (Accessed 29 April 2011).

20. Fischer DG, Pearce FW, Statz DJ, Wood MW. Employment retention of health care providers in frontier areas of Alaska. International Journal of Circumpolar Health 2003; 63(4): 423-435.

21. University of Alaska Anchorage Alaska Center for Rural Health. 2007 Alaska health workforce vacancy study. (Online) 2007. Available: http://acrh-ahec.uaa.alaska.edu/projects/archives/pdf/2007workforce.pdf (Accessed 29 April 2011).

22. Alaska Native Tribal Health Consortium. AFHCAN Telehealth Solutions. (Online) 2011. Available: http://www.afhcan.org (Accessed 27 November 2012).

23. Hudson H. Rural telemedicine: lessons from Alaska for developing regions. Telemedicine Journal and e-Health 2005; 11(4): 460-467.

24. Bischoff R, Hollist C, Smith C, Flack P. Addressing the mental health needs of the rural underserved: findings from a multiple case study of a behavioral telehealth project. Contemporary Family Therapy: An International Journal 2004; 26(2): 179-198. doi:10.1023/B:COFT.0000031242.83259.fa.

25. Cloutier P, Cappelli M, Glennie J, Keresztes C. Mental health services for children and youth: a survey of physicians' knowledge, attitudes and use of telehealth services. Journal of Telemedicine and Telecare 2008; 14(2): 98-101. doi:10.1258/jtt.2007.070815.

26. Swinton J, Robinson W, Bischoff R. Telehealth and rural depression: physician and patient perspectives. Families, Systems, & Health 2009; 27(2): 172-182. doi:10.1037/a0016014.

27. Stamm B. Bridging the rural-urban divide with telehealth and telemedicine. In: B Hudnall Stamm (Ed). Rural behavioral health care: an interdisciplinary guide. Washington, DC: American Psychological Association, 2003; 145-155.

28. Burke B, Bynum A, Hall-Barrow J, Ott R, Albright M. Rural school-based telehealth: how to make it happen. Clinical Pediatrics 2008; 47(9): 926-929. doi:10.1177/0009922808320597.

29. Jennett P, Jackson A, Healy T, Ho K, Kazanjian A, Bates J et al. A study of a rural community's readiness for telehealth. Journal of Telemedicine and Telecare 2003; 9(5): 259-263. doi:10.1258/135763303769211265.

30. Jennett P, Watanabe M. Healthcare and telemedicine: ongoing and evolving challenges. Disease Management & Health Outcomes 2006; 14(1Suppl): 9-13. doi:10.2165/00115677-200614001-00004.

31. Payette C, Desrochers J, Lavoie-Tremblay M, Richer M. Exploring perceptions of healthcare professionals in the implementation of a new professional role of clinical telehealth coordinator within a university integrated healthcare network. Telemedicine Journal and e-Health 2010; 16(5): 614-619. doi:10.1089/tmj.2009.0154.

32. Peddle K. Telehealth in context: socio-technical barriers to telehealth use in Labrador, Canada. Computer Supported Cooperative Work (CSCW) 2007; 16(6): 595-614. doi:10.1007/s10606-006-9030-3.

33. Jarvis-Selinger S, Chan E, Payne R, Plohman K, Ho K. Clinical telehealth across the disciplines: lessons learned. Telemedicine Journal and e-Health 2008; 14(7): 720-725. doi:10.1089/tmj.2007.0108.

34. Moehr J, Anglin C, Schaafsma J, Pantazi S, Anglin S, Grimm N. Video conferencing-based telehealth - its implications for health promotion and health care. Methods of Information in Medicine 2005; 44(2): 334-341.

35. Ricci M, Caputo M, Callas P, Gagne M. The use of telemedicine for delivering continuing medical education in rural communities. Telemedicine Journal and e-Health 2005; 11(2): 124-129.

36. Gattoni A, Tenzek K. The practice: an analysis of the factors influencing the training of health care participants through innovative technology. Communication Education 2010; 59(3): 263-273. doi:10.1080/03634521003605808.

37. Madison R. Behavioral health partnership initiatives funded. University of Alaska Health Resources on the Web. (Online) 2004. Available: http://www.alaska.edu/health/downloads/BHP%20Init%20Final.pdf (Accessed 27 November 2012).

38. Koverola C, Gifford V, Landen P. An innovative approach to behavioral health workforce issues in rural Alaska. Symposium, Western Psychological Association Convention; 5 May 2007; Vancouver, BC; 2007.

39. McCann I, Pearlman L. Vicarious traumatization: a framework for understanding the psychological effects of working with victims. Journal of Traumatic Stress 1990; 3(1): 131-149. doi:10.1007/BF00975140.

40. Nunnally JC, Bernstein IH. Psychometric theory, 3rd edn. New York: McGraw Hill, 1994.

41. Iacobucci D, Duhacheck A. Advancing alpha: measuring reliability with confidence. Journal of Consumer Psychology 2003; 13(4): 478-487.

42. Demiris G, Edison K, Vijaykumar S. A comparison of communication models of traditional and video-mediated health care delivery. International Journal of Medical Informatics 2005; 74(10): 851-856. doi:10.1016/j.ijmedinf.2005.03.018.

43. Hibbert D, Mair F, May C, Boland A, O'Connor J, Angus R et al. Health professionals' responses to the introduction of a home telehealth service. Journal of Telemedicine and Telecare 2004; 10(4): 226-230. doi:10.1258/1357633041424386.

44. Godleski L, Nieves J, Darkins A, Lehmann L. VA telemental health: suicide assessment. Behavioral Sciences & the Law 2008; 26(3): 271-286. doi:10.1002/bsl.811.

45. Frueh B, Monnier J, Yim E, Grubaugh A, Hamner M, Knapp R. A randomized trial of telepsychiatry for post-traumatic stress disorder. Journal of Telemedicine and Telecare 2007; 13(3): 142-147. doi:10.1258/135763307780677604.

46. Monnier J, Knapp R, Frueh B. Recent advances in telepsychiatry: an updated review. Psychiatric Services2003; 54(12): 1604-1609.

47. Shore J, Manson S. The American Indian veteran and posttraumatic stress disorder: a telehealth assessment and formulation. Culture, Medicine and Psychiatry 2004; 28(2): 231-243. doi:10.1023/B:MEDI.0000034413.61037.0d.

48. Lee J, Kim J, Jhoo J, Lee K, Kim K, Woo J et al. A telemedicine system as a care modality for dementia patients in Korea. Alzheimer Disease and Associated Disorders 2000; 14(2): 94-101.

49. Shore J, Savin D, Orton H, Beals J, Manson S. Diagnostic reliability of telepsychiatry in American Indian veterans. The American Journal of Psychiatry 2007; 164(1): 115-118.

50. Shore J, Brooks E, Savin D, Orton H, Grigsby J, Manson S. Acceptability of telepsychiatry in American Indians. Telemedicine Journal and e-Health 2008; 14(5): 461-466. doi:10.1089/tmj.2007.0077

51. Fortney J, Pyne J, Edlund M, Williams D, Robinson D, Henderson K et al. A randomized trial of telemedicine-based collaborative care for depression. Journal Of General Internal Medicine 2007; 22(8): 1086-1093. doi:10.1007/s11606-007-0201-9.

52. Morgan R, Patrick A, Magaletta P. Does the use of telemental health alter the treatment experience? Inmates' perceptions of telemental health versus face-to-face treatment modalities. Journal of Consulting and Clinical Psychology 2008; 76(1): 158-162. doi:10.1037/0022-006X.76.1.158.

53. Ermer D. Experience with a rural telepsychiatry clinic for children and adolescents. Psychiatric Services 1999; 50(2): 260-261.

54. O'Reilly R, Bishop J, Maddox K, Hutchinson L, Fisman M, Takhar J. Is telepsychiatry equivalent to face-to-face psychiatry? Results from a randomized controlled equivalence trial. Psychiatric Services 2007; 58(6): 836-843.