A modular approach to rural and remote research education: a project report

full article:

Introduction

Building research capacity is seen as a means of creating both locally relevant research, which informs practice, and the skills or organisational structures to enable ongoing research activities¹. A number of programs have attempted to solve the issue of building research capacity in the Australian rural health sector, notably the Australian Government's Primary Health Care Research, Evaluation and Development strategy and its Researcher Development Program^2-5, and NSW Health's Rural Research Capacity Building Program⁶. Programs such as these demonstrate that outcomes for research capacity building, while difficult to measure, can be considerable and are often as the result of quite modest investment^4,6,7. In addition to building research skill, capacity building strategies stimulate an interest in undertaking further research⁸ in practice or policy related areas⁹, with small scale research projects leading to more ambitious research questions for larger scale, collaborative work³. Research capacity endeavours create an environment in which health professionals, policy makers and researchers can collaborate⁹.

The challenge of building rural and remote research capacity in Australia is complicated by many issues, many of which are structural issues beyond the scope of influence of the individual with an interest in research. These issues include a shortage of rural researchers, limited funding¹⁰, access to expertise¹¹ and the need for specific training in rural health research¹². Other structural challenges include busy workloads with a lack of dedicated time to research and a lack of organisational support¹³. Many of these issues overlap: without access to knowledge, support and resources research activity does not occur, thus organisations do not see the value of research in action and may not see the need to support research activities within their sites. In these environments there is limited ability for an individual to undertake training in research or conduct research activity.

Given a lack of support structures in the rural health workplace, support may need to come from outside of rural health organisations, and one of the known drivers of successful research capacity building is meaningful collaboration¹⁰. Building meaningful collaborative relationships between university departments of rural health and rural clinical schools can assist health services and health providers to overcome limited access to expertise³.

It was in the context of these challenges that the Centre for Research Excellence in Rural and Remote Primary Health Care Research (CRE) created a Research Capacity Building Program (RCBP). This program aimed to:

• strengthen research capacity in those regions where the CRE operates (Bendigo, Alice Springs and Broken Hill)
• collaborate with key primary healthcare (PHC) organisations such as federally funded networks that are designed to support the development of PHC and health service organisations in the geographical areas of responsibility
• build both individual and organisational capacity to undertake health services research, evaluation and development work
• develop, in collaboration with participating health service organisations, supervised, 'close to practice' research programs for novice researchers aimed at improving access to or performance of health services delivered by that organisation, individually or in partnership with other organisations
• develop a transferable capacity building program with supervision and support based on adult learning principles, to extend skills
• develop relationships with participating health service organisations
• promote the development of research writing skills and improve publications in the field of rural health services research.

The RCBP utilised philosophies of experiential learning and cultural constructivism, which can combine learning and research in a single framework¹⁴, namely that novice researchers learn research skills by undertaking a research project in a field of interest and importance to them and their organisation, and by associating with others undertaking research. This article aims to both describe the approach used within the RCBP and evaluate its effectiveness in meeting its aims as a development program.

The structure of the CRE RCBP was in response to the unique geographical challenges posed by the rural and remote landscape. Each faculty of the CRE hosted two or three trainees, with each site providing a postdoctoral researcher who acted as the research supervisor for trainees at that site. Thus the RCBP was designed in a modular fashion: a single program across three hub sites (Alice Springs, Bendigo, Broken Hill) which acted as self-contained units or 'modules', with both a vertical support structure, which allowed access to supervision and expertise from CRE staff, and a horizontal or peer support network within and across sites.

Research trainees were chosen with the support of the trainee's organisation, and projects needed to demonstrate congruity with CRE goals. Training was by a single short course in research methods, with a range of research topics (Table 1), although additional research methods mentoring was available by regular or ad hoc teleconference. The short course was followed by smaller group meetings at each hub site, which were facilitated at the discretion of the CRE staff at each hub site. Supports for trainees included mentoring over the 2-year period, and regular teleconference support was offered on a monthly basis.

Trainees were funded and supported by their workplaces to dedicate approximately 1 day per week on their research project, either as a day weekly or in blocks of time to balance research and usual workloads. Trainees were able to access library, statistical and technical support from staff at the CRE hub sites.

Projected outcomes of the RCBP included presentation of research findings at a workplace or organisational level and at an appropriate conference. A formal research report was required and trainees were required to submit an article to a peer-reviewed journal on completion of the research reporting phase.

Table 1: Topics taught in research methods short course

Methods

Evaluation of the RCBP used a cross-sectional study, and a questionnaire adapted from that developed by McIntyre et al⁸, who provided permission to use the questionnaire in full or in part. Invitations to participate were sent by email by the one of this study's authors (SK), along with a participant information sheet, consent form and questionnaire to research trainees (n=8), managers (n=4) and CRE staff (n=8).

Only four managers were contacted due to lack of manager contact details or requests from trainees to not contact their manager. Questionnaires were completed and returned by email or completed verbally by phone with either the lead author (DS) or co-author (SK) if preferred. The survey was conducted at the completion of the 2-year RCBP period.

Self-assessed research experienced was assessed using the research spider tool developed by Smith et al¹⁵, which is a commonly used tool for measuring self-reported research experience in 10 domains of research practice. Participants completed the baseline spider after an introductory research methods workshop that also included non-RCBP trainees. The spider was reassessed at program completion. Additional questions on support aspects of the RCBP were rated on a five-point scale from 1 ('very supportive') to 5 ('not at all supportive').

Due to the small numbers in each group of this study no attempt was made to complete detailed statistical analysis of findings, with the exception of accumulated research spider data, which was analysed using the same method applied by Harding et al, who used the Wilcoxon signed rank test to compare pre- and post-training scores and reported scores as mean and standard deviation⁷. All other responses were aggregated where appropriate, with the goal of providing as comprehensive an understanding of the RCBP as possible. Collation and analysis of the study data was completed by the lead author (DS) and analysis was completed using Excel and GraphPad software.

Ethics approval

Ethical approval was granted by the University of Sydney Human Research Ethics Committee (ref 2014/007).

Results

Responses were received from trainees (n=7 from 8), managers (n=4 from 4) and CRE staff (n=8 from 8), giving an overall response rate of 95%. The profile of the trainees engaged in the RCBP is shown in Table 2. All trainees came originally from nursing, allied health or population health backgrounds, all of which are traditionally female dominated.

Trainees applied to the RCBP for the purpose of developing research skill or to evaluate an existing work practice. Some saw an opportunity to experience research without committing to a research higher degree. Managers supported application to the program on the basis of organisational need and benefits, with personal development of the trainee often a secondary consideration.

While trainees were funded at the rate of A$25,000 per participant to their employing organisation to undertake their project for 1 day per week, substantive workloads necessitated flexibility about the allocation of research time at points along the research journey and sometimes involved additional days or additional work out of paid hours. While four of the seven trainees largely used 1 day per week, some trainees used block release from time to time, which allowed the capacity to focus more comprehensively on the project. Inability to quarantine time was reported as major barrier to progress.

Trainees were often required to work away from their regular workplace. Five trainees completed their projects whilst working from home, at a CRE campus or other offsite location. For those continuing to research at their existing workplace, managerial support and intervention facilitated project time.

Impact of undertaking the RCBP on the trainee's substantive role ranged from none to very substantial in the opinion of trainees and managers, with little agreement between managers and the trainees where responses could be paired.

Aspects of the support and supervision provided to trainees through the RCBP are summarised in Table 3.

At completion of the RCBP, two trainees had completed their research report, with three more in draft or partially complete and a further two trainees yet to complete. Four of the seven trainees had presented their research in a work or scientific forum, which included professional meetings, health service board meetings and a PHC research conference. Publication of research findings was a goal for the trainees, with one article accepted to the journal Rural and Remote Health, two stating that their manuscript was ready to submit and a fourth indicating two articles would be written in the coming months.

All trainees indicated that they would consider future research. Managers indicated that the RCBP experience was useful in encouraging future research by the trainee and within their organisation, provided future research activity is compatible with the organisational goals and sufficient alignment between the supporting organisation and the CRE is available.

Table 2: Demographic characteristics of Research Capacity Building Program trainees




Table 3: Supervision and support ratings of trainees, managers and Centre of Research Excellence staff

Discussion

This project has shown that in a rural and remote area structural limitations to research training can be overcome by using a decentralised program spread across three hub sites in a modular fashion. Research activity was increased in those regions where the CRE operates and collaborating with key PHC organisations in the geographical areas of responsibility built both individual and organisational capacity to undertake health services research, evaluation and development work.

The structure of the RCBP had some advantages, including the ability to adapt processes to suit the local needs of each site, the availability of both supervision and resources at a (relatively) local site and a cluster of peers at the same site. However, peer support between trainees appeared to occur primarily within clusters around the hub sites, thereby limiting the opportunities to build a broader support network, share resources or coordinate learning approaches between sites.

Demonstrating the effectiveness of programs such as the RCBP to build research capacity at a system or organisational level can be difficult. Webster et al examined the perspective of candidates, their managers and providers of NSW Health's Rural Research Capacity Building Program and found that participants reported that their research skills had improved and that they felt supported⁶. However, the consensus was that the processes within the participating organisations worked against achieving the goal of improving research capacity. Similarly, McIntyre et al surveyed participants of the PHC Research Development Program and found that most of participants agreed that the program was a valuable experience and that they were interested in undertaking further research⁸. The authors reported that their results indicated that this program had a positive effect on the program Fellows in terms of knowledge of research, attitude to research, and the way they used research in their work.

When the RCBP is examined alongside the criteria for measuring research capacity building in health proposed by Cooke¹ and further developed by Sarre and Cooke¹⁶ there are some elements of the RCBP that are likely to lead to increased research capacity. These include:

• skill development of the individual
• research that is close to practice for the individual
• research activity within the organisation
• dissemination of research findings
• leadership (on behalf of the CRE).

Less evident were aspects such as:

• continuity and sustainability of research activity
• establishment of a research culture within the organisations
• linkages and partnerships (beyond the relationship between the CRE and partnering organisations)
• investments in (research) infrastructure.

When considering self-reported research experience, several trainees indicated they had some prior research involvement through their previous work roles. However, the group as a whole displayed limited self-rated research experience at commencement. Figure 1 displays mean research spider response for the group at commencement and completion. The mean research spider scores improved from 2.2/5 (standard deviation 0.5) to 2.8/5 (standard deviation 0.5), a change that was significant (z=-2.8, p=0.005).

Some perspective on the growth of trainees can be achieved by comparing the work of Smith et al¹⁵. At commencement RCBP short course trainees indicated research experience equivalent to that of a 'recipient of [a small] bursary and/or author of non peer-reviewed article(s) or letter(s)'¹⁵.

At completion of the RCBP trainees reported experience equivalent to 'a holder of research grant funding between £5000 to £10,000 and/or author of at least two peer-reviewed publications'¹⁵.

This growth is particularly notable given the lack of funding opportunities and support otherwise available in a remote or outer rural area.

It is evident from this project that the RCBP was valued by trainees who participated in the RCBP. The level of engagement was extremely high and a program completion rate of 87% compares extremely favourably with the program's closest parallel, NSW Health's Rural Research Capacity Building Program, which has a completion rate of 72%¹³. Outputs from the RCBP are comparable in terms of reports, publications and presentations to other Australian workplace-based novice researcher programs^4,7,17.

Many of the challenges facing the RCBP are those experienced by other education or training programs operating in the rural health environment and include isolation, the tyranny of distance and the challenge of remote support^6,12,13,18. Increased use of technologies such as video or web conferencing might have overcome some of the usual difficulties associated with rural and remote professional development for health professionals.

Continuity and sustainability are key elements of successful research capacity building^1,16. Whilst this program continues to see outputs from the research undertaken, both in terms of publications and ongoing practice change, the future of the capacity building efforts is contingent upon ongoing funding.

While this evaluation had a high engagement rate, with 19 of a possible 20 participants responding, the RCBP is a small program with a single cohort so there is a limit to how much can be drawn from the data. The use of a survey based data collection was a pragmatic choice which by its nature limits the depth of the data being collected; however, this is offset by the involvement of a range of key stakeholders, which provided a comprehensive overview of the program and its outcomes.

Figure 1: Trainees' mean self-assessed research experience at commencement and completion of Research Capacity Building Program, rated from 1 (no experience) to 5 (very experienced).

Conclusions

The program has produced outcomes that are consistent with building research capacity in health. It has met many of its establishment goals, demonstrating that through collaboration and a decentralised, modular model rural and remote research training is achievable. Measurable improvements in research experience created organisational and individual potential for ongoing PHC research activity. Ongoing support for local research training would build on this potential by enabling ongoing rural and remote research activity.

References

1. Cooke J. A framework to evaluate research capacity building in health care. BMC Family Practice (Internet) 2005; 6: 44. Available: http://www.biomedcentral.com/1471-2296/6/44 (Accessed 11 September 2011).

2. Bailey J, Veitch C, Crossland L, Preston R. Developing research capacity building for Aboriginal & Torres Strait Islander health workers in health service settings. Rural and Remote Health (Internet) 2006 6(4): 556. Available: www.rrh.org.au (Accessed 18 April 2012).

3. Birden HH. The researcher development program: how to extend the involvement of Australian general practitioners in research? Rural and Remote Health (Internet) 2007; 7(3): 776. Available: www.rrh.org.au (Accessed 18 April 2012).

4. Cameron HE, Boreland FT, Morris JR, Lyle DM, Perkins DA, Magin PJ, et al. New South Wales and Australian Capital Territory Researcher Development Program 2005-07: modest investment, considerable outcomes. Australian Journal of Primary Health 2013; 19(1): 59-67.

5. Brown LJ, McIntyre EL. The contribution of primary health care research, evaluation and development-supported research to primary health care policy and practice. Australian Journal of Primary Health 2012; 20(1): 47-55.

6. Webster E, Thomas M, Ong N, Cutler L. Rural research capacity building program: capacity building outcomes. Australian Journal of Primary Health 2011; 17(1): 107-113.

7. Harding KE, Stephens D, Taylor NF, Chu E, Wilby A. Development and evaluation of an allied health research training scheme. Journal of Allied Health 2010; 39(4): e143-148.

8. McIntyre E, Brun L, Cameron H. Researcher development program of the primary health care research, evaluation and development strategy. Australian Journal of Primary Health 2011; 17(1): 114-121.

9. Jansen MWJ, Hoeijmakers M. A masterclass to teach public health professionals to conduct practice-based research to promote evidence-based practice: a case study from The Netherlands. Journal of Public Health Management and Practice 2013; 19(1): 83-92.

10. Dunbar JA. Building capacity for rural and remote research. Australian Journal of Rural Health 2010; 18(4): 133.

11. Brett T, Sherrard R, Bower A. Research capacity building in general practice: a new opportunity in Fremantle, WA. Australian Family Physician 2006; 35(1-2): 65-66.

12. Taylor J, Hughes C, Petkov J, Williams M. Unique issues in research and evaluation in rural and remote locations: is there a place for specific research training? Rural and Remote Health (Internet) 2005; 5(2): 351. Available: www.rrh.org.au (Accessed 6 August 2013).

13. Schmidt D, Robinson K, Webster E. Factors influencing attrition from a researcher training program. International Journal for Researcher Development 2014; 5(1): 56-67.

14. Stauffacher M, Walter AI, Lang DJ, Wiek A, Scholz RW. Learning to research environmental problems from a functional socio-cultural constructivism perspective. International Journal of Sustainability in Higher Education 2006; 7(3): 252-275.

15. Smith H, Wright D, Morgan S, Dunleavey J, Moore M. The 'Research Spider': a simple method of assessing research experience. Primary Health Care Research and Development 2002; 3(3): 139-140.

16. Sarre G, Cooke J. Developing indicators for measuring research capacity development in primary care organizations: a consensus approach using a nominal group technique. Health & Social Care in the Community 2009; 17(3): 244-253.

17. Ried K, Montgomery BD, Stocks NP, Farmer EA. General practice research training: impact of the Australian Registrar Research Workshop on research skills, confidence, interest and involvement of participants, 2002-2006. Family Practice 2008; 25(2): 119-126.

18. Lloyd B, Pfeiffer D, Dominish J, Heading G, Schmidt D, McCluskey A. The New South Wales Allied Health Workplace Learning Study: barriers and enablers to learning in the workplace. BMC Health Services Research (Internet) 2014; 14(1): 134. Available: http://www.biomedcentral.com/1472-6963/14/134 (Accessed 27 March 2014).