The allied health rural generalist pathway: a cost consequence analysis

Introduction

Demand for allied health services in Australia has increased due to the expansion of disability and aged care services, increasingly complex health needs and an ageing population¹. Rural and remote health services face challenges recruiting and retaining allied health professionals to service their local communities². A range of workforce strategies is required to improve access to allied health in rural services^3-5. Furthermore, rural allied health professionals need skills across a broad range of clinical areas to meet the variable and complex needs of their communities, which can be overwhelming early in a career⁶. Postgraduate training is one strategy that can be used to develop skills relevant to rural allied health roles, and to help advance allied health professionals’ careers, facilitated by appropriate support structures.

Government services face economic challenges as they attempt to work within allocated funding, while also meeting the needs of communities. Significant funding is allocated to rural and remote health workforce initiatives and training^7-9 and judicious use of public funds is essential; however, there is a scarcity of research investigating the economic impact of these initiatives¹⁰. Economic evaluation explicitly compares the outputs or outcomes of a program, initiative or process to the resources used in its delivery by using systematic, transparent and quantifiable approaches¹¹. When health services are clear about the economic value of the services they provide, it is easier to justify the cost to funding bodies or to make decisions about where resources should be allocated for maximum benefits^10,12.

A range of methods of economic evaluation is available depending on the nature of the initiative to be evaluated, and the relevant costs and outcomes to consider from the decision-maker’s perspective. Regardless of the approach chosen, an economic evaluation will consider the costs and the outcomes (anticipated benefits) of the activity compared to a comparator (eg usual care or the status quo level of support)¹¹. Usually, best practice for an economic evaluation will involve aggregating and narrowing costs and outcomes of an activity into a single summary ratio, which then represents the incremental cost per unit of benefit, such as in a cost-effectiveness analysis or cost–utility analysis (considered a special kind of cost-effectiveness analysis where a standardised unit of benefit is used known as a quality-adjusted life year or QALY)¹³. Use of the QALY is considered an advantage of the cost–utility analysis method in some senses, such that it allows policymakers to compare the cost per QALY gained across different activities to identify that which provides the highest QALY for the lowest cost¹³. When analysing the impact of a service development program or initiative, there is often a range of inputs and outputs considered important, depending on the perspective of the decision-maker, and some of these are not easily converted into a QALY or valued as a cost^14-16. Additionally, for some activities there might not be a single benefit or outcome that can be identified as the key to combine with information about the costs into a single ratio as required by cost-effectiveness analysis and cost–utility analysis. In these cases, use of cost-effectiveness or cost–utility analysis would potentially obscure the value of a program, by focusing attention on a single ratio as the outcome, to the detriment of the wide range of other benefits the program is providing. A cost–consequence analysis is an approach to economic evaluation that enables researchers to outline a range of the costs and consequences of an activity in comparison to usual care or the status quo situation, in a table or balance sheet for the reader to consider depending on their own context or values^14,17. In a cost–consequence analysis, the benefits can relate to different stakeholders using a range of outcome measures¹⁴. Researchers should include as many of the costs and consequences as feasible, including qualitative and quantitative measures¹². The flexibility and practicality of the cost–consequence analysis are some of the reasons some researchers have advocated for its increased use when analysing the value for money of multicomponent, health promotion activities, or interventions that have diverse impacts across multiple agents (eg an intervention that provides support for families of very sick children)^18,19. When undertaking a cost–consequence analysis, it is important to describe the context in which the analysis was done so that the reader can decide whether the research conditions were relevant or transferable to their setting or situation¹⁴, and they can also consider the costs and consequences that are most relevant to them¹².

In 2019, Rural Health Workforce Strategy funding, provided by the government of South Australia, supported a trial of the Allied Health Rural Generalist Pathway (AHRGP) in SA Health regional local health networks (LHNs). The program introduction was coordinated centrally by an AHRGP program manager from the Rural Support Service (RSS), SA Health. The AHRGP was originally developed through a collaboration between the Allied Health Professions Office of Queensland, Services for Australian Rural and Remote Allied Health, Australian state and territory healthcare sectors, and other stakeholders, including universities and the Australian Healthcare and Hospitals Association. The education component of the AHRGP is provided by James Cook University, with two training levels available, one for newly qualified and a second for more experienced allied health professionals. Rural generalist trainees undertake course work and work-based projects throughout the program. They have quarantined study time within their workload as well as dedicated profession-specific supervision.

The AHRGP has the potential to provide opportunities for allied health training, support and career advancement pathways^20,21. There has been limited research undertaken to investigate the experience and outcomes of the pathway for allied health professionals, organisations and consumers^22-24. Previous research has identified a range of outcomes, including improved recruitment and retention of trainees. In Queensland, all of the rural generalist trainees completed the AHRGP, and 6 months after completing the pathway 78% continued to work in rural or remote areas²⁵. In New South Wales, physiotherapy rural generalist trainees were recruited into chronically vacant positions, which allowed health services to offer a physiotherapy service²⁴. A range of service development outcomes has also been achieved since the introduction of the AHRGP in these states, including the development of new and innovative service types, increases in allied health service access and occasions of service, the development of new allied health services, and a reduction in consumer need to travel for services^24,26.

Retaining existing clinicians in rural and remote areas saves employers considerable costs relating to attracting and recruiting new staff. Although researchers have reported the benefits of retention⁴, it is challenging to measure retention in terms of costs and benefits. In 2011, Chisholm et al measured the cost, in Australian dollars, of allied health professional turnover in regional, rural and remote areas². They included vacancy costs (locums and overtime of other staff working during the vacancy), recruitment costs (advertising, attracting applicants, interviewing and relocation costs) and costs relating to orientating and training new allied health professionals once recruited. While Chisholm and colleagues have outlined turnover patterns and costs and the AHRGP has demonstrated early positive workforce and service outcomes^24,26, there have been no economic evaluations of postgraduate professional development programs or workforce retention strategies for rural and remote allied health workers. Understanding the economic impacts of workforce development and postgraduate training initiatives will enable health services to make informed choices about investing in workforce strategies and increase the sustainability and perceived value of these investments. This study aimed to undertake the first evaluation of the resources and outcomes of the AHRGP by using a cost–consequence approach.

Methods

Study design

A multi-phase, mixed-methods research project was conducted between 2019 and 2022, investigating the experiences and outcomes of the AHRGP. Allied health rural generalist trainees’ experience and perceptions were explored over four phases from the beginning of the training pathway to 6 months after completion. All trainees who enrolled in the pathway were included in the study. The economic evaluation is one aspect of the comprehensive research^27-29, For this research, the resources and outcomes to be analysed were discussed with the RSS program management team to ensure the elements were comprehensive and meaningful to the relevant stakeholders. As regional LHNs were the key stakeholders, the analysis needed to be presented in a way that would assist managers to make an informed decision about whether the AHRGP was value for money and an effective workforce initiative.

Setting and participants

All six regional LHNs across rural and remote South Australia, and allied health professionals from five disciplines (occupational therapy, physiotherapy, podiatry, speech pathology and social work), participated in the AHRGP. At the beginning of the pathway, the 10 trainees participating in the level 1 program were working in base-grade allied health positions (AHP1) while the five level 2 participants were employed in mid-career level positions (AHP2)²⁹. Workforce data relating to regional LHN allied health professionals working at the same classification as the trainees at commencement of study, who were not participating in the pathway, were used a comparator. Allied health workforce data relating to the trainees and all allied health professionals employed at AHP1 and AHP2 in regional LHNs were provided by the RSS for analysis.

Intervention

AHRGP trainees participated in the rural generalist program, which involved completing an online work-integrated training program through James Cook University (level 1: 12-module rural generalist program over 1–2 years part time; level 2: eight-subject graduate diploma in rural generalist practice recommended over 2–3 years part time). Trainees were provided with 0.1–0.2 full-time equivalent study time at work, dedicated clinical supervision to support their engagement in the pathway, and support from the RSS AHRGP program manager. Allied health professionals who were not engaged in the pathway received standard allocations of training and supervision^30,31.

Outcomes

Outcomes were measured at phase 1 (enrolment in 2019), phase 2 (midpoint in 2020, approximately 12 months after starting), phase 3 (end point (2020–2022)) and phase 4 (6 months after completing). The end point varied with each trainee and the average time between enrolment and completion was 27 months with a range of 17–43 months. Trainees who discontinued were followed up via interview at the time of their withdrawal.

Retention outcomes were measured through length of tenure and turnover data, comparing the AHRGP trainees to the rest of the regional LHN allied health professional population. The RSS provided comparative allied health professional workforce data across all the regional LHNs in South Australia for the same evaluation period as the AHRGP trainees. This included the number of allied health professionals employed at each classification level, the number of separations at each classification level, and the average length of time allied health professionals were employed by the regional LHN during the evaluation period.

Intended tenure outcomes, career progression, service development project time, confidence and competence were collected via survey and interview at enrolment, and at midpoint and end point of the pathway. Trainees were asked to record how long they intended to continue working in a regional LHN on a 7-point scale ranging from less than 1 year to more than 10 years. They were also asked to rate their confidence working as a rural generalist on a 10-point scale with 0 being not at all confident and 100 being extremely confident. Supervisors and managers were asked to rate the trainees’ competence in providing a large variety of services and confidence as a rural generalist on the same rating scale. Job satisfaction was also rated on a 10-point scale with 0 being extremely dissatisfied and 100 being extremely satisfied. Service development time was recorded as a self-reported average number of hours participating in service development activities per month, and career progression was reported anecdotally during the interviews at each research phase.

Resource use and valuation

The analysis was undertaken from the perspective of regional LHNs as the key stakeholder. The resources considered included tuition, program manager salary, quarantined study time, and time required for supervision and support of the trainees. Tuition was the actual charges from the education provider, which were calculated on the basis of total cost to complete the rural generalist program. For trainees who discontinued partway through, tuition fees were based on the number of modules they enrolled in.

Study time and time spent receiving supervision were collected via trainee survey in phases 2 and 3. Supervisors, managers and profession leads also recorded the average number of hours they were spending supervising or supporting a trainee, as well as any associated meetings and administrative tasks that they reported at interview at the research midpoint and end point.

The SA Health Allied Health Clinical Supervision Framework³⁰ was used as the minimum standard for allied health professional clinical supervision. Only supervision costs additional to these standard levels were considered costs associated with the AHRGP. There is no minimum standard for support provided by managers or profession leads and so each individual reported subjectively whether they were providing additional support to trainees above what they would usually provide to other early- and mid-career allied health professionals.

Unit costs for the recruitment of a new allied health professional were drawn from Chisholm’s rural and remote allied health turnover cost calculations² updated to 2020 using consumer price index data from the Australian Bureau of Statistics^31,32. For the purposes of the calculations, the ‘all health services’ category was used because the trainees were working across regional, rural and remote locations. The unit cost of recruitment for one allied health professional was $26,721 in 2011, updated to $32,867 for 2020.

Salaries of trainees, supervisors, managers, profession leads and the AHRGP program manager were based on employment classification reported by each individual during research interviews and the corresponding South Australian Public Sector Enterprise Agreement^31,32.

Statistical analyses

Descriptive statistics:  Descriptive statistics are presented for continuous outcomes and resources used, including mean, standard deviation and range. Tuition resources and program manager salary are presented as a total spend over the follow-up period for all trainees as well as an average cost per trainee, along with study and supervision time.

Intended lengths of tenure are presented as mean number of years as well as a range. Proportions were used to describe the number of trainees who completed the pathway as well as the number continuing to work in regional LHNs beyond the follow-up period. Percentage changes in confidence and competence are reported to demonstrate changes from across the research phases. Total number of hours spent undertaking service development projects are presented for trainees who participated in the midpoint and end-point phases.

Calculations:  Tuition fees per trainee were calculated by utilising the costs per program and the cost per module provided by the education provider. Tuition fees for trainees who did not complete the AHRGP were based on the number of modules each trainee completed multiplied by the cost per module provided by the RSS.

AHRGP program manager cost per trainee was calculated by dividing the total cost of the manager salary by the total number of trainees who participated in the pathway.

Quarantined study time was calculated by multiplying the reported hours of study time per week with the total number of weeks each trainee participated in the pathway. The total number of hours reported was then multiplied by the trainee’s hourly wage to give a total cost per trainee.

Time for supervision and support of the trainees was calculated similarly. Reported hours per week were multiplied by the total weeks each trainee participated in the pathway, which was then multiplied by their hourly wage to give a total cost per trainee. Likewise, the cost attributed to the additional time supervisors, managers or professional leads spent supporting the trainees was calculated by multiplying the reported hours per month with their hourly wage.

Calculating return on upfront investment on recruitment of allied health professional:  To calculate monetary benefits due to increased length of tenure for the trainees working within regional LHNs, the average length of tenure of the trainees was compared to the average length of tenure for the broader allied health professional population at the same salary classification as the trainees at commencement of study, over the same time period (drawn from workforce data provided by the RSS). The cost for recruiting an allied health professional was divided by the average length of tenure for each group to give the return expected on the recruitment investment per year for each group. The difference in the return on investment was then multiplied by the number of years of the program and the number of trainees to calculate the benefits over the life of the program.

Calculating saving on costs of recruitment for allied health professional:  Additionally, the rate of turnover per year within the AHRGP was compared to the turnover of the broader allied health professional population at the same salary classification as the trainees at commencement of study, over the same period (drawn from workforce data provided by the RSS). The difference in turnover rate was then multiplied by the number of level 2 trainees to estimate the expected additional number who would have resigned and needed to be replaced if this group had turned over at the same rate as the broader regional LHN allied health professional population. This additional number who would have been expected to resign was than multiplied by the turnover cost drawn from Chisholm et al² updated to 2020 prices to give the estimated savings in recruitment costs.

Intended tenure benefits were calculated by adding the completing trainee’s actual length of tenure to their phase 3 future intended length of tenure. This was compared to the average length of tenure of AHP1 and AHP2 allied health professionals working in regional LHNs over the same period.

Assumptions were made for discontinuing trainees. If a trainee reported, at the point of leaving, that they were moving to work in a different setting, then it was assumed they remained there for the follow-up period.

Ethics approval

Ethics approval was granted by the Southern Adelaide Clinical Human Research Ethics Committee (21 August 2019, HREC/19/SAC/170.) All methods were performed in accordance with the relevant ethical guidelines and regulations.

Results

During the follow-up period, 55 stakeholders participated in the research, including 15 trainees, nine clinical supervisors, 13 managers, seven profession leads, four RSS program managers and seven consumer representatives. Of the 15 trainees, three completed the level 1 pathway, four completed the level 2 program, and one was continuing the level 2 program beyond the follow-up period because of deferral.

Resource use and costs

AHRGP program manager salary and tuition fees are outlined in Table 1. In total, the program manager salary over 3 years from 2019 to 2022, including on-costs (superannuation and leave), was $143,498, and the total cost paid to the education provider for tuition to June 2022 was $162,777.

On average, level 1 trainees reported twice as much quarantined study time at work per month in phase 2 (21.2 hours) than in phase 3 (10 hours), while the level 2 trainees reported study time was relatively consistent across phase 2 (20.8 hours) and phase 3 (19.2 hours). No additional hours of supervision or management were reported by trainees, managers, supervisors or profession leads above the range recommended by the supervision framework²⁶ or normal expectations for managers and profession leads.

Resource analysis

The total months each trainee spent undertaking the pathway ranged from 3 to 42 months, depending on the level of program enrolled in and proportion of program completed. This range accounts for the resulting significant differences in costs. During the 3-year follow-up period, the average cost of supporting one trainee position regardless of whether they completed was $37,600 while the cost of a completing level 1 trainee was $34,875 and a completing level 2 was $70,469.

Outcomes

Outcomes for the participants and the regional LHNs are presented (Fig1). Seven of the 15 participants completed the pathway and were surveyed at phase 3. All seven continued to work in a regional LHN and had progressed to a higher position with higher duties and salary. One additional level 1 trainee who had discontinued the AHRGP was continuing to work in a regional LHN at the follow-up date (see Fig1 for proportion details). The remaining six discontinuing trainees ceased employment with the regional LHNs. In total, level 1 trainees reported participating in 158 hours of service development time while the level 2 trainees reported 2300 total hours. Trainee confidence and competence increased throughout the pathway.

Cost–consequence analysis

The findings of the cost–consequence analysis are in Table 1. On average, the pathway cost $37,600 per trainee who enrolled in the program regardless of how long they participated. This is inclusive of tuition costs, study time and program manager time. When calculated only for trainees who completed or who were planning to complete, the average level 1 program cost ($34,875) was approximately half the average level 2 program ($70,469) per participant, attributable to significantly higher tuition costs and more time to complete studies. Considering all 15 trainees involved, the total pathway cost was $563,985 over the 3-year research period. The level 2 program includes a graduate certificate qualification requiring a larger volume of study than the level 1 program, which does not include a postgraduate qualification and is designed for newly qualified allied health professionals. Level 1 participants in this study took on average 19.9 months to complete the program while level 2 participants took on average 32.5 months.

As of the follow-up date, overall, the level 1 trainees (AHP1s) participating in the AHRGP had an average length of tenure 82% greater than the general AHP1 population working in the six regional LHNs (2.55 v 1.4 years). Of the 10 level 1 trainees who started, 40% continued working within a regional LHN beyond the follow-up period compared to 35% of all AHP1s across the six regional LHNs.

The level 2 (AHP2) trainees stayed for an average of 6.5 years at the follow-up date. In the three years from 2019 to 2022, 157 AHP2s across the six regional LHNs resigned, which equated to 17.6% of the total AHP2 workforce per year. In this cohort of five level 2 trainees, there were no resignations. As none of the level 2 participants resigned, there was an annual total saving of $28,924 during the follow-up period, estimated from a reduction in costs to recruit. As an alternative calculation, when considering the cost to recruit an AHP2 as an upfront cost, the total rate of return per year in the general regional LHN AHP2 population is $32,867/8.4 years or $3912 per year. Therefore, the saving per AHRGP level 2 position during the 3 years is $11,736. For either calculation, a saving is evident for the level 2 participants.

At the 6-month follow-up point, on average the level 1 trainees intended to work within a regional LHN for an additional 2.3 years (range 1.5–4 years) and the level 2 trainees planned to work within a regional LHN an additional 7.25 years (range 4.5–10+ years). If the graduated trainees remain working as long as projected, the level 1 trainees who have completed the pathway will have a tenure 385% longer than the general AHP1 professional population, and the level 2 trainees will have a tenure 47% greater than the general AHP2 professional population.

Reduced turnover of staff was a significant outcome of the program. Considering the estimated return on investment and the number of allied health professional positions involved, the increased return was $317,610 for the level 1 trainees and $58,680 for the five level 2 trainees over the 3-year follow-up period. Overall the program produced a total return of $376,290 over the 3-year follow-up period.

Discussion

The aim of this study was to understand the costs and consequences of the AHRGP in regional LHNs in South Australia to identify if the AHRGP is an effective allied health workforce strategy. Recruitment, retention, career development and service development are key challenges in rural and remote areas and the significant investment in the AHRGP is unlikely to be sustainable in the long term without clear, measurable and positive outcomes.

A cost–consequence analysis of the AHRGP was conducted to outline the resources and outcomes of the pathway for South Australian regional LHNs. This approach to analysis has not previously been published in allied health professional training and workforce research although other industries have used similar approaches to measure the impact of workforce initiatives^15,16. While the tuition fees for AHRGP training are available from the education provider for organisations or individuals who are considering the AHRGP, associated costs were previously unknown and the economic outcomes had not been described in research findings. This research outlined the resources relating to tuition, quarantined study time and program manager time. Time devoted to supervision and management of trainees was found to be within standard expectations and not an additional cost to be factored in. This was an unexpected finding requiring further exploration and contrasts with similar research conducted in other jurisdictions^24,26.

Outcomes of the AHRGP were found to be extensive, including reduced allied health professional turnover costs, increased tenure intentions, career advancement as measured by promotion, and increased confidence and competence of trainees. A study conducted by Chisholm and colleagues² outlined the costs of allied health professional turnover in rural and remote areas. This study was used as a basis for calculating turnover costs of trainees completing AHRGP compared to the general allied health professional population employed in regional LHNs at the same employment classifications and at the same time. This research found that AHRGP trainees had a lower turnover rate than the rest of the allied health professional population, which was then calculated as a benefit in dollar terms. While there was a turnover of level 1 trainees in this study, on average their length of tenure was longer than that of the greater AHP1 population regardless of completion, indicating a positive result. It is pleasing to note that all five level 2 participants continue to be employed beyond the follow-up date. Whether there are any benefits for length of tenure for the level 2 participants is not yet clear, because of the average tenure for AHP2s in the six regional LHNs being longer (8.4 years) than the follow-up period for this study. Any impact on tenure will become clearer in the next few years. It should be noted that while the differences in retention of allied health professionals are promising, we are unable to assign direct causality for this to the AHRGP. Reasons for this caution include the fact that our participants self-selected to participate in the program, our sample is relatively small with a short follow-up period, and confounding factors that could have also influenced retention were not controlled for. However, the findings are encouraging, and demonstrate the importance of conducting larger-scale evaluations of the potential economic benefits of similar programs in the future. A range of additional outcomes that could not be calculated economically were outlined. Participants were afforded opportunities for career advancement during the pathway. This provides a range of benefits for participants as well as regional LHNs, through improved organisational leadership and contributions. Research suggests that allied health professionals with career advancement opportunities in rural and remote areas are more likely to have greater intentions to remain working rurally than those who have limited career options⁶. Additionally, trainees became more confident and competent working as rural generalists, which has positive impacts for the rural and remote communities in which they work. The time trainees spent participating in service development projects also benefited the teams and regions in which they worked. Without the AHRGP, these projects may not have been completed. Trainees planned to continue working in rural and remote areas longer in phase 3 than they had in phase 1, which indicates the retention outcomes are likely to increase over time.

These findings provide valuable insights into benefits that were realised for trainees as well as the employers. Our sample size was relatively small (n=15 trainees) but did include the whole cohort undertaking the program during the follow-up period. It allowed us to collect detailed information on each individual participant and their path through the program for this evaluation. However, we recognise that our small sample size is a limitation with regard to the generalisability of our findings. We recommend that more research evaluating these programs with larger samples of participants be undertaken to provide more certainty regarding the costs and benefits. Readers of this research can consider the relevant or important factors for their circumstances when weighing up the benefits and costs of the pathway. Turnover savings were also calculated in a study investigating the economic value of introducing a formalised mentoring program for early career teachers; the study identified a range of costs and benefits, including the cost benefit of reduced turnover of teachers who participated in the program¹⁶. There is otherwise a scarcity of research investigating the costs and benefits of workforce training and this new research offers a novel approach for future research to consider.

Conclusion

In this first economic evaluation of the AHRGP, a range of outcomes has been outlined demonstrating significant value and benefits for the regional LHNs. While overall the initiative cost $563,985, the program demonstrated reductions in recruitment costs of $376,290 in the first 3 years of implementation. Given most trainees were intending to continue working with regional LHNs, it is expected that these recruitment savings will continue to grow, with the expected tenure of the level 1 trainees who have completed the pathway calculated as 385% longer than the general AHP1 population working across the six regional LHNs and the level 2 trainees estimated to have a length of tenure 47% greater than the general AHP2 population.

In addition, the AHRGP had a high completion rate for the level 2 program. Participants were promoted during or immediately after the pathway and their confidence and competence improved. Participants developed and implemented innovative service development activities that provided benefits for the organisation and consumers. In addition to these multiple outcomes, AHRGP has been demonstrated to provide an excellent return on investment, with benefits expected to grow as graduate trainees are further retained in the workforce.

Funding

The authors wish to acknowledge funding support provided by the Rural Health Workforce Strategy, SA Health, Government of South Australia to undertake this research.

Conflicts of interest

The authors declare no conflicts of interest.