Rationale and protocol for the Nursing and Allied Health Graduate Outcomes Tracking (NAHGOT) study: a large-scale longitudinal investigation of graduate practice destinations

Introduction

In many countries, the health workforce is inequitably distributed and disproportionately concentrated in metropolitan settings¹. Although health workforce distribution varies between professions, fewer types and smaller numbers of health professionals limit access to health care in underserved rural areas and contribute to adverse health outcomes². Health workforce supply and distribution is a complex interplay of multiple factors related to ‘producing, attracting, sustaining and retaining’ healthcare practitioners (p. 8)³. The WHO posits that change in health employment, education and service delivery is enabled by adequate funding, intersectoral collaboration, facilitated migration of health practitioners, and robust health workforce and labour market research⁴. The WHO also suggests that interventions designed to address health workforce availability should focus on three modifiable phases of the working lifespan: producing and preparing health professionals to enter the workforce; improving and enhancing the performance of active health workers; and managing issues that influence workforce attrition³. Further, the WHO advocates establishing institutional information systems for health workforce research that allow tracking of health professionals from the point of entry into prevocational training and over the course of their careers³.

Gaebel et al define graduate outcomes tracking as a process that involves maintaining information on ‘students and/or graduates, with regard to their learning progress, skills acquired, perceptions, jobs, between at least two points in time, through aggregated or individual-level data, collected mainly via administrative processes and surveys’ (p. 9)⁵. The TRACKIT study found that European universities generally used methods that combined student administrative data routinely collected by universities with graduate information, supplemented by quantitative and qualitative survey data⁵. National-level approaches to tracking outcomes of nursing and midwifery courses exist^6,7; however, most are limited to short-term outcomes (less than 3 years) from single educational institutions and report data collected at isolated points in time. There is a paucity of studies tracking the longitudinal outcomes of allied health students and graduates.

To date in Australia, the main focus of health student tracking research has been medical students and particularly on understanding why graduates decide to practise in rural and remote locations, with most studies reporting outcomes for single institutions^8-10. That research has identified three key indicators of graduate rural and remote health professional practice: rural background, exposure to rural settings during training, and rural practice intention at commencement of studies^11-18. Rural background or origin is the most influential factor^11,13,16,17, although the mechanisms underlying this association remain unclear; one study found no social, environmental or economic factors to explain the rural background effect¹¹. Rural practice is also associated with the length of time students spend on rural-based placements, with a possible exposure–response effect evident in several studies^12,13,16,18. Having a rural background increases the effects of rural experience, while longer duration of immersion is associated with an increased uptake of graduate rural practice^16,18. Medical students’ rural practice intention has also been shown to be a strong predictor of rural practice, one large-scale study¹⁴ finding that preference for rural practice at commencement of medical studies was a consistent predictor of being in rural practice in the first and third postgraduate years, independent of rural background. Another study found that while rural clinical placements significantly enhanced students’ preference for rural practice, the effect was only significant if coupled with their original rural practice intentions¹⁵.

There is less evidence about the factors that influence rural practice intentions in nursing and the allied health professions in Australia, as well as internationally. Some studies suggest similar indicators to those found in medicine, including the effect of rural background and an association between length of placement and graduate practice location^19,20. Also, early findings from a longitudinal study of allied health students²¹ suggested that, compared with students of metropolitan background, those of rural origin were 2.35 times more likely to be working in a rural or remote area after graduation. Another study of allied health and nursing graduates found that after correcting for the effects of rural background, rural employment was significantly associated with the quality rather than duration of placements²⁰. Meanwhile, an investigation of change in pharmacy students’ rural practice intentions between the first and final year found that rural background, rural degree and rural placement all had a positive influence on students’ rural practice intentions²². Some previous studies have suggested that early career employment in a rural setting, particularly a graduate’s first job, may be a predictor of future rural health practice^23,24.

Informed by WHO guidelines, the Nursing and Allied Graduate Outcomes Tracking (NAHGOT) study has been developed and implemented to address the apparent evidence gap related to why allied health and nursing graduates choose to work in particular locations. This study seeks to address perceived shortcomings in previous health workforce research by linking existing student and graduate data, targeting entire pre-registration student cohorts in multiple disciplines, involving multiple universities across multiple jurisdictions, and creating a repository of data over an extended period of time.

Methods 

The NAHGOT study is a cross-jurisdictional, multi-university, multidisciplinary, longitudinal, prospective cohort study^25,26. The outcome variable of interest is where graduates are practising, assessed annually according to the Australian Health Practitioner Regulation Agency (Ahpra) principal place of practice data. The exposure variables of particular interest are whether participants were of rural background or had rural professional placements, or both, although the effect of other variables will also be considered. Figure 1 shows a conceptual model of health professional student and graduate experience pipeline or pathway^27,28 that informed the study design. The model illustrates the many different variables to be considered, including core demographic characteristics, student and graduate learning experiences, evolving life and career opportunities, and changing values, aspirations and practice intentions.

Study aim and objectives

The aim of this study is to investigate the factors that influence Australian nursing and allied health graduates’ practice locations over an extended period of their careers compared with their location of origin, locations of their professional placements during their studies, and their practice location intentions before and after graduation. Specific objectives are to:

• aggregate enrolment and professional placement data routinely collected at multiple universities over the duration of the students’ degrees
• compile survey data from students and graduates about their intended and actual practice locations before and after graduation, respectively, as well as about factors that influenced their decision-making
• source data annually from the professional registration authorities about the graduates’ principal place of practice
• link data from the various sources to track entire cohorts of students during their studies and as registered practitioners each year for 10 years after they graduate
• analyse the data as to the relationships between demographic variables at entry to university, location of placement experiences during their studies, changes in their postgraduate practice location intentions, and their actual graduate practice destinations.

A key element of the study aim is to collect data from entire cohorts of students, regardless of demographic background, practice intentions or location of their professional placement experiences during their studies.

Context

The NAHGOT study involves three Australian universities – the University of Newcastle, Monash University and Deakin University – all of which receive funding under the Australian Government’s Rural Health Multidisciplinary Training (RHMT) program to maintain separate University Departments of Rural Health (UDRHs). The primary purpose of the RHMT program is to ‘deliver effective rural training experiences for medical, dental, nursing and allied health students’, with a view to building future rural health workforce capacity²⁹. Under the funding agreements, each university is required to meet targets for enrolment into health professional degrees of students of rural origin, the number of students who complete UDRH-based rural placements, and the average duration of rural placements. In addition, each university receiving RHMT funding is required to ‘collect and maintain data on rural workforce outcomes resulting from rural training activity’²⁹. These requirements motivated the three universities involved in this study to form a collaboration, to aggregate their data and thus increase the sample size and generalisability of the findings.

Participants

Participants are those students and graduates from the three participating universities who are studying or have completed nursing and allied health degrees that lead to registration with Ahpra. Students of allied health professions that are not registered with Ahpra will not be included, as reliable tracking of postgraduate practice location is limited. Table 1 lists the 15 Ahpra National Health Practitioner Regulation Boards³⁰ and shows the in-scope disciplines from each university. International students and graduates will also be excluded.

Each year, a new cohort of commencing students is included, with every cohort tracked throughout their studies and for 10 years after completion. Although it varies from year to year, at each of the three universities, a total of approximately 2000 students are enrolled annually into the health professional degrees listed in Table 1. It is not possible to accurately predict the ultimate sample size over the duration of the study but, even with loss to follow-up, it will include thousands of individual participants.

Data collection

As well as the conceptual model of the student and graduate pipeline, Figure 1 summarises the recruitment and data collection process at each stage, from university admission, undergraduate attendance at university and after graduation. Items highlighted in bold italic font are variables about which data are to be collected for the NAHGOT study.

University admissions data:  Admissions data are accessed via the student administration units at each university under the terms and conditions of enrolment agreed by students on admission. Universities routinely collect relevant data, including age, gender, Aboriginal and Torres Strait Islander status, home address, high school attended or other entry pathway, highest previous educational attainment, and that of parents, and whether they are the first in their family to attend university.

Locational data for home address and high school attended are each classified according to the Australian Statistical Geographical Classification – Remoteness Area (ASGS-RA)³¹ and Modified Monash Model (MMM)³² schemes. In addition, home address and high school location are categorised for socioeconomic status according to the Socio-Economic Index for Areas and Index of Relative Social Disadvantage (IRSD)³³.

Professional placement data:  Data will be collected annually about students’ professional placements, after all placements have been completed for the year. The professional placement administrative units of each faculty at each university are the source of data for location, duration, stage or year of their degree, and type of agency in which placements are undertaken by each student over the entire course of their studies. Based on the dates provided, placement duration is converted to number of placement days, assuming a 5-day working week, unless otherwise informed. The geographical location of student placements is categorised according to both ASGS-RA and MMM classifications, and placement type or facility is categorised as public, private, non-government and other relevant categories, depending on the data provided by each university.

Survey data:  University administrative data do not include information about students’ intended practice location after they graduate or factors that influence their decision-making. Therefore, that information is collected by surveying both students and graduates by using questions that have been informed by the literature^34-36. Relevant NAHGOT study questions have been incorporated in the Student Experience Survey (SES)³⁷, which is administered annually to students of all Australian universities by Quality Indicators for Learning and Teaching (QILT), an external organisation, on behalf of the Australian Government Department of Education, Skills and Employment. Figure 2 details the questions added to the SES distributed to students enrolled in the targeted degrees at the universities participating in the NAHGOT study. Final year students are asked if they agree to longitudinal tracking after graduation and, if they agree, to provide two sets of contact details, one for themselves and, as a back-up, one for one other person with whom they are likely to remain in contact.

Graduates are surveyed by two means, using the survey questions shown in Figure 3. Shaded NAHGOT study questions in Figure 3 have been added to the Graduate Outcomes Survey (GOS)³⁸ for graduates in the target professions from the participating universities. As with the SES, the GOS is a national survey administered via QILT to Australian university graduates in their first and third years after graduation. Consenting participants may also volunteer to receive an online survey that includes all the questions shown in Figure 3, administered by the NAHGOT team in the second and fourth-to-tenth years after graduating.

Registration data:  The main source of data for longitudinal tracking of the practice location of entire graduate cohorts is Ahpra. Each of the universities involved in the NAHGOT study has entered into a confidentiality agreement with Ahpra. Under that agreement, the university provides a list of student identification numbers of new graduates in the relevant professions and Ahpra provides a matched list of the graduates’ registration numbers. Having the registration numbers enables each university to download the ‘principal place of practice’ of graduates, the key outcome variable in this study, which is coded according to both ASGS-RA and MMM classifications. Although their registration number remains the same, all practitioners in the professions listed in Table 1 must renew their registration annually to be eligible to practise and, in doing so, update their principal place of practice. Thus, by downloading data annually from Ahpra, it is possible to track graduates’ practice locations over extended periods.

Data linkage:  Student identification number is anonymised and converted to a unique NAHGOT identification number, which, together with the name of the degree in which the student is enrolled, enables university administrative and placement data, student survey data and SES data to be linked. Graduate data are linked to student data via the traceable connection between a participant’s NAHGOT identification number and their Ahpra registration number.

Ethics approval

Ethics clearance for all components of the NAHGOT study was obtained from Monash University (ref: 7962), the University of Newcastle (ref: H-2017-0332) and Deakin University (ref: 2019-411) Human Research Ethics Committees. Access to internal university enrolment and administrative data, as well as SES and GOS data, are subject to the terms and conditions of enrolment (available on request) and in accordance with the requirements of the RHMT Program. The Ahpra data are made available for research purposes strictly according to the confidentiality agreements with each university. For the university graduate online surveys, an explanatory statement is provided on the opening page, assuring potential participants of confidentiality, with consent implied by commencing the survey. Each university maintains their own datasets containing identifiers. Only de-identified data are aggregated into a common database, managed by a central data manager who has no direct involvement with students, graduates or the research itself.

Results

All data are cleaned and coded at each university separately before being entered into a common database for storage and analysis. The NAHGOT study database will enable a variety of investigations and study designs covering different time periods, disciplines, institutions and geographical regions. Database access is currently restricted to formal collaborative partners, with the longer-term intent to permit external data access based on predefined study protocols, similar to the process for Medicine in Australia: Balancing Employment and Life (MABEL), Australia's national longitudinal survey of doctors³⁹. The first peer-reviewed publications from NAHGOT were cross-sectional studies^40,41 and reflected the relatively short period of data collection at the time. It is expected that future NAHGOT study outputs will capitalise on the ongoing annual data collection and permit robust longitudinal analyses.

Study governance

A NAHGOT Research Management Committee, including representatives of all participating universities, was established at the outset to oversee the study. A Data Management Sub-Committee exists to coordinate data aggregation and analysis, guided by the NAHGOT Research Protocol Manual and Data Dictionary. The three-way partnership is the subject of a collaborative agreement developed by legal counsel and signed by senior personnel of each university. While recognising the rights of each partner as an independent entity, that agreement acknowledges the common goals and collaborative processes, including a requirement of ethics committee clearance addressing issues of privacy and confidentiality in data sharing, as well as authorship and intellectual property. Each university remains custodian of their own data but agrees to share deidentified data with partner universities for the purpose of the NAHGOT research study. As above, each university has separate arrangements with Ahpra and QILT for data access.

Discussion

This study has a number of significant advantages over previous studies similarly aimed at investigating the relationship between health professional graduate practice destinations and predictors such as rural origin and rural placement experience.

First, although many studies have investigated the factors that influence graduates’ choice of rural medical practice, few comprehensive studies have specifically focused on nursing and allied health students and graduates. Evidence from medical studies supports the understanding that rural background is the strongest predictor of future rural practice location^42-44. Considerably less evidence exists about the factors that lead to rural practice in nursing and allied health, although rural origin and positive rural health undergraduate experiences have been identified as similarly influential^45-48. What appears to be lacking to inform nursing and allied health workforce planning is a robust, longitudinal, multifactorial and multicentre study, as described in this protocol article.

Second, this study will be multisite, large scale and comprehensive, as well as longitudinal. The research protocol described in this article targets eight different health professions across three universities, as listed in Table 1, with combined annual first-year enrolments of the order of 6000 students. The study tracks participants from entry into university, throughout their pre-registration education, and well into their postgraduate careers. The scale and complexity of the study demands high levels of rigour and diligence, necessitating development of a cogent research protocol and data dictionary. The results are expected to provide insights into evolving values, beliefs and attitudes as participants transition from pre-registration students to mature practitioners. There is potential for the study to inform future university selection and admission processes, as well as curriculum design and identification of those nursing and allied health students who will benefit most from rural placement experiences. Consequently, in addition to disseminating the findings across the broader academic and health professional networks, internal university reports will be produced to inform quality improvement initiatives.

Third, a noteworthy advantage of the NAHGOT study is that it makes use of multiple existing sources of data integrated with primary, cross-sectional survey data. Using linked datasets permits detailed analyses based on a wide range of variables drawn from university enrolment and professional placement data and, ultimately, Ahpra data. In addition, linked data from the SES, GOS and online NAHGOT graduate survey enable exploration of influences in the decision-making process about choice of practice location. There appears to be no evidence in the literature that all these sources have been linked in previous studies; however, other studies have used Ahpra data to investigate recruitment and retention, although they have focused almost exclusively on medicine and single rural clinical schools^15,49. A comparison between the use of Ahpra data and personal contact found that Ahprdata were not sensitive enough to detect locational changes due to prevocational and vocational training over the 10 years after graduation⁵⁰. This is not likely to be the case with nursing and allied health practitioners, who do not undergo intensive specialist postgraduate training regimes. In one recent nursing and allied health graduate tracking study, only 31% of the participants were identified from Ahpra data 15–17 years after they were initially surveyed, although that cohort study included several disciplines that are not nationally registered²⁴ and targeted only students who had a placement supported by one particular UDRH.

Fourth, and perhaps most importantly, the NAHGOT study is a prospective cohort study. This means that it will be possible to draw comparisons and test null hypotheses, principally that there is no significant difference in the principal outcome variable of graduate practice location between exposed and non-exposed study participants. This is an important step forward in being able to draw conclusions about whether rural exposure, either before admission or as part of students’ educational experiences, makes a difference to where they take up their initial, as well as ongoing, professional practice. The study design goes beyond the contractual student tracking requirements of the Australian Government’s RHMT program²⁹, which could have been met by tracking only students that undertake rural placements in particular locations. The NAHGOT study has much more research potential because it includes the capacity to compare students undertaking metropolitan and rural placements anywhere in Australia.

Limitations

The main strengths of the study are discussed above but the study protocol is not without limitations. While the use of existing datasets may reduce the risk of self-reporting or recall bias, there is dependence on the accuracy of the data that are entered into the databases. This may result in some participants being excluded if, for example, details are entered incorrectly, or data have been omitted. It is unlikely that this will affect a large proportion of participants, and in such a large sample it is not expected to bias the results appreciably. The study will also not be able to track graduates who choose to not register with their profession; however, these data may inform the rate of loss to in-scope professions of graduates not entering the health workforce.

Another challenge inherent in longitudinal studies⁵¹ is that of loss to follow-up in recurrent surveys. The risk of survey fatigue is lessened, however, through the incorporation of NAHGOT questions into the SES and GOS, which are well-established national surveys. In 2019, the response rates for the participating universities for the SES were all slightly above 45%³⁷ and GOS response rates ranged from 45.8% to 51.9%³⁸. Loss to follow-up is less of a concern in relation to the Ahpra data because all practitioners in the target professions are required to update their principal place of practice annually. The data are directly accessed from Ahpra, not from the participants themselves.

A further limitation to generalisability of the findings is that certain allied health professions, such as dietetics, social work and speech pathology, are not nationally registered under Ahpra. They are, therefore, excluded from the study, even though they have a valuable healthcare role and the degrees are currently offered by more than one of the universities involved in the study.

Conclusion

This study is an example of a systematic multi-institutional approach to health workforce tracking and research that is advocated by the WHO to address the inequitable distribution of health professionals. The perceived long-term benefit is in ‘producing, attracting, sustaining and retaining’ health workforce³, with a particular focus on rural practitioners. In its current form, the NAHGOT study has the potential to build evidence that affects all phases involved in producing and preparing practitioners to enter the workforce, including prevocational admission and education processes, targeted interventions such as the RHMT program, and graduate pathways and ongoing support. In the future, however, the flexible and inclusive study design is intended to enable other universities, as well as those allied health professions not regulated by Ahpra, to join the study, thus increasing the generalisability. Furthermore, although the protocol only addresses the first phase of the working lifespan of a health professional, the project could be expanded to explore health workforce attrition and thereby influence health workforce planning overall.

Acknowledgements

The authors acknowledge administrative and academic staff from Monash University, the University of Newcastle and Deakin University for support and advice about data access. Monash University, the University of Newcastle and Deakin University receive funding through the Australian Government’s RHMT program, a requirement of which is to ‘maintain data on rural workforce outcomes resulting from rural training activity’. The funder had no role in establishing the NAHGOT study, the scope of which extends beyond the funding requirement.