The Pharmacy Community Apgar Questionnaire: a modified Delphi technique to develop a rural pharmacist recruitment and retention tool

Introduction

Healthcare services are necessary to the functioning, vibrancy, and both individual and public health of rural communities. Providing these services requires healthcare professionals such as medical practitioners, nurses and allied health professionals (eg occupational therapists, physiotherapists and pharmacists). Maintaining an adequate and stable healthcare workforce in rural communities remains essential for the continuity of care and improved health outcomes of these populations^1-5. Although fundamental, rural communities continue to struggle to have these essential professionals living and working in these areas despite increased training, funding and programs to support these goals^1,6-9.

Strategies implemented to address health workforce shortfalls in rural communities have often centred on addressing medical and nursing workforce deficits, thereby overlooking another critical workforce, pharmacists¹⁰. Pharmacists are often the first point of contact in rural areas and play a critical role in providing primary care as well as triage and referral of community members to other health professionals within the healthcare system^11-13. In some cases, a pharmacist may be the only health professional that is easily accessible in a rural or remote community¹⁴.

Healthcare professional workforce initiatives over the past two decades have resulted in an increase in the number of pharmacists in Australia^14,15, while maldistribution has resulted in a persistent undersupply across rural communities. Although increasing the pharmacist supply partially mitigates these longstanding challenges, key issues remain with poor attraction, recruitment and retention of pharmacists to rural areas^3,10,15,16. To date, there has been limited research focused specifically on the recruitment and retention of pharmacists in rural communities^3,14-16.

Literature reviews have investigated the drivers and challenges of rural pharmacist recruitment, retention and related decision-making^10,14,15. The findings indicate that pharmacists seek job satisfaction, good working conditions, a desire for responsibility, training and an adequate but not overbearing workload¹⁵. Additionally, beyond the workplace-identified geographical and family related themes that influence the choice to practice rurality, there are other key drivers of rural employment. These include economic drivers of rural employment such as wages and incentives, an expanded scope of practice afforded to pharmacists in rural areas, and the rural practice environment. Characteristics of the community itself and its support for pharmacists who join and become part of the town or area are also important factors^10,14,17-19. Although there are many pull factors that may bring pharmacists to contemplate, consider and undertake rural practice, there currently remains a need to define how these factors may be uniquely and practically utilised among individual rural communities to recruit and retain their much-needed workforce^10,18,19.

An approach to addressing these challenges among physicians and nurses has been use of the Community Apgar Questionnaire (CAQ) and the Nursing Community Apgar Questionnaire (NCAQ), respectively^7,20,21. These instruments, each consisting of 50 individual factors or key elements that influence practice location decision-making, have been used as objective measurement tools to assess the assets and challenges of individual rural communities to successfully recruit and retain doctors and nurses. Just as an Apgar test is designed to assess a newborn’s health and indications for action, the CAQ and NCAQ were developed to quantitatively scale the important resources and capabilities of rural communities that impact recruitment and retention^1,7,20. The CAQ and NCAQ assist rural communities to be more effectively equipped to recruit medical and nursing staff that are the ‘best fit’ with their unique service demands and community, while also identifying longer term strategies that services can undertake to improve recruitment and retention^7,20.

Research aim

The aim of this study was to develop the Pharmacy CAQ (PharmCAQ) and examine its reliability, while also understanding the similarities with the CAQ and NCAQ.

Methods

In this exploratory study, a modified Delphi technique was employed to formulate and develop the PharmCAQ, informed by the development process undertaken with the CAQ and NCAQ^7,20. The ‘Delphi’ is an established method employed to achieve clear consensus regarding healthcare policies and guidelines, can be applied qualitatively, and is regularly utilised to create solutions in the healthcare environment^22-25.

Modified Delphi technique

A Delphi remains fluid and flexible in its application due to the absence of a concrete theoretical framework. To mitigate this lack of a framework, an adaptation of the Conducting and Reporting of Delphi studies (CREDES) guideline was utilised to inform this study²⁶. Overall, three key elements are essential, with at least 8–12 participants who (1) are experts within the field, (2) come from a variety of backgrounds and (3) are heterogeneous in terms of their specialities, insights and understandings regarding the subject matter^24,25,27.

Along with these three participation elements, the Delphi process is achieved through four distinct steps²⁸. Initially an exploration of the subject matter occurs, where each expert contributes information that they consider important. In the second step, the group collectively works together to gain an understanding of the issue and discuss disagreements. It is these differences that need to be embraced, commended and explored in greater detail. This informs the third step, where insights and the development of creative solutions or alternatives occur. The final step in the process encompasses the gathering and analysis of the data, which is fed back to the group for further clarification and understanding. This process is then repeated several times until an artifact is created or solution is developed and agreed upon. The process undertaken is summarised in Figure 1. Although anonymity within the Delphi process is ideal to reduce influence or social bias, in some circumstances this may not always be possible given experts may already know each other, or it may be essential for a group of experts to seek consensus on key matters²⁵.

Participants

In this study, a panel of experts were purposively selected through known contacts of the various professional organisations both across the study area as well as nationally, and included eight stakeholders. These known contacts comprised representatives from professional organisations with comprehensive rural experience relevant to the study. They were in contact with many individuals and organisations across the country in their current and past roles, while some had or were also working clinically in rural contexts. In addition, they either represented or regularly met with rural individuals and organisations, which included areas classified as Modified Monash MM4 to MM7. These representatives were from the Society of Hospital Pharmacists of Australia (n=1), the Pharmaceutical Society of Australia (n=1), the Pharmacy Guild of Australia (n=1), the Pharmacy Board of Australia (n=1) and a government health agency and public hospital (n=1). Other participants included local and international academics with decades of health policy and rural health workforce expertise (n=3). No demographic data are reported here, to maintain the confidentiality of all participants.

Procedure for questionnaire development and refinement

Open discussion, as with previous CAQ and NCAQ development processes^1,20, was used rather than survey-type questioning. This was essential to understand and define the key factors vital to rural pharmacist recruitment and the nuances regarding how each factor was defined^24,25. As such, the Delphi technique facilitated the capture of vital information regarding those factors considered essential and as defined within the literature¹⁰. The group leader, an independent researcher (DS) and other researchers (DT and EB) met collectively with all participants for three separate focus groups of 45–60 minutes duration.

Initially, a draft set of factors and their description, based on a systematic literature review by Terry et al¹⁰, was introduced to the group prior to the first meeting. Throughout each group discussion, individual responses were written and summarised in real time and later analysed, with recommended changes made directly to the draft of the PharmCAQ. The revised questionnaire was then redistributed for the next meeting round. This exploratory process was repeated until consensus was achieved regarding the key factors that were to be included in the PharmCAQ and how each was defined. As a part of the process, each meeting session was recorded to facilitate ease of recall regarding key suggestions and points made by each participant.

Three rounds of discussions were anticipated to be achieved^22,23, with the process being completed over a 5-week period. Additional review and refinement of each of the factors occurred via email in the subsequent weeks to ensure accuracy of description as well as the ordering of the factors, and to support the flow of conversation when the PharmCAQ was being implemented in practice. Overall, the process enabled an in-depth discussion of each factor associated with the PharmCAQ, its definitional development, and fine-tuning as it was prepared to be piloted in the field.

In addition to the ordering to support conversation flow, each factor was assigned values using a four-point scale according to a community’s advantages or challenges (major advantage = 2, minor advantage = 1, minor challenge = –1, major challenge = –2) and then its level of importance based on a four-point scale (very important = 4, important = 3, unimportant = 2, very unimportant = 1). This was in line with the CAQ and NCAQ previously developed and allows each factor to be examined and provided a weighted Apgar score between +8 and –8 according to the following algorithm:

advantage or challenge score × importance score = Pharmacist Community Apgar Score^1,7,20

Reliability testing

An essential element of the PharmCAQ development process was its initial face and content validity, which was achieved through the participation of representatives with rural experience from professional organisations. In addition, internal consistency was achieved when the tool was initially piloted among 10 rural pharmacists in rural Victoria (MM4 to MM5), where numerical rankings for each factor within the tool were utilised to calculate a weighted Apgar score. Each rural pharmacist was recruited through known contacts, informed of the study, and invited to participate in the testing of the PharmCAQ. This also provided further face and content validation of the tool. The process highlighted the important unique features identified as advantages and challenges of a community that may be modifiable or require additional support to mitigate their potential for a negative impact.

Four of the ten interviews were conducted face-to-face, while the remaining occurred by video-conferencing technology. Each interview was 45–90 minutes in duration and was audio- or video-recorded. After interviews were conducted, data were cleaned, checked and analysed using Microsoft Excel and the Statistical Package for the Social Sciences v25.0 (IBM; http://www.spss.com). As outlined through the procedure in Schmitz et al²⁰, data were scored accordingly, and the internal consistency of the PharmCAQ was assessed by determining the Cronbach’s alpha coefficient, a standard measurement of reliability, where alpha scores greater than 0.7 are considered acceptable²⁹.

Ethics approval 

Ethics approval was provided by the Federation University Australia Human Research Ethics Committee (#A21-023) and the University of Tasmania Human Research Ethics Committee (#26068), with all elements of human research being conducted in line with the National Statement on Ethical Conduct in Human Research 2007 (updated 2018)³⁰.

Results

All eight experts participated in rounds 1 and 2 using video-conference software. A third round was conducted through a series of email exchanges that occurred between the lead researcher (DT) and each participant. The bulk of the tool’s development was achieved by the second-round discussion, with the third round aimed at refining minor elements of the PharmCAQ. The outcome of the process is detailed below, including all three rounds of development leading to the final development of PharmCAQ.

Round 1: discussing and defining factors

Prior to the first meeting with the various content experts, a draft list of 90 factors, based on the systematic literature review conducted by Terry et al¹⁰, was sent to each participant. Each factor was grouped according to five classifications, consistent with the format of the CAQ and NCAQ^7,20. All participants were then invited to read and review all identified factors considered most important to recruitment and retention of rural pharmacists for discussion at the initial meeting.

The outcome of the first meeting highlighted that, out of the initial 90 factors, many were similar and should be combined and then further clarified among participants. All factors were then rated by level of importance for recruitment and retention. This led to the initial list of factors being reduced to 35 considered very important, with an additional 25 being considered important. All other factors were excluded as they were considered less important or unimportant: they pertained to attributes of the pharmacist rather than the community, which is what the tool seeks to measure. For example, starting a business with friends and being married with children are unimportant when assessing a community.

In addition to discussing, defining and ranking key factors drawn from the literature, the expert group also identified several factors that were missing or not well defined within the literature yet were also known to have an impact on recruitment and retention of pharmacists in rural areas. These included opportunities to participate in research, the obligation or ability to provide discounted or value-added pharmacy services, and the health and wellbeing of the pharmacist themselves.

After the list of factors was identified, the nuances regarding each factor were discussed with the experts, providing greater context or details that may have been missing from the literature. For example, the factor associated with the availability of support staff was considered very important, however, it was indicated that ‘availability’ was only one facet of the factor itself. Specifically, it was suggested that ‘competency’ of both technical and other support staff, such as front of house or retail assistants, was also essential. In this way, definitional details were provided by each expert regarding each factor identified.

Round 2: further discussion and refining

Two days prior to the second meeting, an updated set of factors was sent to all experts. At the second meeting, there was a greater and increased level of discussion regarding key factors and their relevance, while seeking better ways to differentiate the various factors. In so doing, the factors were reduced from 60 to 50 through further clarification and combining of similar factors. This process also ensured the meaning of factors was clarified and more succinctly described. For example, multiculturalism identified from the literature did not accurately outline what this meant for recruitment and retention of pharmacists to a rural area. As such, the team of experts was able to more carefully define this as acceptance of cultural diversity within a community, specifically pertaining to a pharmacist who may be from a diverse cultural background.

Beyond carefully defining what each factor meant and how it could be assessed when examining a rural community, each factor was again scrutinised to ensure it was grouped according to the most appropriate classification and that the classifications were adequately described. As such, minor adjustments were made and each factors was grouped into one of the following five classifications: (1) geographic, (2) economic and resources, (3) practice and scope of practice, (4) practice environment and (5) community practice support factors.

Round 3: final revision and approval

As with previous rounds, each expert suggested nuances and further adjustments, which resulted in minor wording changes for several factors. This discussion and feedback were used to develop the final definitions along with key tag lines, used to briefly describe each of the factors that were to be present in the final questionnaire. The final group of factors, tag lines and definitions were sent to all experts prior to the next meeting. However, given the smaller number of participants at the final meeting, such an endeavour provided a limited number of suggestions. To overcome this, any additional information or modifications that were provided from the various experts via email or conversation were further incorporated into the final list of factors. In most cases, any modifications suggested were minor in nature, and focused on wording changes and clarifying statements where ambiguity may have remained. After final consensus, the factors, tag lines and definitions were used to develop the final questionnaire along with a glossary of terms. Additionally, three open-ended questions used to identify specific factors for an individual community, identical to those on the CAQ and NCAQ^7,20, were included (see Appendix I and Appendix II).

Reliability testing – final revisions and acceptability

As the PharmCAQ was piloted among the 10 pharmacists, numerical rankings for each factor within the tool were utilised to calculate a weighted Apgar score. In this case, the process further informed the internal consistency of the tool in that it measured accurately what it intended to measure, while feedback on the PharmCAQ was sought from each pharmacist in relation to how well the questions were measuring each classification and associated factor. In addition, the reliability of the PharmCAQ was assessed using Cronbach’s alpha coefficient, resulting in an overall Cronbach’s alpha of 0.852, which is above the minimum threshold of acceptability (0.7), and further increasing the reliability was not achieved if any of the individual items were removed (Appendix III).

Discussion

The outcome of the modified Delphi technique was the development of a PharmCAQ that has been suitably refined, assessed for reliability and validated for use across rural communities in Australia, to facilitate the identification of key factors considered most vital to the recruitment and retention of pharmacists. While the development and use of the Apgar questionnaire for the recruitment and retention of health professionals is not a novel idea^7,18,20, seeking to specifically focus on pharmacists is unique^10,14. The discussion, deliberation and collaboration between organisations with varied perspectives provided an opportunity for nuanced insights regarding the many recruitment and retention factors during the questionnaire development. In addition, the dialogue that occurred as part of the Delphi process allowed the sharing of challenges and methods used to address these mutual concerns, all of which provided a deeper level of engagement. All participants in the study identified a need for a sharper focus upon the pharmacist as a health professional who also encounters related but unique challenges compared to their medicine and nursing counterparts in coming to and working in a rural community^1,6-9.

This finding was particularly evident in the development of the PharmCAQ. For example, it is noted that, among the 50 factors identified and developed, 10 factors were similar to those found in both the CAQ and NCAQ. These included five geographic factors: communication infrastructure, social and cultural opportunities, recreation and adventure opportunities, spouse or partner opportunities, and availability or accessibility of schools^7,20 (Fig2).

Three factors similar to the CAQ and NCAQ were economic and resource factors: financial income, contract flexibility and support for continued professional development. The remaining two factors that were similar were associated with workplace diversity (the demographic mix among clients or patients) and community recognition (appreciation for and community support of a new pharmacist)^7,20.

Although 10 factors were shared between all health professions, pharmacist and physicians shared eight factors that were not shared with nurses, including moving allowance, support staff, teaching and having a sense of purpose in the community²⁰. Further, pharmacists and nurses shared six additional factors not found among physicians in their respective questionnaires, including day care, housing and community size⁷. Although these similarities and differences did not always have identical wording, the consistency demonstrates some factors remain important for considering a rural career for any health profession^7,20.

Despite the similarities and differences, various factors are considered specifically important to pharmacists. These include, but are not limited to, pharmacy ownership, mutual support and collaboration with other health professions, the presence of a multipurpose service or local health hub, and the obligation or ability to provide discounted or value-added pharmacy services. Other unique factors include the availability and competency of technical and ancillary staff, and separate credentialing that enables the pharmacist to deliver diabetes education, immunisations, and alcohol and other drug services¹⁰.

Regardless of similarities or differences between health professions in terms of recruitment and retention, improving access to primary health care is an essential yet complex conundrum for rural communities in Australia and internationally³¹. People in rural communities, globally, typically experience higher levels of complex and chronic disease, have lower levels of health literacy and reduced access to health services^31,32. The combination of these factors further emphasises how vital rural pharmacists are in providing health care, the need for an expanding or a more rural generalist scope of practice, while ensuring improved recruitment and retention strategies are in place^33,34.

Limitations

The Delphi technique was insightful and used several heterogeneous experts from key organisations across Australia and partners in the USA. However, the findings of the Delphi and the developed PharmCAQ may not be representative of the nuanced differences found in other countries that experience challenges in recruiting and remaining pharmacists in rural areas. Due to this recognition, the PharmCAQ may require slight wording changes and further adaptations to meet local needs. Further, pre-testing to verify the 50 factors may be vital to ensure they are also relevant to country-specific issues and challenges. Additional testing may also be essential in more rural communities (MM7), where key drivers of what makes a rural or remote community differ to less remote communities. However, the open-ended questions within the tool are designed to capture key nuances of specific communities, including those who may be considered remote.

Despite these limitations, the study has resulted in the development of a tool that can assist in understanding the fundamental factors associated with pharmacist recruitment and retention within the Australian context, and how these may be addressed within rural community settings. Lastly, the authors have shown that the factors that influence the pharmacy workforce in a rural area are proposed here as being unique to the discipline of pharmacy. However, a close discrimination of those factors more important to the differing settings of pharmacy practice, such as hospital pharmacy compared to community pharmacy, have not been provided. While there are clear differences between the roles of the pharmacist in these diverse settings^33,35, development of additional tools that are more setting specific could be a benefit in those particular circumstances.

Conclusion

Pharmacists represent a cornerstone of the healthcare system and provide a first port of call into this system for many of those residing in rural areas of Australia. As a mechanism for addressing the worsening health professional shortage currently experienced in rural areas, the opportunities for the pharmacist to play an expanded role are compelling. Despite the pivotal role of pharmacists, recruitment and retention have proven troublesome for rural communities, despite an increased number of graduates from pharmacy education programs. Using a Delphi technique, the PharmCAQ was developed, refined, assessed for reliability and validated as a tool for supporting the recruitment and retention of the pharmacist workforce in rural areas. The authors believe the key factors contained within the PharmCAQ may offer – for the first time – an opportunity to better support communities to identify and address the key factors considered most vital to the recruitment and retention of a pharmacy workforce. More research is required to further test and demonstrate the overall impact of the tool across various rural communities in their capacity to recruit and retain pharmacists to further support the healthcare needs of the community.