Developing conceptually sound items for a clinical courage questionnaire

Introduction

With the worldwide shortage of physicians living and practising in rural and remote areas, it is important to understand better the nature of rural practice. A recent qualitative study examined the lived experience of rural doctors when their patients’ needs and the extent of their capabilities intersect¹. This study identified an overarching construct called clinical courage. This phenomenon resonates strongly with rural clinicians, who have found it difficult to explain the nature of rural practice where they seek to balance quality and safety of health care with patients’ access to services. However, the clinical courage literature prior to this study was limited to conceptual pieces^2-4. 

The six attributes of clinical courage experienced by rural doctors have recently been described further in the literature:

1. ‘Standing up to serve anybody and everybody in the community’¹. Rural doctors often have a deep commitment to providing inclusive health care in the communities they identify as belonging to^5-7. Interconnected relationships enable doctors to have place-based knowledge of their patients^1,8. However, managing intersecting professional and personal relationships, while commonplace in rural practice, is considered problematic by urban-centric codes of ethics and practice standards^8-10.
2. ‘Accepting uncertainty and persistently seeking to prepare’. Current skills and knowledge are challenged by the broad range and complexity of cases seen¹¹. Rural doctors seek to maintain, extend and expand their clinical skills, seeking to develop adaptive expertise through integrative wisdom^7,11.
3. ‘Deliberately understanding and marshalling resources in the context’⁵. Working in resource-limited environments, rural doctors describe requiring an understanding of the physical and human resources available and the adaptability of these in serving patients’ needs^7,8. Thin resources, which can shift in substantial ways over short periods of time, mean constantly balancing individual needs with broader health service and community needs⁸.
4. ‘Humbly seeking to know one’s own limits’. Self-awareness of what one can and cannot do enables a rural doctor to articulate her capability to herself, her local colleagues and tertiary referral services⁸.
5. ‘Clearing the cognitive hurdle when something needs to be done for your patient’. When no action is not an option, and identifying that other resources are not available and no one else is better qualified or experienced, the rural physician makes the choice to act.
6. ‘Collegial support to stand up again’. Rural physicians consciously seek out peer support from both those in their community of practice and others who are familiar with rural practice to maintain their resilience and capacity to practice^5,7.

Some attributes of clinical courage have been discussed in rural health professionals other than doctors, including rural paramedics and rural physiotherapists^9,12. An exemplar of clinical courage is shown in Box 1.

A related concept, rural practice self-efficacy, has been found to be associated with rural and remote clinical practice^13,14. Self-efficacy is having the belief that you can achieve what you set out to do within a defined context – in this case, rural practice. Clinical courage and self-efficacy overlap to some extent, with clinical courage covering a broader range of specific concepts. In a similar research vein, a study in Australia identified the motivations of existing practitioners who worked in remote communities for 3 years or more¹⁵. The quantitative study found 3 years or more of remote clinical practice more likely in males, in nurses (compared with other health professionals) and those with high clinical self-containment and intercultural interest scores and lower needs to accommodate relationships or have balanced lifestyles¹⁵. Thus, there is a growing body of knowledge that indicates a key role for psychosocial behavioural concepts contributing to ongoing rural practice. These do not however describe the praxis of rural medicine. Clinical courage has been identified as a potentially relevant concept that rural clinicians seem to identify. Potentially the attributes described above and the overall concept of clinical courage could be applied to support the engagement, development, support and retention of rural doctors. Being a relatively new concept there is no existing questionnaire that can be used to measure or gauge its role. A focused survey that is psychometrically sound and relatively short will enable the rural health sector to measure the six attributes consistent with the lived experience of clinical courage.

The overarching aim of this research is to develop a quantitative measure of clinical courage, based on a theoretical understanding of the previously described six aspects of clinical courage, combined with psychometric principles to produce a refined initial questionnaire. The purpose of this article is to detail the initial in-house development of survey items used to create a clinical courage questionnaire using a nominal group technique, which facilitated consensus building regarding questionnaire items among members of the research team¹⁶. Subsequent work will undertake psychometric evaluation and development of the questionnaire.

Methods

The questionnaire development involved two key concepts: a second-order latent factor model structure¹⁷ and the nominal group method of creating survey items^18,19.

Latent factor model

The questionnaire development here focuses solely on the first-order factors, identified as aspects of clinical courage in the qualitative research. Evidence indicates that, for initial stability, five questions are required for each of the six identified domains, called first-order factors^20,21. Three rather than five or more questions are shown in the clinical courage questionnaire in Figure 1.

Nominal group method

Nominal groups usually involve a structured face-to-face interaction of 5–12 participants¹⁶. For the in-house development of survey questions, based on the researchers’ previous qualitative research, the nominal group consisted of four of the five researchers who undertook the original qualitative study of clinical courage, all of whom are rural doctors (DC, IC, JK, LuW)^1,5. Also included in the group were an academic general practice registrar (IW) and an experienced qualitative researcher, who had both been involved in further research on clinical courage within the context of COVID^22,23. The team are from three countries: Australia (DC, LuW, IW, SW), South Africa (IC) and Canada (JK), bringing a range of cultural perspectives to the process of survey item design.

Moderator

The moderator (RB) is a remotely based academic researcher with a strong background in questionnaire development and psychometrics. As a research psychologist he is familiar with working with and understanding latent constructs. RB has a history of small-group work, clinically, educationally and in research. He ensured all participants were fully engaged and that their voices were heard. The group accepted RB as an appropriate facilitator.

The group facilitation process iteratively consolidated the knowledge of the group and built agreement around all elements of the questionnaire. All meetings were conducted via Zoom to allow for the real-time, virtual, face-to-face interaction of the group members. This is a suitable approach for complex tasks when the interaction of experts is likely to be productive¹⁴.  

Consensus process

To ensure methodological rigour, the steps in the nominal group method rely on drawing on expertise to generate ideas, confidentiality and consensus formation¹⁶. This was achieved by the moderator managing the flow of information in the meetings and between each meeting, as outlined in Table 1. Usual practice with consensus research is for a predetermined definition of consensus²⁴. In this study, consensus was defined as the highest ranking questions, provided that no member of the nominal group membership objected to the inclusion of the item. The three to six highest ranked items for each domain were then returned to the group, where group members were invited to object to the inclusion; if this did not occur, a final review and wordsmithing of the item were completed.

Project team tasks

Six main activities were undertaken during this nominal group process.

1. Developing and agreeing on the definitions of six attributes of clinical courage:

• Expertise  The group followed standard brainstorming processes to ensure all contributed and no-one was censored.
• Confidentiality  All the ideas were worked with independently by the moderator, who constructed draft definitions distributed prior to the next meeting.
• Consensus formation  Open discussion of each definition occurred, challenging meaning and structure; this was followed by rewriting each definition as a group. The final definitions were then written by the moderator and distributed for final approval by the group.

2. Generation of a pool of items for each of the domains:

• Expertise  All group members contributed to generation of items.
• Confidentiality  Group members returned suggested questions directly to the moderator, who then presented the group with items in a random order within each domain in turn.

3. Decisions about inclusion of items:

• Consensus formation  The items were discussed by the group and consensus reached about whether any questions should be excluded or modified.
• Expertise: All group members were asked to choose their top five questions and provide this information to the moderator. The highest ranked items for the domain were calculated by the moderator and the top five for each domain provided to the group.
• Confidentiality  The process of ranking was undertaken as a group activity. Although choosing was independent, reporting to the moderator was not confidential.

4. Decisions about item construct:

• Expertise  Questions for each domain were presented to the group and assessed as being good questions for the domain or not.
• Confidentiality  This was an open process relying on group dynamics to generate discussion.
• Consensus formation  Each question was examined and modified as needed on the basis of discussion, and the final questions were then put into a draft questionnaire by the moderator.

5. Construction and review of the initial questionnaire items: 

• Expertise  The clinical courage research group examined the questionnaire and moved items or modified items as too close to other items, or too similar to another construct.
• Confidentiality  This was an open process, using the group process to generate diverse contribution.
• Consensus formation  The final questionnaire was drafted by the moderator.

6. Initial piloting and item revision: 

• Expertise  This was based on lived experience from rural doctors in three countries who were colleagues of the clinical courage research group. Each clinician in the group approached rural colleagues to complete and review the questionnaire.
• Confidentiality  Comments were returned to the colleagues, who asked them to comment, and then de-identified and sent to the moderator. The comments were used to modify the questionnaire as necessary.
• Consensus formation  This modified questionnaire was distributed to the participants for sign-off, which all did, and this was the version used in the next stage of the development of the clinical courage questionnaire.

These activities were undertaken across a series of weekly meetings across a 3-month period with the facilitator (RB), providing a summary of consensus work and next steps prior to each meeting, as outlined in Table 1. 

Ethics approval

This research project received low-risk ethics approval from The University of Adelaide Human Research Ethics Committee (project number H-2022-086). 

Results

The question generation process resulted in 73 questions across the six domains (Table 2). For each domain, between 7 and 19 draft questions were reviewed by the group to eliminate questions that were (1) not a measure of the domain or (2) overlapped too much with each other within the domain. After exclusion and refinement, individual ranking and review after post-pilot feedback, 30 domain-specific questions remained.

After considerable discussion, the introductory statement for each question was modified to the form of the affective judgement (Table 3, option 3): ‘In my current practice how like me is it …’ The consensus was that clinical courage was most likely a praxis associated with contemporary work (‘in my current practice’), rather than a description of an individual’s characteristics (option 1) or values (option 2).

A trial questionnaire was constructed from the final questions in each domain. This trial questionnaire was modified in view of feedback from eight rural clinicians, who completed and commented on it. At the penultimate group meeting, the nominal group discussed the feedback and through consensus decided to split one question and delete a question that was perceived as very similar to another question. Based on experience with rural and remote clinicians, the primary author suggested a global construct item: ‘to experience personal pleasure from meeting the significant clinical challenges of rural and remote patients’. A global item is one that is not directly related to any single underlying construct and may be found to serve as a proxy for the concept (global construct) of clinical courage. If it is determined to be a proxy, it can be used in situations when survey space is critically short in place of the questionnaire. Further, a global item may be useful in developing the scoring structure of the six factors. The process resulted in the initial clinical courage questionnaire (Table 4), which consists of 30 items covering the six domains and one global item.

Discussion

Clinical courage has been identified through a number of qualitative articles as one way of experiencing work as a rural doctor^1,5. Developing a clinical courage survey tool will allow researchers to better understand the frequency, intensity and stability of this praxis in rural medicine and potentially explore clinical courage in other professional groups. Recruiting and retaining clinicians to work in rural and remote areas is a multifaceted problem with factors internal and external to the clinician that influence this decision²⁵. Many of the factors, particularly the external ones, are not amenable to change – such as urban background, partner preference and location of amenities²⁶. Thus, efforts to understand intrinsic psychosocial emotional factors that attract and retain physicians working in rural and remote communities may provide a means to increase the number of clinicians in rural practice.

The steps outlined in this article provide an example of the internal processes a research team can undertake to improve construct validity of an initial questionnaire. Building valid questionnaires requires a methodical approach based on existing literature, expert knowledge and psychometric principles²⁷. In consensus research, a small group well known to each other may be at risk of persuasion bias due to the social influence of stronger personalities of individuals within the group²⁸. The authors recognise this is a study limitation; however, this group has spent several years undertaking critical discourse for meaning-making during qualitative data analysis²⁹.

Questionnaires are a ubiquitous research tool and it is clear that they have been widely misused³⁰. The nominal group approach, using a small number of knowledgeable participants, provided a solid foundation to achieve construct validity in the questionnaire design. Careful development of agreed definitions of the domains involved provided a strong base on which to write and rewrite survey items, as there was a definition against which to test each question. The selection and ranking processes provided a reasonably quick method of identifying the preferred items. However, this voting process was not confidential. Confidential voting is an expectation of nominal group processes^16,28. The research group has worked well together over several years and our familiarity with working together resulted in this intentional omission.

Another potential weakness to this study is that consensus does not equate to correctness²⁸. The nominal group members are all researchers who bring their previous common research history to this process. This limitation will be compensated for in the second phase of this research, when the clinical courage questionnaire is released for testing by rural doctors. The purpose of sharing this questionnaire at this time is, first, to inform other rural researchers about the questionnaire development process and, second, to show the questionnaire at this stage to other rural researchers and rural clinicians for further discussion and feedback.

Future developments

Physicians working in rural and remote communities around the world will be invited to complete the initial questionnaire. Physicians will be recruited through a number of rural physician organisations, such as the Australian College of Rural and Remote Medicine, Rural WONCA (World Organization of Family Doctors), and the Society of Rural Physicians of Canada. When sufficient numbers of surveys have been returned, the questionnaire will be modified based on psychometric principles. The phenomenological and psychometric approaches will not have a 1–1 correspondence and it is anticipated there may be substantial differences. Ultimately, modifications will be made to include other specified populations such as students undertaking rural/remote placements, nurses undertaking similar placements, and for non-physician clinician groups.

The authors hope the results of this questionnaire will ultimately help to inform health professional education for rural and remote practice as well as recruitment and retention strategies. It may also be a useful, relevant tool that will provide valuable information to guide program evaluation and support future research.