Development of a questionnaire measuring student attitudes to working and living in rural areas

full article:

Introduction

The shortage of doctors and other health professionals working in rural areas is well documented¹. There have been a number of strategies implemented which attempt to address the mal-distribution of health professionals in rural areas. One such strategy has been to increase exposure of undergraduate health students to rural practice and rural health issues via curricula and rural placements². Student attachments in rural locations have been instigated around Australia to provide positive rural experiences for students. These positive experiences are designed, in turn, to foster positive attitudes to rural practice and encourage students to work in a rural area when they graduate^3-5. Retrospective research has identified links between positive attitudes to rural practice and intention to work or working in a rural area⁶.

Silagy and Piterman's work with medical students has shown that attitudes to rural practice are multi-dimensional and can be categorised as relating to: family and personal issues; professional issues; and community and social issues⁷. Work with allied health students indicate that these categories are also relevant to other health students. Advantages and disadvantages of rural practice reported in the literature^3-5,8 include:

1. Community and social issues: relaxed rural lifestyle, friendly community, sense of community and social atmosphere, surrounding environment, community respect, social and cultural opportunities and lack of privacy.
   
2. Family and personal issues: safety, cost of living, isolation from family and friends, difficulties for family members to find employment and distance from home.
   
3. Professional issues: breadth and variety of work, nature of work, professional education and advancement, helpful staff and teamwork, autonomy and opportunities for innovative work practice, greater intellectual challenge, professional isolation, peer support, access to equipment, longer hours, more on-call and too much responsibility.

It is reasonable to hypothesise that measuring a change in student attitude towards rural practice would assist in evaluating the impact of rural placements. However, we were unable to find a reliable and valid tool (searching the databases Medline, CINAHL, AEI and RURAL) that examined health students' attitudes to rural work and life covering the three domains of community and social issues, family and personal issues and professional issues. The aim of the present study was to develop a reliable and valid tool that measures students' attitudes to rural practice and rural life across all three dimensions. Use of this tool to evaluate the impact of a rural placement may provide evidence to support increased rural exposure to undergraduate health students. This article reports on the development, reliability and validity of such a tool to fill this gap in the rural health research literature.

Methods

Participants

The Mount Isa Centre for Rural and Remote Health (MICRRH) in Queensland, Australia, and the Spencer Gulf Rural Health School (SGRHS) in South Australia are part of the national University Department of Rural Health network. Both centres organise, facilitate and/or track health students going on placement to rural or remote areas. MICRRH and SGRHS simultaneously conducted the study in South Australia and North West Queensland. The data sets from the two sites were combined to enable a larger sample size within the timeframe of the project and to sample a larger cross-section of students.

All health students who commenced a rural placement in South Australia or a rural placement organised by MICRRH during the period 1 January 2001 to 31 December 2001 were invited to participate in the study. A rural placement was defined as a minimum of a 1 week block spent in a rural location for reasons associated with their current study. Visiting international and interstate students were excluded from the study for the SGRHS sample, because ethical approval was only obtained for South Australian students. The Queensland ethical approval included visiting students.

MICRRH distributed the questionnaire to 121 students, of these, 114 students completed pre-placement questionnaire (94% participation rate) and 86 completed the post-placement questionnaire (71% participation rate). SGRHS distributed the questionnaire to 122 students, 68 students completed pre-placement questionnaire (56% participation rate) and 36 completed the post-placement questionnaire (30% participation rate).

There were no significant differences in sex, age and length of placement for MICRRH respondents for both pre- and post-placement questionnaires. However, medical students were more likely to not respond than nursing or allied health for pre-placement questionnaires, c² (2, n = 121) = 6.456, p <.05 and post-placement questionnaires, c² (2, n = 121) = 6.785, p < .05.

SGRHS did not collect information from those students who did not respond to the pre-placement questionnaire. However, for those who did not complete the post-placement questionnaire there were no significant differences in gender, age and discipline. There was a significant difference for length of placement. Independent t-test showed that those who completed the post questionnaire were on placement one week less (M = 4.06, SD = 1.55) than those who did not complete the post questionnaire (M = 5.06, SD = 1.78), t (68) =-2.504, p< .05.

Instrument

A questionnaire, named the Student Attitudes to Rural Practice and Life Questionnaire was designed to assess student attitudes to rural practice and life both before and after a rural placement. A literature review informed the structure and content of the questionnaire. A 37-item, pre- and post-questionnaire was developed initially. A panel of five academics associated with student placements rated the relevance of each item on a four-point scale. Items rated low in importance, or items where the panel failed to reach a consensus on their relevance were discarded. The questionnaire was piloted with samples of students from both SGRHS and MICRRH, and their feedback was sought on the appropriateness of the items. This process saw the questionnaire reduced to 14 items. Final refinement of the questionnaire (via further discussion with SGRHS academics and student piloting) involved re-wording of eight of the items, and the addition of five new items. The final version of the pre- and post-questionnaire (Fig 1), contained 18 items (with items 1-7, and 14 relating to professional issues, items 8-13, 15-16 relating to community and social issues and items 17 and 18 relating to family and personal issues). Five of the 18 items were worded negatively to avoid bias. Respondents were instructed to indicate how strongly they agreed or disagreed with each statement on a Likert scale of 1 (strongly disagree) to 6 (strongly agree). SGRHS also included a response option of not applicable.



Figure 1: Student Attitudes to Rural Practice and Life Questionnaire.


Procedure

Following approval from the University of South Australia's Human Research Ethics Committee, SGRHS administered their survey via postal mail to students undertaking placement in rural areas across the state, and surveys were returned via reply-paid post. SGRHS chose this data collection method because it suited the geographically distributed sample. SGRHS contacted the host organisation prior to posting the survey to facilitate the student receiving the questionnaire. MICRRH administered the survey to students directly for completion on-site, after approval from the Mount Isa Health Services District Ethics Committee. All students completed the questionnaires during their first week of placement (pre-placement survey) and last week of placement (post-placement survey).

Data analysis

The assessment of validity and reliability involved three main processes. The first stage was the assessment of content validity. Content validity relates to the relevance of the items included in the questionnaire and whether the instrument assesses the whole domain of interest^9,10. Content validity was assessed through the discussion and rating process conducted with the academic panel, and through student pilots of the tool. The second stage was to assess construct validity, the conceptualisation of the issue. The items were hypothesised to cluster into the three themes¹¹. Exploratory factor analysis was used to provide evidence of construct validity. The third stage was assessment of reliability; the ability of the instrument to create reproducible results¹¹. In this instance, homogeneity or the internal consistency of the questionnaire was most important¹¹. STATISTICA vers. 6 was used to perform the statistical tests.

Results

Characteristics of the sample

One hundred and eighty-two students participated in the study by completing the pre-placement questionnaire and 122 completed the post-placement questionnaire. A breakdown of respondents' age, gender, discipline and placement length is given (Table 1). The proportion of male and female students from SGRHS was almost equal, however MICRRH had a far greater proportion of females. Overall, age was positively skewed and this distribution was similar when broken down by gender and site. Medical students consisted of almost half the sample when MICRRH and SGRHS participants were combined. Nursing students consisted of approximately one-third of the sample, and other disciplines included pharmacy, physiotherapy and other allied health courses. The small number of questionnaires completed that contained 'not applicable' responses or 'missing data' were excluded from the factor analysis. In all, 110 respondents completed all items on both pre- and post-questionnaires.

Table 1: Sample summary statistics



Validity

Exploratory factor analysis was conducted on all 18 items of the pre- and post-placement questionnaires (n = 110). The factor analysis on the pre-placement items showed that there were five components with eigenvalues of greater than 1.0. The scree plot, factor loadings and interpretation however, indicated that a three factor solution (explaining 49.87% of the total variance) was the best fit for the data. Overall, the interpretation of the pre-placement questionnaires was not very clear. However the factors identified with the post-placement questionnaire were interpretable. Factor analysis revealed the presence of five factors with eigenvalues of greater than 1.0, explaining 66.56% of the total variance. Examination of the scree plot indicated, however, that a three or four factor solution would be appropriate (Fig 2). The factors were rotated using the normalized varimax rotation method to assist in the interpretation of the extracted factors. Examination of the factor loadings found stronger loadings and clearer interpretation with a four factor solution (in comparison with 3 factors). The four-factor solution explained 60.82% of total variance.



Figure 2: Scree plot of eigenvalues for post-placement questionnaire


The four factors and the items that load on these factors are illustrated (Table 2). There were good factor loadings on all factors (0.54 to 0.86). The content of the items were used to interpret the four factors, which varied from the three originally hypothesised. Factor One relates to health staff and the general community in rural areas and was interpreted as 'friendliness and support in rural areas'. Five items loaded on this factor. Factor Two was interpreted as 'isolation and socialisation problems associated with living and working in rural areas'. Five items loaded on this factor. This factor contains all the negatively worded items from the questionnaire. The three items on Factor Three were community and social items and were interpreted as the 'enjoyable aspects of living in a rural area'. The four items on Factor Four related to the 'opportunities that working in a rural area provides'. Item Two did not load on to any factor.

Table 2: Item breakdown and factor score component loadings for post-placement questionnaire (n = 110)



Reliability

Cronbach's alpha coefficient was used to assess the reliability of the questionnaire by correlating each item with all other items, to show whether each item is related to the one concept. The Cronbach's alpha coefficient for the post-questionnaire was 0.68, which is considered acceptable for newly developed scales¹¹.

Limitations

The different questionnaire distribution method (face-to-face versus postal) is a potential limitation of the study and it is hard to predict what effect this had on results overall. However, the postal distribution method may have resulted in the low response rate among the SGRHS sample and it reflects the mobility of this sample. We are aware that some students did not receive their questionnaire because they changed their placement location or date at short notice, or they completed the placement earlier than expected. In some services, administration staff did not know where to deposit students' mail and students did not know where to collect their mail. The SGRHS low response rate is a limitation in that those who completed the post-questionnaire were on placement for on average one week less. This may mean they had less formed attitudes. However it is unlikely that the extent of this bias will impact on the results.

Discussion

Based on literature, the current study hypothesised three important components of students' attitudes to rural practice and rural life, namely, professional issues, personal issues, and community and social issues. Although the factors interpreted are not those hypothesised, the results are meaningfully interpreted and provide a more detailed reflection of the rural experience. The main deviation from the three hypothesised attitude components was the identification of a new factor associated with isolation and socialisation problems of living and working in rural areas. The remaining advantageous items grouped into more integrated factors than those hypothesised.

Factor One combined both professional, and community and social issues that are associated with friendliness and support found in a rural community. Factor Two combined items from the three hypothesised components and reflect the negative aspects of rural life and work, 'isolation and socialisation'. The remaining community and social items that form Factor Three (enjoyable aspects of rural living) appear to be more personalised items than those in Factor One, such as Item 9 ('There are people I could be friends with in rural areas'), and focus on 'enjoyable' aspects. No professional items loaded onto this factor. Factor Four grouped items relating to professional opportunities.

Item Two ('There are good opportunities for employment in rural areas') did not load onto this factor, or any other factors. This may suggest that this issue was outside the experience of the students on rural placement in this study sample.

The four factors identified reflect the experience of living and working in a rural area, demonstrating the professional, personal and community spheres of a student's experience in the rural setting are not mutually exclusive. While the literature clearly supports that these factors exist in rural health practice, it also supports that these factors are interrelated and should be considered collectively in efforts to promote rural practice^11-13. These results also emphasise the importance of isolation and support issues in determining attitudes to rural life and work.

The Student Attitudes to Rural Practice and Life Questionnaire has shown good reliability for a newly developed scale. The inability to meaningfully interpret the pre-placement questionnaire via factor analysis is not surprising. The likely variations in students' preconceived ideas and previous experiences of living and working in rural areas may have resulted in quite differing attitudinal responses. Students who lack rural experience may have uncertain and ill-formed attitudes. Post-placement, however, students' attitudes may have firmed as identifiable and interpretable key constructs. For example, the items relating to isolation and socialisation problems failed to load together in the pre-placement analysis, prior to students' rural experience, but formed a distinct factor in the post-placement analysis.

Given the reliability and validity findings, the tool appears suitable for measuring students' attitudes. Future studies should consider the following. First, researchers could further develop the tool by re-examining the items to ensure new factors such as Factor Two (isolation and socialisation problems) and Factor Three (enjoyable aspects of rural living) are adequately represented and not demarcated by negatively versus positively worded items in the questionnaire. One consideration may be the inclusion of professional-related items in Factor Three, where rural work may be seen as an enjoyable aspect of rural living. Second, in order to examine the tool's usefulness to measure change, test-retest reliability is important and this requires examination. Third, in order to meaningfully interpret factors relating to pre-placement attitudes, future studies should ensure they have a sufficient sample size to differentiate students based on previous rural experiences. Finally, to increase the response rate to postal surveys, evidence-based strategies can be utilised to increase the response rate, such as incentives not conditional on response, pre-contact and follow up¹⁴. Further studies should also consider more uniform sampling procedures in combining data sets.

Initiatives have been implemented to address shortages of rural health professionals and the effectiveness of these initiatives must be evaluated¹⁵. There has been an increase in the number of rural placements undertaken by tertiary level health students, creating a need for a valid and reliable measure of the impact of these placements. Measuring long-term outcomes such as rural recruitment is difficult due to the time frame required. The measurement of attitudinal change to rural practice and rural life provides a short-term indicator to evaluate rural placements¹⁶.

The Student Attitudes to Rural Practice and Life Questionnaire shows potential to measure change in student attitudes towards rural practice and life, and we are not aware of similar tools. This tool is being used by the Spencer Gulf Rural Health School in South Australia to measure undergraduate medical students' attitudes, pre- and post-exposure to rural health practice. Their results have been similar to the present study's, with the pre-placement questionnaire factor analysis being uninterpretable and the post-placement questionnaire forming the same four factors. Together, these initial findings support the use of the tool to measure student attitudes towards rural life and work.

Conclusions

This study has identified four factors associated with health student attitudes to living and working in a rural area. While these differ from those hypothesised, they are meaningfully interpreted and supported in the literature. The importance of experiencing friendliness and support, and also isolation and socialisation problems in forming students' attitudes to rural practice and life has also been highlighted. The study has met the main aim of establishing the validity and reliability of a newly developed tool to measure health students' attitudes and change in attitudes following rural placement, across a broad spectrum of professional, social and personal related factors.

Acknowledgements

The authors thank the students who completed the questionnaires. We acknowledge Dr Susan Shannon and Associate Professor Jonathan Newbury of the Spencer Gulf Rural Health School for providing feedback on the final manuscript.

References

1. Wilkinson D. Inequitable distribution of general practitioners in Australia: analysis by state and territory using census data. Australian Journal of Rural Health 2000; 8: 87-93.

2. Dunabin J, Levitt L. Rural origin and rural medical exposure: their impact on the rural and remote medical workforce in Australia. Rural and Remote Health 3: 212. (Online) 2003. Available: http://rrh.org.au (Accessed 28 February 2005).

3. Russell M, Clark M, Barney T. Changes in attitudes and skills among occupational therapy students attending a rural fieldwork unit. Australian Occupational Therapy Journal 1996; 43: 72-78.

4. McAllister L, McEwen E, Williams V, Frost N. Rural attachments for students in the health professions: are they worthwhile? Australian Journal of Rural Health 1998; 6: 194-201.

5. Barney T, Russell M, Clark M. Evaluation of the provision of fieldwork training through a rural student unit. Australian Journal of Rural Health 1998; 6: 202-207.

6. Lee S, Mackenzie L. Starting out in rural New South Wales: the experiences of new graduate occupational therapists. Australian Journal of Rural Health 2003; 11: 36-43.

7. Silagy C, Piterman L. Attitudes of senior medical students from two Australian schools towards rural training and practice. Academic Medicine 1991; 66: 417-419.

8. Boonyawiroj E, Haven M, Freeman V et al. Allied health faculty attitudes toward rural clinical practice. Journal of Allied Health 1996; 25: 263-274.

9. Pierce A. Measurement. In: L Talbot (Ed). Principles and Practice of Nursing Research. St Louis, US: Mosby, 1995; 265-291.

10. Luecht R, Madsen M, Taugher M, Petterson B. Assessing professional perceptions: design and validation of an interdisciplinary education perception scale. Journal of Allied Health 1990; 19: 181-190.

11. Hays RB, Veitch PC, Crossland LJ, Cheers B. Why doctors leave rural practice. Australian Journal of Rural Health 1997; 5: 198-203.

12. Jensen CC, DeWitt DE. The reported value of rural internal medicine residency electives and factors that influence rural career choice. Journal of Rural Health 2002; 18: 25-30.

13. Tolhurst H, Stewart M. It depends what you expect from life - factors which influence medical students to choose a rural career. In: Proceedings, 7th National Rural Health Conference. Canberra: National Rural Health Alliance (Online) 2003. Available: http://www.ruralhealth.org.au/nrhapublic/publicdocs/CD-ROM/data/papers/7_conf/tolhurst.pdf (Accessed 28 February 2005)

14. Edwards P, Roberts I, Clarke M et al. Increasing response rates to postal questionnaires: systematic review. BMJ 2002; 324: 1183.

15. Wilkinson D. Evidence-based rural workforce policy: an enduring challenge. Rural and Remote Health 3: 224. (Online) 2003. Available: http://rrh.org.au (Accessed 7 July 2004).

16. Somers G, Young A, Strasser R. The medical student's intention to practice in rural Australia: A questionnaire. In: Proceedings, 6th National Rural Health Conference. Good Health - Good Country Canberra: National Rural Health Alliance. (Online) 2001. Available: http://www.nrha.net.au/nrhapublic/publicdocs/conferences/6thnrhc/somerspaper.htm (Accessed 28 February 2005)