Why do medical graduates choose rural careers?

full article:

Introduction

This article reports on research conducted in 2007 to 2008 that assessed international and national best practice in the selection of students for graduate entry medical courses, to investigate correlations between medical student selection procedures and exposure to rural medical practice during medical training with choice of careers in rural medicine. The study was performed to guide the development of a selection process for the new Deakin University Medical School, which aims to produce a cohort of graduates for regional and rural settings.

Central to the study was the issue of the medical workforce shortage in Australia's rural communities. Current medical student selection processes and medical course training experiences have failed to deliver sufficient medical practitioners with a commitment to rural medical practice. The critical gaze of the study was, therefore, on those selection processes and within-training experiences deemed to promote the likelihood that, on graduation, medical students would pursue a medical career in rural communities.

Methods

For the purposes of this study, rural communities were those classified under the Rural Remote Metropolitan Area (RRMA) classification system as RRMA 3 to RRMA 7^1,2.

The study adopted the metaphor of the 'rural pipeline into medical practice'³ as a template. The rural pipeline, as a concept in graduate medicine education, has a history in northern America dating at least from 1998. At that time the US Council on Graduate Medical Education⁴ noted:

The key seems to be the creation of a pipeline that reaches out to rural communities to encourage the selection and success of rural students, gives them opportunities throughout medical school and residency to work in rural settings, and supports them in practice after they do settle in rural areas.

The four stages of the rural pipeline begin with structured contact between rural secondary schools and the medical profession, followed by rural student selection into medical programs, then rural exposure during medical training and, finally, on graduation, measures to address retention of the rural medical workforce. This study provides evidence-based commentary on all four stages with particular emphasis on the second and third stages of the pipeline.

The research was conducted in three phases: (i) a review of international and Australian literature on medical student selection methods; (ii) interviews with selection officers and other key staff in the 9 Australian medical schools offering graduate entry programs in 2007; and (iii) interviews with graduates working in Victorian hospitals in 2007 near the end of their internship year.

Interns were recruited into the project by an invitation in the form of a plain language statement describing the project and an attached consent form that was distributed by medical administration staff in Victorian hospitals with a return-mail envelope addressed to the researchers. The self-selecting interns had undertaken their medical courses at the University of Melbourne, Monash University and the University of Tasmania; 11 had been attached to a rural clinical school (RCS) during their medical course and six had completed short rural rotations during attachment to a metropolitan clinical school (MCS). The interviewed interns were completing either a rural (n = 9) or a metropolitan (n = 8) hospital internship.

Although the focus of the study was on graduate entry medical programs, the literature reviewed was sourced more broadly and interviews were conducted with interns who had completed both undergraduate and graduate entry courses.

Results

Phase 1: Condensed review of background literature

This article is a condensation of a much more substantial review included in the original report of this project⁵. The literature review was structured in two dimensions: (i) a review of the literature on the power of selection practices to predict successful graduation; and (ii) a review of the literature on the impact of the components of the rural pipeline, including student selection, on the choice of rural practice as a career.

Comprehensive reviews of the literature relevant to these two dimensions were published in 2002⁶ and 2004⁷. Relevant literature published from 2002 to 2007 has also been reviewed for this study.

The conclusions to be drawn on the first dimension of this literature review phase of the study are fourfold. First, academic performance during the undergraduate degree as measured by grade point average (GPA) has the highest predictive power for student academic performance, including performance in clinical assessments^8-11. Second, the predictive power of interviews for student academic performance is modest but there is some evidence that their predictive potential can be enhanced by refinements that include clarity for applicants and interviewers on the role of interviews in the overall selection process¹², the use of the multiple mini-interview (MMI) format^13,14, less structured one-on-one interviews¹⁵, broader interview panel membership^16,17 and pre-interview training for panel members to prevent bias¹⁸. Third, a range of non-cognitive qualities thought to be important for medical practice have been assessed during interviews¹⁹ and by psychometric/personality tests^20,21. Methods of assessment include batteries of psychometric tests, refereed autobiographical statements and situational judgement instruments. Claims of predictive reliability of these measures of non-cognitive qualities²² have been tempered by concerns about coaching and fraud²³, particularly in the selection elements reliant on personal testimony²⁴. It has also been suggested that it is necessary to blind interviewers to applicants' academic performance to allow unbiased assessment^25,26. Fourth, there has been very limited study of medical course admission strategies incorporating applicants' field of undergraduate study or previous professional training.

The impact of components of the rural pipeline on choice of a rural medical career is strongly supported by the literature^27-30. The first stage of the rural pipeline, structured contact between rural secondary schools and the medical profession or medical schools, has been implemented in a variety of ways^31-34. For example, American Health Education Centres have operated health career promotion activities through their 'Pipeline to Practice' program with years K-12 and college students for many years³⁵. These approaches vary in their intensity of contact and programmatic structure. All claim to influence rural school students' career choices and pathways although only three of the four studies referenced here surveyed participants to support these claims.

There is an extensive literature on the second stage in the rural pipeline, selection of rural students into medical school programs. A period of rural residence of unspecified length prior to entry into medical school is the strongest predictor of a career in rural medicine after graduation^6,36-38. Acting on this finding, a number of medical schools have developed selection procedures with positive discrimination towards rural applicants, including quotas for rural applicants, adjusting selection scores according to rurality and separate selection scores for rural and remote applicants^39-42. There is evidence that positive discrimination can be undertaken in ways that maintain adequate academic entry standards and that the academic and clinical performance of these students during their medical training is satisfactory^26,43.

A range of other measures designed to increase the likelihood of selecting students who will choose a rural career have been reported, including weekends in rural locations prior to selection¹⁶, interview questions assessing understanding of rural issues⁴⁴, additional autobiographical statements⁴⁵ and referees' reports⁴⁶. Broadening the interview panel with rural community representatives and/or rural GPs has also been described¹⁷.

A number of reports support the efficacy of the third stage of the rural pipeline, rural experience during medical training, to promote choice of a career in rural practice. There is some evidence that exposure to rural practice may also influence urban students towards a rural career^47,48. The effect of rural exposure is claimed to be strongest for clinical placements in the later years of medical training⁴⁹ and for prolonged rural placements that are thought to increase opportunities for rural connectedness^40,50.

Obligatory bonding on completion of specialist or general practice training has recently been introduced as a strategy to increase the Australian rural medical workforce. There are scant research studies supporting this approach and its long-term effectiveness has been questioned^37,51.

Several authors have addressed the final stage of the rural pipeline, measures to improve retention of the rural medical workforce. Retention of rural medical practitioners is improved by professional support at national, state and local levels^38,52,53, and availability of career pathway opportunities⁵⁴. The practitioner's spouse's contentedness in rural communities⁵⁵, preparedness to adopt a rural lifestyle and success in connecting with the local community^55,56 are key factors affecting retention, as are concerns about educational opportunities for children and proximity to extended family and social circle^56,38. Medical schools have limited capacity to address many of these external issues.

Phase 2: Analysis of Australian graduate entry medical school selection methods

Selection officers and academic staff involved in student selection into Australian medical schools offering graduate entry programs in 2007 were interviewed. These staff were from the Australian National University, Flinders University, Griffith University, University of Melbourne, University of Notre Dame (Fremantle), University of Queensland, University of Sydney, University of Western Australia and University of Wollongong. Their responses are reported according to four themes which emerged during analysis of the interviews.

Selection into all 9 medical schools was based on a combination of academic performance in the undergraduate degree measured by GPA, performance in the Graduate Australian Medical Schools Admissions Test (GAMSAT) and an interview. All schools used GPA and GAMSAT scores to select applicants for interview. One school also requires applicants to submit a personal portfolio.

Theme 1: Rurality as a factor in selection of applicants for interview: All graduate entry schools reported that they meet the Australian Government Rural Undergraduate Support and Coordination (RUSC) scheme requirement that 25% of government supported places are awarded to applicants from rural locations (defined as 5 years residence in an RRMA 3-7 site). One school calculates a rurality score and a second is developing a rural index based on the RRMA classification and the Accessibility/Remoteness Index of Australia. The remaining universities do not attempt to quantify applicants' 'rurality', although the school using a personal portfolio includes consideration of suitability for rural practice in its assessment of the portfolio. Medical schools with designated rural cohort streams give preference to applicants with rural backgrounds.

Whether quantified or not, rurality comes into play after applicants are ranked according to their GPA and GAMSAT scores, and again after interview. The universities' selection officers adjust the pre- and post-interview rankings based on applicants' rural backgrounds in order to meet rural quotas. No university accepts rural applicants whose GPA or GAMSAT score is below a published minimum level.

Theme 2: Assessment of rurality in selection interviews: Two interview formats are used: 6 medical schools use a prolonged (usually 45 min) structured interview of each applicant by a panel of two to three trained interviewers; 3 use the MMI format, where applicants are interviewed by single interviewers at 8 to 10 stations, of 5 to 10 mins' duration, each addressing attributes thought to be important for medical practice. Similar attributes are assessed in the structured panel interviews. In both formats these attributes are assessed by responses to scenarios, case studies and direct questions. One school also uses a mock problem-based learning tutorial in addition to the MMI.

Six medical schools do not directly explore rurality or understanding of rural issues during interviews, although 3 report further exploration during the interview if applicants introduce rural-related experiences or intentions. Two medical schools that use the MMI include aspects of rurality in interview stations and, therefore, as a criterion for assessing applicants. The remaining medical school, which uses panel interviews, undertakes a more extensive exploration of rurality for applicants for an RCS cohort but not for the majority of applicants. One medical school delays any consideration of rurality until after interviews have been completed and short listing has occurred. Then a rurality-oriented questionnaire is sent to all candidates along with a request for preferences for the range of student places on offer, including RCS places and scholarships. Final selection decisions take into account applicants' responses.

Theme 3: Rural representation on interview panels: Five of the 9 medical schools include rural medical practitioners and/or non-medical rural community members on their interview panels. Interviewees commented that this can be challenging, particularly in states with larger geographic areas.

Theme 4: Rural experience during graduate entry medical courses: All medical schools offer an elective or compulsory rural placement in the initial campus-based years of their graduate-entry program. These placements range from 1 to 8 weeks. Students in the later clinical years are offered longer placements in a variety of ambulatory and hospital-based generalist and specialist rotations. Such placements can range from 6 weeks to 2.5 years. The longer placements are undertaken by students in designated rural streams or attached to RCSs.

Of the 9 universities sampled, four have rural medicine integrated across the entire course of study. Four universities conduct full or partial dedicated rural streams where the course of study is centred on rural medicine. The ninth university is planning a new course of study framework for 2008 that includes rural medicine.

In summary, all graduate entry medical schools offer short rural placements for mainstream students during the clinical years of the course and much greater rural exposure and variety for students undertaking a rural stream. Rural placements and rural streams build on the experience of shorter rural placements in the pre-clinical years. Such placement practices are consistent with the rural pipeline strategy, which aims to reinforce rurally oriented students' intentions towards rural practice at the time of enrolment, and to encourage students with metropolitan backgrounds to consider a rural medical career.

Phase 3: Interviews with Victorian interns

Structured interviews were held with 17 interns who had an average age of 26 years (range 23-32 years). Fifteen had completed undergraduate medical degree programs and nine had completed graduate entry programs. All were highly articulate and had well-considered views about the factors that had influenced their career aspirations. Further background details of the interns are included (Table 1).

Table 1: Backgrounds of intern interviewees



Review of interns' life histories and reflections

The review of the participating interns' life histories and reflections on rural practice identified a mix of influential factors that can be broadly divided into pre-entry biographically related factors, current non-medical training biographically related factors and medical training related factors.

Pre-entry biographically related factors are disposition forming in the sense of predisposing medical students towards rural medicine at admission to their program. From the interview data available, rural residency prior to commencing medical studies is a powerful disposition forming factor, particularly rural residency outside of country towns. In addition to rural residency, participation in country-oriented outdoors recreational activities is a further disposition forming factor. A concomitant factor is a full school education in the a rural area.

Current non-medical training biographically related factors may further enhance 'upon admission' rural dispositions or, alternatively, may over-ride rural dispositions and set an individual medical student on a metropolitan career direction. Among the factors at play here for a rurally disposed medical student, the choice of a partner is critical. If the partner has a rural residency background and a career that can be co-located in the country, then the medical student's entry disposition may well be further enhanced. Whether or not the partner has a rural residency background, if the partner is secured in a city-based lifestyle and occupation, then this has the strong potential to over-ride a medical student's disposition towards rural medicine.

Additional factors in this category are family, friends and lifestyle, and perceptions of country lifestyles. The impact of these factors is conditional on the disposition of the medical student towards country practice. If family, friends and lifestyle preferences are, in the main, country located then these factors reinforce a student's rural disposition. If family and friends are more city located then students with strong rural dispositions seem to find ways to cope. A positive perception of country lifestyles came through the review of interviewee responses as significant. Interns with non-rural residency backgrounds seemed to need positive perceptions of country lifestyles in order to advance upon their developing positive dispositions about rural medicine.

The third category of factors influencing medical students' career choices includes those related to their medical training experiences. These within-medical program factors are both disposition enhancing and disposition forming. For students with entry dispositions favouring rural medicine, positive educative and social experiences during rural rotations can be disposition strengthening. For students with weak or negative dispositions towards rural practice on admission, these rural rotations, if extended, well organized and resourced, can be life- and career-changing experiences. Rural rotations through the medical program and within the internship year were clearly the most significant rural dispositional development factor in this category. Medical students' perceptions varied depending on their initial dispositions, but the extended rural rotations available through RCSs had a far greater impact than short-term rotations offered by MCSs. However, RCS rotations must be carefully structured to provide the powerful 'work-based' medical learning outcomes reported by the interns in this study.

An additional positive rural-orienting factor was shorter-term placements with rural GPs. Medical training experiences that expose students in positive ways to general practice appear, from the interview data, to correlate with a rural medicine career choice at the internship stage. Rotations that do not provide well-organized, work-based medical learning may introduce a degree of negativity into a medical student's perception of rural medicine.

Finally, rural rotations within the internship itself were also a significant factor within this third category. Several interviewees with a current positive disposition towards rural practice identified their extended, collegiate placements at a regional hospital or their rotation to a small community hospital as powerful affirming or career-changing experiences.

Thematic analysis of interviews of interns

Review of the interns' biographical details and their reflections on rural practice identified a number of influential factors that can be divided broadly into pre-entry personal factors, personal factors impacting on training, and medical training factors.

Three themes were identified from analysis of the interviews. Given the limited size of the sample, extrapolation of the conclusions should be undertaken with caution. However, the themes overlap with and reinforce the conclusions drawn from the first two phases of this study.

Theme 1: The efficacy of the rural pipeline: The rural pipeline, from recruitment of applicants with experience of rural communities through to the internship year, appears to have educative and social outcomes that favour a career in rural medicine. Interns who have lived in rural communities, attended rural schools and participated in rurally based recreational activities before entering medical school, who then enrol in a medical training program with an extended, well-organized RCS culminating in an internship year which includes an extended rural hospital rotation, were more likely to express an intention to become rural practitioners.

Medical students with no previous rural exposure who enter the rural pipeline during their training program, can be influenced to choose a career in rural practice. The impact appears to be greater for students attached to RCSs but the rural pipeline can also be entered at the internship stage.

Theme 2: Community connectedness through the rural pipeline: Extended rotations are important to establish connectedness with medical and other professional communities in rural locations. Extended rotations during medical courses are only available through RCSs.

Theme 3: Impediments to the effect of the rural pipeline: Impediments to choosing a career in rural practice can emerge during training or can be external to the rural pipeline. Interviewees stated that the most significant impediment during training was exposure to poor clinical learning experiences: for example, seemingly irrelevant material, less supportive clinical teaching staff, or tension in the workplace. A perception of limited career opportunities arising from rural medical training was also noted.

However, most interns commented that external factors were more significant, particularly living with a partner who is not committed, or able, to work in a rural site and/or is committed to a metropolitan lifestyle. Other less significant barriers were a personal preference for a metropolitan rather than a rural lifestyle, and isolation from friends and family.

Discussion

The three phases of this study (a review of the literature; interviews of staff involved in selection into Australian graduate entry medical schools; and interviews of interns who have recently graduated from Victorian medical schools) provide evidence to support the use of the rural pipeline to address Australia's regional and rural medical workforce shortage.

The first and third phases of the study support the widely held view that rural residency prior to selection into a medical course is the strongest predictor of working in rural or regional settings after graduation. This finding has been incorporated into the selection procedures of Australian graduate entry medical schools; all reserve 25% of government supported places for applicants who have lived for an extended period in a RRMA 3-7 location. The intern interview phase of the study suggests that the predictive value of prior rural residency is further strengthened by rural or regional schooling and by interest in rural-oriented recreational pursuits.

The second phase of the study reveals that some, but not all, graduate entry medical schools include rural representatives on selection interview panels; a minority explore rurality or understanding of rural issues during the interviews.

There is currently no definitive longitudinal information on the effect of selection methods on eventual choice of a rural career.

Australian graduate entry medical school curricula include exposure to rural medicine during short placements in the pre-clinical years and longer placements or rural streams during the clinical years. Taken together, the first and third phases of the study suggest that the following features of a rural clinical placement have a positive impact on students' and interns' choice of a career in rural medicine:

• extended, collegiate and well-coordinated clinical placements in rural workplaces for medical students, particularly placements in RCS
  
• internships in regional hospitals or regional hospital term rotations for metropolitan hospital interns
  
• student and intern perception of a supportive approach from supervisors and teaching staff as graduates take on increased responsibilities
  
• opportunities for medical students and interns to interact with local health professionals during rural placements
  
• opportunities for pursuing a career interest in general practice or other specialist training while working in rural or regional settings after graduation.

The following appear to have a negative impact on students' and interns' choice of a career in rural medicine:

• student and intern perception of lack of support from supervisors and/or teaching staff (most commonly because of workload pressures)
  
• perceived professional and personal tensions between health professionals in rural workplaces
  
• a belief that rural placements limit career options
  
• preference for a metropolitan lifestyle and perceived isolation from metropolitan-based family and friends
  
• most significantly, a partner who is not committed or able to work in a rural site and/or is committed to a metropolitan lifestyle.

Conclusion

This study provides support from three intersecting sources of data for a range of strategies to increase the number of medical graduates choosing a career in rural medicine. This 'triangulation' methodology adds weight to the recommended strategies in a field in which the strength of evidence for the efficacy of interventions is limited. A number of these strategies are already in place in the medical schools participating in the study and in other Australian and international medical schools. Most would require increased resources, including an expansion of teaching staff and clinical placements in rural and regional settings.

The suggested strategies are listed as components of each stage of the 'rural pipeline' into medical practice.

Stage 1: Contact between rural secondary schools and the medical profession:

1. Development of programs to promote medical careers to rural secondary school students. These programs should incorporate follow up during the later years of secondary school, initially targeting middle secondary school students so that subject choices during years 10 to 12 are appropriate for prerequisite study pathways for medicine.

Stage 2: Selection of rural student into medical programs:

2. Adoption of the rural pipeline concept to guide medical student selection processes.
3. Development of a rurality index for use in selection into medical schools, based on the nature and duration of applicants' rural and regional residency and attendance at rural and regional schools and universities.
4. Inclusion of rural medical practitioners and community representatives on interview panels and incorporation of rural issues in the content of selection interviews.

Stage 3: Rural exposure during medical training:

5. Progressively longer rural placements during medical courses, culminating in full year rotations in the clinical years. Ideally there should be opportunities for all students based in metropolitan settings to be exposed to rural medical practice. As this may not be possible following the recent expansion of Australian medical schools, it may be more appropriate to provide this rural exposure to those most likely to choose a rural career.
6. Provision of support for clinical teachers and clinical supervisors to improve the quality of medical students' learning experiences during rural rotations.
7. Activities to increase awareness of rural lifestyle issues, particularly during the later years of medical course, addressing issues relevant to rural medical practice including housing, regional schools, professional and social networks, and cultural and recreational opportunities.
8. Promotion of interaction of medical students and interns with rural community members and professional groups, including regular social activities.
9. Promotion of rural medicine career opportunities during or after rural rotations by provision of opportunities to discuss rural medical practice with rural clinicians and/or careers counsellors.
10. Expansion of internships and intern rotation terms in rural and regional hospitals. These positions will only be successful if interns are provided with clinical experience, educational support and supervision at least equivalent to metropolitan intern posts.
11. Expansion and improved support of general practice and other specialist training programs in rural and regional sites. These positions will only be successful if trainees are provided with clinical experience, educational support and supervision at least equivalent to metropolitan training posts.

Stage 4: Measure to address retention of the rural medical workforce:

12. Improved support for established rural practitioners, including access to continuing professional development and specialist advice, and measures to provide cover during periods of absence.

Acknowledgements

The authors acknowledge the Victorian Department of Human Services and Department of Innovation, Industry and Regional Development for jointly funding this research. The authors also thank the medical school staff and interns who made themselves available for interview.

References

1. Australian Government Department of Health & Ageing. Review of the Rural, Remote, and Metropolitan Areas (RRMA) Classification. Discussion paper. Canberra, ACT: AGDHA, 2005.

2. Australian Institute of Health and Welfare. Rural, regional and remote health: a guide to remoteness classifications. Canberra, ACT: AIHW, 2004.

3. Tesson G, Curran V, Pong R. Advances in rural medical education in three countries: Canada, the United States and Australia. Education for Health: Change in Learning & Practice 2005; 18(3): 405-415.

4. Council on Graduate Medical Education. Tenth report, physician distribution and health care challenges in rural and inner-city areas. Rockville, MD: CGME, 1998; 23.

5. Henry J, Edwards B, Crotty B. Assessing best practice in the selection of students for graduate medical programs: Final report. Melbourne, VIC: Deakin University, 2008.

6. McDonald J, Bibby L, Carroll S. Recruiting and retaining general practitioners in rural areas: improving outcomes through evidence-based research and community capacity-building. Evidence-based review: final report. University of Ballarat, VIC: Centre for Health Research and Practice, 2002.

7. Curran V, Bornstein S, Jong M, Fleet L. Rural medical education: a review of the literature. Component 1 of strengthening the medical workforce in rural Canada: the role of rural/northern medical education. Newfoundland: Faculty of Medicine, Memorial University of Newfoundland, 2004. Available: http://www.cranhr.ca/pdf/Rural_Med_Ed_Literature_Synthesis_FINAL_DRAFT_June_2004.pdf (Accessed 8 March 2007).

8. Ferguson E, James D, O'Hehir F, Sanders A. Pilot study of the roles of personality, references, and personal statements in relation to performance over the five years of a medical degree. BMJ 2003; 326(7386): 429-432.

9. Cohen-Schotanus J, Muijtjens A, Reinders J, Agsteribbe J, van Rossum H, van der Vleuten CP. The predictive validity of grade point average scores in a partial lottery medical school admission system. Medical Education 2006; 40(10): 1012-1019.

10. Salvatori P. Reliability and validity of admissions tools used to select students for the health professions. Advances in Health Sciences Education. Theory Practice 2001, 2006; 6(2): 159-175.

11. Peskun C, Detsky A, Shandling M. Effectiveness of medical school admissions criteria in predicting residency ranking four years later. Medical Education 2007; 41(1): 57-64.

12. Salafsky BM, Glasser M, Ha J. Addressing issues of maldistribution of health care workers. Annals Academy of Medicine Singapore 2005; 34(8): 520-526.

13. Eva K, Reiter H, Rosenfeld J, Norman G. The ability of the multiple mini-interview to predict preclerkship performance in medical school. Academic Medicine 2004; 79(10 Suppl): S40-42.

14. Harris S, Owen C. Discerning quality: using the multiple mini-interview in student selection for the Australian National University Medical School. Medical Education 2007; 41(3): 234-241.

15. Courneya CA, Wright K, Frinton V, Mak E, Schulze M, Pachev G. Medical student selection: choice of a semi-structured panel interview or an unstructured one-on-one. Medical Teacher 2005; 27(6): 499-503.

16. Flinders University. Flinders University Parallel Rural Community Curriculum: a proven model of successful medical education in Australia. (Online) 2006. Available: http://www.dest.gov.au/NR/rdonlyres/A6390CB4-F0AD-42A8-81D2-921AD7B9A7CE/14086/Submission49.pdf (Accessed 4 March 2007).

17. Gupta T, Grant M, Alberts V, McKenzie A, Veitch C, Murphy et al. The role of medical schools in recruitment and retention: rural, remote, Indigenous & tropical health at the James Cook University School of Medicine. In: Proceedings, ACRRM 2002 Scientific Forum. Available: http://www.acrrm.org.au/downloads/Forum 02 Presented/Speaker 05-Sen Gupta final.doc (Accessed 10 March 2007).

18. Turnbull D, Buckley P, Robinson JS, Mather G. Increasing the evidence base for selection for undergraduate medicine: four case studies investigating process and interim outcomes. Medical Education 2003; 37(12): 1115-1120.

19. Rippentrop AE, Wong MYS, Altmaier EM. A content analysis of interviewee reports of medical school admissions interviews. Medical Education Online 2003; 8(10). Available: http://www.med-ed-online.org/res00063.htm (Accessed 2 April 2007).

20. Powis DA, Bore M, Munro D. Selecting medical students: evidence based admissions procedures for medical students are being tested. BMJ 2006; 332(7550): 1156.

21. Lumsden MA, Bore M, Millar J, Jack R, Powis D. Assessment of personal qualities in relation to admission to medical school. Medical Education 2005; 39(3): 258-265.

22. Lievens F, Buyse T, Sackett PR. The operational validity of a video-based situational judgment test for medical college admissions: illustrating the importance of matching predictor and criterion construct domains. Journal of Applied Psychology 2005; 90(3): 442-452.

23. Cullen M, Sackett P, Lievens F. Threats to the operational use of situational judgment tests in the college admission process. International Journal of Selection and Assessment 2006; 14(2): 142-155.

24. Kyllonen P, Walters AM, Kaufman JC. Noncognitive constructs and their assessment in graduate education: a review. Educational Assessment 2005; 10(3): 153-184.

25. Shaw DL, Martz DM, Lancaster CJ, Sade RM. Influence of medical school applicants' demographic and cognitive characteristics on interviewers' ratings of noncognitive traits. Academic Medicine 1995; 70(6): 532-536.

26. Longo DR, Gorman RJ, Ge B. Rural medical school applicants: do their academic credentials and admission decisions differ from those of nonrural applicants? Journal of Rural Health 2005; 21(4): 346-350.

27. Strasser R. Attracting future rural practitioners to health care. In, Proceedings, Transforming Rural Practice Through Education: 7th WONCA Rural Health Conference. 8-15 September 2006; Seattle, WA, USA. 2006.

28. Hsueh W, Wilkinson T, Bills J. What evidence-based undergraduate interventions promote rural health? New Zealand Medical Journal 2004; 117(1204): U1117.

29. Murray RB, Wronski I. When the tide goes out: health workforce in rural, remote and Indigenous communities. Medical Journal of Australia 2006; 185(1): 37-38.

30. Rabinowitz HK, Paynter NP. The role of the medical school in rural graduate medical education: pipeline or control valve? Journal of Rural Health 2000; 16(3): 249-253.

31. Eley R, Hindmarsh N, Buikstra E. Informing rural and remote students about careers in health: the effect of Health Careers Workshops on course selection. Australian Journal of Rural Health 2007; 15(1): 59-64.

32. Bly J. What is medicine? Recruiting high-school students into family medicine. Canadian Family Physician 2006; 52: 329-334.

33. University of Washington School of Medicine. What is the U-DOC summer program? Seattle, WA: Office of Multicultural Affairs, 2007. Available: http://depts.washington.edu/omca/UDOC/ (Accessed 22 March 2007).

34. Fraser J, Alexander C, Simpkins B, Temperley J. Health career promotion in the New England area of New South Wales: a program to support high school career advisers. Australian Journal of Rural Health 2003; 11(4): 199-204.

35. Sienkiewicz MJ, NYS AHEC System Sustains Record Outreach. New York State Area Health Education Center Newsletter. September 2008. Available: http://www.ahec.buffalo.edu/newsletters/Monthly%20Newsletters/Sep08_AHEC_Nwsltr_final1.pdf (Accessed 16 January 2009).

36. Laven G, Wilkinson D. Rural doctors and rural backgrounds: how strong is the evidence? A systematic review. Australian Journal of Rural Health 2003; 11(6): 277-284.

37. Curran V, Rourke J. The role of medical education in the recruitment and retention of rural physicians. Medical Teacher 2004; 26(3): 265-272.

38. Richards HM, Farmer J, Selvaraj S. Sustaining the rural primary healthcare workforce: survey of healthcare professionals in the Scottish Highlands. Rural and Remote Health 5: 365. (Online) 2005. Available: www.rrh.org.au (Accessed 25 February 2008).

39. Florence J, Goodrow B, Wachs J, Grover S, Olive K. Rural health professions education at East Tennessee State University: survey of graduates from the first decade of the community partnership program. Journal of Rural Health 2007; 23(1): 77-83.

40. Wilkinson D, Laven G, Pratt N, Beilby J. Impact of undergraduate and postgraduate rural training, and medical school entry criteria on rural practice among Australian general practitioners: national study of 2414 doctors. Medical Education 2003; 37(9): 809-814.

41. Gunn L. Diversity within diversity: equity programmes in eight Australian universities. Perspectives: Policy and Practice in Higher in Education 2003; 7(3): 83-87.

42. Basco W, Gilbert G, Blue A. Determining the consequences for rural applicants when additional consideration is discontinued in a medical school admission process. Academic Medicine 2002; 77(10 Suppl): S20-S22.

43. Hutten-Czapski P, Pitblado R, Rourke J. Who gets into medical school? Comparison of students from rural and urban backgrounds. Canadian Family Physician 2005; 51: 1240-1241.

44. Kreiter C, Solow C, Brennan R, Yin P, Ferguson K, Huebner K. Examining the influence of using same versus different questions on the reliability of the medical school preadmission interview. Teaching and Learning in Medicine 2006; 18(1): 4-8.

45. Dore K, Hanson M, Reiter H, Blanchard M, Deeth K, Eva K. Medical school admissions: enhancing the reliability and validity of an autobiographical screening tool. Academic Medicine 2006; 81(10 Suppl): S70-S73.

46. McManus IC, Iqbal S, Chandrarajan A, Ferguson E, Leaviss J. Unhappiness and dissatisfaction in doctors cannot be predicted by selectors from medical school application forms: a prospective, longitudinal study. BMC Medical Education 2005; 13(5): 38.

47. Chan B, Degani N, Crichton T, Pong R, Rourke J, Goertzen J et al. Factors influencing family physicians to enter rural practice: does rural or urban background make a difference? Canadian Family Physician 2005; 51: 1246-1247.

48. Tolhurst HM, Adams J, Stewart SM. An exploration of when urban background medical students become interested in rural practice. Rural and Remote Health 6: 452. (Online) 2006. Available: www.rrh.org.au (Accessed 25 February 2008).

49. Somers G. Do rural mentors, rural undergraduate clubs and rural rotations increase the medical student's intention to practise in the country? In: Proceedings, 2002 ACRRM Academic Scientific Forum. Available: http://www.acrrm.org.au/downloads/Forum%2002%20Presented/Speaker%2014%20-%20Somers%20final.doc (Accessed 13 March 2007).

50. Denz-Penhey H, Shannon S, Murdoch C, Newbury J. Do benefits accrue from longer rotations for students in Rural Clinical Schools? Rural and Remote Health 5: 414. (Online) 2005. Available: www.rrh.org.au (Accessed 25 February 2008).

51. Productivity Commission. Australia's Health Workforce, Research Report. Canberra, ACT: Productivity Commission, 2006.

52. Eley D, Young L, Shrapnel M, Wilkinson D, Baker P, Hegney D. Medical students and rural general practitioners: congruent views on the reality of recruitment into rural medicine. Australian Journal of Rural Health 2007; 15(1): 12-20.

53. Wainer J, Bryant L, Strasser R. Sustainable rural practice for female general practitioners. Australian Journal of Rural Health 2001; 9(Suppl 1): S43-S48.

54. Schoo AM, Stagnitti KE, Mercer C, Dunbar J. A conceptual model for recruitment and retention: allied health workforce enhancement in Western Victoria, Australia. Rural and Remote Health 5: 477. (Online) 2005. Available: www.rrh.org.au (Accessed 25 February 2008).

55. Mayo E, Mathews M. Spousal perspectives on factors influencing recruitment and retention of rural family physicians. Canadian Journal of Rural Medicine 2006; 11(4): 271-276.

56. Heng D, Pong RW. The More things change, the more they stay the same: Key factors influencing choice of practice locations of family physicians at four different career points. In: Proceedings, Northern Health Research Conference; 2-3 June 2006; Sault Ste Marie, Ontario. 2006.