The influence of loan repayment on rural healthcare provider recruitment and retention in Colorado

full article:

Introduction

Rural communities worldwide are experiencing an ongoing shortage of healthcare providers¹. Many countries have been found to have lower healthcare provider or physician densities in their rural communities than in their urban communities, including Tanzania², India³, Nicaragua⁴, Australia⁵, and Japan⁶. The USA is experiencing similar rural healthcare provider shortages^7-11. In 2000, over half of all US frontier counties with hospitals were classified as nurse shortage counties, while only 30% of non-frontier counties carried this designation⁹; the population density of dentists in non-metropolitan areas is less than half of what it is in metropolitan areas¹²; and for every 100 000 people in 2005, there were 210 physicians in urban locations but only 52 in more remote rural areas⁸. This disparity is likely to worsen, because a greater proportion of generalist physicians are nearing retirement in rural US counties, compared with urban counties¹³.

Medical educators and healthcare policymakers have sought strategies to promote recruitment and retention of the rural healthcare workforce. Several provider characteristics and systems interventions have been linked with rural practice choice, most notably being raised in a rural area^14-23, exposure to rural areas during medical training^14,24-32, and being offered financial incentives to work in a rural community^33-38. Financial incentives have become a popular intervention to enhance recruitment, given the rising debt levels of US medical graduates (the median debt level of indebted 2008 medical school graduates was US$155,000, up 53% from 1998 after inflation adjustment)³⁹. Examples of such incentives include direct financial incentives, resident support, scholarships, and loan repayment programs (LRPs), all with or without associated service commitments³¹.

For the past three decades, LRPs have increased in popularity as a means to enhance recruitment and retention in medically underserved areas, including rural communities³⁴. This trend has been attributed to both the rising debt load of medical graduates and the tendency of these incentives to target providers in the stage of their career when they are ready to make new commitments³⁴. Indeed, several studies have reported medical students and residents' willingness to consider changing their field of practice or clientele if offered loan repayment^40-43. Fewer studies have retrospectively correlated the influence of loan repayment on actual decisions involving practice location^14,33,44. To our knowledge, no studies have followed loan repayment recipients to investigate the motivations and attitudes that led them to their current practice location and to assess the influence of these values on provider retention.

The purpose of this study was to investigate the effects of loan repayment on recruitment and retention of healthcare providers in rural versus urban communities in Colorado. We hypothesized that LRPs would be influential in recruitment of providers to rural areas but ineffective at retention beyond the service commitment or loan repayment period.

Methods

This study was a retrospective cohort study of the participants of three Colorado LRPs. It was approved by the Colorado Multiple Institutional Review Board as an exempt protocol with a waiver of informed consent.

From July to October 2007, a survey was administered through the Colorado Rural Health Center (CRHC) to all healthcare providers who had most recently participated in any of three Colorado LRPs, chosen because they were the major LRPs in Colorado and had existing relationships with the CRHC. The three Colorado LRPs were the:

1. Colorado Health Professional Loan Repayment Program (CHPLRP)
   
2. Colorado Rural Outreach Program (CROP)
   
3. Dental Loan Repayment Program (DLRP)

The CHPLRP, beginning in 1992, rewards physicians, dentists, physician assistants (PAs), nurse practitioners (NPs), certified nurse midwives, and mental health specialists with up to $35,000 per year of educational loan repayment (with a maximum of $70,000) for working in any rural or urban Health Professional Shortage Area (HPSA) in Colorado⁴⁵ with a minimum two-year service commitment. The program is funded federally with community dollar-for-dollar matching, and the funds are distributed by Colorado Area Health Education Center (AHEC)^46,47. Surveys were mailed to all providers enrolled between 1992 and 2007.

Colorado Rural Outreach Program

The CROP, created in 1998, currently rewards all types of healthcare professionals (including but not limited to physicians, NPs, PAs, nurses, providers of mental health services, providers of dental health services such as dentists and dental hygienists, allied health professionals such as lab or radiology technicians, and pharmacists) up to $10,000 per year of educational loan repayment for up to 3 years. The provider must be working in a rural community in Colorado to be eligible, and the program requires a one-year service commitment. Funding comes from private donations, grants, and community matching, and distribution changed from the Colorado Medical Society Foundation to the Colorado Rural Health Center (CRHC) in 2005^46,47. Thus, surveys were mailed to all providers enrolled between 2005 and 2007.

Dental Loan Repayment Program

The DLRP of Colorado, beginning in 2002, rewards dentists up to $25,000 per year and dental hygienists up to $6,000 per year up to the amount of the provider's full outstanding educational loans. The providers must work with underserved populations in either rural or urban areas to be eligible. The program is state funded, and awards are distributed by the Colorado Department of Public Health and Environment (CDPHE). There is no service commitment^46,47. Surveys were mailed to all providers enrolled between 2002 and 2007.

The survey

The survey was mailed to all subjects and returned via mail or fax. Non-responders were mailed a second survey, emailed, and phoned twice before considered 'missed'. Subjects for whom contact information could not be obtained were excluded from the study. Respondents were asked to provide demographic information and to identify why they decided to participate in a LRP, if they were planning on going to similar communities if no loan repayment had been offered, and if they stayed in their initial communities after their loan repayment awards were received. Five-point Likert scales were used throughout the survey for the respondents to rate the relative importance of factors such as loan repayment and various community characteristics on their decision making (1='not important' and 5='very important'). The importance of such factors was assessed for decisions such as where to practice during LRP enrollment as well as why to leave the community after loan repayment awards were received, if applicable. Other data such as amount of educational debt upon enrolling in a LRP, year of graduation from training, and the length of time the participant expected to stay in the community in which they enrolled in a LRP was collected by write-in responses.

Respondents were also asked to indicate where they attended high school, locations of previous practice, and their current practice location. Rural and urban locations in the USA were quantified and differentiated using the Rural Urban Commuting Area Codes (RUCA), developed by the University of Washington's Rural Health Research Center and the Economic Research Service (ERS)⁴⁸. These codes differentiate rural and urban census tracts using the standard Bureau of Census Urbanized Area and Urban Cluster definitions in combination with work commuting information (v 2.0 used in this analysis). For this study, 'rural' was defined as any zip-code with a RUCA designation above and including 4.0 (to include both the 'large rural' and 'small rural' categories of the RUCA coding system), and 'urban' was defined as any zip-code with a designation below 4.0.

Surveys for which greater than 50% of the questions were left blank were excluded from the analysis (and considered as non-responders).

Statistical analysis

All analyses were performed using STATA 10.1 (StataCorp; College Station, TX). Data are presented using descriptive statistics. Respondents were classified by type of provider (physician, dentist, or non-physician, with registered dental hygienists being considered non-physicians), and by LRP practice location (rural participant or urban participant). The LRP practice location represents the type of community the provider practiced in during enrollment in the LRP, which is not necessarily indicative of the current practice location of the provider. Additionally, Likert scores of 4 and 5 were pooled and simply labeled as 'important' in the results. Chi-square testing was used to compare differences among the survey responses of relevant groups. Two-tailed p<0.05 was considered statistically significant.

Results

Those surveyed were 46 CROP participants (27 physicians, 1 dentist, and 18 non-physicians), 42 CHPLRP participants (27 physicians, 2 dentists, and 13 non-physicians) and 52 DLRP participants (39 dentists and 13 non-physicians). Included were two participants who participated in more than one program (one CROP and CHPLRP, the other CHPLRP and DLRP). Of 138 total LRP participants, 16 were unable to be located. Of the 122 surveys that were sent, 97 were returned (80% response rate). Of these 97, four surveys (all CROP participants) were excluded from the analysis due to the respondent's lack of recollection of participating in a LRP and thus inability to complete the survey.

Of those unable to be located, 3 were CROP participants, 5 were CHPLRP participants, and 8 were DLRP participants; 2 were physicians, 5 were dentists, and 8 were non-physicians. Of those who didn't respond, 9 were CROP participants, 10 were CHPLRP participants, and 7 were DLRP participants; 11 were physicians, 8 were dentists, and 7 were non-physicians.

The characteristics of the remaining 93 respondents are summarized (Table 1). The two professions most heavily represented in our data are physicians and dental professionals. The rural participants were mostly physicians (31; 54%), while the urban participants were mostly dentists (22; 61%).

Table 1: Demographics of survey respondents



Motivational factors

Over half of all survey respondents (52; 56%) reported they were already working in an eligible community prior to LRP application. Further, an additional 17 (18%) of all providers were planning to work in a specific community eligible for loan repayment when they became aware of the program. Thus, only 24 (26%) of the respondents were not already working in or intending to work in an eligible community when they became aware of the LRP. Nineteen (79%) of these respondents were physicians.

Overall, 32 (34%) reported having a desire to serve underserved populations as a primary reason in choosing to enroll in a LRP. Given the opportunity to write-in other motivations for enrolling in a LRP, 9 (24%) of the physicians cited 'money' or 'debt'.

Recruitment

The importance of several factors in the providers' choices of specific loan repayment-eligible community is shown, stratified by location and provider type (Table 2). Overall, the most important factors were the location of the community, scope of practice, and fit between family and community. Rural participants rated location, scope of practice, and family fit with the community as the most important factors; while urban participants rated location, salary, and scope of practice as important most often. Almost all (29; 94%) rural physicians rated scope of practice as an important factor.

Several questions addressed the effects that loan repayment had on decision-making. Excluding respondents already working in an eligible community when they applied for the LRP, 22 (69%) of those who enrolled in a LRP in a rural community reported that the opportunity for loan repayment was an important influence on their choice of where to practice; however, 21 (66%) reported that they planned to work in a rural community regardless of any loan repayment option. Of the providers who planned to practice in a rural community, 8 (38%) cited the LRP as having an important influence on their choice of the specific rural community in which to practice.

Rural participants were more likely to have gone to high school in a rural area than urban participants (21 [38%] vs 3 [9%]; p=0.007). Of the providers who stated they were planning to practice in a rural area regardless of any loan repayment option, 16 (38%) went to high school in a rural area.

Table 2: Numbers of providers rating the following factors as important in choosing where to work



Retention

Of the 66 providers who had fulfilled their terms of service at the time of the survey, 30 (45%) had left their original community; 9 of these were urban participants and 21 were rural participants. Of these who left, 47% (4 urban, 10 rural) had stayed 0-1 additional years beyond their obligation; 20% (2 urban, 4 rural) stayed 2-4 years, and 33% (3 urban, 7 rural) stayed 5 years or longer. Of the 15 physicians who had left, 6 (40%) stayed 0-1 years, 4 (27%) stayed 2-4 years, and 5 (33%) had stayed 5 years or longer.

Of the 36 (55%) providers who were still at their original site after completing their terms of service, 15 were urban participants and 21 were rural participants. Of these who stayed, 42% (5 urban, 10 rural) had stayed 0-1 additional years, 28% (2 urban, 8 rural) had stayed 2-4 years, and 31% (8 urban, 3 rural) had stayed for 5 years or longer. Of the 12 physicians who had stayed, 4 (33%) had stayed 0-1 additional years, 4 (33%) had stayed 2-4 years, and 4 (33%) had stayed 5 years or longer.

Of the providers who had completed their service commitment at the time of the survey, 27 (64%) of the rural participants were still practicing in a rural community and 23 (96%) of the urban participants were still practicing in an urban community. For those rural participants who stayed in rural communities, 11 (41%) said the LRP was an important factor in their decision to do so.

Of the 23 rural participants finished with their commitment who were already working in an eligible community when they enrolled in a LRP, 21 (91%) had remained in that practice at the time of the survey, and 8 (38%) reported that participation in a LRP was important in their decision to stay.

Thirty-eight of the rural participants who had completed their commitment specified on the survey where they had attended high school. Of the 25 participants who had attended urban high schools, 16 (64%) were currently practicing in a rural community; while, of the 13 participants who attended rural high schools, 10 (77%) were currently practicing in a rural community. This difference in current rural practice rates between the urban and rural high school groups was not significant (p=0.42).

Thirty-nine of the rural participants who had completed their commitment answered whether or not they were initially planning on practicing in a rural community regardless of any loan repayment option. Of the 6 who answered they were not, 3 (50%) were practicing in rural communities at the time of the survey. Of the 33 who answered that they were, 21 (64%) were practicing in rural communities.

When asked to recall how long they initially intended to stay in the community when they enrolled in a LRP, 29 (31%) of the respondents reported 'indefinitely', with no difference between rural and urban participants or physicians and non-physicians (data not shown).

The 41 respondents who eventually left the community in which they received their loan repayment awards were asked to rate various factors on their decisions to leave (Table 3). The factors most frequently rated as important were 'family's desire to move' for rural participants and 'desire for a higher income' for urban participants (however, notably, two-thirds of the urban participants who desired higher incomes were dentists). Physicians most often rated their family's desire to move as an important factor, while dentists most often desired a higher income.

Table 3: Of providers who left the community where they enrolled in a loan repayment program, numbers rating the following factors as important in their decisions to leave



Discussion

Influence of loan repayment on rural recruitment and retention

Intuitively, financial incentives such as loan repayment given to healthcare providers should influence where they choose to practice, especially considering the high costs of education currently. Two-thirds of the physicians responding to the survey reported educational debt loads of more than $100,000, with over one-third more than $150,000; dentists were statistically similar. Prior studies have shown there to be a significant influence of loan repayment on rural physician recruitment^14,44, while several more have shown that medical students and residents are interested in financial incentives, and many would consider serving underserved populations in exchange for loan repayment^40-43,49.

The present data are consistent with these reports, because a majority of rural participants cited loan repayment as being an important factor in deciding where to practice. However, 21 (66%) of the rural participants said they were planning on practicing in a rural area regardless of whether they received loan repayment. These proportions do not even include the 52 providers who enrolled in a LRP while already working at an eligible practice. Thus, it appears from our survey that the vast majority of loan repayment awards for rural Colorado are awarded to providers who would have been likely to practice in a rural community regardless of any loan repayment distribution. However, although most rural participants planned on practicing in a rural community, 8 (38%) still reported that loan repayment had an important influence on the specific rural community in which they chose to practice. This suggests that while LRPs may only have a limited influence on the recruitment of providers to rural Colorado in general, rural communities that offer loan repayment may attract more providers than those that do not.

The present data suggest that LRPs may be more important in rural provider retention. The vast majority of newly recruited rural participants stayed in rural communities for some time after their LRP was complete, and 11 (41%) of these reported that the LRP was important in their decision to do so. The LRP also appears to have had a significant influence on the providers who were already working at an eligible rural practice when they signed up for the program; 8 (38%) of these providers also said that the LRP was important in their decision to stay. The influence of LRPs on retention has been previously supported in the literature, and it has been shown that they may have higher retention rates than many other types of financial incentive programs³¹. A study of Maine nurses also revealed that 65% considered loan repayment an important reason for staying at their current practices⁵⁰.

Influences of rural upbringing

Rural upbringing has been correlated with higher rates of physician recruitment to rural community practice^14-23,25. Regarding LRPs in Colorado, the present data shows that while 21 rural participants (38%) came from rural high schools, only 3 urban participants (9%) came from rural high schools. Though the present data agrees that there is some influence of rural upbringing on likelihood to practice in a rural area, the majority of rural participants came from urban high schools. Thus, the survey results suggest that rural recruitment strategies should not be limited solely to providers with rural upbringing.

These data also showed that rural or urban upbringing had no influence on the likelihood of retention in a rural area. The lack of correlation between rural retention and rural upbringing has been supported by some^15,51 but not all prior studies¹⁸.

Other factors influencing recruitment

There were differences between rural and urban participants in the importance of some factors involving recruitment. Rural participants placed more importance on their family's fit with the community as well as the opportunity for recreation, while urban participants placed more importance on having friends and colleagues in the area. Spousal influences may factor heavily on rural provider recruitment^25,44,52,53, but multiple authors have argued against any correlation between marital status and rural recruitment^15,20. The present data suggest family influences are important, but few respondents rated their spouse's job as an important factor in practice location decision-making.

Physicians and non-physicians also rated several factors much differently in terms of their importance in the decision of where to practice. Notably, physicians nearly universally rated scope of practice as important, a trend that has been previously reported^53-55.

Other factors influencing retention

Most of the LRP recipients in our study were still practicing in the communities where they received their loan repayment awards. However, the retention of participants in rural communities has been less than that of participants in urban communities. After enrollment in a LRP has ended, rural participants were more likely to move to urban areas than vice versa.

Of the rural participants who changed practice after their LRP enrollment was over, 11 (48%) reported that their family's desire to move was important in their decision. The next most important factors were professional/personal isolation and dissatisfaction with the medical community, factors which have been previously emphasized for their importance in retention^15,51,56-59 Although some authors have demonstrated the importance of income on rural provider retention^15,58, desire for higher income was an important factor for only 5 non-retained rural participants (22%), suggesting that money was not as important for rural retention as satisfaction with the professional environment in general.

Limitations

This study is limited by only assessing LRP participants in Colorado, so the results may not generalize to other areas of the country or beyond the USA.

Because this survey asked providers to recall decisions made in the past, recall bias must be considered. Although the response rate was high, response bias may also influence results; non-responders or those that could not be located may have been more likely to have left their original LRP practice location or have different attitudes and motivations.

Of the 66 respondents who had completed their terms of service at the time of the survey, only 21 (32%) were more than 5 years from completion of their LRPs. Thus, inferences about long-term retention are limited.

By design, this study included a cohort of health providers who received loan repayment. Therefore, the attitudes and motivations of providers who have declined participation in LRPs could not be assessed. It is possible that a significant number of providers would be interested in and satisfied with rural practice but selected urban practice due to financial or other constraints.

Due to the methods by which participants were collected, there was a correlation between rural participants and physicians as well as urban participants and dentists. While there was an attempt to stratify these groups, the location-provider type association may have potentially confounded the results. Due to the relatively small sample sizes, further stratification or multivariable modeling to explore confounding by location or provider type was not possible.

Marital status was not measured in the survey. This limits the interpretation of family and spousal influence, as it cannot be differentiated if a participant reported a low influence of these factors simply because there was no spouse or family involved in decision-making.

Conclusions

Loan repayment programs have increased in popularity as a means to enhance recruitment and retention of healthcare providers in medically underserved areas. From the survey responses obtained in this study, it appears that loan repayment in rural Colorado is most often distributed to providers who are already practicing or planning to practice in a rural community regardless of the award. Although loan repayment appears to have limited influence on the decision of a provider to practice in a rural community, it does appear to have notable influence on the specific rural community the provider chooses as well as on the retention of the provider in a rural community. Rural provider recruitment was most heavily influenced by the location of the community, the size and scope of practice, family fit, and recreational opportunities, while retention was most dependent on family preferences, personal and professional support, and satisfaction with the medical community. The desire for a higher income was not a leading factor in the retention of rural providers. These data are important to inform policymakers and administrators of LRPs and to better tailor loan repayment for optimal provider recruitment and retention to rural communities in the USA and worldwide.

Acknowledgements

The authors thank Ryan Gamble and Clint Cresawn for assistance in data collection.

This research was supported by the Colorado Rural Health Center.

References

1. Dussault G, Franceschini MC. Not enough there, too many here: understanding geographical imbalances in the distribution of the health workforce. Human Resources for Health 2006; 27(4): 12.

2. Munga MA, Maestad O. Measuring inequalities in the distribution of health workers: the case of Tanzania. Human Resources for Health 2009; 7(1): 4.

3. De Costa A, Al-Muniri A, Diwan VK, Eriksson B. Where are healthcare providers? Exploring relationships between context and human resources for health Madhya Pradesh province, India. Health Policy2009; 93: 41-47.

4. Nigenda G, Machado MH. From state to market: the Nicaraguan labor market for health personnel. Health Policy and Planning 2000; 15(3): 312-318.

5. Smith K, Tennant M. Demographic analysis of currently registered dentists in Western Australia: rural urban divide. Australian Journal of Rural Health 2006; 14(3): 126-128.

6. Kobayashi Y, Takaki H. Geographic distribution of physicians in Japan. Lancet 1992; 340(8832): 1391-1393.

7. Thompson MJ, Lynge DC, Larson EH, Tachawachira P, Hart G. Characterizing the general surgery workforce in rural America. Archives of Surgery 2005; 140: 74-79.

8. Fordyce MA, Chen FM, Doescher MP, Hart LG. 2005 Physician Supply and Distribution in Rural Areas of the United States. Final Report #116. Seattle, WA: WWAMI Rural Health Research Center, University of Washington, 2007.

9. Sherman JE. Addressing the Nursing Shortage: Impacts and Innovations in Rural America. Ojo Sarco, NM: Frontier Education Center, National Clearinghouse for Frontier Communities, 2004.

10. Knapp KK, Hardwick K. The availability and distribution of dentists in rural zip codes and primary care health professional shortage areas (PC-HPSA. zip codes: comparison with primary care providers. Journal of Public Health Dentistry 2000; 60(1): 43-48.

11. MacDowell M, Glasser M, Fitts M, Nielsen K, Hunsacker M. A national view of rural health workforce issues in the USA. Rural and Remote Health 10(3):1531. (Online) 2010. Available: http://www.rrh.org.au/ (Accessed 29 August 2010).

12. National Rural Health Association. Meeting oral health care needs in rural America. Kansas City, MO: NRHA Policy Brief, 2005.

13. Doescher M, Fordyce M, Skillman S. The aging of the rural generalist workforce: are some locations more vulnerable than others? Final Report #127. Seattle, WA: WWAMI Rural Health Research Center, University of Washington, 2009.

14. Daniels ZM, Vanleit BJ, Skipper BJ, Sanders ML, Rhyne RL. Factors in recruiting and retaining health professionals for rural practice. Journal of Rural Health 2007; 23(1): 62-71.

15. Rabinowitz HK, Diamond JJ, Hojat M, Hazelwood CE. Demographic, educational and economic factors related to recruitment and retention of physicians in rural Pennsylvania. Journal of Rural Health 1999; 15(2): 212-218.

16. Navin TR, Nichols AW. Evaluation of the Arizona Medical Student Exchange Program. Journal of Medical Education 1977; 52: 817-823.

17. Halaas GW, Zink T, Finstad D, Bolin K, Center B. Recruitment and retention of rural physicians: outcomes from the rural physician associate program of Minnesota. Journal of Rural Health 2008; 24(4): 345-352.

18. Matsumoto M, Inoue K, Kajii E. A contract-based training system for rural physicians: follow-up of Jichi Medical University graduates (1978-2006). Journal of Rural Health 2008; 24(4): 360-368.

19. Hsueh W, Wilkinson T, Bills J. What evidence-based undergraduate interventions promote rural health? New Zealand Medical Journal 2004; 117(1204): U1117.

20. Looney SW, Blondell RD, Gagel JR, Pentecost MW. Which medical school applicants will become generalists or rural-based physicians? Journal of the Kentucky Medical Association 1998; 96(5): 189-193.

21. Fryer GE Jr, Stine C, Vojir C, Miller M. Predictors and profiles of rural versus urban family practice. Family Medicine 1997; 29(2): 115-118.

22. Stratton TD, Geller JM, Ludtke RL, Fickenscher KM. Effects of an expanded medical curriculum on the number of graduates practicing in a rural state. Academic Medicine 1991; 66(2): 101-105.

23. Jarman BT, Cogbill TH, Mathiason MA, O'Heron CT, Foley EF, Martin RF et al. Factors correlated with surgery resident choice to practice general surgery in a rural area. Journal of Surgical Education 2009; 66(6): 319-324.

24. Rabinowitz HK, Diamond JJ, Markham FW, Hazelwood CE. A program to increase the number of family physicians in rural and underserved areas: impact after 22 years. JAMA 1999; 281(3): 255-260.

25. Henry JA, Edwards BJ, Crotty B. Why do medical graduates choose rural careers? Rural and Remote Health 9(1):1083. (Online) 2009. Available: http://www.rrh.org.au/ (Accessed 23 September 2009).

26. Fryer GE, Stine C, Krugman RD, Miyoshi TJ. Geographic benefit from decentralized medical education: student and preceptor practice patterns. Journal of Rural Health 1994; 10(3): 193-198.

27. Rabinowitz HK. Recruitment, retention, and follow-up of graduates of a program to increase the number of family physicians in rural and underserved areas. New England Journal of Medicine 1993; 328(13): 961-963.

28. Verby JE, Newell JP, Andresen SA, Swentko WM. Changing the medical school curriculum to improve patient access to primary care. JAMA 1991; 266(1): 110-113.

29. Bowman RC, Penrod JD. Family practice residency programs and the graduation of rural family physicians. Family Medicine 1998; 30: 288-292.

30. Rosenthal TC, McGuigan MH, Osborne J, Holden DM, Parsons MA. One-two rural residency tracks in family practice: are they getting the job done? Family Medicine 1998; 30(2): 90-93.

31. Rosenthal TC, McGuigan MH, Anderson G. Rural residency tracks in family medicine: graduate outcomes. Family Medicine 2000; 32(3): 174-177.

32. Rabinowitz HK, Diamond JJ, Markham FW, Wortman JR. Medical school programs to increase the rural physician supply: a systematic review and projected impact of widespread replication. Academic Medicine 2008; 83(3): 235-243.

33. Pathman DE, Konrad TR, King TS, Taylor DH Jr, Koch GG. Outcomes of states' scholarship, loan repayment, and related programs for physicians. Medical Care 2004; 42(6): 560.

34. Pathman DE, Taylor DH, Konrad TR, King TS, Harris T, Henderson TM et al. State scholarship, loan forgiveness, and related programs: the unheralded safety net. JAMA 2000; 284(16): 2084.

35. Sampowski IP. Effectiveness of financial incentives in exchange for rural and underserviced area return-of-service commitments: systematic review of the literature. Canadian Journal of Rural Medicine 2004; 9(2): 82-88.

36. Bass M, Copeman W. An Ontario solution to medically underserviced areas: an evaluation of an ongoing program. Canadian Medical Association Journal 1975; 113: 404-407.

37. Wade T, Sauer ML, Kushner C. Recruitment and retention of physicians and primary care practitioners for North Carolina: a partnership approach. North Carolina Medical Journal 2007; 68(3): 187-190.

38. Grobler L, Marais BJ, Mabunda SA, Marindi PN, Reuter H, Volmink J. Interventions for increasing the proportion of health professionals practicing in rural and other underserved areas. Cochrane Database of Systematic Reviews 2009; 21(1): CD005314.

39. King KM, Scott GA. Graduate medical education: trends in training and student debt. US Government Accountability Office Report GAO-09-438R. Washington, DC: 2009. Available: http://www.gao.gov/new.items/d09438r.pdf (Accessed 23 September 2009).

40. Sibbald B, Slater J, Gosden T, Williams A, Parke S, Philpin S. Solving inequalities in provider distribution: loan repayment. Health and Social Care in the Community 2002; 10(3): 162.

41. Miller JB, Crittenden RA. The effects of payback and loan repayment programs on medical student career plans. Journal of Rural Health 2001; 17(3): 160.

42. Price MA, Cohn SM, Love J, Dent DL, Esterl R. Educational debt of physicians-in-training: determining the level of interest in a loan repayment program for service in a medically underserved area. Journal of Surgical Education 2009; 66(1): 8-13.

43. Tierney EP, Kalia S, Kimball AB. Assessment of incentives for student loan debt repayment among recent dermatology residency graduates. Archives of Dermatology 2009; 145(2): 208-209.

44. Szafran O, Crutcher RA, Chaytors RG. Location of family medicine graduates' practices. What factors influence Albertans' choices? Canadian Family Physician 2001; 47: 2279-2285.

45. Colorado Department of Public Health and Environment. Shortage area designation. Colorado Department of Public Health and Environment. (Online) 2009. Available: http://www.cdphe.state.co.us/pp/primarycare/shortage/ (Accessed 23 September 2009).

46. Western Interstate Commission for Higher Education. Inventory of Rural Health Practice Incentives in the Western WICHE States. (Online) 2007. Available: http://www.wiche.edu/info/publications/stateInventory.pdf (Accessed 31 October 2010).

47. Colorado Rural Health Center. Comparison of Colorado Healthcare Provider Loan Repayment Programs. Denver, CO: Colorado Rural Health Center, 2009. Available: www.coruralhealth.org (Accessed 27 July 2009).

48. WWAMI RUCA Rural Health Research Center. Rural-urban commuting area codes (RUCAs). (Online) 2009. Available: http://depts.washington.edu/uwruca/ (Accessed 23 September 2009).

49. Rosenthal MP, Diamond JJ, Rabinowitz HK, Bauer LC, Jones RL, Kearl GW et al. Influence of income, hours worked, and loan repayment on medical students' decision to pursue a primary care career. JAMA 1994; 271(12): 914-917.

50. Ryder RL. An educational loan repayment program for registered nurses. Nursing Economics 1990; 8(6): 397-403.

51. Pathman DE, Konrad TR, Dann R, Koch G. Retention of primary care physicians in rural health professional shortage areas. American Journal of Public Health 2004; 94(10): 1723.

52. Scammon DL, Williams SD, Li LB. Understanding physicians' decisions to practice in rural areas as a basis to developing recruitment and retention strategies. Journal of Ambulatory Care Marketing 1994; 5(2): 85.

53. Hemphill E, Dunn S, Barich H, Infante R. Recruitment and retention of rural general practitioners: A marketing approach reveals new possibilities. Australian Journal of Rural Health 2007; 15: 360-367.

54. Chauban TS, Jong M, Buske L. Recruitment trumps retention: results of the 2008/09 CMA rural practice survey. Canadian Journal of Rural Medicine 2010; 15(3): 101-107.

55. Laurence CO, Williamson V, Sumner KE, Fleming J. 'Latte rural': the tangible and intangible factors important in the choice of a rural practice by recent GP graduates. Rural and Remote Health 10(2):1316. (Online) 2010. Available: http://www.rrh.org.au/ (Accessed 29 August 2010).

56. Gill D, Palmer C, Mulder R, Wilkinson T. Medical student career intentions at the Christchurch school of medicine. The New Zealand Wellbeing, Intentions, Debt and Experiences (WIDE. survey of medical students pilot study. Results part II. New Zealand Medical Journal 2001; 114: 465-467.

57. Eley D, Young L, Shrapnel M, Wilkinson D, Baker P, Hegney D. Medical students and general health practitioners: Congruent views on the reality of recruitment into rural medicine. Australian Journal of Rural Health 2007; 15: 12-20.

58. Stenger J, Cashman SB, Savageau JA. The primary care physician workforce in Massachussetts: implications for the workforce in rural, small town America. Journal of Rural Health 2008; 24(4): 275-283.

59. Cutchin MP, Norton JC, Quan MM, Bolt D, Hughes S, Lindeman B. To stay or not to stay: issues in rural primary care physician retention in eastern Kentucky. Journal of Rural Health 1994; 10(4): 273-278.