Increasing the enrolment of rural applicants to the faculty of medicine and addressing diversity by using a priority matrix approach to assign values to rural attributes

full article:

Introduction

Approximately half of the global population lives in a rural area, but these areas are served by less than a quarter of the total physician workforce¹. While healthcare inequity is especially severe in low-income countries, even high-income countries experience shortage of health workers in remote and rural areas. Approximately 20% of the populations in the USA and Canada live in rural areas^1,2.The health status of the rural populations in both countries is thought to be lower than that of their urban counterparts^2,3. Yet, only 9-10% of these nations' physicians serve the healthcare needs of the rural populations^1,2.

Shortages in the physician workforce have been reported in the rural and remote areas of the Canadian province of Manitoba⁴. In addition, specialist care is concentrated in Winnipeg and Brandon and, therefore, is not easily accessible to rural Manitobans⁴. The University of Manitoba operates under the social accountability vision adopted by all Canadian medical schools, and one principal aim is to improve access to physician services in rural Canada⁵. Accordingly, the university reaffirmed its ongoing commitment to develop a sufficient and sustainable physician workforce, and re-examined its efforts to recruit, enroll and educate students more likely to practise in rural Manitoba.

Beginning in 2001, there have been comprehensive internal and external reviews of the Faculty's admission policies and procedures. The Brownell-Reiter External Review Report made a recommendation, among others, that policies and processes for medical school admissions be modified to increase enrolment of qualified students who would be more likely to practise in rural and remote areas in Manitoba⁶. Increasing the proportion of medical students from rural areas has been recommended elsewhere as a focused way of addressing rural physician shortage^1,7,8.

A 2002 study showed that students from rural Canada are under-represented in medical schools in Canada compared with those whose parents are from urban Canada⁹. Only 10.8% of medical students have ever lived in a rural area compared with 22.4% of the Canadian population⁹. Such statistics suggest that people in rural regions may not have an equitable opportunity to gain entry into medical school. However, among medical school applicants, grades of rural-origin applicants and the proportion admitted were similar to those of applicants from urban backgrounds^10,11. Perhaps the issue is not that rural applicants are disadvantaged or discriminated by the admission process, but that a smaller-than-representative proportion of prospective students from rural areas apply to medical school than their urban counterparts¹². In fact, only 7.3% of Ontario applicants to medical school were of rural origin while 13% of the population of Ontario is rural¹¹. Cultural and financial barriers likely play a bigger role in the decision to apply to medical school.

As in Ontario^10,11, rural applicants from Manitoba, despite holding academic qualifications similar to applicants from urban Manitoba, were persistently under-represented in the medical school applicant pool and in the medical class, relative to the general population. Historically at the Manitoba medical school, rural classification, used merely to gather demographic data on Manitoba applicants, was based solely on location of high school graduation. In the years 2000 to 2007, while 28.5% of Manitobans lived in rural Manitoba, only 444 Manitoba applicants to medical school (21.8%) graduated from rural high schools in Manitoba. Similar to the proportion successful among urban applicants, 32.4% (n=144) of the rural applicants were successful in obtaining admission into medical school, and they represented 20% of the students in incoming classes in those years (Table 1). That is, the proportion of rural applicants offered admission showed no evidence of inequity compared with urban applicants, but the demographics of Manitoba medical school's registrants failed to represent the rural-urban demographics of Manitoba's population. In the absence of specific policies, rural applicants, under-represented in the applicant pool, were also under-represented in the incoming classes. Therefore, as a first step at the Faculty level, it was deemed critically appropriate to narrow the rural demographic gap in incoming classes by increasing offers of medical school admissions to qualified applicants from rural Manitoba. By integrating available evidence and physician workforce policy recommendations, the Faculty implemented new policies and procedures related to assessment of admissions criteria and ranking of applicants to medical school. The methodology adopted at this Faculty to bring about these changes may be of interest to educators and administrators responsible for rural health training and education in Canada and elsewhere.

Table 1: University of Manitoba Faculty of Medicine admission statistics for Manitoba applicants, 2000-2007



The Working Group for personal assessment and scoring for admission to medical school

In April 2008, a Working Group was formed to develop criteria for personal assessment and scoring of applicants to increase the number of students admitted with a potential for rural practice. The specific objectives of the Group's mandate were:

1. To identify personal attributes reflecting a potential for living and working in rural areas and to determine a hierarchy of importance among these attributes expressed in their assigned relative weights.
2. To develop a method to distinguish applicants with the personal attributes valued by the Working Group.
3. To determine a numerical formula that could be applied to applicants' final composite scores (applicant ranking for admission to the Faculty of Medicine is based on applicants' composite scores which consist of the adjusted grade point average (AGPA), scores from the Medical College Admission Test (MCAT), and the Personal Assessment Score (PAS); the PAS is solely determined from a personal interview) to increase the number of admission offers made to applicants with the valued rural attributes, to the point where enrolment would mirror the province's demographics.

The 12 member group was broadly represented by clinicians from tertiary-care teaching hospitals, faculty members from basic and clinical sciences, rural physicians, senior faculty administrators from allied health professions' programs, and representatives of Manitoba Health and the Office of Rural and Northern Health.

Methodology of the Working Group

Prior to the inaugural meeting, it was proposed that the methodology of the Working Group would include the following:

1. Literature review and environmental scan of practices used by Canadian and international medical schools to select applicants likely to serve in rural areas after completion of degrees.
2. Construction of a priority matrix to compare and rank personal attributes reflecting a potential for living and working in rural areas.
3. Design of a data-capture questionnaire focusing on personal attributes valued by the Working Group.
4. Establishment of minimal criteria to be demonstrated by applicants receiving rural scores derived from the priority matrix.
5. Selection of the magnitude of adjustment to applicants' composite scores via mathematical modeling of admissions data from past applicants.
6. The outlining of procedures to be followed by the Admissions Committee while assessing and screening applicants for rural attributes, assigning rural scores, and applying adjustment constants before final ranking and selection of applicants.

The Working Group unanimously agreed to the proposed methodology (Fig1) at the first meeting.



Figure 1: Methodology followed by the working group for personal assessment and scoring of applicants with rural attributes for admission at the University of Manitoba Faculty of Medicine.

Literature review and environmental scan: Literature relevant to physician training for, and retention in, rural areas was circulated prior to the inaugural meeting. At the inaugural meeting, criteria^13-19 utilized by Canadian, American and Australian institutions to selectively admit applicants more likely to serve rural areas were discussed. Rural practitioners and students interested in rural practice were more likely to have rural connections such as rural upbringing and family in rural areas, to have graduated from rural high schools, to have lived in smaller communities, and to have rural service experience, irrespective of an urban or rural background^13-19.

Construction of a priority matrix: The priority matrix approach is used to plan for optimum results by providing a framework for setting priorities in complex situations with many factors²⁰. It allows for the integration of available evidence with feelings, intuitions, and logic over multiple levels of decision making. The effective use of the priority matrix approach requires clarity of purpose and knowledge of options and choices that most contribute towards the successful accomplishment of that purpose. In this case, the purpose was to attribute relative weights to personal characteristics that reflect a potential for living and working in rural Manitoba. The personal attributes to be evaluated were chosen based on those identified in the literature (rural community service, rural connections, rural employment) as well as those honored historically at the University of Manitoba through our social accountability framework and commitments to educate the province's diverse communities (history of interprofessional work in health care, advanced degree, commitment to the Canadian Forces) (Table 2).

Table 2: Construction of priority matrix to rank personal attributes reflecting a potential for life and work in rural Manitoba among applicants to medical school



The matrix construct (Table 2) allowed for systematic, one-on-one comparisons of attributes in a pair-wise manner. Attributes were listed, one on each row, and again on each column, and compared with every other attribute in a pair-wise manner (row-to-column comparison) to answer the question, 'which attribute, in the pair, is a more important indicator of potential for life and work in rural Manitoba?' Numerical values were entered to indicate whether the row attribute was considered by the Working Group to be more important than, less important than, or equally important as, the column attribute, in each pair compared. The numerical values ranged from 1/10 (much less important) to 10 (much more important) and were selected through a process of consensus rather than voting. Negative signs were introduced when attributes were thought to facilitate migration away from Manitoba rather than influence retention within Manitoba. For each pair-wise comparison, values were entered simultaneously in the intersection of row with column, as well as column with row as the values were reciprocals of each other.

The chairperson ensured that the Working Group focused on the question at hand and that it did not change its thought process anytime during the exercise. At the completion of all possible pair-wise comparisons, an attribute score (ie a row total) was calculated for each attribute. The attributes with negligible or negative row totals were not considered further. Row total relative to the matrix total yielded the relative importance of the attribute as an indicator of potential for life and work in rural Manitoba compared with all other attributes in the context of the matrix. The relative importance of the attribute was also expressed as a percentage of the matrix total.

Personal Attributes Questionnaire to assess applicants' rural attributes: After establishing comprehensive definitions, a questionnaire to serve as a supplement to the main application form was prepared to enable the Admissions Office to elicit detailed, verifiable information on the attributes of interest. The questionnaire was constructed to encompass the different sections on the priority matrix:

1. Rural connections, namely: rural birth, graduation from a rural high school, current primary address in a rural region, whether the first decade of life was spent in a rural area, whether the second decade of life was spent in a rural area.
2. Employment in rural areas during high school, following high school, following an undergraduate degree. Only employment that lasted more than one semester or more than one summer was deemed as appropriate for consideration in this context.
3. Rural community service during high school, following high school, following undergraduate degree. Service could be: (a) rural volunteer experience that lasted more than one semester or more than one summer; and/or (b) rural leadership experience that lasted more than one academic year or more than one season of sports.

'Rural' was defined, in the Manitoba context, as being outside the city of Winnipeg. The Faculty of Medicine reserved the right to verify all information reported on the supplementary questionnaire, and applicants were so advised.

Establishment of minimal criteria to be met during assessment of applicants' rural attributes and assignment of rural scores: A defined minimum number of personal attributes within each section of the questionnaire was to be demonstrated before an applicant was assigned rural scores derived from the priority matrix. An applicant who reported at least 2 attributes under the rural connections section would be assigned a rural score of 40.0 (46.2%) of the total possible rural score of 86.6 from the matrix. One or more eligible rural employment experiences entitled an applicant to a rural score of 21.4 (or 24.7%). If any 2 eligible rural community service experiences were listed, then 25.2 (or 29.1%) would be the rural score assigned to the applicant. Applicants received scores for all the sections of the questionnaire for which they were qualified. For example, applicants qualified to collect scores on all 3 sections of the questionnaire would receive a rural score of 86.6 (or 100.0%). Applicants who do not meet any of the above criteria would receive a rural score of 0 (zero).

Mathematical modeling of historic data on applicants' composite scores: While the magnitude of scores assigned to applicants was derived from the row values in the priority matrix, the magnitude of the adjustment factor 'p' to enhance the ranking of applicants with rural attributes was determined through mathematical modeling of historic admissions data. Factor 'p' had to be large enough to move a sufficient number of competitive, wait-listed applicants with rural attributes to the list of accepted applicants and achieve a student body more representative of the province's demographics. Composite scores of rural applicants in the years 2006 and 2007 were systematically increased by varying percentages to determine the additional (hypothetical) number of rural applicants who would have been offered admission in those years if such adjustments had been made (Table 3). For admission cohorts of 2006 and 2007, a 'p' no greater than 15% resulted in an increase in the number of students with rural attributes by 29% and 33%, respectively. Adjustments to composite scores would have to be proportionate to the rural scores of applicants with maximum adjustment given to those with a rural score of 86.6 (or 100%).

A formula to adjust the composite score was established as follows: [1 + (p*rural score)] * [composite score], where factor 'p' will be no greater than 15%. That is, (15%*rural score) would represent a series of numerical constants whose values range from 0 for applicants with a rural score of 0, to 0.15 for applicants with a rural score of 86.6. In the end, the absolute increments to composite scores gained by applicants with rural attributes will be determined by applicants' rural scores as well as their composite scores.

Table 3: Mathematical modeling of historic admissions' data from the University of Manitoba Faculty of Medicine: determining the percent increase to be applied to composite scores of competitive applicants with rural attributes so a sufficient number can be moved from waiting list to admit list



Implementation of policy and procedures

The new policy and procedures were implemented during the application and admission cycle for the year 2009. In all, 129 of 327 applicants (39.4%) invited for personal interviews had rural scores in one or more of the 3 categories of rural attributes (until the year 2009, all Manitoba applicants were invited for interview). To enroll a class of 110 students, 136 offers of admission were extended to applicants. Sixty of the 136 admission offers (44.1%) were made to applicants with rural attributes. Without adjustment of composite scores, only 49 applicants with rural attributes (36%) would have been offered admission. These 11 applicants represented a 22.4% increase in admission offers to applicants with rural attributes that could be directly attributed to our new policy and procedures. The mean (standard deviation) of the unadjusted and the adjusted composite scores of applicants with rural attributes did not differ significantly from the mean (standard deviation) of the composite scores of applicants without rural attributes (p-value =0.07 and p-value=0.15, respectively) (Table 4). The change in the relative ranking of applicants after composite score adjustment is illustrated (Fig2). Forty-nine among the 60 accepted the offer of admissions and represented 44.5% of the 110 students in the incoming class. Thirty-three of the 110 incoming students (30%) graduated from a rural high school in Manitoba (Table 5), compared with the overall 20% of the incoming classes in the past who graduated from a rural high school in Manitoba (Table 1). This amounted to an approximate 50% increase in the representation of students who graduated from a rural high school this year, compared with previous years, and was closer to the proportion of Manitobans living in rural areas (28.5%). A short-term surrogate measure of success of the Working Group's work was that the rural demographic gap observed in previous incoming classes was met.

Table 4: Distribution of composite scores of applicants with and without rural attributes at the University of Manitoba Faculty of Medicine in the year 2009




Table 5: Distribution of rural attribute scores among the 2009 cohort of applicants with offers of admission to the University of Manitoba Faculty of Medicine, by location of high school graduation of applicants.






Figure 2: University of Manitoba Faculty of Medicine: distribution of composite scores and relative ranking of applicants considered for medical school admissions in 2009.

Strengths of the priority matrix approach to enhancing diversity in medical class cohorts

By determining applicants' rural attributes and assigning corresponding rural scores, these revised admissions process identified and recruited 18 more students with rural attributes in the incoming class than would have been possible if relying on location of high school graduation alone (Table 5). Thus, the ability of the matrix approach to positively distinguish applicants as those characterized by rural attributes appears to be greater than the use of location of high school graduation alone. Moreover, the matrix approach does not exclude applicants from urban backgrounds who have demonstrated a rural orientation by their past life choices. It has been acknowledged that, in order to narrow the gap in rural physician shortage, a significant portion of physicians choosing rural practice will likely originate from urban backgrounds since students from urban backgrounds form the majority in medical schools^15,17. Yet another strength of the priority matrix approach was that it allowed for proportional adjustment of composite scores secondary to ordered increments in rural scores of applicants.

Generalizability of the Manitoba approach

Although the focus of this article has been specific to the Manitoba context, the approach can be easily applicable to all regional needs. The attributes evaluated in the priority matrix can be replaced or redefined, and made specific to any local context. The rural scores associated with attributes have the potential to change based on the number and type of other attributes in the matrix and also based on the Working Group evaluating the attributes. The minimal criteria to be demonstrated by applicants before rural scores can be assigned will also depend on local needs, values and communities of origin of applicants. For example, instead of stipulating that at least a decade of life must have been spent in a rural area in order to be eligible for rural scores under section 'rural connections,' other schools may wish to assign graded amounts of points for increasing numbers of years. Finally, the priority matrix approach is highly adaptable and would be equally applicable to enhance diversity related to other attributes among eligible admission cohorts. To maintain transparency during admission process, the use of rurality scoring and the magnitude of adjustments made to the composite scores while ranking applicants should be communicated clearly to applicants, parents and other stakeholders. The procedures should preferably be described in the Faculty's applicant information bulletin.

Future directions

Having modified its recruitment and selection policies to moderately increase the proportion of students with rural attributes in incoming medical classes, the Manitoba Faculty is equally committed to monitoring and evaluating the effectiveness of this policy change, revising the policy as necessary, building the evidence base, and improving understanding about the extent to which admission interventions succeed in improving the health outcomes of rural populations.

An initial measure of the success of our intervention will be a sustained increase in individuals with rural attributes among enrolled students, followed by a demonstrable decrease in the shortage of rural physician workforce over time. Already a study is in place at the University of Manitoba medical school to track graduate outcomes such as residency choices and practice locations. In the future, outcomes will be related to rurality scores to evaluate effectiveness of the rural-cohort admission policy and to establish if selection of applicants by matrix methods ultimately results in greater levels of rural workforce. It is estimated that if at least half the additional 11 or so applicants with rural attributes who are offered admission annually due to the new policy choose rural practice in Manitoba, over the next 12 years there will be a minimum of 66 additional practitioners in rural Manitoba. This will be a 50% increase on the estimated number of Manitoba graduates entering rural practice in Manitoba currently.

Conclusion

Consistent under-representation of medical students from rural Manitoba during a period of rural physician shortage led the University of Manitoba Faculty of Medicine to use a qualitative, group decision-making tool to identify and rank attributes that were reported to select medical school applicants with a potential for living and working in rural areas. Using mathematical modeling on historic admissions data, the group recommended an adjustment factor that was hypothesized to narrow the rural gap in incoming classes. The policy, at implementation, resulted in a class that was representative of the rural-urban demographics of Manitoba while maintaining the quality of selected applicants. As applicants with rural attributes were at least as academically qualified as applicants with no rural attributes, the authors agree with others²¹ who suggested that medical schools can maintain competitive admission criteria while at the same time accepting students more likely to enter rural practice.

Acknowledgement

The authors gratefully acknowledge the contribution of the following members of the Working Group for personal assessment and scoring for admission to medical school: Dr Samia Barakat, Dr Charles Bernstein, Dr Chris Burnett, Dr Dauna Crooks, Dr Lawrence Elliott, Ms Moni Fricke, Dr Don Klassen and Dr Daniel Sitar.

References

1. World Health Organization. Increasing access to health workers in remote and rural areas through improved retention: global policy recommendations. Paris, France: WHO, 2010.

2. Government of Canada. Rural Canada: Access to Health Care. (Online) 2002, revised 2004. Available: http://dsp-psd.pwgsc.gc.ca/Collection-R/LoPBdP/BP/prb0245-e.htm (Accessed 30 June 2010).

3. Issue brief (Grantmakers Health). Rural health care: innovations in policy and practice. Grantmakers Health 2009; 34: i-v, 1-37.

4. Roos NP, Fransoo R, Carriere KC, Frohlich N, Bogdanovic B, Kirk P et al. Needs-based planning: the case of Manitoba. Canadian Medical Association Journal 1997; 157(9): 1215-1221.

5. Health Canada. Social accountability: a vision for Canadian medical schools. (Online) 2001. Available: http://www.afmc.ca/fmec/pdf/sa_vision_canadian_medical_schools_en.pdf (Accessed 9 September 2010).

6. Brownell K, Reiter H. Report on the Admissions Process at the University of Manitoba Faculty of Medicine. Winnipeg, MB: Canada, 2007.

7. Rourke J. How can medical schools contribute to the education, recruitment and retention of rural physicians in their region? Bulletin of the World Health Organization 2010; 88(5): 395-396.

8. Rabinowitz H. Recruitment and retention of rural physicians: How much progress have we made? Journal of the American Board of Family Practice 1995; 8(6): 496-499.

9. Dhalla IA, Kwong JC, Streiner DL, Baddour RE, Waddell AE, Johnson IL. Characteristics of first-year students in Canadian medical schools. Canadian Medical Association Journal 2002; 166(8): 1029-1035.

10. Hensel JM, Shandling M, Redelmeier DA. Rural medical students at urban medical schools: too few and far between? Open Medicine 2007; 1(1): e13-17.

11. Hutten-Czapski P, Pitblado R, Rourke J. Who gets into medical school? Comparison of students from rural and urban backgrounds. Canadian Family Physician 2005; 51: 1240-1241.

12. Wright B, Woloschuk W. Have rural background students been disadvantaged by the medical school admission process? Medical Education 2008; 42: 476-479.

13. Rabinowitz HK, Diamond JJ, Hojat M, Hazelwood CE. Demographic, educational and economic factors related to recruitment and retention of physicians in rural Pennsylvania. The Journal of Rural Health 1999; 15(2): 212-218.

14. Rabinowitz HK, Diamond JJ, Markham FW, Paynter NP. Critical factors for designing programs to increase the supply and retention of rural primary care physicians. JAMA 2001; 286(9): 1041-1048.

15. Rourke J, Dewar D, Harris K, Hutten-Czapski P, Johnston M, Klassen D et al. Strategies to increase the enrollment of students of rural origin in medical school: recommendations from the society of rural physicians of Canada. Canadian Medical Association Journal 2005; 172(1): 62-65.

16. Laven G, Wilkinson D. Rural doctors and rural backgrounds: How strong is the evidence? A systematic review. Australian Journal of Rural Health 2003; 11(6): 277-284.

17. Rourke JT, Incitti F, Rourke LL, Kennard M. Relationship between practice location of Ontario family physicians and their rural background or amount of rural medical education experience. Canadian Journal of Rural Medicine 2005; 10(4): 231-240.

18. Laven GA, Beilby JJ, Wilkinson D, McElroy HJ. Factors associated with rural practice among Australian-trained general practitioners. Medical Journal of Australia 2003; 179(2): 75-79.

19. Carter RG. The relation between personal characteristics of physicians and practice location in Manitoba. Canadian Medical Association Journal 1987; 136: 366-368.

20. Hoffherr GD. The toolbook: decision-making and planning for optimum results, 1st edn. Windham, NH: Markon, 1993.

21. Longo DR, Gorman RJ, Ge B. Rural medical school applicants: do their academic credentials and admission decisions differ from those of non-rural applicants? Journal of Rural Health, 2005; 21(4): 346-350.