Performance of rural health clinics: an examination of efficiency and Medicare beneficiary outcomes

full article:

Introduction

Rural Health Clinics (RHCs) are among the major employers in the rural economy of the USA, where an individual rural physician generates 23 jobs¹. Established through the Rural Health Clinic Program (Public Law 95-210, 1977), RHCs are intended to provide access to primary care in underserved rural areas². Almost 3800 RHCs existed in 2011 throughout 45 states. Although many are small, they vary in size, years of operation, ownership, and system membership.

The RHCs contend with a variety of issues that challenge their operational performance. They serve complex communities. Compared with urban populations, rural residents are generally poorer and older, and have higher rates of chronic disease such as diabetes, congestive heart failure (CHF), and certain types of cancer³. In addition, because they are located in rural areas, RHCs often struggle to attract and retain needed healthcare professionals.

Faced with such challenges, it is imperative that RHCs find ways to maximize their performance in order to remain financially viable and continue to provide high quality healthcare services. Yet, to date little research has been conducted to identify the variability in RHC performance. Previous studies have stressed the importance of improving performance among primary healthcare clinics^4-6; however, few focus on RHCs specifically. Two national survey research studies summarize financial, operational, and other characteristics of RHCs^7,8. Other studies are regional or case-specific in scope. For example, Sinay analyzed the characteristics of efficient and inefficient Rural Health Clinics in the Midwest region of the USA⁹.

In an effort to address these knowledge gaps, a national, longitudinal study of RHCs was conducted using multiple sources of data. The goals of the study were to: (i) identify the factors that influence the variation in RHC performance; and (ii) determine the relationship between two aspects of performance: efficiency and effectiveness. The term 'efficiency' has been defined in a variety of ways depending on the discipline (economics, statistics, health services etc). In general terms, efficient healthcare organizations or systems are those that optimize resources while providing healthcare services. For purposes of this study, 'efficiency' is defined as outcomes per unit input. The term 'effectiveness' has also taken on a variety of definitions. For purposes of this study, 'effectiveness' is defined as the timely receipt of appropriate care¹⁰.

Using a context-design-performance framework to guide the development and measurement of the study variables, the interrelationships were examined among the context (community or demographic factors), design (organizational structure and other mediating factors), and performance (efficiency and effectiveness) of RHCs (Fig1). Multivariate analysis was applied to identify the characteristics of RHC design that contribute to efficiency and effectiveness while controlling for context variables such as poverty rate and the ratio of physicians to population.

Figure 1: The context-design-performance framework for assessing RHCs' performance.

Research hypotheses

Three major research hypotheses were tested:

1. The effectiveness of RHCs is positively influenced by efficiency, holding constant the community and organizational characteristics.
2. There is a reciprocal relationship between RHC efficiency and effectiveness.
3. Large RHCs are more efficient than small RHCs.

Hypothesis 1: Effectiveness of RHCs is positively influenced by efficiency, holding constant the community and organizational characteristics. The literature suggests that high-performance healthcare facilities show high levels of accomplishments in efficiency and effectiveness^11,12. To ascertain how this relationship applied to RHCs, the effect was examined of the prior level of efficiency on the later measure of effectiveness (efficiency → effectiveness) while the effects of community and organizational characteristics were held constant.

Hypothesis 2: There is a reciprocal relationship between RHC efficiency and effectiveness. In addition to the causal relationship between RHC efficiency and effectiveness, a reciprocal relationship between efficiency and effectiveness was postulated and tested in this model.

Hypothesis 3: Large RHCs are more efficient than small RHCs. The determinants of RHCs' performance were specified by context (community) and design (organizational) factors in the formulation of the proposed model. The literature suggests that larger clinics are more efficient than small ones. In large community health centers (CHCs), higher patient volumes may contribute to higher levels of productivity, and enable fixed costs to be distributed over more patient visits⁵. A recent study examined the efficiency scores of 163 Midwestern RHCs using data envelopment analysis, and found larger RHCs to be more efficient and more productive (as measured by patient visits) than others⁹.

Methods

A longitudinal, non-experimental study of RHC performance was conducted using 2 years of secondary data (for 2006-2007). A panel of RHCs formed the basis of this analysis. This panel was composed of all RHCs continuously in operation during the period 2006-2007 using data from the CMS Online Survey and Certification Reporting System (OSCAR)¹³. Approval for the investigation was obtained through the Institutional Review Board of the University of Central Florida. The individual RHC was the unit of analysis throughout the study.

Data sources

Three principal sources of secondary data were used: (i) the Medicare Cost Report^14-17; (ii) Medicare beneficiary claims¹⁸; and (iii) the Area Resource File (ARF System)¹⁹. Aggregated indicators for effectiveness measures were derived from the Medicare Beneficiary Claims file for each RHC. Additional data sources included the CMS Online Survey and OSCAR System; the Bureau of Labor Statistics; the Department of Labor; and the Bureau of Census databases.

Study variables

Study variables were developed from the literature. A description of the primary study variables follows, and a complete listing of study variables and definitions is provided (Appendix I).

Performance: Two aspects of RHC performance were analyzed: efficiency and effectiveness.

Efficiency Efficiency is described in managerial epidemiology as outcomes per unit input, where input is largely composed of labor. Two components of efficiency were evaluated: technical and cost efficiency. Technical efficiency describes the relationships between various inputs and the related output²⁰ and is measured by a technical efficiency 'score'. To arrive at the score for each RHC, the linear programming technique 'data envelopment analysis' was used, where the inputs were physician, nurse practitioner (NP), and physician assistant (PA) full-time equivalents (FTEs), and the outputs were physician, NP, and PA visits. Cost efficiency refers to the ability of a center to produce a given level of output at minimum possible cost. It was calculated as a ratio of the total cost of RHC operations to the total number of visits.

Effectiveness Effectiveness has been defined as the timely receipt of appropriate care²⁰. In developing measures of effectiveness, Medicare beneficiary outcomes related to chronic illness were focused on because rural area residents in the USA have higher rates of chronic disease than do urban area residents¹⁰. Six variables were constructed to measure effectiveness: three to measure utilization of secondary preventive services by RHC patients diagnosed with diabetes, CHF, or chronic obstructive pulmonary disease (COPD); and three to measure hospitalizations for ambulatory care sensitive conditions (ACSCs) for RHC patients diagnosed with diabetes, CHF, or COPD. For each of the three chronic conditions (diabetes, CHF, COPD), a 'utilization-prevention' variable was calculated as the ratio of claims for related prevention services to the total number of claims for services provided to patients coded for that chronic condition. The three 'ACSC rate' variables were calculated as the ratio of inpatient Medicare claims by RHC patients diagnosed with any of the three chronic conditions to the total of all claims for services provided to patients coded for that condition. Only claims for relevant International Classification of Diseases-9 (ICD-9) codes that were responsible for the majority of the hospitalization were used. Access to other Medicare/Medicaid primary care providers was controlled for by the inclusion of the variable 'physician : population ratio'. This variable was calculated as the ratio of primary-care physicians to the population in the county of location of each study RHC.

Context: The context variables (representing RHC community or demographic characteristics) are treated as variables that constitute multiple contingencies within organizations, and can either facilitate or impede their performance⁹. Eight context variables were included: median income; % over age 65 years; % in poverty; physician : population ratio; % female; % at-risk: social; mortality rate; and region. With the exception of 'region', all context variables were calculated on the county level.

Design: The design variables represented RHC organizational factors over which management has greater control, such as technology use, financial structure, staffing patterns, and participation in integrated systems. Seven design variables were included: size; % NP; % PA; type of control; age; independent; and integration.

Analytical approaches

Prior to the multivariate analyses, several steps were completed. The multivariate analysis of effectiveness focused on Medicare beneficiaries of over age 65 years. Thus, the observations for clinics that provided pediatric services only were eliminated. (These clinics accounted for only 2.6% of the study panel). The study panel were then divided into two subsets - one for the provider-based clinics and another for the independent clinics. (This step was important particularly for the analysis of the cost component of RHC efficiency. RHCs have distinct reimbursement mechanisms depending, in part, on whether they are classified as independent [generally stand-alone clinics] or provider based [clinics operated by a hospital, nursing home, or home health agency]. Provider-based RHCs that are based in hospitals with fewer than 50 beds are not subject to the per-visit payment limit that applies to other RHCs²).

Univariate and bivariate analyses: Descriptive statistics were calculated for each subset. Next, bivariate analyses of the relationships were conducted between the clinic characteristics and the performance outcome measures, as well as the interrelationships among various clinic characteristics using χ², t-tests, Cramer's V, Pearson correlation, and Spearman correlation statistics. The research team used SPSS v18.0 (www.spss.com) for these analyses. The statistical significance of estimated parameters was determined at the α level of 0.05 or lower.

Multivariate analysis: Multivariate analyses were conducted of longitudinal data using structural equation modeling (SEM). This statistical method allowed the researchers to analyze the dynamic components of RHC operations, including multiple causes and multiple outcomes simultaneously over time. The analyses were completed using Amos 17.0 software (www.spss.com). The statistical significance of estimated parameters was determined at the α level of 0.05 or lower. The analysis proceeded as follows.

1. Develop and validate the measurement models of RHCs' performance. To measure the technical efficiency component of the efficiency construct, the research team used Data Envelopment Analysis (DEA) to construct Technical Efficiency Scores ('scores'). A measurement model of the efficiency construct was established and validated with panel analysis of repeated measures using 2 years of data, as detailed in Wan's Evidence-based Healthcare Management: Multivariate Modeling Approaches¹². A measurement model of the effectiveness construct was developed and validated using cross-sectional analysis. Confirmatory factor analysis was executed to determine the validity and reliability of the efficiency and effectiveness constructs.

2. Identify factors influencing the variation in efficiency and effectiveness in RHCs, and determine the relationship between efficiency and effectiveness. Two covariance structure models were developed, one for each RHC classification. This approach allowed empirical evaluation of the hypothesized relationships among the constructs (latent variables). For example, the model assumed that efficiency at Time 1 (2006) affected effectiveness at Time 2 (2007). The modeling also examined the direct linkages between the design variables (eg size and age), the context variables (eg poverty rate and physicians per population), and the performance variables (efficiency and effectiveness).

Results

The study panel was formed with the 3565 RHCs found to be continuously in operation during the time period 2006-2007. The majority (57.8%) of the clinics in the panel were independent; 42.2% were provider based. Almost identical proportions were located in the Midwest (39.4%) and South (39.7%); 17.5% were in the West; and 3.3% were in the Northeast. Most (46%) were for-profit, whereas 37.9% were non-profit, and 16.1% were government controlled. The mean number of years that clinics had been Medicare certified was 8.8. Compared with the US population overall, the RHCs in the study panel were located in counties with higher proportions of elderly, persons below poverty level, and families with female heads.

From the original panel, two study samples (one for independent; the other for provider-based RHCs) were then created using clinics for which complete data existed. The data were examined for outliers and other influential observations which were consequently removed. The final samples were composed of 623 independent RHCs, and 730 provider-based RHCs. Descriptive statistics for each classification of RHC are listed (Tables 1 & 2).

Table 1: Descriptive statistics: independent RHCs (n = 620)




Table 2: Descriptive statistics: provider-based RHCs (n = 731)

Measurement models for efficiency and effectiveness were constructed and validated. Efficiency was measured using two indicators for each of the 2 years: technical efficiency score and (the inverse of) cost per visit. Confirmatory factor analysis revealed partially satisfactory fit statistics (for the independent RHC model: χ² = 415.736, df = 2, p = 0.000, minimum discrepancy function C [CMIN]/df = 207.868, goodness of fit index [GFI] = .804, root mean square error of approximation [RMSEA] = .577; for the provider-based RHC model: χ² = 326.496, df = 2, p = 0.000, CMIN/df = 163.248, GFI = .851, RMSEA = .465.)

Effectiveness was measured using five indicators: prevention-diabetes, prevention-CHF, and ACSC rates for diabetes, CHF, and COPD. Confirmatory factor analysis revealed partially satisfactory fit statistics (for the independent RHC model: χ² = 0.000, df = 0, p = 0.000, RMSEA = .251; for provider-based RHC model: χ² = .673, df = 1, p = 0.412, CMIN/df = .673, RMSEA = .000.)

It was postulated that efficiency has a direct positive impact on effectiveness (Hypothesis 1), and that there be a reciprocal relationship between efficiency and effectiveness (Hypothesis 2). Two covariance structure models (one for independent RHCs and one for provider-based RHCs) were created to analyze these relationships. However, efficiency was not found to have a positive direct impact on effectiveness, nor was there evidence of a reciprocal relationship between efficiency and effectiveness. To confirm the lack of relationship between efficiency and effectiveness, the initial models were then revised by eliminating the reciprocal relationship for Year 2. However, the revised models were not supported by the data.

It was further postulated that clinics of larger size have a positive impact on efficiency (Hypothesis 3). Two additional efficiency-based structural equation models (one for independent RHCs and another for provider-based RHCs) were created to test this hypothesis. Both models were supported by the data (independent model: χ² = 13.8, df = 8, p = 0.088, relative χ² = 1.723, adjusted GFI [AGFI] = .981, RMSEA = .034 ; provider-based model: χ² = 19.011, df = 8, p = 0.015, relative χ² = 2.376, AGFI = .978, RMSEA = .043). Estimated parameters are provided (Table 3). Thus, Hypothesis 3 was supported by the data: larger clinics of both classifications (independent and provider-based) had a positive impact on efficiency (independent model: standardized β = .238, p <0.000; provider-based model: standardized β = .416, p <0.000). This result is consistent with findings in the literature that suggest that larger clinics are more efficient than smaller ones¹¹.

Table 3: Parameter estimates of the structural equation model (standardized regression weights)

In addition to 'size,' the (inverse of) 'cost per visit' and 'score' (technical efficiency score) were associated with higher efficiency for clinics of both classifications. Interestingly, the analyses also revealed that some performance-related factors varied depending on the clinic classification as independent or provider-based. For example, for independent RHCs, two characteristics were associated with higher efficiency: for-profit control and the percentage of persons in poverty in the county where the RHC is located. Among provider-based RHCs, one characteristic was associated with lower efficiency: location in the South region. There were additional factors associated with effectiveness for provider-based clinics. A higher proportion of NPs (as a percentage of total physicians + PAs + NPs) was positively associated with effectiveness. This finding must be interpreted with caution, however. It does not imply that the proportion or PAs or mid-levels as a whole are not making an impact on effectiveness. Rather, it is an indication that among provider-based clinics there was less variability in the proportion of PAs or mid-levels compared with NPs. Finally, provider-based clinics located in counties with higher percentages of persons in poverty were associated with lower effectiveness.

Discussion

Although rural communities vary considerably from region-to-region across the country, much can be learned from aggregated and trend data, such as that used in this study. These findings have several implications for the management of RHCs. These concern optimizing RHC performance, providing quality services to rural residents, and enhancing the value of available RHC data.

Optimizing rural health clinic performance

The lack of relationship between RHC efficiency and effective prevention services and outcomes suggests that there is little variation in the levels of efficiency or effectiveness measured in this study. This lack of variation may be explained by the considerable constraints RHCs experience in their financial and human resources. Furthermore, efficiency in management and operations may be optimized only if optimal performance measures (benchmarks) are known and achievable through the improvement of the production function of RHC practice.

The analyses found large clinics to be more efficient than small ones. However, many rural communities are too sparsely populated to support large RHCs. In fact, the mean size of RHCs in the panel was 2.63 FTEs (total of physician + PA + NP FTEs) for independent clinics, and 2.08 for provider-based clinics. This finding suggests that small clinics that choose to join integrated systems, or formal or informal networks, may achieve efficiencies through sharing of human, technological, and other resources, and/or by maximizing purchasing power and opportunities for developing human resources.

The findings seem to indicate that independent and provider-based clinics respond differently to type of control and demographic factors. Perhaps because they have a greater focus on revenue and expenses, independent clinics under for-profit control were associated with higher efficiency. Those independent clinics located in counties with a higher proportion of persons in poverty were also found to be more efficient. Neither of these findings held true for the provider-based clinics in the study panel, however.

Providing quality services

Pay-for-performance program measures must incorporate adjustments for the regional variation in demographics of RHC service areas. The results indicate that a higher proportion of persons in poverty in the RHC's county is related to lower effectiveness but higher efficiency. In addition to poverty, the percent of migratory populations, elderly, and other at-risk groups must be taken into account in measuring performance.

Developing RHC services requires ongoing assessment of community health needs. Community level health will continue to be enhanced through the development of the rural health infrastructure including information technology. The RHC providers need more ready access to information and developments in disease prevalence and incidence, environmental health, and other population health issues. Comparisons by region (eg US Department of Health and Human Services or US Census Bureau region) of the incidence of chronic illnesses or conditions such as diabetes will give insight for enhancing rural health services delivery.

Enhancing the value of rural health clinic data

There is a need for RHCs, health planners, policymakers, and researchers to have access to comprehensive sources of RHC data. Fulfilling this need must be taken on balance with the potential hardship providers would experience if they were expected to routinely collect, enter, and report additional operational data. One approach might be to share the services of a trained health informatics specialist among multiple RHCs for the data entry, organization, and reporting functions.

It is important that data collection tools capture the distinctions between independent and provider-based clinic operations. In an effort to meet community needs, services provided by a clinic that is free-standing (independent) and/or located far from other healthcare services may vary from those of provider-based clinics. For example, the free-standing clinic often provides basic emergency services that a provider-based clinic (located at or near a hospital) can refer to their emergency room.

Finally, there is a need to use long-term trend data to better anticipate how rural systems will react and behave in the future²¹. Trend data on utilization, productivity, costs, and staffing at the clinic, regional, and national levels is needed to enhance the ability of RHC administrators and policy-makers to plan for the future.

Limitations

This study was not a comprehensive evaluation of RHC performance; a limited number of indicators were used to measure efficiency and effectiveness. Furthermore, the final samples were limited to those clinics for which complete data existed. Consequently, although the present findings may not be generalized to the entire population of RHCs, they do describe the experiences of large samples of RHCs.

The calculations of ACSC rates used the referring RHC as the provider; the ACSC rate have not been adjusted to account for the migratory patients who may have received services at multiple RHCs. In addition, the effectiveness measures were not risk-adjusted because the Medicare claims files did not contain specific information pertaining to the severity of illness.

Finally, the lagged effect of technical efficiency on clinical outcomes was not ascertained in this study because the study period was limited to 2 years. Had longitudinal Medicare data been available for a period of 3 or more years, this lagged effect of technical efficiency on clinical outcomes could have been determined. Ideally, the assessment of RHC performance should be based on specific 'gold standards' in terms of operational and process efficiencies, provision of preventive services, and patient outcomes. The lack of more than 2 years of observational data on operational and process efficiencies prevented full examination of the net influence of efficiency on clinical outcomes, holding organizational characteristics constant.

Future research

There is a critical need for designing and implementing a summative evaluation of RHCs' performance, as evidenced by the federal budget allocations and investments in the RHC program. However, efficiency and effectiveness of RHCs cannot be concomitantly achieved without identifying best practices under the highly diverse financial, demographic, and political conditions and constraints found throughout the USA. The analytical approach incorporates these factors and is recommended for a future research study that would analyze multiple years of RHC data.

Conclusions

The RHCs play a key role in serving the medically underserved. This study clarified the relationship between efficiency and effectiveness of RHCs. In addition, it identified several factors that influence the variation in RHC performance. The present is a critical time in the history of RHCs as they transition to meet the goals and expectations of the US health system reform. Additional research is needed to quantify and trend RHCs' contribution to the rural health delivery system in order to optimize their service to rural populations.

Funding for this research was received from the Office of Rural Health Policy of the Health Resources and Services Administration, US Department of Health and Human Resources, award number 4 R04RH10661-01-01. The authors acknowledge John Gill, PA, President of the National Association of Rural Health Clinics, as Advisory Consultant on this article. Graduate research assistants who assisted with this project were Natthani Meemon, Seung Chun Paek, and Abiy Agiro. The analysis for this article was supported by the Targeted Rural Health Research Grant Program, Office of Rural Health Policy, Health Resources and Services Administration, US Department of Health and Human Services. The opinions and statements made in this article reflect the authors' views only and do not represent those of the funding agency.

References

1. National Rural Health Association (NRHA). Support all rural providers with HIT money in stimulus. (Online) 2010. Available: http://www.nrha.org (Accessed 26 October 2010).

2. Centers for Medicare and Medicaid Services. Rural health clinic fact sheet. (Online) 2009. Available: http://www.cms.gov (Accessed 1 September 2009).

3. U. National Library of Medicine and National Institutes of Health. Rural health concerns. (Online) 2008. Available: http://www.nlm.nih.gov (Accessed 21 February 2008).

4. McAlearney JS. Financial Performance of Community Health Centers, 1996-1999. Health Affairs 2002; 21(2): 219-225.

5. Finkler SA, Knickman JR, Hanson KL. Improving the financial viability of primary care health centers. Hospital & Health Services Administration 1994; 31(1): 117-131.

6. Parsons R, Barnes J, Highley H. The community health center in a changing health care environment: the price of survival. Health Marketing Quarterly 1989; 6: 127-135.

7. Thometz A. A Summary Report of the First National Survey of Rural Health Clinics. Office of Rural Health Policy. Bethesda, MD: Global Exchange, 1994.

8. Gale JA, Coburn AF. The characteristics and roles of rural health clinics in the United States: A Chartbook. Portland, ME: University of Southern Maine, 2003.

9. Sinay T. Productive efficiency of rural health clinics: the Midwest experience. Journal of Rural Health 2001; 17(3): 239-250.

10. Institute of Medicine. Primary care: America's health in a new era. Washington, DC: National Academy, 1996.

11. Wan TTH. Analysis and evaluation of health care systems: an integrated approach to managerial decision making. Baltimore: Health Professions, 1995.

12. Wan TTH. Evidence-based health care management: multivariate modeling approaches. Boston, MA: Kluwer Academic, 2002.

13. Centers for Medicare and Medicaid Services. CMS Online survey, certification, and reporting, OSCAR. Baltimore, MD: CMS, 2007.

14. Centers for Medicare and Medicaid Services. Independent rural health clinic and freestanding federally qualified health center cost report. Baltimore, MD: CMS, 2006.

15. Centers for Medicare and Medicaid Services. Hospital and hospital complex cost report. Baltimore, MD: CMS, 2006.

16. Centers for Medicare and Medicaid Services (CMS). Independent rural health clinic and freestanding federally qualified health center cost report. Baltimore, MD: CMS, 2007.

17. Centers for Medicare and Medicaid Services. Hospital and hospital complex cost report. Baltimore, MD: CMS, 2007.

18. Centers for Medicare and Medicaid Services. Chronic care condition data warehouse. Baltimore, MD: CMS, 2007.

19. Bureau of Health Professions. Area resource file (ARF) access system. Rockville, MD: Bureau of Health Professions, 2007.

20. Ozcan YA. Quantitative methods in health care management: techniques and applications. San Francisco, CA: Jossey-Bass, 2005.

21. Ricketts TC. Rural health research and rural health in the 21st Century: the future of rural health and the future of rural health services research. Journal of Rural Health 2002; 18(5): 140-146.

Appendix I: Variables and operational definitions