Quality assessment of primary care for common mental disorders in isolated communities: Taking advantage of health records

full article:

Introduction

Depression and anxiety disorders can have a negative effect on an individual's professional, academic, social and family life, with significant consequences for loved ones^1,2. The high prevalence of these disorders is pushing health systems to increase the roles of primary health care (PHC) in order to recognize, assess, treat and follow up the majority of those affected by these conditions³. For individuals and families, PHC represents the first level of contact with the system of care and services (SCS)⁴. These services are regarded as less costly, more accessible and often more acceptable than specialized mental health services, which should be reserved for individuals with more complex conditions and needs^3,5. The WHO recommends evaluation organizational methods and clinical practices to ensure the effectiveness and quality of PHC for people with common mental disorders⁶.

In the province of Quebec, Canada, little information is available on the nature and quality of PHC offered to people with mental health problems living in isolated rural communities. In addition, few assessment tools are available to decision-makers regarding quality of care adapted to their context. Researchers worked in collaboration with decision-makers involved in two isolated regions of Quebec to adapt procedures and research tools so as to provide them with a way to assess the quality of PHC offered within their populations to those with various mental health-related needs. This article focuses mainly on the methodology developed for assessing PHC quality associated with common mental disorders, such as depression and anxiety disorders.

Primary health care in isolated rural communities in Quebec

Quebec province is a vast area of 1 667 441 km². The province is sparsely populated: the majority of its eight million residents live in the southern part and occupy approximately one-fifth of the territory. The official language is French. Approximately 1% of Quebec's population live in isolated areas and 50% of this segment of the population belong to one of the 11 indigenous groups in the province⁷. From the health services perspective, isolated rural communities are defined as those situated more than 3.5 hours of travel time from a major urban center⁸. Health services are generally offered by a multidisciplinary team within the healthcare providers (nurses, social workers and community workers). To meet the health and social needs of the population an expanded clinical practice is adopted^9-14. Most health centers in isolated rural communities have no doctor or psychologist on site, but remote support is available to the PHC team and medical visits are organized on a regular basis. However, the provision of comprehensive and effective care for people with mental health needs can pose challenges, given the scarcity of local mental health resources, geographical distance from specialized resources, inadequacy of proposed guidelines, and the lack of support for initial clinical management. This situation is further complicated by a high employee turnover, because a large number of PHC providers come from elsewhere^9-11,15-18. In many cases, problems could remain undiagnosed¹⁸.

Measuring the quality of care and services

Quality of care is about 'delivering the best possible care and achieving the best possible outcomes for people every time they deal with the healthcare system or use its services. Essentially, it means doing the best possible job with the resources available'¹⁹. Accordingly, quality care is effective, person-centered, accessible, timely, efficient, safe and fair^20,21. Quality is typically measured based on indicators used as reference points. Quality indicators can relate to healthcare structures, to technical (prevention, detection, access, assessment, treatment/adherence, coordination, continuity, safety) and interpersonal (communication, decision-making, interpersonal style) clinical processes, as well as to outcomes for patients or healthcare systems²². Indicators can be developed through systematic or non-systematic methods²³. In systematic methods, indicators are first identified using evidence-based data from empirical studies or practice guidelines. Then, the indicators are reviewed by experts in the field to retain the most relevant²³. In non-systematic methods, indicators are identified through available data and critical incidents documented from case studies²³.

Taking advantage of health records in remote regions

Three data sources - patient-reported data, administrative records and health records - are frequently used to assess quality indicators in PHC settings^20,24,25. Patient self-reported data are collected using questionnaires or by conducting interviews. These can provide information on technical and interpersonal aspects of care as well as their impact on patient experiences of care²⁵. Such an approach has been used in mental health research in isolated populations^18,26. However, concerns were expressed by decision-makers involved in the present project about repeated solicitation from the research community and its potentially negative impact on the population under investigation. The advantage of medical administrative databases is that information can be systematically collected for all service users across a given clinical setting or system. In mental health, this is based on the Diagnostic and statistical manual of mental disorders (DSM)²⁷ and the International classification of diseases (ICD)²⁸, standardized diagnostic coding systems recording hospitalizations, outpatient visits and prescribed medications²⁵. However, the databases currently available in Quebec do not consider the particularities of PHC in isolated rural communities, such as the expanded role of healthcare providers^9-14.

This study aimed to better support local and regional stakeholders in their efforts to improve their population's access to quality mental health services by addressing the research question, 'Are health records a valuable source for assessing quality of PHC offered to individuals with common mental disorders who live in Quebec's isolated rural communities?' The goal was to identify useful, measurable and valid indicators^12,24,25,29 adapted to the context of health services in isolated rural communities and that could be measured from information found in health records.

The focus was primarily on developing indicators for technical-type clinical processes that reflect the four steps for managing service needs for those with common mental disorders: (1) recognition, (2) assessment, (3) treatment and (4) follow-up. Recognition involves early detection so that common mental disorders can be treated early to minimize functional impairment^30,31. Information is collected during the assessment to identify the best treatment for the individual^32-34. This information includes current medical history, signs and symptoms, risk of suicide or hetero-aggressiveness, health, illness and treatment history, lifestyle, diagnoses, functional impairment intensity and the individual's personal preferences. Stepped care models are recommended in several practice guidelines for common mental disorders^33,35,36. This approach suggests varying treatment types and levels based on the individual's needs and preferences^37,38. The PHC providers have also to take into consideration their own clinical experience/skills, current legislative policies and available resources³⁸. Consequently, patients with mild functional impairment are typically offered low-intensity interventions (such as bibliotherapy, supportive self-management, lifestyle interventions, light therapy, brief psychological treatment, all of which are readily available to all care providers). Conversely, high-intensity interventions (pharmacology, psychotherapy, combination therapies, electroconvulsive therapy, hospitalizations) are available to individuals with more severe symptoms or who have relapsed^33,36,39. They require specialized medical care or mental health resources. Finally, the effectiveness and efficiency of care relies on the progression of treatment intensity based on an individual's health status and the implementation of sustained personalized follow-up^37,38.

Methods

Using a systematic method to create quality indicators could result in assessment criteria that are too rigid and unsuitable for remote isolated regions. On the other hand, a non-systematic method based on case studies could limit the potential for comparison of various remote isolated regions and identication of promising ways for improvement. Therefore, indicators were developed using a hybrid approach that integrates evidence, opinions from key informers and case studies derived from health records.

Locate evidence

A preliminary list of quality-of-care indicators adapted to isolated rural communities was created using scientific evidence sources, institutional records and case studies. Evidence was identified on the basis of recognized practice guidelines for depression and anxiety disorders^40,41 and indicators from the National Inventory of Mental Health Quality Measures (NINHQM)²⁵. Institutional records refer to two qualitative studies on mental health services done in isolated regions of Quebec^10,11 and that were consulted in order to identify contextualized aspects deemed to be good quality indicators. Case studies were reconstructed using data collected from the health records of people with common mental disorders in the two isolated rural regions being studied. Means undertaken to gather information from health records are presented later in this section.

Ensure adaptation of quality indicators to the context of rural isolated communities

The adaptation of quality-of-care indicators to rural isolated communities was appraised according to their relevance, measurability and construct validity in this context.

Relevance was examined by analysing each indicator on the preliminary list of quality-of-care indicators. This was done together with three members of the mental health team (one psychologist, a nurse and a social worker) from two participating health centers. The aim was to retain the relevant indicators and make adjustments as necessary. Each indicator was presented and discussed in face-to-face meetings.

Measurability involves verifying the possibility of reconstructing indicators deemed relevant using the data collected from health records and establishing a measurability score for each indicator, if applicable. The allotment of the measurability scores aims to promote use of the indicators by research partners. Figure 1 presents the measurability chart where the score (1=easy, 2=moderate, 3=complex) took into account the nature of the denominator and the type of variables. Denominators could refer to the entire observation period, specific events or care episodes, ie a consolidation of related events that could extend over time. Variables can be primary variables stemming directly from the collected data (Fig2) or secondary variables created from primary variables or narrative text.

The construct validity ultimately relies on the known-groups method, which considers an indicator's construct to be valid when it is possible to demonstrate statistically significant differences between groups known to be theoretically different^42,43. For this, the results of relevant and measurable indicators are subjected to a series of bivariate analyses based on individual characteristics (sex, age group, presence of chronic physical illness, presence of substance abuse disorders, presence of social problems, predominance of anxiety symptoms), organizational characteristics (presence of a local permanent physician) or contextual characteristics (regions) likely to cause measurement differences and documented from health records. When the denominator referred to the observation period, bivariate analyses were conducted using the χ² test. When the denominator referred to events or episodes of care, the generalized estimating equation (GEE) was used to account for multicollinearity data, given the possibility for a single individual to accumulate many events/episodes⁴⁴.

Figure 1: Measurability chart Figure 2: Fields for the data entry form

Gather data from health records

A primary focus was on individuals aged 14 years or more presenting depression or anxiety disorder and using PHC in isolated rural communities throughout Quebec. The choice of this minimum age satisfies the need of research partners to check if there are breaks in the continuum of care between youth and adult services. The data collection in health chart was deployed in an iterative three-step process: (1) feasibility analysis, (2) development of the data collection process and tool and (3) the actual data collection.

1. Feasibility analysis: In this first step, health records of patients with a common mental disorder were examined to verify the type of sociodemographic and clinical information they contained, to assess their relevance, clarity and format^45,46, and to identify elements, such as biases or potential errors, that might affect their validity^45,47. This step was conducted over a 5-day period in 2007 in an isolated rural community health center in Quebec. Since no registry exists to identify patients with common mental disorders, a systematic review of the archives was performed to identify patients presenting a mental disorder. In health center archives, health records are organized by file number, in ascending order. These numbers are assigned chronologically by the regional health service: at birth for all those born in a community, and at first consultation for those from outside the community.

Notes were reviewed for all selected individuals aged 14 years or more on 1 January 2006, to identify those who presented with mental disorders. In the absence of a relevant medical diagnosis, a set of psychological counselling, medication(s), clinician impressions (ie differential diagnosis), or signs and symptoms associated with mental disorders, was used to identify potential cases. During the first day of the search, health records for 305 people were reviewed. Of these, 29 (9.5%) had a confirmed or suspected diagnosis of mental health disorders (severe or common). During the remaining four days, these 29 records were examined in detail to document each step of the clinical management process: recognition, assessment, treatment and follow-up. Data collected included sociodemographic characteristics, nature of the consultations, services consulted, reason(s) for initial consultation, current medical history, clinical signs and symptoms. Information retrieved based on documentation practices varied between providers, especially when it came to assessment of comorbidity, physical/mental/social history, substance abuse disorders and the individual's resources. Level of functional impairment was rarely documented; however, this could often be established retrospectively from clinical notes (ie symptoms and illness trajectory). More importantly, the record review showed that a 12-month observation period was not sufficient to assess a longitudinal clinical process such as follow-up. Confounding issues were frequent and included use of acronyms, illegible notes, and dual language documentation (eg dates are written differently in French and in English: YYY-DD-MM in English and YYYY-MM-DD in French).

2. Developing a data collection process and tool: Observations made in step 1 influenced how health records were selected, the length of the observation period, the type of variables required, and also the competencies of those who would be assigned to data collection. The secondary purpose of step 2 was to develop data collection tools. Given the absence of a registry for individuals with common mental disorders and the imprecision of certain clinical diagnoses, a tracking method based on experiences from step 1 was developed for locating health records. Records meeting the following criteria were selected:

1. Individual was aged 14 years or over on 1 January of the reference year.
2. Individual received health services during a reference year. At least one note, during the reference year, had to have indicated one of five conditions:

a. medical diagnosis of depression or anxiety disorder

b. psychiatric hospitalization, or referral to or consultation with a psychiatrist, psychologist or other mental health specialist

c. prescription for anxiolytic or antidepressant medication

d. presence of suicidal ideation, a suicide attempt or a suicide

e. showing at least two signs and symptoms of a major depressive disorder and anxiety disorders²⁷, present most of the time for at least 2 weeks (persistent) or documented on two occasions over a period of 6 months (recurring). Signs and symptoms cannot be due to physical causes, medication, substance abuse or a normal reaction to a difficult situation.

The second criterion was observed starting from the reference year end (December to January). A 2-year observation period was considered sufficient to obtain valid care trajectories. This period of observation was applied by creating a window of 12 months before and after the first observation of the second selection criterion.

The need to record a large amount of information over a long time period justified the use of a computerized tool. The data entry form was set up using the freeware program Epi Info v3.5.1 (Center for Disease Control and Prevention; http://wwwn.cdc.gov/epiinfo) that automatically generates an Access database. This database consists of two tables linked by a numerical code assigned to each record. Each row in the first table corresponds to a person and each row of the second table documents a note to the health record.

A first pre-test was done to ensure the feasibility and reliability of the data collection process and tools. In the pre-test, the average time to identify and process a record was set at 180 minutes. An inter-rater reliability test was carried out afterward in an isolated rural community by two bilingual (French, English) research nurses with experience working in remote regions. Using six records, an inter-rater agreement was measured on 15 statements corresponding to types of events (n=6), assessment aspects (n=4), treatment (n=4) and follow-up (n=2) from 30 randomly determined events. Adjustments were made to all statements generating an inter-rater agreement (kappa) less than 0.80. Whenever research nurses had trouble classifying certain information because of wording differences between health notes and the data entry form, some check boxes were replaced by text boxes. Figure 2 presents the fields from the final version of the data entry form.

3. Data collection: This step involved collecting data in 21 isolated rural community clinics that are associated with three health centers in two of Quebec's health regions. The population of 21 communities contained 11 500 individuals aged 14 years or more⁴⁸ . Data collection was carried out between October 2009 and August 2010. Reference year was set as the year 2007 and therefore the 24-month observation period was between January 2006 and December 2008. Based on the findings in step 1 (ie prevalence of 9.5%), the target population was estimated at 1090 records. Given the processing time (180 minutes), the estimated sample that could be obtained in the allowable time (918 hours) was established at 360 records. Records were selected using a randomly drawn number and reviewing all records that ended with that particular number. The selection process lasted until the pre-established number of records for each community in proportion to its population was reached.

Each day, the information gathered was submitted to a member of the research team in order to correct any input errors before the research nurse moved on to another community. The database included an electronic communication platform for sharing observations on processed records, sites visited and notes on the data collection process.

Health records from 3669 people aged 14 years or more were reviewed for all clinics and 290 records (7.9%) met the selection criteria. For these records all care or service events associated with mental health were documented. The collected data were anonymized and an alphanumeric code assigned to each health record.

All diagnoses rendered during the observation period were reviewed to identify cases of depressive or anxiety disorders. If no diagnosis was available, research diagnoses were established using signs and symptoms documented during the 2-year observation period. These signs and symptoms were included as free text in the database (in narrative text format) and had to be classified beforehand based on diagnostic criteria from DSM-IV-TR (4th edition, text revision) for a major depressive disorder and the most common anxiety disorders²⁷. Records where there were at least two signs and symptoms corresponding to diagnostic criteria for depression or anxiety disorder unexplained by other conditions were retained. The final sample for common mental disorders included 218 records (5.9%).

Ethics approval

This research received administrative approval from participating health centers for the access to health records and ethics approval from the Research Ethics Committees of the Agence de la Santé et des Services Sociaux de Montréal (Project 162) and Université Laval. Researchers and nurses who accessed the health records for this research signed a confidentiality agreement beforehand.

Results

Selection of indicators

A total of 36 quality indicators for clinical procedures that could be adapted to isolated rural communities were initially selected after reviewing relevant practice guidelines for depression and anxiety disorders^40,41 and using indicators from the NINHQM²⁰. Eleven new indicators were added following the analysis of the mental health services qualitative studies^10,11 and the 10 case studies from health records. Finally, a total of 47 quality indicators were set at the start of the study process.

Relevance of study

Members of the mental health team rejected three of the 47 indicators as irrelevant: rapid access to services (treatment), assessment of psychotic features in depression (assessment) and involuntary detention (treatment). First, rapid access to care in an emergency was not deemed an issue in this context since every person has immediate access to a nurse 24 hours a day and 7 days a week. Access to a second level of care can vary significantly from one context to another and establishment of a standard is difficult. Second, the assessment of psychotic features in depression was not deemed a priority as indicators for assessing suicide risk or concomitant disorders were preferred. Third, involuntary detention was considered descriptive data rather than a quality-of-care indicator.

Measurability and construct validity studies

Of the 44 indicators remaining, 10 could not be measured using the health record data. Their nature and the reasons for rejection are specified in Table 1. The definitions, origins, nature of the denominators, primary and secondary variables needed to create them, measurability scores as well as measures of the 34 indicators considered relevant and measurable are illustrated in Table 2. Of these, 11 indicators could not be subjected to bivariate analyses because a denominator was less than 30 (n=5), there was an insufficient number of occurrences per compared group (n=3), or the services were not available at the time of the research (n=3).

The remaining 23 indicators were cross-referenced for various individual, organizational and contextual characteristics documented from health records. Eighteen indicators were significant (p≤0.05) for at least one comparison group (ie construct validity) (Table 2). These indicators were mainly related to the assessment (n=5) and follow-up (n=8) processes.

Table 3 shows that scientific resources (empirical studies and practice guidelines) helped identify 12 relevant, measurable and valid indicators. This represents a third of the 36 indicators initially identified using this source. Consultation of institutional records would have led to the selection of 9 indicators, 41% of the 22 identified using this source. As for case studies, 10 indicators would have been identified, 55% of the 17 identified using this source. This table also shows that all the indicators identified using institutional records were identified with the two other sources as well. These indicators were all derived from the practice guidelines/NINHQM and case studies.

Table 1: Non-measurable quality-of-care indicators for common mental disorders (n=10) Table 2: Relevant and measurable quality indicators (n=34) tested for construct validity Table 3: Number of quality indicators selected according to function and from data sources

Discussion

This article addresses the question of whether health records are valuable sources for assessing the quality of care offered to individuals with common mental disorders who live in isolated rural communities. Making publicly available the tools developed and the selected indicators necessitates a discussion on the strengths and challenges associated with data sources, the representativeness of the sample, the method used to identify quality-of-care indicators and the usefulness of developed indicators.

Health records as a data source

In the field of mental health, health records have been described as underexploited data sources⁴⁷. They provide an abundance of clinical information while keeping participants anonymous, which is critical for conducting research in small communities. In addition, the use of health records is facilitated by the simplicity of clinical structures in isolated rural communities. Isolated areas have an advantage over non-isolated areas where this type of exercise is complicated by the multiple access points to healthcare systems. However, health records also raise methodological challenges and limitations to be considered. First, records are not systematic and documentation varies widely. The quality of notes is an important factor/variable that is not controlled for. Moreover, attempting to decipher why certain information is missing is arbitrary. Did the PHC provider forget to write it down or does it reflect the inconsistency of clinical processes? Therefore, results are based on the feeble assumption that PHC providers, at least, noted information on what was reported to them, what they observed and what they did.

Accordingly, a major challenge in using records relates to the complexity of data collection^45,47,49. The credibility of collecting data from health charts is based largely on the documentation of biases and potential entry errors during previous steps, which helped develop ways to control them. Recording data in a free/narrative text format significantly increased data collection time and complicated data preparation for analysis. Staff motivation, isolation and fatigue were also a challenge in the data collection process and, as a result, time spent in the field was reduced from 6 weeks to 4 weeks. In short, health records are valuable for assessing the quality of care offered to individuals who have common mental disorders and live in remote regions, but the data collection process is demanding. Also, additional data collection for monitoring quality of care in participating regions could be simplified. A review of primary and secondary variables needed to create indicators (Table 2) revealed that all the categories of collected data (Fig2) were useful. However, documentation of non-mental health-related events could be trimmed (eg dates, PHC providers involved and reasons for generic consultation). Check boxes could also be reintroduced since the most frequently used terms to describe signs and symptoms, as well as assessment and treatment components, were documented during data collection. These terms could be specified in the reference manual.

Representativeness of the sample

The method used for identifying records involving individuals with common mental disorders circumvents the under-diagnosis of these conditions in primary care^18,32. The rate of 5.9% of records (n=218) obtained for people 14 years or more appears to be consistent with national prevalence rates, which show that about 12% of Canadians undergo a high level of psychological distress each year, indicating a possible major depressive disorder or other mental problems⁵⁰. Of these individuals, 45% will consult healthcare services for these matters⁵¹. The selection of records based on signs and symptoms associated with common mental disorders identified 22% of the cases. These selection criteria for cases go beyond the medical diagnosis to assess the recognition function and expand the quality-of-care study to include those individuals who had not met all the diagnostic criteria for common mental disorders, but who had symptoms that could develop into a disorder if nothing was done. However, these 'expanded' criteria promote sensitivity (selecting real cases) at the expense of specificity (rejecting false cases). Use of clinical and research diagnostics revealed an efficient way to identify and reject any such false cases.

Studying a large enough sample to translate results to the general population poses scientific and logistic challenges when research is carried out in remote settings, largely due to limited populations and expenses. The fact that certain indicators might be measured from events or care episodes enhanced statistical power. Also, while statistical significance was not reached in the retrospective study of 10 records, the review produced clinically useful results^45,47,52,53. Considering these limitations, any quality-of-care assessment in remote regions would benefit from a mixed research design to triangulate results, either statistically or clinically, including qualitative data and input/interpretation by local stakeholders.

Method used to identify quality-of-care indicators

In the present study, the hybrid method of reviewing scientific evidence and case studies from health records would have been appropriate to identify relevant, measurable and valid quality indicators. This hybrid selection method underscores the importance of contextualizing indicators because certain indicators initially deemed measurable theoretically (eg mention of respect for the person's preferences²⁵) were not in fact measurable. Therefore, a strength of the approach used in this study was to compare data with the opinion of key informers.

Usefulness of developed quality-of-care indicators

Current mental health reforms advocate integration of resources and between service levels^3,39. Yet, service integration experiences producing the best results are those that place an added value in clinical practices rather than in structures⁵⁴. Assessing care is therefore a useful way to target clinical practices in need of improvement⁵⁵. Also, data collection tools and databases using accessible software (Access and Epi Info) as well as measurability scores^20,56 are designed to promote transferability to research partners from participating regions. With measurability scores (Fig2), research partners can easily identify indicators for which data are easier to collect and analyze, and include them in an audit process. The transferability of developed indicators and deployed method to other settings with similar processes of care and contexts need to be studied.

It is also evident that more work is needed to better document the relationship between clinical procedures and desired outcomes in the context of research⁵⁵, to ensure the sensitivity of the indicators to change²³ and to adjust them over time²⁰. Several indicators are used to measure follow-up and assessment functions. However, few of these measure treatment aspects, including adequacy of care. To achieve this, research partners from isolated regions must work to adapt guidelines to their context (culture, available resources, etc.). While the approach in this study uses data collected a few years ago, the indicators developed are still relevant. Measurements from the 2006-2008 data can be used as a benchmark for assessing progress. Descriptive information can be drawn from the data collected in health records in support of this study. Most work on the adequacy of treatment for a common mental disorder is derived from a medical diagnosis²⁴. Use of the functional impairment level, which can be established after the fact from narrative text and the Global Assessment of Functioning (GAF) scale²⁷ , is an avenue to explore for assessing future treatments. The GAF scale allows PHC providers or researchers to rate subjectively the social, occupational and psychological functioning of adults from their health history on a hypothetical continuum of mental health-illness (1 through 100). Such an approach appears well suited to multidisciplinary primary care especially when no diagnosis is available, as is often the case in isolated rural communities^18,26.

Conclusions

Depression and anxiety disorders affect society's most productive age groups and represent a burden in small isolated rural communities^57-60. To target areas for improvement in care, and for providers to offer solutions adapted for affected individuals and their families^54,61,62, they must be relevant, measurable and valid quality indicators that are contextualized to local challenges. The approach presented in this paper has identified 18 quality indicators that may be useful for better understanding the PHC offered in rural isolated communities. These indicators are a suitable monitoring tool for those involved in improvement of quality of care in these contexts.

Acknowledgements

The authors would like to thank the research partners for their broad support. These are the Nunavik Regional Board of Health and Social Services, the health care centers in Tulattavik, Inuulitsivik, La Romaine and Pakuashipi, the Centre de santé et des services sociaux de la Basse-Côte-Nord, the Cree Board of Health and Social Services of James Bay, and the Institut National de santé publique du Québec. The Trajectoires project, which provided most of the data for this research, was financed by the Canadian Institutes of Health Research (CIHR) and the Ministère de la Santé et des Services sociaux du Québec (MSSS) within the Partnerships for Health System Improvement program. L. Lessard received doctoral scholarships from the Fondation de recherche en sciences infirmières du Québec (FRESIQ), Quebec's provincial education ministry, the National Network of Aboriginal Mental Health Research and the Canada Research Chair on Comparative Aboriginal Conditions. L. Fournier held a CIHR-MSSS-Fond québécois de recherche en santé (FQRS) Research Chair in Public Health at the time of this project.

References

1. Langlois K, Samokhvalov A, Rehm J, Spence S, Connor Grober S. Health state descriptions for Canadians. Ottawa: Statistics Canada, 2012.

2. Mathers C, Loncar D. Projections of global mortality and burden of disease from 2002 to 2030. PLOS Medicine 2006; 3: 2011-2030.

3. World Health Organization. Integrating mental health into primary care: a global perspective. Geneva: WHO, 2008.

4. Starfield B. Primary care: balancing health needs, services and technology. New York: Oxford University Press, 1998.

5. World Health Organization. Declaration of Alma-Ata. Alma-Ata, USSR: WHO, 1978.

6. World Health Organization. Human resources and training in mental health. Geneva: WHO, 2005.

7. Duhaime G, Godmaire A. Les modèles de développement du nord. Analyse exploratoire au Québec isolé. Recherches sociographiques 2004; 43(2): 329-351.

8. Gauthier J, Haggerty J, Lamarche P, et al. Entre adaptabilité et fragilité: les conditions d'accès aux services de santé des communautés rurales et éloignées. Québec: Institut national de santé publique du Québec, 2009.

9. Strasser R. Rural health around the world: challenges and solutions. Family Practice 2003; 20(4): 457-463.

10. Lessard L, Aubé D. Contextual study of mental health services in the Cree Iiyiyiu Aschii. Québec: Institut national de santé publique du Québec, 2008.

11. Lessard L, Bergeron O, Bruneau S, Fournier L. Contextual study of mental health services in Nunavik. Québec: Institut national de santé publique du Québec, 2008.

12. Boothroyd L, Kirmayer L, Spreng S, Malus M, Hodgins S. Completed suicides among the Inuit of northern Quebec, 1982-1996: a case-control study. Canadian Association Medical Journal 2001; 165(6): 749-755.

13. Brown R. Australian Indigenous mental health. Australian and New Zealand Journal of Mental Health Nursing 2001; 10(1): 33-41.

14. Kirmayer L, Corin E, Corriveau A, Fletcher C. Culture et maladie mentale chez les Inuit du Nunavik. Santé Mentale au Québec 1993; 18(1): 53-70.

15. Topping S, Calloway M. Does resource scarcity create interorganizational coordination and formal services linkages? A case study of a rural mental health system. Advances in Health Care Management 2000; 1: 393-419.

16. Harris A, Robinson G. The Aboriginal Mental Health Worker Program: the challenge of supporting Aboriginal involvement in mental health care in the remote community context. Australian e-Journal for the Advancement of Mental Health 2007; 6(1): 1-11.

17. Martin G. On rural services for mental health. Australian e-Journal for the Advancement of Mental Health 2007; 6(1): 1-4.

18. Kirmayer L, Paul K. Mental health, social support and community wellness. Quebec: Institut national de santé publique du Québec, 2008.

19. Health Canada. Quality of care. (Online) 2011. Available: http://www.hc-sc.gc.ca/hcs-sss/qual/index-eng.php (Accessed 18 March 2014).

20. Institut national d'excellence en santé et services sociaux. Des indicateurs de qualité à l'intention des professionnels et des gestionnaires des services de première ligne. ETMIS 2012; 12(1): 1-68.

21. Bengoa R, Kawar R, Key P, Leatherman S, Massoud R, Saturno P. Quality of care: a process for making strategic choices in health systems. Geneva: World Health Organization, 2006.

22. Donabedian A. The quality of care: how can it be assessed? JAMA 1988; 260(12): 1743-1748.

23. Campbell S, Braspenning J, Hutchinson A, Marshall M. Research methods used in developing and applying quality indicators in primary care. Quality and Safety in Health Care 2002; 11(4): 358-364.

24. Duhoux A, Fournier L, Menear M. Quality indicators for depression treatment in primary care: a systematic literature review. Current Psychiatry Reviews 2011; 7: 104-137.

25. Hermann R. Improving mental healthcare: a guide to measurement-based quality improvement. Arlington, VA: American Psychiatric Publishing, 2005.

26. Haggarty J, Cernovsky Z, Kermeen P, Merskey H. Psychiatric disorder in an Arctic community. Canadian Journal of Psychiatry 2000; 45(4): 357-362.

27. American Psychiatric Association. Diagnostic and statistical manual of mental disorders, 4th edn, text revision. Washington, DC: American Psychiatric Association, 2000.

28. World Health Organization. The ICD-10 classification of mental and behavioural disorders: clinical descriptions and diagnostic guidelines. Geneva: WHO, 1992.

29. Worster A, Haines T. Advanced statistics: understanding medical record review (MRR) studies. Academic Emergency Medicine 2004; 11(2): 187-192.

30. Kavanagh M, Beaucage C, Cardinal L, Aubé D. La dépression majeure en première ligne. Québec: Agence de la santé et de services sociaux de la Capitale-Nationale & Institut national de santé publique du Québec, 2006.

31. Joffres M, Jaramillo A, Dickinson J, et al. Recommendations on screening for depression in adults. Canadian Association Medical Journal 2013; 185(9): 775-782.

32. Fournier L, Aubé D, Roberge P, et al. Vers une première ligne forte en santé mentale: messages clés de la littérature scientifique. Montréal: Institut national de santé publique du Québec, 2007.

33. National Institute for Health and Clinical Excellence. Commissioning stepped care for people with common mental health disorders. London, UK: National Institute for Health and Clinical Excellence, 2011.

34. Shea S. La conduite de l'entretien psychiatrique: l'art de la compréhension. Paris: Elsevier, 2005.

35. National Institute for Health and Clinical Excellence. Common mental health disorders: identification and pathways to care. London: National Institute for Health and Clinical Excellence, 2011.

36. Fournier L, Roberge P, Brouillet H. Faire face à la dépression au Québec: protocole de soins à l'intention des intervenants de première ligne. Montréal: Centre de recherche du CHUM, 2012.

37. Von Korff M, Tiemens B. Individualized stepped care of chronic illness. Western Journal of Medicine 2000; 172(2): 133-137.

38. Brouillet H, Roberge P, Fournier L, Doré I. Les soins en étapes, une approche prometteuse pour le traitement des troubles anxieux et dépressif. Quintessence 2009; 1(1): 1-2.

39. Ministère de la santé et des services sociaux. Plan d'action en santé mentale 2005-2010: les forces des liens. Québec: Bibliothèque Nationale du Québec, 2005.

40. Canadian Psychiatric Association. Clinical practice guidelines : management of anxiety disorders. Canadian Journal of Psychiatry 2006; 51(8) (suppl. 2): 1s-97s.

41. National Institute for Health and Clinical Excellence. Quick reference guide. Depression: management of depression in primary and secondary care. London, UK: National Institute for Health and Clinical Excellence, 2007.

42. Hattie J, Cooksey R. Procedures for assessing the validities of tests using the 'known-groups' method. Applied Psychological Measurement 1984; 8(3): 295-305.

43. Fortney J, Sullivan G, Williams K, Jackson C, Morton S, Koegel P. Measuring continuity of care for clients of public mental health systems. Health Services Research 2003; 38(4): 1157-1175.

44. Hanley J, Negassa A, Edwardes M, Forrester J. Statistical analysis of correlated data using generalized estimating equations: an orientation. American Journal of Epidemiology 2003; 157(4): 364-375.

45. Engel L, Henderson C, Fergenbaum J, Colantonio A. Medical record review conduction model for improving interrater reliability of abstracting medical-related information. Evaluation & the Health Professions 2009; 32(3): 281-298.

46. Allison, J, Wall ,T, Spettell, C, et al. The art and science of chart review. Joint Commission Journal of Quality Improvement 2000; 26(3): 115-136.

47. Gearing R, Mian I, Barber J, Ickowicz A. A methodology for conducting retrospective chart review research in child and adolescent psychiatry. Journal of the Canadian Academy of Child and Adolescence Psychiatry 2006; 15(3): 126-134.

48. Statistics Canada. 2006 census. (Online) 2006. Available: http://www.statcan.gc.ca (Accessed 6 November 2009).

49. Wu L, Ashton C. Chart review. A need for reappraisal. Evaluation & the Health Professions 1997; 20(2): 146-163.

50. Patten S, Wang J, Williams J, et al. Descriptive epidemiology of major depression in Canada. Canadian Journal of Psychiatry 2006; 51(2): 84-90.

51. Kohn R, Saxena S, Levav I, Saraceno B. The treatment gap in mental health care. Bulletin of World Health Organization 2004; 82(11): 858-866.

52. Agence nationale d'accréditation et d'évaluation en santé. Réussir un audit clinique et son plan d'amélioration. La Plaine: Agence nationale d'accréditation et d'évaluation en santé, 2003.

53. Dixon N, Pearce M. Guide to ensuring data quality in clinical audits. London: Healthcare Quality Improvement Partnership, 2011.

54. Pineault R, Tousignant P, Roberge D, et al. Research Collective of the organization of Primary Care Services in Québec. Montréal: Agence de développement de réseaux locaux de services de santé et de services sociaux de Montréal, 2005.

55. Mainz J. Defining and classifying clinical indicators for quality improvement. International Journal of Quality of Health Care 2003; 15(6): 523-530.

56. Ebidia A, Mulder C, Tripp B, Morgan M. Getting data out of the electronic patient record: critical steps in building a data warehouse for decision support. Proceedings AMIA Symposium 1999; 745-749.

57. World Health Organization. The mental health context. Geneva: World Health Organization, 2004.

58. OECD. Improving value in health care: measuring quality forum on quality of care. Paris: OECD Publishing, 2004.

59. Pong R. Rural health research in Canada: at the crossroad. Australian Journal of Rural Health 2000; 8: 261-265.

60. World Health Organization. The world health report 2001 - mental health: new understanding, new hope. Geneva: WHO, 2001.

61. Sturmberg J, O'Halloran D, Martin C. Understanding health system reform - a complex adaptive systems perspective. Journal of Evaluation in Clinical Practice 2012; 18(1): 202-208.

62. World Health Organization. Systems thinking for health systems strengthening. Geneva: WHO, 2010.