Retrospective bibliometric review of rural health research: Australia's contribution and other trends

full article:

Introduction

In an era of rapid globalization and urbanization approximately half the world's 6 billion people still live in rural areas¹. Although rural populations in different settings vary considerably in their exposure to adverse social, environmental and biomedical conditions, there is nevertheless a worldwide disparity in health status in favour of urban populations. In addressing global health disparities the World Health Organization has broadened its focus beyond public health and has emphasized the important role of family practitioners in primary health care, and this includes those working in rural environments².

The health status of the 6 million Australians living in the 7.5 million km² that constitute rural and remote communities is demonstrably poorer than that of metropolitan residents³. In Australia an appreciation of the need to develop strategies to address rural health disparities began with the formation of the Rural Doctors Association of Australia in the mid 1980s, the first National Rural Health Conference in 1991 and the launch of the Australian Journal of Rural Health (AJRH), the context for these developments being well described in an editorial in the journal itself⁴. Emerging in the early 1990s as an 'identifiable field of activity focusing on improving the health status and meeting the specific health needs of people living "out back" of metropolitan' areas⁵, rural medicine is gaining acceptability as a distinct and separate discipline in Australia⁶.

At a national level, the Regional Health Strategy was the centrepiece of the federal budget in 2000, and this included funding for significant investment in rural and remote academic infrastructure through the University Department of Rural Health and Rural Clinical School programs⁷. Australia is now considered to be at the forefront of developed nations in developing rural health within policy frameworks that coordinate different levels of government⁸.

One measure of a specialty's standing in the medical community is its performance in research⁹. What constitutes research and research performance can be assessed in many ways. Bibliometric analyses are one of the ways in which the research performance of Australian researchers is traditionally analyzed¹⁰ and may even be used to formulate policy regarding research funding¹¹. Metric-based formulae may even replace traditional methods in the United Kingdom¹². Bibliometric analyses have been used to monitor research publications in domains such as medical informatics¹³ and general practice¹⁴. We have developed a simple method and reported on its utility to track national health and medical research expenditure¹⁵.

A key systematic overview of rural health research in Australia from 1990 to 2000 was published by Patterson¹⁶. A manual search of 14 Australian journals found 519 publications, of which 284 (55%) were considered research. An electronic search including international journals found an additional 151, of which 102 were classed as research. The AJRH and the Medical Journal of Australia were the journals in which publications appeared most frequently. Forty percent of the articles addressed the National Health Priority Areas (NHPAs), more than 30% addressed Indigenous health specifically, and 50% were directed towards public health or health services issues. The author suggested the paper could form a baseline for future research. (The NPHA project was a collaborative effort involving Commonwealth and state and territory governments in the mid-1990s to focus on common health risk factors. In 2002 these included arthritis, asthma, cancer, cardiovascular diseases, diabetes, injury and mental health. In 2006 the NHMRC identified a broader list of National Health Issues, including the pre-existing NHPAs but also adding health workforce, Indigenous health, influenza, stem cell research and water quality).

Given the major initiatives in Australia concerning rural health since 2000, the baseline review by Patterson, and our newly developed method, we undertook a bibliometric analysis of Australian rural health publications from 1990 to 2005, including determining changes since 2000. In addition we compared Australian trends with international rural health research output, especially with two other countries (USA and Canada) where 'rural health' practice and research are identified areas of special interest.

Methods

We used a bibliometric method that can be utilized by others which counts publications under specific Medical Subject Headings (MeSH) of 'rural health' in Medline, the largest bibliographic database in the public domain containing more than 16 million citations from 5000 journals in the life sciences with a concentration on biomedicine¹⁷. Medline is the starting point of many systematic reviews, meta-analyses and practice-guidelines that brings biomedical research findings to the point of care¹⁸.

PubMed is the web interface of Medline that is commonly used by clinicians, academics and increasingly the public. We used PubMed queries to retrieve the relevant data from Medline. A Medline publication record consists of many fields or 'tags' that identify specific aspects of the publication. Examples of some of the fields are: PubMed article number (PMID), journal source (SO), author affiliation (AD), publication type (PT), MeSH, title (TI), abstract (AB) and all fields (ALL). The MeSH tags are the main keywords of the Medline database. Every year, the MeSH tags are revised. Medline currently has 22 997 MeSH words. The specificity of PubMed searches can be increased when precise MeSH tags are used¹⁹. We used the MeSH and AD tags to retrieve data about rural health publications from specific countries using the queries given below.

It is also possible to search using 'text word' which check for the occurrence of a particular phrase or word in all fields. This increases the sensitivity of the search in the event that the a specific MeSH tag does not exist or has not been applied. The LIMIT option was used to retrieve publications for different time periods.

Query 1 - Australian publications

The first query retrieved the number of publications by primary authors linked to an Australian institution or with an Australian address.

Australia [MeSH] OR Australia [AD]

Query 2 - Rural health publications

The second query retrieved the rural health publications. In this regard we relied on authors (or editors) identifying their publications as pertaining to rural or remote health using the relevant keywords. We did not specify that the publications should be limited to the English language, and therefore the data for all publications is referred to as 'total world publications'.

"rural health" [MeSH] OR "rural health services" [MeSH] OR "rural health" [Text Word] OR "remote health"[Text Word]

Query 3 - Australian rural health publications

By combining query 1 & 2 (using AND) all rural health publications from Australia were obtained.

Queries 4 - Rural health publications by other countries

Similarly we obtained publications for Canada and United States of America. We only used the country MeSH word and country AD to increase the specificity and reduce the false positives.

Canada [AD] OR "Canada"[MeSH] OR Quebec[AD]

"United States" [MeSH] OR "United States" [AD] OR USA [AD] OR US [AD]

Query 5 - Publications relating to national health priority areas

In retrieving publications relating to specific NPHAs, we wished to increase the specificity of the queries and used only the recognised MeSH tags. We used the top MeSH for each domain.

"Asthma"[MeSH]

"Neoplasms"[MeSH]

"Cardiovascular Diseases"[MeSH]

"Mental Health"[MeSH] OR "Mental Health Services"[MeSH] OR "Community Mental Health Services"[MeSH] OR "Community Mental Health Centers"[MeSH]) OR "Mental Disorders"[MeSH]

"Diabetes Mellitus"[MeSH] OR "Diabetes Mellitus, Type 2"[MeSH] OR "Diabetes Mellitus, Type 1"[MeSH]

"Wounds and Injuries"[MeSH]

"Arthritis"[MeSH] OR "Musculoskeletal Diseases"[MeSH]

Query 6 - Publications by journal

The Query 3 results (all rural health publications from Australia) were downloaded in Medline format. Using the software PubMed Grabber²⁰ written by one of the authors (KM), the Medline format text file was converted to a MS Access database file with the Medline 'tags' as fields. Using SQL queries we created a journal list (descending frequency) from the tag 'journal source' (SO). Once we created the list of journals in descending frequency of numbers of publications, we crossed-checked again each journal by running a PubMed journal query with the particular journal and the time period (1990-2005). This confirmed the number of the journal article that we obtained from PubMed Grabber and the online PubMed journal query.

The same method was used with Query 2 (total rural health publications) results.

Query 7 - Indigenous health publications by Australian authors

We combined query 1 with terms relating to Indigenous health to determine Australian publication numbers about Indigenous health and then combined this with Query 3 to identify publications relating to rural Indigenous health issues "Health Services, Indigenous"[MeSH] OR "Medicine, Traditional"[MeSH] OR "Indigenous Health" [Text Word]

Query 8 - Publication types

After running query 3 for the time period, we chose the relevant Medline-determined 'Publication Types' (Journal article/Letter/RCT/CT etc) from the drop-down menu to produce numbers of each type.

Results

We retrieved 20 913 rural health publications between 1960 and 2005 from PubMed using query 2. There were 5601 (26.8%) from the US, 1442 (6.8%) from Australia, and 871 (4.2%) from Canada. We limited our main analysis to the period between 1990 and 2005 because of the low counts between 1960 and 1989.

The respective numbers for the total world publication count and for the three identified countries (Australia, Canada, US) are shown in Figure 1. For Australia there appears to be a steeper upward trend starting in 2000 while there is a smaller upward trend for Canada starting in 2004. The US publications increased at a constant rate until the number plateaued between 1996 and 2001, after which the original trend resumed. For both Australia and the US (and for the total world) there is a higher than trend number of publications in 2002 followed by a compensatory drop in 2003. This is likely to be due to technical factors in the PubMed database rather than increased publications in 2002.



Figure 1: The frequency of total world, Australian, Canadian and US rural health publications in PubMed from 1990 to 2005.

Between 1990 and 2005, total world yearly publications increased from 410 to 1207, while the contribution from Australia increased from 17 (4.1%) to 198 (16.4%), compared with Canada, from 10 (2.4%) to 110 (9.1%). Even though the US publications increased from 131 to 298, the percentage of US publications relative to the world total decreased from 32.0% to 24.7% over the period.

For the period 1990 to 2005, the journals that had more than 100 'rural health' publications are listed in Table 1. Two of the top three and three of the top 13 were Australian publications.

The journals with the largest number of Australian rural health publications are given in Table 2. The top five journals were AJRH (374), Medical Journal of Australia (177), Australian Family Physician (101), Rural Remote Health (55) and Journal of Telemedicine Telecare (54) had 57% of the total publications.

The frequency of publication type is shown in Table 3. The 'journal article' was the commonest followed by the 'letter'. The relative proportions of types of publications are generally similar across countries, although Australia had the lowest proportion of publications in the 'clinical trial' category.

Table 4 shows the rural health publications from Australia that addressed the NPHAs²¹ for the period 1990-2005, and also allows direct comparison of our results for the period 1990-1999 with similar data from the review by Patterson¹⁶. For the comparable period, Patterson identified both a greater number of articles (670 vs 454) and a greater proportion addressing the NPHAs (34.6% vs 22.0%) than our study. Mental health was the most frequent NPHA-item addressed in both studies. Our study shows that over the following 6 years the total number of publications had almost tripled (454 to 1290).

Table 5 compares numbers and proportions of publications that address the Australian NPHAs by Australian, US and Canadian journals. The proportion of total publications from the USA and Canada were slightly lower than for Australia - 980 of 4374 (22.4%) and 105 of 638 (16.5%), respectively, vs 317 of 1290 (24.9%) - although the relative proportions of studies across the NPHAs were remarkably similar among the countries.
The number of articles relating to Indigenous health in Australia for the period 1990-2005 was 368, but only 58 specifically related to rural Indigenous health. For the period 1990-1999 there were 104 articles relating to Indigenous health and 11 to rural Indigenous health.

Table 1: Journals with the greatest number of rural health articles from 1990 to 2005



Table 2: Rural health publications in Australian journals from 1990 to 2005



Table 3: Frequency of PubMed publication types from 1990 to 2005 in the world, Australia, Canada and USA



Table 4: Rural health publications in Australian journals addressing National Health Priority Areas



Table 5: Rural health publications in US, Canadian and Australian journals from 1990 to 2005 addressing national health priority areas



Discussion

Australia's contribution to the international rural health literature is increasing, both in terms of the numerical contribution of Australian authors and the prominence of selected Australian journals as the destination for articles on rural health topics. Rural health articles address NHPAs in approximately 25% of the publications.

Patterson's systematic review categorized articles into four defined types: review, editorial, commentary and research¹⁶. However we counted Medline publications according to the classification imposed by Medline. A paper may be categorized as a 'research paper' in one journal but a 'letter to the editor' in another more prestigious journal merely because of journal space. Some letters would identify important errors in methodology or ethical concerns of a research paper and may play an invaluable part in the research process. Unless a hand search of every publication in each journal is performed, it is difficult to be consistent, and even then it will be a subjective decision of the researcher about the categorization. For numerous reasons indicated we did not want to classify publications especially in a young domain such as rural health. In comparison with the technique of Patterson, the bibliometric method we used is easy, reproducible by any investigator and applicable to a range of user-defined searches and cost effective.

On a per capita basis, Australia is producing proportionally more papers on rural health issues than countries such as Canada and the US. Interestingly a visual analysis of the time series graph of publication numbers shows an upward inflexion at 2000. This was the year in which rural and remote health achieved major prominence in Australia when the Regional Health Strategy was the major focus of the federal budget. Following this period there has been ongoing major investment into rural and regional health activities at a national level.

The AJRH, first published in 1992, has almost caught up with the Journal of Rural Health (JRH) which dates back to 1985. In the top two rural health journals (Table 1), only a little more than two-thirds of articles were classified as relating to rural health on the basis of a MeSH word indexing. The AJRH had a total of 525 publications of which 417 (79.4%) were indexed using the MeSH words, 'rural health' or 'rural health services'. The JRH published a total of 632 articles for the same time period but had only 453 (71.7%) similarly indexed. It would be interesting to undertake a detailed analysis of the one-quarter of articles in rurally oriented journals that are not indexed with the MeSH words in query 2.

Internationally among the most frequent publishers of rural health articles were some of the world's leading general medical journals such as The Lancet, BMJ, JAMA and Medical Journal of Australia. The highest number of publications in our study relating to the NHPAs is in the domain of mental health. Increasing rural mental health problems, including suicides from various aetiologies²² and the Commonwealth Government's recent special allocation for counselling for the farming community²³, are both indications that further research in this area is appropriate and timely.

Between 1992 and 2003, all-cause death rates declined in regional, rural and remote areas by approximately 3% per year for males and approximately 2% per year for females²⁴. These improvements have been driven mainly by reductions in circulatory disease and cancer death rates. These two areas of gain have been responsible respectively for about 80% and 11% of the total improvement in regional and rural areas, and approximately 60% and 20% in remote areas. This may be one reason why the rural cardiovascular and cancer publications were relatively low when compared with mental health publications in our analysis.

Telemedicine and general practice journals were among the top ten journals that published rural health articles. This highlights the increasingly prominent role played by information and communication technologies in the delivery of rural health care in general practice settings. Liaw and Humphreys²⁵ warn about the 'eHealth paradox' in that although rural areas stand to benefit from eHealth but have the poorest infrastructure, resources, capacity and capability for successful implementation and uptake. However a recent Cochrane review about the role of telemedicine versus face-to-face patient care concludes that telecommunication technologies are feasible, but there is little evidence for clinical benefit²⁶. The review further states the feasibility of randomized trials on telemedicine in clarifying issues and advised policy-makers about recommending increased use and investment in unevaluated technologies.

Our objective was not to analyze all rural health publications in Australia but to obtain an overview of the contribution of publications originating from Australia in the peer-reviewed world health literature using PubMed. Our methodology is partly dependant on the accuracy of the author affiliation using the 'AD' tag. Incomplete mention of the affiliation will lead to errors of omission. Since only the first author's affiliation is recorded in Medline if a non-Australian author was the first author it would not have been counted as an Australian publication.

In our previous publication using this method we used a sophisticated query that increases the sensitivity of Australian publications by using the 'AD' tag to retrieve 'state' names in Australia¹⁵, when the institution address may not contain the country name. We could not obtain such queries for the other two countries from the published literature and decided to restrict the search on the AD tag to country names only for all countries. However the increase for Australia publications when the more sophisticated query was used was relatively minor, increasing the number only from 1290 to 1325. It should be noted that our query identified publications from Australian institutions or where the authors' address was in Australia. This could include publications by overseas authors resident in Australia but also excludes publications by Australian authors living overseas.

The bibliometric methods that we used are based on the assumption that 'scientists who have something important to say do publish their findings in the international peer-reviewed literature'²⁷. One can argue that our analysis is incomplete and potentially missing local work which was not indexed in Medline. However it has been well-established that Australia's health and medical research has high international visibility¹¹ and, therefore, probably only a small number would have been missed.

Comparison of our results with those of Patterson is interesting and highlights some of the issues associated with literature reviews. Although our technique has its limitations, especially if authors do not use the appropriate key words, it is replicable by others. However the technique used by Patterson is time-consuming and not suitable for the type of time series or international comparisons that we have been able to produce. Furthermore Patterson has not provided enough information to allow her technique to be undertaken by others with any certainty that the same results would be obtained. Our results show a much lower proportion of articles addressing the NHPAs of cardiovascular diseases, diabetes and injuries than for Patterson. The reason for this is unclear and could only be explained by a hand search of the journals accessed by Patterson to determine why we did not detect the relevant articles.

Two journals mentioned in Patterson's article that were frequent sources of published articles drew our interest - Aboriginal and Islander Health Worker Journal and the Health Promotion Journal of Australia. These were the fourth and sixth most frequently published journals in the study, yielding 48 and 20 articles, respectively. Our analysis shows that the former is no longer indexed in Index Medicus and, therefore, articles do not appear in Medline. The latter commenced indexing only in 2005. The inclusion of these journals by Patterson may partly explain the difference in numbers between the two studies. Of the 386 research articles identified by Patterson, approximately one-third were related to Indigenous-only rural health issues. Such articles may have been coded as relating to Indigenous health services, rather than to rural health or rural Indigenous health services, which might also partly explain the lower number identified using our technique. In the comparable period, our study identified approximately same number of publications as Patterson's that related to Indigenous health (approximately 100) but only one-tenth were identified as being specifically rural Indigenous health. For the longer time period we identified more than three times as many publications relating to Indigenous health, but only 15% were related to rural Indigenous health.

An Australian Institute of Health and Welfare report in 1998 suggested that poorer rural health statistics are primarily the result of the disadvantage for all Australians living in this area, rather than being the result of poorer Indigenous health²⁸. Of the total 145 823 Australian publications from 1990 to 2005 found using PubMed, only 1290 (0.9%) were about rural health and, therefore, more focused research into rural health, particularly concerning the new national health issues, is needed. We agree with Humphreys et al that 'there is still a pressing need for more rural research in health priority areas to ascertain the best possible interventions without depending on matters of faith and health anecdotes'²⁹.

Australia can learn from international experience of rural health research. This has been well documented in the US by Hartley³⁰ and in Canada by MacLeod et al³¹. In the former publication, Hartley refers to the establishment of the federal Office of Rural Health Policy, the existence of the National Rural Health Association, and issues networks as helping promote rural health research in the USA. All these elements are present in Australia. He also mentions that rural health research in the US is now moving from the documentation of rural-urban differences in health to advocate for more resources for rural areas, to population-based studies that include determinants of health status, and studies relating to quality of care.

The Canadian Rural Health Research Society³¹ as a network of networks is seen as being inclusive and building capacity in a way that is less possible in metropolitan areas; as such it has the capacity for significant innovation.

However those involved in rural health research can also learn from Australia, which now has two dedicated (and indexed) rural health publications: AJRH and Rural and Remote Health, the latter being an online journal, reflecting the increasing value of e-health in rural health services and education. Our comparison of relative proportions of articles from Australia, the US and Canada addressing the Australian NHPAs is interesting. It is unlikely the identified areas would not be priority health areas in the other countries, even if not explicitly stated, but it is clear that Australia has a higher proportion of publications addressing asthma (than both USA and Canada), cancer (Canada), diabetes (USA) and mental health (Canada).

With a continuing focus on rural health in Australia and overseas, it will be valuable to monitor the output of research through publications in an ongoing way into the future. We believe our method is ideally suited to allow this to occur in real time.

References

1. United Nations Department of Economic and Social Affairs. World Urbanization Prospects: The 2005 revision. Executive summary. (Online) 2005. Available: http://www.un.org/esa/population/publications/WUP2005/2005WUPHighlights_Exec_Sum.pdf (Accessed 17 August 2007).

2. Strasser R. Rural health around the world: challenges and solutions. Family Practice 2003; 20: 457-463.

3. Australian Institute of Health and Welfare. Australia's Health 2004. Ninth biennial health report of the Australian Institute of Health and Welfare, paper 2004a. Canberra, ACT: AIHW, 2004.

4. Harris B. Magic 'A'. Australian Journal of Rural Health 2000; 8: 286-293.

5. Wakerman J, Humphreys JS. Rural health: why it matters. Medical Journal of Australia 2002; 176: 457-458.

6. Pegram RW, Humphreys JS, McLean R. Meeting the needs of rural and remote Australians for specialist medical care: issues and options. Journal of Royal College of Physicians Edinburgh 2005; 35: 298-308.

7. Lawson KA, Chew M, Van Der Weyden MB. A revolution in rural and remote Australia: bringing health education to the bush. Medical Journal of Australia 2000; 173: 618-624.

8. Australian Health Ministers' Conference. Healthy horizons 1999-2003: a framework for improving the health of rural, regional and remote Australians. Canberra, ACT: Australian Government Publishing Service, 1999.

9. Van Der Weyden MB. Australian general practice: time for renewed purpose. Medical Journal of Australia 2003; 179: 6-7.

10. Bourke P. Evaluating university research practice: the British research assessment exercise and Australian practice. National Board of Employment Education and Training commissioned report no 56. Canberra, ACT: National Board of Employment Education and Training, 1997.

11. Butler L. Explaining Australia's increased share of ISI publications-the effects of a funding formula based on publication counts. Research Policy 2003; 32: 143-155.

12. Ford L. Group to develop new research funding model. The Guardian (Digital edition) 23 March 2006. Available: http://education.guardian.co.uk/higher/news/story/0,,1738114,00.html (Accessed 1 August 2007).

13. Otero P, Pedernera F, Montenegro S, Borbolla D. Garcia Marti S, Luna D. Citation of medical informatics in bibliographic databases. Medinfo 2004; 11(pt 1): 301-305.

14. Mendis K, Solangarachchi I. PubMed perspective of family medicine research: where does it stand? Family Practice 2005; 22: 471-473.

15. Mendis K, McLean R. Increased expenditure on Australian health and medical research and changes in numbers of publications determined using PubMed. Medical Journal of Australia 2006; 185: 155-158.

16. Patterson C. Systematic overview of rural health research in Australia published in the serial literature. Australian Journal of Rural Health 2000; 8: 286-293.

17. National Library of Medicine. Facts Sheet. National Library of Medicine. (Online) 2004. Available: http://www.nlm.nih.gov/pubs/factsheets/medline.html (Accessed 1 August 2007).

18. Haynes R. Of studies, syntheses, synopses, and systems: the "4S" evolution of services for finding current best evidence. Evidence-Based Medicine 2001; 6: 36-38.

19. Robinson KA, Dickersin K. Development of a highly sensitive search strategy for the retrieval of reports of controlled trials using PubMed. International Journal of Epidemiology 2002; 31: 150-153.

20. Mendis K, Solagarchchi I, Weerabaddana C. PubMed Grabber / Analyzer - a tool for off-line analysis of PubMed. Journal of Korean Society of Medical Informatics 2003; 9(Supp 2): s249-s244.

21. Government of Australia. National Health Priority Areas. (Online) 2005. Available: http://www.aihw.gov.au/nhpa/index.cfm (Accessed 1 November 2007).

22. Judd F, Cooper AM, Fraser C, Davis J. Rural suicide--people or place effects? Society of Psychiatry and Psychiatric Epidemiology 2006; 41: 911-917.

23. ABC NewsOnLine. Farmers welcome NSW drought funding. (Online) 2006. Available: http://www.abc.net.au/news/newsitems/200610/s1777568.htm (Accessed 8 November 2006).

24. Australian Institute of Health and Welfare. Australia's health 2006. AIHW cat. no. AUS 73. Canberra: AIHW, 2006; 244.

25. Liaw ST, Humphreys JS. Rural eHealth paradox: It's not just geography! Australian Journal of Rural Health 2006; 14: 95-98.

26. Currell R, Urquhart C, Wainwright P, Lewis R. Telemedicine versus face to face patient care: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews 2000, iIssue 2. Art. no: CD002098. DOI: 10.1002/14651858.CD002098.

27. van Raan AFJ. The use of bibliometric analysis in research performance assessment and monitoring of interdisciplinary scientific developments. Technikfolgenabschätzung -Theorie und praxis [Technology Assessment -Theory and Practice] 2003; 1(12): 20-29.

28. Australian Institute of Health and Welfare. Health in rural and remote Australia. AIHW Cat. No. PHE 6. Canberra: AIHW, 1998; iv.

29. Humphreys J, Hegney D, Lipscombe J, Gregory G, Chater B. Whither rural health? Reviewing a decade of progress in rural health. Australian Journal of Rural Health 2002; 10: 2-14.

30. Hartley D. Rural health research: building capacity and influencing policy in the United States and Canada. Canadian Journal of Nursing Research 2005; 37: 7-13.

31. MacLeod MLP, Dosman JA, Kulig JC, Medeves JM. The development of the Canadian Rural Health Research Society: creating capacity through connection. Rural and Remote Health 2007; 7: 622.