Evaluating Australian Indigenous community health promotion initiatives: a selective review

full article:

Introduction

Understanding what works in improving Indigenous health in rural and remote settings is a high priority for individuals, communities and governments. Health promotion initiatives are one avenue of pursuing health improvement, and evaluative research is an important mechanism for determining the effectiveness and impact of these activities.

Many evaluations of Indigenous health promotion initiatives occurring in remote or rural Australia are not published. As a result positive outcomes and methodological developments are not widely disseminated and used in other regions and communities. The objectives of this brief review were to examine the methodological approaches utilised in evaluations of Indigenous health promotion initiatives in the Australian context between 2000 and 2005, and to identify the use of key ethical guidelines for evaluation practice. We hoped that this would provide a snapshot of current evaluation practice in the Australian context and offer suggestions for ongoing improvements in this area.

Ethical guidelines for research with Aboriginal and Torres Strait Islander people have been available in Australia for a considerable time. The need for these stems from the fact that Indigenous Australians have been widely researched by non-Indigenous researchers and, in many instances, the research has resulted in the appropriation of Indigenous knowledge, or has used methodologies that were culturally insensitive or inappropriate¹. The guidelines also offer a means of improving evaluation practice or quality, particularly for non-Indigenous researchers. These guidelines can also have implications for the design of evaluation studies, many of which are restricted by time and resources. This review provides an overview of recent approaches to evaluation that have been utilised in the area of Aboriginal and Torres Strait Islander health promotion, and discusses the importance of applying ethical guidelines in evaluation practice.

Background

Health promotion practice emerged in the context of the modern primary healthcare movement, beginning with the Declaration of Alma-Ata in 1978. The declaration recognised health as a fundamental human right, and the need to move to a more population-focused healthcare system². In 1986, WHO made a second declaration, the Ottawa Charter for Health Promotion³. This charter has since provided an international framework for health promotion practice built on the principles of primary health care, as developed at Alma-Ata.

The Ottawa Charter describes health promotion as the process of enabling people to increase control over the determinants of health and thereby improve their health. To reach a state of complete physical, mental and social wellbeing, an individual or group must be able to identify and realise aspirations, to satisfy needs and to change or cope with the environment³. Five categories of health promotion interventions are frequently described in the literature⁴:

1. Screening for individual risk factor assessment and immunization.
   
2. Social marketing and health information provision.
   
3. Health education and skill development.
   
4. Community action (for social and environmental change).
   
5. Creating settings and supportive environments.

While this description of health promotion and its interventions fits many Indigenous health promotion initiatives, advocates of Indigenous health argue that efforts to improve Indigenous health require Indigenous models of health promotion that are culturally appropriate, community controlled, self-determining and based on the goals of Indigenous communities^5-7. Moreover, health promotion must adopt a model of health improvement that recognises the strengths, assets and capacities of Indigenous people⁸.

Health promotion for Indigenous people must take into account culture, diversity within the populations; socio-economic circumstances; languages and dialects; geographic location and the consequences of colonisation. Relationships within the community (particularly those with Elders) and spiritual connections to the land and ancestors need to be considered in the interpretation of health issues⁹.

Method

A selective review of the literature over a five-year period was undertaken. It examined the broad range of approaches to the evaluation of Aboriginal and Torres Strait Islander health promotion initiatives and the utilisation of ethical guidelines for research, as reported in literature. Two search strategies were used. The first entailed identifying available ethical guidelines, and the second required a search for Aboriginal and Torres Strait Islander evaluation studies, reports or publications.

The methodology for the review involved searching a range of databases, online sources and journals to locate published and unpublished literature, including PubMed, MEDLINE, Google Scholar, Health Source: Nursing and Academic, Health Reference Centre, Aboriginal and Torres Strait Islander Health Bibliography, Australian Indigenous Health Bulletin, the Medical Journal of Australia, Indigenous Australia, Aboriginal and Islander Health Worker Journal, and the Health Promotion Journal of Australia. Search terms included: evaluation, health promotion, Aboriginal, health, public health, ethical, and guidelines. As many Indigenous health projects are not published, a number of non-published documents, such as evaluation reports, were also collected through conference proceedings and internet searching.

Publications were included in the review if they reported on Aboriginal and Torres Islander health promotion programs or interventions that:

• published between 2000 and 2005
  
• had sufficient discussion of evaluation methods and programs outcomes
  
• used any of the following strategies: screening, social marketing, health education and skill development, community action, creating supportive environments.

Papers excluded from the review were those reporting on clinical trials, service, economic and policy evaluations, illness management program evaluations, commentaries on evaluations and publications that contained insufficient details about evaluation methods.

Ten documents were identified that provided details of ethical guidelines for the conduct of research with Aboriginal and Torres Strait Islander communities. Five of these were selected as key documents for content analysis with a focus on the common principles identified for ethical research practice.

One hundred and five abstracts were examined for inclusion in the review and 21 were selected for inclusion by a single reviewer. Each paper was read for inclusion against the criteria, and data were extracted about: the description of the program or initiative, evaluation methods, outcomes reported, acknowledgement of ethical guidelines or ethics committee review and collaborations and partnerships in Aboriginal and Torres Strait Islander evaluations.

Results

A number of important guidelines for ethical conduct of research were identified that are of specific relevance to evaluation practitioners. However an examination of the selected evaluation literature found relatively few references to the utilisation of these guidelines. The majority of evaluation studies were only able to report on short- to medium-term impacts of health promotion interventions. Difficulties in reporting longer-term health outcomes were effected by time and resources and by the selection of evaluation designs used to measure change. The methodologies used were often insufficiently robust and, as a result, many excellent initiatives were only partially able to demonstrate positive outcomes through a rigorous evaluation of the intervention.

Utilisation of guidelines for research and evaluation practice

In 1989 the National Aboriginal Health Strategy in Australia endorsed a set of principles for Indigenous research and evaluation. These principles included requirements for¹⁰:

• Indigenous involvement and control of research
  
• community consultation and endorsement
  
• the use of holistic understandings of health
  
• project-linked action research
  
• sensitivity to the social and cultural context.

Later publications affirmed and expanded upon these core principles, resulting in comprehensive guidelines for research and evaluation involving Indigenous people and their communities. The most recent and notable of these guidelines for Australia include:

• Australian Institute for Aboriginal and Torres Strait Islander Studies (AIATSIS) Guidelines for Ethical Research in Indigenous Studies (2000)¹¹
  
• National Public Health Partnership, Guidelines for the development, implementation and evaluation of national public health strategies for Aboriginal and Torres Strait Islander peoples (2002)¹²
  
• The NHMRC Road Map: A Strategic Framework for Improving Aboriginal and Torres Strait Islander Health through Research (2002)⁷
  
• NSW Health, Principles for Better Practice Aboriginal Health Promotion (2002)¹³
  
• National Health and Medical Research Council, Guidelines for Ethical Conduct in Aboriginal and Torres Strait Islander Health Research (2003)¹
  
• Australian Government, Keeping Research on Track: a guide for Aboriginal and Torres Strait Islander peoples about health research ethics (2005)¹⁴.

The common feature of these documents is a set of principles founded on respect for Indigenous people's inherent right to self-determination and maintenance of culture and heritage. Table 1 summarises the broad principles for ethical research showing the emergence of relatively common principles for the conduct of ethical research over time.

Table 1: Guidelines for ethical research and evaluation with Aboriginal and Torres Strait Islander Communities^1,11,12,14



These guidelines provide a rigorous and ethical framework within which evaluative research should be conducted but the responsibility for maintaining ethical standards cannot depend solely on rules or guidelines, but rests with researchers and the people with whom they engage¹.

The NHMRC guidelines¹ reiterate the continued concerns of Aboriginal and Torres Strait Islander peoples about poor consultation, lack of communication and infringements of deeply held values arising from cross-cultural insensitivity. Such concerns ought to be at the forefront of those conducting evaluations and in this respect the guidelines offer specific challenges for non-Indigenous evaluators who are engaged as external consultants.

The AIATSIS guidelines¹¹ similarly direct evaluators, as a sub-group of researchers, to engage in high levels of consultation and negotiation with Indigenous communities and ensure that Aboriginal and Torres Strait Islander people maintain input and control of the evaluation process. Evaluators need to respect the cultural property rights of Indigenous peoples in relation to knowledge, ideas, cultural expression and cultural materials. Further to this, there must be a clear benefit to Indigenous peoples at a local level and, more generally, from the evaluation overall. Each of these requirements demands collaboration and partnership, commitment to adequate time by all parties and culturally sensitive practice on the part of the evaluator, regardless of the methods to be employed.

Finally, the most recent guide Keeping on Track (2005)¹⁴ draws on the previous documents to assist Aboriginal people and communities to ensure that any research undertaken respects Aboriginal values, is relevant to needs, aspirations and priorities, and develops long-term ethical relationships.

The types of principles identified in Table 1 should not only influence the processes used by evaluators, but also the methods, the timeframes and the resources within which research or evaluation is conducted. From the literature reviewed for this paper it is difficult to ascertain how extensively these guidelines have been adopted for evaluation research. Only seven of 21 publications reviewed acknowledged that the research had received ethical approval and only two reported the use of NRMRC ethical guidelines for the conduct of the evaluation study. This does not mean that the guidelines were not used or that ethics clearance was not gained prior to the study, rather that the authors did not make such an acknowledgement in the publication.

The implications of not reporting on the utilisation of these guidelines, even if they have been used in practice, are manifold. Most notable is the fact that their importance may be undermined at several levels. For example, other researchers and evaluators may perceive the work as deficient, including journal reviewers who may not give credence to the findings if they are not located explicitly within an ethical framework. Of greater significance is a lack of information about how the methods employed may not be wholly in keeping with the core values enshrined in these guidelines. In those papers that did acknowledge the use of ethical guidelines or of having submitted the evaluation protocol to ethics committee review, there was no discussion of how the evaluation study design and process was shaped or influenced by the guidelines. In research and evaluation of Aboriginal and Torres Strait Islander health initiatives this would appear to be indicated.

In the field of health promotion more broadly, most who engage in evaluation would recognise their work as research. However, at a program delivery level, some may see evaluation and research as distinct and may not see the connection between ethics and evaluation as central to their work. From Table 1 what remains paramount in Aboriginal and Torres Strait Islander research is consultation, negotiation, participation and ownership of the process and outcomes. In this context ethical guidelines are central to the conduct of evaluations and ought to be at the forefront of considerations for design and process. In the papers reviewed almost all health promotion interventions or initiatives were a collaborative project between Aboriginal and Torres Strait Islander people and a range of organizations (health, education, councils, local government).

From the publications reviewed here it was difficult to ascertain the role that Indigenous people and communities had played in the research component of projects. However, eight of the 21 studies examined indicated that Aboriginal people had participated in the research. In some cases this involved Aboriginal health workers or community members liaising with communities, collecting data and working together with non-Indigenous researchers. There was limited discussion about building research capacity of Aboriginal and Torres Strait Islander people. Again, this is not to say it was not a component of the studies, rather that it was not discussed in the evaluation publications.

Characteristics of Indigenous health promotion evaluations

A report for the National Public Health Partnership¹² published during 2002 indicated that most public health strategies in relation to Aboriginal and Torres Strait Islander peoples are not evaluated, or evaluations are not published. Where evaluations have been completed, they were often limited, done inadequately or inadequately funded¹². Evaluation of complex, multi-faceted health promotion and public health initiatives have long been recognised as challenging to appraise on the basis of difficulties with attribution and the application of study designs that minimise bias¹⁵.

Table 2: Evaluations of Indigenous health promotion initiatives^16-36



Table 2 summarises 21 evaluation studies of Indigenous health promotion initiatives. Of these, a total of 10 used qualitative methods only, two employed quantitative methods only, eight reported using mixed method approaches and one study was a systematic review. Most of the programs evaluated were addressing complex problems with multiple causal factors. The Aboriginal and Torres Strait Islander health promotion initiatives targeted health issues such as smoking, alcohol and drug use, healthy weight, diabetes type II, social and emotional wellbeing and family violence.

A wide range of data collection methods were reported including surveys and questionnaires, focus groups, interviews, arts based strategies, narrative inquiry, screening, tests and biological measures. The majority of evaluations were able to report on short- to medium-term program impacts with only two demonstrating longer-term health outcomes^31,32. The types of impact or outcome indicators reported in the publications reviewed are summarised (Fig 1).



Figure 1: Indicators of change in Aboriginal and Torres Strait Islander health promotion program evaluations.


A number of limitations and challenges were reported that impact upon the evaluation process and quality of data. These included:

• long lead times between interventions and targeted health outcomes¹⁶
  
• inadequate time and other resources compromising the effectiveness of both programs and their evaluations^16-20
  
• a disparity between western and traditional perspectives about health and language, limiting what could be evaluated¹⁷
  
• sensitivity to Aboriginal and Torres Strait Islander's position as researchers within their own communities²¹
  
• a need to assess whether the implementation process was participatory and empowering for participants²²
  
• a need for better and more subtle indicators of change for measuring small improvements^18,23
  
• insufficient base-line data, lack of control groups, use of self reporting, self selection of participants, poorly defined impact and outcome measures, gender balances, sample sizes, lack of time-series or longitudinal data collection^{20,22,24-26,36}.

In essence, despite the fact that many of the programs were innovative, well received and supported by communities, the majority of evaluation designs were insufficiently robust to measure the intermediate or long-term impacts or desired program outcomes of the complex and multifaceted interventions described. This, however, is not specific to evaluations of Indigenous health promotion initiatives but a challenge for evaluators more generally. Furthermore, some of the studies used study designs that made it difficult to identify the specific components of the programs that contributed to program success. In this sample of studies mixed method approaches did appear to have greater success in isolating the elements of program success.

Discussion

An examination of the ethical guidelines and the evaluation studies in this selective review revealed a range of factors that were attributed to the success of Aboriginal and Torres Straight Islander health promotion initiatives^1,7,11-14. These included:

• the role of community Elders and the importance of widespread community support
  
• the value of participatory research
  
• the designation of Aboriginal health workers as cultural brokers
  
• the need to make explicit the way in which ethical guidelines have influenced methodology
  
• the potential of mixed method approaches in evaluation.

A number of the evaluation studies identified the establishment of steering (or reference) groups led by community Elders and program participants as a critical factor in the success of the program and evaluation process^17,30,31. In accordance with guidelines for the conduct of ethical research, it is not only important to establish widespread community support for the implementation of programs but that the involvement, support and participation of community members must be maintained throughout the evaluation process. Where Aboriginal and non-Aboriginal researchers and organisations are engaged in a research partnership greater attention could be focused on building research capacity within the community from the early stages of the health promotion project.

Participatory evaluation practices were reported to strengthen community ownership and sustainability of the intervention as well as fostering participation and cooperation in evaluations. In some cases the use of participatory action research was reported as an empowering process for program participants and community members participating in the evaluation^21,23,31, but this approach can require longer time periods for the completion of the evaluation^35,36.

Participatory approaches to evaluation appear to be a requirement if following any of the ethical guidelines for research with Aboriginal and Torres Strait Islander communities. Across these documents, practice guidelines stipulate the need for the involvement of Indigenous individuals and communities as collaborators in the research¹¹, evaluation protocols being community driven¹², and negotiating agreed research plans collaboratively¹⁴.

Another factor reported to be of value in evaluation practice included the designation of Indigenous health workers acting as cultural brokers between non-Indigenous evaluators and the community. This strategy facilitated the opportunities for non-Indigenous researchers with particular skills to be appropriately introduced or integrated into the project evaluation process and the community³⁰. Indeed, successful programs and evaluations appeared to be those in which the program and the evaluation remained community driven and controlled and increasingly non-Indigenous researchers diminished their role over time but ensured that feedback and results were provided back to the community promptly and regularly.

Attempts to bring greater rigor to evaluations of health promotion initiatives has been a challenge to the field of health promotion for the last decade and are not unique to evaluations of Indigenous health promotion initiatives. This might be addressed to some extent by the use of critical appraisal frameworks³⁷ during the design phase of evaluations or for enhancing the quality of publications. Many critical appraisal tools are now widely available and free³⁷. They generally comprise of criteria or checklists or standards that are used to evaluate evidence. They can be used to assess the value of a single study or several studies. While not widely discussed in the literature they could also be used to assist in the design and development of evaluation protocols to improve methodological rigour³⁸.

Alternatively greater use of mixed methodology approaches could be appropriate in evaluations of complex interventions¹⁵. This involves mixing or combining quantitative and qualitative research techniques, and the use of multiple data using different strategies or approaches in a way that the combination is likely to produce complementary strengths and reduce methodological weaknesses. This approach may answer a broader range of research questions and provide a stronger evidence base for conclusions about outcomes³⁹. Evaluators working in the area of Indigenous community health promotion could look to utilising a broader range of qualitative methods including ethnography, case studies, narrative approaches and arts-based methods within evaluation practice, given the reported value and success of such approaches within programs themselves. Arts-based evaluation methods were successfully used within programs and, in some instances, as part of evaluation data collection methods; however, the discussion of this approach has been somewhat limited^17,18. The possibilities for arts-based strategies within evaluations should be considered in much greater detail, as should the use of oral, narrative or story telling as data collection methods.

Conclusion

A wide range of innovative Aboriginal and Torres Strait Islander health promotion programs are conducted and evaluated in Australia each year. An analysis of selected evaluation literature revealed a number of issues that are relevant for those involved in the evaluation of Aboriginal and Torres Strait Islander health promotion initiatives. Evaluations need to be explicitly underpinned by guidelines for ethical research with Aboriginal and Torres Strait Islander people, as these will help inform choices about the design of evaluations, as well as processes used. A more explicit discussion of how the guidelines have shaped evaluation practice within publications would be useful in improving evaluation practice for early or new researchers in the area as well as demonstrating good practice.

Challenges or barriers to effective evaluation practice were clearly described in the literature, the most notable being the inadequate time and resources available for program implementation and evaluation. Establishing realistic expectations of what can be evaluated from the outset of programs is important. A second major challenge is the need to establish effective partnerships to work collaboratively with participants, not only in the period of program implementation, but also for evaluation. Building Aboriginal and Torres Strait Islander research and evaluation capacity could be a component of all Indigenous research. The third major challenge centres on the need to establish, in overtly demonstrable ways, the rigor of evaluation study designs so that findings and perspectives can stand up to external scrutiny. Here the use of critical appraisal tools may prove useful in the early design of evaluation studies.

One of the most important issues to emerge from this review is the need to re-emphasise the interdependent relationship between appropriate standards of ethical behaviour in health promotion research, and rigorous methods of conducting that research. While this review has raised this issue for discussion and debate, further on-going reviews such as this will need to be undertaken to ensure that standards are improved and maintained.

Recommendations

As a result of this review, the following recommendations are presented:

1. In accordance with guidelines for the conduct of ethical research with Aboriginal and Torres Strait Islander people, it is not only important to establish community support for the implementation of health promotion programs, but also throughout the evaluation process. The establishment of steering groups led by community Elders and community members may enhance evaluation practice.
   
2. It may be beneficial to other researchers if those publishing reports on evaluations of health promotion initiatives describe how ethical guidelines were used in the program or initiative, and how they influenced the evaluation process. This could encourage a broader use of the guidelines and better dissemination of good practice.
   
3. Community participation, collaboration and control ought to be the defining feature of evaluations of Aboriginal and Torres Strait Islander health promotion initiatives. While Indigenous cultural brokers have been shown to be an effective means of enhancing the work of cross-cultural research teams, evaluators should give more consideration to building research capacity within Indigenous communities from the early stages of the health promotion project or evaluation.
   
4. Developing rigor in evaluations of health promotion initiatives is a challenge to all evaluators. Those engaged in working with Aboriginal and Torres Strait Islander communities could look to utilising critical appraisal tools to improve the quality of evaluation designs, mixed method approaches and a broader range of qualitative methods including ethnography, case studies, narrative approaches or arts based methods. These may offer creative ways of responding to the complexity of multifaceted community interventions.

Acknowledgements

The authors acknowledge the support and assistance of Kylie Pryde of the Healthpact Research Centre for Health Promotion and Wellbeing for administrative assistance in the writing of this paper.

References

1. National Health and Medical Research Council. Guidelines on ethical matters in Aboriginal and Torres Strait Islander health research. Canberra: NHMRC, 2003.

2. World Health Organisation. Declaration of Alma-Ata 1978. (Online) no date. Available: World Health Organisation (Accessed: 16 December 2006).

3. World Health Organisation. Ottawa Charter for Health Promotion. (Online) 1986. Available: World Health Organisation (Accessed: 16 December 2006).

4. Victorian Government, Department of Human Services. Health promotion interventions and capacity building. (Online) 2005; Available: Victorian Government, Department of Human Services (Accessed: 24 November 2005).

5. Durie M. An indigenous model of health promotion. Proceedings, 18th World Conference on Health Promotion & Health Education; 26-30 April 2004, Melbourne, Vic; 2004.

6. McLennan V, Khavarpour F. Culturally appropriate health promotion: its meaning and application in Aboriginal communities. Health Promotion Journal of Australia 2004; 15: 237-239.

7. Commonwealth of Australia. The NHMRC Road Map: A strategic framework for improving Aboriginal and Torres Strait Islander health through research. Canberra, ACT: National Health and Medical Research Council, 2002.

8. Brough M, Bond C, Hunt J. Strong in the City: towards a strength-based approach in Indigenous health promotion. Health Promotion Journal of Australia 2004; 15: 215-220.

9. Australian Indigenous Health Info Net. Indigenous Health. (Online) 2005. Available: Australian Indigenous Health Info Net (Accessed: 16 December 2005).

10. Hearn S, Wise M. Health promotion: a framework for Indigenous health improvement in Australia. In: R Moodie, A Hulme (Eds). Hands-on health promotion. Melbourne: IP Communications, 2004; 313-330.

11. Australian Institute of Aboriginal and Torres Strait Islander Studies. Guidelines for Ethical Research in Indigenous Studies. Canberra: Australian Institute of Aboriginal and Torres Strait Islander Studies, 2000.

12. National Public Health Partnership. Guidelines for the development, implementation and evaluation of national public health strategies in relation to Aboriginal and Torres Strait Islander peoples. Melbourne: NPHP, 2002.

13. NSW Health. Principles for better practice in Aboriginal health promotion: the Sydney Consensus Statement. (Online) 2002. Available: NSW Health (Accessed: 16 December 2006).

14. Australian Government. Keeping research on track: a guide for Aboriginal and Torres Strait Islander peoples about health research ethics. Canberra: Australian Government, 2005.

15. Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D et al. Framework for design and evaluation of complex interventions to improve health. BMJ 2000; 321: 694-696.

16. Tyrrell M, Grundy J, Lynch P, Wakerman J. Laramba Diabetes Project: an evaluation of a participatory project in a remote Northern Territory community. Health Promotion Journal of Australia 2003; 14: 48-53.

17. Davis B, McGrath N, Knight S, Davis S, Norval M, Freelander G et al. Aminina Nud Mulumuluna ("You Gotta Look After Yourself"): Evaluation of the use of traditional art in health promotion for Aboriginal people in the Kimberley region of Western Australia. Australian Psychologist 2004; 39: 107-113.

18. Murphy L, Kordyl P, Thorne M. Appreciative inquiry: a method for measuring the impact of a project on the well-being of an Indigenous community. Health Promotion Journal of Australia 2004; 15: 211-214.

19. Harvey D, Tsey K, Cadet-James Y, Minniecon D, Ivers R, McCalman J et al. An evaluation of tobacco brief intervention training in three Indigenous health care settings in north Queensland. Australian and New Zealand Journal of Public Health 2002; 26: 426-431.

20. Gray D, Saggers S, Sputore B, Bourbon D. What works? A review of evaluated alcohol misuse interventions among Aboriginal Australians. Addiction 2000; 95:11-22.

21. Poelina A, Perdrisat I. A report of the Derby/West Kimberley Project: working with adolescents to prevent domestic violence. Canberra: Commonwealth of Australia, 2004.

22. Victoire A. Issues in evaluation of a health promotion intervention: 'Taking Big Steps'. Aboriginal and Islander Health Worker Journal 2003; 27: 10-14.

23. Tsey K, Wenitong M, McCalman J, Whiteside M, Baird L, Patterson D et al. A participatory action research process with a rural Indigenous men's group: monitoring and reinforcing change. Australian Journal of Primary Health 2004; 10: 130-136.

24. Begley L. 'Yarning for better health' - Improving the health of Aboriginal and Torres Strait Islander population. Australian Family Physician 2005; 34: 27-29.

25. Mark A, McLeod I, Booker J, Ardler C. The Koori tobacco cessation project. Health Promotion Journal of Australia 2004; 15: 200-204.

26. Mooney N, Bauman A, Westwood B, Kelaher B, Tibben B, Jalaludin B. A quantitative evaluation of Aboriginal cultural awareness training in an urban health service. Aboriginal and Islander Health Worker Journal 2005; 29: 23-30.

27. Tsey K, Every A. Evaluating Aboriginal empowerment programs: the case of family wellbeing. Australian and New Zealand Journal of Public Health 2000; 24: 509-514.

28. Adams K, Dixon T, Guthrie J. Evaluation of the Gippsland regional indigenous hearing health program - January to October 2002. Health Promotion Journal of Australia 2004; 15: 205-210.

29. Dunn S, Dewis E. Healthy weight program evaluation, Queensland 1996-1999. Aboriginal and Islander Health Worker Journal 2001; 25: 26-28.

30. Mikhailovich K, Arabena K. Evaluating an indigenous sexual health peer education project. Health Promotion Journal of Australia 2005; 16: 189-193.

31. Curtis S, Pegg D, Curtis O. 'Aunty Jean's Good Health Team' - listening to the voices of the Elders to create an Aboriginal chronic and complex care program. Wollongong: Illawarra Health, 2004.

32. Shannon C, Canuto C, Young E, Craig D, Schluter P, Kenny G et al. Injury prevention in Indigenous communities: results of a two-year community development project. Health Promotion Journal of Australia 2001; 12: 233-237.

33. Ivers R, Castro A, Parfitt D, Baile RS, Richmond RL, D'Abbs P. Television and delivery of health promotion programs to remote Aboriginal communities. Health Promotion Journal of Australia 2005; 16: 155-158.

34. Smith J. Wadja warriors football team's healthy weight program. Aboriginal and Islander Health Worker Journal 2002; 26: 13-15.

35. Goo EC. Self-development in order to improve community development: an evaluation of a personal empowerment pilot initiative in far north Queensland indigenous communities. Aboriginal and Islander Health Worker Journal 2003; 27: 11-16.

36. Rowley KG, Daniel M, Skinner K, Skinner M, White GA, O'Dea K. Effectiveness of a community-directed 'healthy lifestyle' program in a remote Australian Aboriginal community. Australian and New Zealand Journal of Public Health 2000; 24: 136-144.

37. National Health Service, Public Health Resource Unit. Critical Appraisal Tools. (Online) 2000. Available: NHS, Public Health Resource Unit (Accessed: 10 December 2005).

38. Rychtnik L, Hawe P, Waters E, Barratt A, Frommer M. A glossary for evidence based public health. Journal of Epidemiology and Community Health 2004; 54: 538-545.

39. Johnson RB, Onwuegbuzie AJ. Mixed method research: A research paradigm whose time has come. Educational Researcher 2004; 33: 14-26.