Original Research

Contextual factors and health service performance from the perspective of the provincial health administrators in Papua New Guinea

AUTHORS

name here
Emma Field
1 MAppEpi, Epidemiologist *

name here
Jethro Usurup
2 MBBS, Senior Medical Officer

name here
Sally Nathan
3 PhD, Senior Lecturer

name here
Alexander Rosewell
4 PhD, Senior Lecturer

CORRESPONDENCE

* Emma Field

AFFILIATIONS

1 Menzies School of Health Research, Spring Hill, QLD 4000, Australia and Abt Associates and School of Public Health and Community Medicine, University of New South Wales, UNSW Sydney, NSW 2052, Australia

2 Abt Associates

3, 4 School of Public Health and Community Medicine, University of New South Wales, UNSW Sydney, NSW 2052, Australia

PUBLISHED

5 October 2018 Volume 18 Issue 4

HISTORY

RECEIVED: 2 June 2017

REVISED: 29 March 2018

ACCEPTED: 25 April 2018

CITATION

Field E, Usurup J, Nathan S, Rosewell A.  Contextual factors and health service performance from the perspective of the provincial health administrators in Papua New Guinea. Rural and Remote Health 2018; 18: 4484. https://doi.org/10.22605/RRH4484

AUTHOR CONTRIBUTIONSgo to url

This work is licensed under a Creative Commons Attribution 4.0 International Licence


abstract:

Introduction:  The Rural Primary Health Services Delivery Project aims to improve the quality and coverage of health services to rural populations in Papua New Guinea. There are limitations in measuring performance of such projects through analysis of health information system data alone due to data quality issues and a multitude of unmeasured factors that affect performance. A mixed methods study was undertaken to understand the contextual factors that affect health service performance.
Methods:  A performance assessment framework was developed including service delivery indicators derived from the National Health Information System. Prior to implementation, a baseline analysis of the indicators was undertaken. Subsequently, semi-structured interviews were conducted with health administrators, in which they were asked about factors they perceived to influence health facility performance. During the interviews, key informants were provided with health indicators for their province and asked to interpret the performance of facilities. Interviews were transcribed and inductive thematic analysis performed.
Results:  Performance indicators varied greatly within and between districts. Key informants cited a number of reasons for this variation. Health facilities accessible by road in urban areas, with competent and/or higher level staff and health services operated by churches or private companies, were cited as contributors to high performance. For high performing districts, key informants also discussed use of health information, planning and targeted strategies to improve performance. Inadequate numbers of staff, poorly skilled staff, funding delays and challenging geography were major contributors noted for poor performance.
Conclusion:  Analysis of quantitative indicators needs to be performed at health facility level in order to understand district level performance. Interpretation of performance through key informant interviews provided useful insight into previously undocumented contextual factors affecting health delivery performance. The sequential explanatory mixed methods design could be applied to evaluations of other health service delivery programs in similar contexts.

Keywords:

contextual analysis, health service delivery, Papua New Guinea, sequential explanatory mixed methods.

full article:

Introduction

The Rural Primary Health Services Delivery Project is an initiative of the Government of Papua New Guinea (PNG). The project aims to improve the coverage and quality of rural primary healthcare services through a range of initiatives at national, provincial, district, health facility and community levels1. The project is being piloted in two districts in eight provinces, with the intention of further roll-out if effective. The project is expected to improve health system performance according to a range of access, quality, and maternal and child health indicators.

While health programs are largely measured through quantitative indicators, it is acknowledged by WHO that ‘Health systems are complex and their performance and impact are difficult to capture using only quantitative indicators’2. The context within which a program is implemented can have an effect on the success or otherwise of the program. In order to measure progress and impact of a program, or to scale up an intervention, it is important to understand the contextual factors that may influence performance3,4. In PNG, several factors have been identified to negatively impact health facility performance, including remoteness; access to funds; and cultural beliefs of the community, particularly around family planning, pregnancy and birth5-7. These factors have largely been identified through quantitative studies where data on predefined variables were used but may not have captured the full range of factors that could influence performance. A key challenge for this evaluation was to measure changes in coverage and quality of primary health care in eight very different provinces.

Qualitative data are regularly used in monitoring and evaluation of health programs4. Health administrators can offer a useful perspective on the reasons for high or low performance at the health facility and district level due to their contextual knowledge. In order to understand the contextual factors that may affect performance of the project, the performance of key indicators were analyses and a contextual analysis was conducted through interviews with senior provincial health administrators to document perceived factors affecting performance at baseline. The authors propose that this sequential explanatory mixed methods approach to the contextual analysis could be beneficial for evaluating complex programs in low-resource settings.

Methods

Setting

The project commenced in 2012 and operates in two districts in each of eight provinces of PNG: Central and Southern Region in the Autonomous Region of Bougainville, Wewak and Maprik in East Sepik, Okapa and Kainantu in Eastern Highlands, Laiagap-Porgera and Kompian-Ambum in Enga, Alotau and Kiriwina-Goodenough in Milne Bay, Bulolo and Menyamya in Morobe, Talasia and Kandrian-Gloucester in West New Britain and Tambul Nebilyer and Mul-Baiyer in Western Highlands1. These eight provinces are spread across the four regions of PNG (Highlands, Islands, Momase and Papua regions). The project supports the review or development of national policies and standards, establishment of partnerships at provincial level with state and non-state partners in health; health worker training, infrastructure development through construction of two community health posts in each district and community-level health promotion activities to improve demand for services1.

Study design

Two health system consultants were engaged to conduct a formative evaluation during implementation from 2013 to 2017. A performance assessment framework was developed for the project. In line with best practice, and to avoid duplicate data collection systems, the indicators were aligned with the National Health Plan Monitoring and Evaluation Framework and data for the indicators were taken from the existing National Health Information System (NHIS) at the facility and district level8. A sequential explanatory mixed-methods study design was employed involving analyses of the indicators in the performance assessment framework for the baseline year (2013) and subsequent key informant interviews with provincial health administrators from participating provinces to help understand the context of the performance indicator9.

Indicator analysis

The NHIS is used to record data on outpatient and other services provided at health facilities. Data were transcribed from automated reports from the NHIS for 2013 into Microsoft Excel. The indicators calculated were outpatients per person per year, outreach clinics per 1000 children aged less than 5 years, measles immunisation coverage at 9–11 months, family planning couple years protection per 1000 women of reproductive age, proportion of pregnant women who had at least one antenatal care visit (ANC1) and proportion of pregnant women who had a supervised delivery using the methods defined in the National Health Plan Monitoring and Evaluation Plan8. All indicators were adjusted for missing monthly summary reports by dividing by the percentage of reports submitted for the year. Health facilities with no catchment population provided were excluded because rates or percentage coverage could not be calculated. Indicators at the facility and district level for each indicator were compared to national targets8.

Key informant interviews

Face-to-face key informant interviews were conducted with all provincial health advisors or provincial health authority chief executive officers (interviewees) from the project provinces. The interviews were conducted in English by three trained monitoring and evaluation staff. The interviewees were considered key informants, having in-depth knowledge of the health system and more than 10 years of experience.

The interviews were semi-structured with questions about factors affecting access, utilisation and quality of health services. The interviewees were provided with a table that included the indicators from the NHIS data at the facility level for districts within their provinces. Some of the probing questions were derived from these tables, specifically asking about the context in which facilities with exceptionally high or low performance in indicators operate. Audio recordings were transcribed and data were coded using NVivo v10 (QSR International; http: www.qsrinternational.com/nvivo). First, the data were coded according to the question. Second, inductive thematic analysis was conducted for three of the interviews, which means that themes were identified from the data10. From this initial analysis, the main themes identified were geography, cultural and religious beliefs, human resources, financing and health information, with further themes identified within these main themes. A coding framework with these themes was developed and applied to all interviews.

Ethics approval

The key informant interviews were conducted only after informed consent was obtained. The audio recording was used for transcription purposes only and the transcripts were de-identified. A diversity of quotes was sought for presentation in this article, and interviewee numbers have been removed to protect anonymity. The study was approved by the PNG Medical Research Advisory Committee (15.10) and the University of New South Wales (HC15208).

Results

NHIS analysis

For brevity, only four of the most relevant indicators from the performance assessment framework are presented, covering health system access, and maternal and child health. The indicators presented are outreach clinics per 100 children aged less than 5 years, measles vaccination coverage at 12 months of age, ANC1 coverage and supervised delivery coverage. The performance indicators varied widely across the 16 districts (Tables 1,2). Both districts in Milne Bay met all four targets for the indicators presented. Conversely, no targets were met in Central Region in the Autonomous Region of Bougainville, Kandrian-Gloucester in West New Britain and both Bulolo and Menyamya in Morobe.

Six districts met the outreach clinic rate coverage target. In these, 100% (7/7) of facilities in Kiriwina-Goodenough and 29% of facilities (2/7) in Kompian-Ambum met the target. Seven districts met the target for measles vaccination with three districts notably exceeding 100%. Similar to results for the outreach clinic rate, a variable proportion of facilities within those districts met the target for measles vaccination: from 29% (2/7) in Maprik to 100% (3/3) in Kainantu. Seven districts met the ANC1 target. In Kainantu, the ANC1 target was not met despite all facilities listed in the NHIS having met the target. Six districts met the target for supervised deliveries and of these districts Laiagap-Porgera and Wewak had only one facility that met the target (1/6 and 1/9 respectively) while in Kiriwina-Goodenough 86% of facilities (6/7) met the target.

Table 1:  Indicator performance by district, 2013

Table 2:  Number of facilities where indicator met (%), by district, 2013

Key informant interviews

All provincial health advisors or provincial health authority chief executive officers (interviewees) from provinces covered by the project participated in the interviews. The interviewees (n=8) had been working in health for 12–43 years with a majority of that time working in the province in PNG where they were currently employed. Together they represented all of the eight provinces. The responses presented here relate to discussion of all indicators presented: outpatients per person per year, outreach clinics per 1000 children aged less than 5 years, measles immunisation coverage at 9–11 months, family planning couple years protection per 1000 women of reproductive age, proportion of pregnant women who had at least one antenatal care visit and proportion of pregnant women who had a supervised delivery.

Geography

Difficult terrain for travel, either mountainous regions or locations only accessible by sea, was a reason given by many interviewees for poor performance of health facilities (Table 3). The negative impacts cited from challenging geography include people presenting to the health facility only if they are seriously ill; restriction of health workers’ ability to conduct outreach in the catchment communities of their facilities; limited support and distribution of essential supplies from the district and provincial level to remote facilities; and increased costs for providing health services in remote locations, particularly if air travel is required. By contrast, high performance was attributed to the health facility location being accessible by road and in urban areas, where people frequently travel for other purposes.

Table 3:  Quotes from key informant interviews, demonstrating themes

Cultural and religious beliefs

When asked if cultural beliefs affect access and utilisation of health services, interviewees often cited beliefs in sorcery, the preference of women to deliver babies in the village and religious beliefs around the use of family planning. For example, in some communities people believe that illness is caused by sorcery rather than believing in traditional Western beliefs about causation, which influences their health-seeking behaviour, including late presentation to health facilities. Several cultural issues were attributed to women preferring to deliver babies in the village, noted by two of the interviewees. These were the expectation by male health workers that they do not deliver babies and that women prefer to deliver in the village with a known person rather than delivering at a facility with the possibility of a male health worker in attendance. The low uptake of family planning was attributed to the community beliefs around family planning and some church-run health facilities not providing contraceptives.

Human resources

Human resources was the theme mentioned most frequently as impacting people’s access and utilisation of health services. Having an adequate number and distribution of skilled staff was seen as a vital component for improving health services and access by the community. Half of the interviewees noted that the existing health workers and support staff did not have adequate skills to do their jobs.

In remote locations, some interviewees said health workers face issues with accommodation and limited support. Health workers in remote locations may frequently travel away from their facility to go to urban areas, and when staff are not at their health facility people will bypass the facility and go to a higher level facility such as a hospital. The corollary is that health workers were perceived to be the reason for high performance of some facilities. For two interviewees, the availability of staff was a reason for high performance and they reported that people will seek out these facilities even if they have to travel further. In addition to the presence of formal health workers, high performance was attributed to village health volunteers by one respondent, particularly in encouraging women to attend antenatal care and have supervised deliveries.

Financing

The majority of interviewees mentioned funding as a factor contributing to low performance. In particular, delays in funds were mentioned as a factor affecting the delivery or services. One interviewee stated there were sufficient funds but the main issue was with the delay in receiving the funds. Another interviewee stated that the issue was not insufficient funding but the use of the funds available and the capacity of staff. The experience of the recently introduced free healthcare policy demonstrates the possible impact of the misuse and poor management of funding, with the reintroduction of user fees to ensure facilities can remain open after funds have been depleted.

Health information

Several interviewees stated that the NHIS data were not accurate either because of inaccurate reporting from facilities or inaccurate population estimates for calculating percentages and rates. Regardless of questions around data quality, two interviewees mentioned using data for reviewing performance at the facility level and creating targeted interventions to improve performance on a specific indicator.

Discussion

Provincial health administrators perceive that multiple factors from the health system and the context in which health facilities operate affect performance. Health facilities accessible by road in urban areas with competent and/or higher level staff and health services operated by churches or private companies were cited as contributors to high performance. Conversely, inadequate numbers of staff, poorly skilled staff, delays in funding and challenging geography were the major contributors noted for poor performance. The authors identified a large variation in performance indicators between and within the project districts: while a district may meet the national target, a high proportion of facilities within the district may not, suggesting that coverage of services is not optimal.

Several factors affecting performance identified in this study have been identified previously: workforce of appropriate number, cadre mix and skills11; challenging geographical access to health facilities6,12,13; delays in receiving funds for health services5; and cultural factors, particularly for maternal and reproductive health indicators14. This contextual analysis offered new information on factors affecting performance such as the availability of health workers at a facility and use of data to develop targeted interventions to improve performance. Further, the contextual analysis aided understanding of quantitative data used for the indicator analysis.

The interviewees noted issues with data quality and district and health facility catchment populations used within the NHIS to calculate coverage and rates. Issues with the NHIS data quality and completeness have been documented15,16. The population catchments used for calculating percentage coverage were estimates based on projections from the 2000 population census. In this study there were many coverage rates well above 100%, which could be due to inaccurate numerator data from the NHIS, inaccurate population estimates, or people from outside the catchment area accessing the facility, who would not be included in the population denominator. According to the interviewees, some facilities with coverage rates well above 100% were accessed by people outside the catchment area either due to the location in urban areas or perceived quality of services.

The indicators used in this study are usually only presented at the national, provincial or district level12,16,17. As seen in this study, individual health facility performance is highly variable, and reporting at a district level may mask underperformance in a number of facilities. While previous studies have suggested district-level benchmarking is important, it is also important to review data at each facility. The Reaching Every District strategy encourages monitoring of data and plan development at the health facility and district level to improve immunisation coverage, and has been adapted for prevention of mother-to-child transmission of HIV18,19. The Reaching Every District strategy is supported by the National Department of Health for immunisation20,21. However, most districts had multiple facilities well below the target coverage for measles immunisation, suggesting that the strategy has not been effectively implemented. Two interviewees did talk of using a strategy similar to Reaching Every District, whereby they reviewed indicators, developed interventions and observed improvements in indicators.

There are several limitations to this study. As noted by some of the interviewees, the validity of the NHIS data may be questionable due not only to inaccurate reports but also inaccurate populations for coverage and rate calculations. This limitation highlights the benefit using mixed methods for this evaluation, particularly sequential explanatory mixed methods where qualitative data can explain the quantitative data9. A second limitation is that the key informant interviews provided qualitative information that is the perception of a limited number of people; it does not necessarily mean that these are the only views. These interviews form just one perspective of the health system in PNG.

Conclusion

Undertaking a contextual analysis using a sequential explanatory mixed methods design for the baseline of the formative evaluation of the Rural Primary Health Services Delivery Project proved useful for understanding the context in which the health facilities operate and potential issues with the data for use in the evaluation. Further, analysis of indicators at both district and health facility level enabled a richer understanding of the heterogeneity of performance. Subsequent to this study, the focus of the evaluation methods shifted from predominantly quantitative to predominantly qualitative methods to document the contextual factors that enabled or prevented project progress. Further, analyses of performance in indicators were undertaken at the district and health facility level rather than just the district level as planned. Such ongoing contextual analyses and facility-level analysis of indicators may be useful for health service delivery programs in similar contexts where health information data quality is poor and the context is not adequately documented elsewhere.

Acknowledgements

The authors sincerely thank the health administrators who participated in this study. This work was carried out as part of the formative evaluation of the Rural Primary Health Service Delivery Project. The Rural Primary Health Services Delivery Project is a joint initiative of the Government of PNG, the Asian Development Bank, Australian Department of Foreign Affairs and Trade, OPEC Fund for International Development, World Health Organization, United Nations Children’s Fund and the Japanese International Cooperation Agency.

references:

1 Asian Development Bank. Project administration manual. PNG Rural Primary Health Service Delivery Project, 2011. Available: web link (Accessed 27 August 2018).
2 World Health Organization. Monitoring the building blocks of health systems: a handbook of indicators and their measurement strategies. Geneva: World Health Organization, 2010.
3 de Savigny D, Adam T. Systems thinking for health systems strengthening. Geneva: World Health Organization, 2009.
4 Bamberger M, Rao V, Woolcock M. Using mixed methods in monitoring and evaluation: experiences from international development. 2010. Available: web link (Accessed 16 May 2018).
5 World Bank. Below the glass floor: analytical review of expenditure by provincial administrators on rural health from health function grants & provincial internal revenue. Washington: World Bank, 2013.
6 Howes S, Mako AA, Swan A, Walton G, Webster T, Wiltshire C. A lost decade? Service delivery and reforms in Papua New Guinea 2002–2012. 2014. Available: web link (Accessed 20 October 2017).
7 Australian Doctors International. Barriers to family planning services in PNG: supply and demand report and recommendations. Australian Doctors International, 2010.
8 National Department of Health. Papua New Guinea health sector monitoring and evaluation strategic plan: national health plan (2011–2020). Port Moresby: National Department of Health, 2012.
9 Creswell JW, Clarck VLP. Designing and conducting mixed methods research. 2nd Edn. Thousand Oaks: Sage Publications, 2011.
10 Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology 2006; 3(2): 77-101. DOI link
11 The World Bank. PNG health workforce crisis: a call to action. 2011. Available: web link (Accessed 27 August 2018).
12 Bauze AE, Tran LN, Nguyen KH, Firth S, Jimenez-Soto E, Dwyer-Lindgren L, et al. Equity and geography: the case of child mortality in Papua New Guinea. PLOS One 2012; 7(5): e37861. DOI link, PMid:22662238
13 Gething PW, Johnson FA, Frempong-Ainguah F, Nyarko P, Baschieri A, Aboagye P, et al. Geographical access to care at birth in Ghana: a barrier to safe motherhood. BMC Public Health 2012; 12: 991. DOI link, PMid:23158554
14 Vallely LM, Homiehombo P, Kelly AM, Vallely A, Homer CS, Whittaker A. Exploring women's perspectives of access to care during pregnancy and childbirth: a qualitative study from rural Papua New Guinea. Midwifery 2013; 29(10): 1222-1229. DOI link, PMid:23684099
15 Ashwell HE, Barclay L. Problems measuring community health status at a local level: Papua New Guinea's health information system. Rural and Remote Health 2010; 10(4): 1539. Available: web link (Accessed 17 August 2018). PMid:21214300
16 National Department of Health. Sector performance annual review: assessment of sector performance 2010-2014. Port Moresby: Government of Papua New Guinea, 2015.
17 World Health Organization. Global Health Observatory data repository WHO Western Pacific Region: Papua New Guinea statistics summary (2002 – present). Available: web link (Accessed 18 August 2014).
18 Kanyuuru L, Kabue M, Ashengo TA, Ruparelia C, Mokaya E, Malonza I. RED for PMTCT: an adaptation of immunization's Reaching Every District approach increases coverage, access, and utilization of PMTCT care in Bondo District, Kenya. International Journal of Gynaecology and Obstetrics 2015; 130 Suppl 2: S68-S73. DOI link, PMid:26115861
19 World Health Organization. Microplanning for immunization service delivery using the Reaching Every District (RED) Strategy. Geneva: World Health Organization, 2009.
20 Datta SS, Toikilik S, Ropa B, Chidlow G, Lagani W. Pertussis outbreak in Papua New Guinea: the challenges of response in a remote geo-topographical setting. Western Pacific Surveillance and Response Journal 2012; 3(4): 3-6. DOI link, PMid:23908930
21 Apeng D, Drorit D, Mabong P, Theo L, Fitzwarryne C, Bunat A. The 'Reach Every Village' strategy for community-based health improvement interventions in the Momase Region of Papua New Guinea. Papua New Guinea Medical Journal 2010; 53(1-2): 37-47. PMid:22768478

You might also be interested in:

2016 - The devil's in the demography: comparison and change in the remote and rural nursing workforce in Scotland between 2001 and 2011 using representative longitudinal data

2013 - The role of risk theory in rural maternity services planning

2010 - An evaluation study on the relevance and effectiveness of training activities in Northern Uganda

This PDF has been produced for your convenience. Always refer to the live site https://www.rrh.org.au/journal/article/4484 for the Version of Record.