Rural trauma in Iran: are the data reliable?

full article:

Dear Editor

An Iranian team has reported the results of a rural pre-hospital trauma system¹. In a previous comment to their article I asked the team to come forward with additional information in order to validate the accuracy of the main variable in their study, the physiological severity score (PSS)². The Iranian colleagues responded promptly³ and thus opened an important discussion on quality standards for trauma registry analysis.

It is well documented that injury severity is the main and most accurate risk predictor of trauma death. As such it sets the foundation for any trauma registry analysis and quality control of trauma system performances. Therefore the data, especially for the severity variables, must be 100% reliable. As recommended for trauma registries, the Iranian database also measures injury severity by two variables: (i)anatomical severity measured by the injury severity score (ISS); and (ii)physiological severity measured by PSS, equivalent to the revised trauma score (RTS).

There are two plain ways to screen data reliability for the main variables. The first is that there should be a reasonable correlation between anatomical and physiological injury severity. Our center runs trauma systems in Iraq and Cambodia. In these trauma registries (Iraq, n=3500; Cambodia, n=1800) the two scores correlate well (Pearson r 0.6 and Spearman's r_s at 0.55). In the Iranian dataset, Saghafinia reports a more moderate correlation between the two (Spearman's r_s at 0.4)³. The discrepancy does not necessarily indicate data flaws; it may well be explained by mixed subgroups and ties to other variables, such as variations in pre-hospital transit times among different scenarios⁴.

Second, receiver operating curve (ROC) statistics is a more solid validation method as it does not take account of distributional assumptions. We use ROC analysis to measure the accuracy of medical tests. In this case, how well does a severity indicator predict the ultimate trauma outcome - survival or death? The accuracy is expressed as the area under curve (ROC-AUC). By tradition we classify AUC values as follows: <0.7 as poor; values in the range of 0.7-0.8 as fair accuracy; and AUC >0.9 as good accuracy⁵. In trauma registry validations we analyze the accuracy of physiological severity (PSS) as trauma death predictors. In our trauma registry in Iraq (n=3500) - which draws data from a population of land mine and war injured very much similar to the Iranian study population - the PSS variable proves to have very high accuracy predicting trauma death, ROC-AUC 0.93 (M Murad, H Husum, pers. data, 2009). The finding corresponds well with other studies of large trauma cohorts^6-8. For the Iranian data set, however, Saghafinia reports the ROC-AUC for PSS to be very low, 0.6, close to the predictive value of flipping a coin (ROC-AUC=0.5). We should thus conclude that the main result variable in the Iranian study performs poorly as trauma outcome predictor.

Comparing results reported by the Iranian team and results from other studies, we believe that the data quality in the Iranian study should be scrutinized.

Hans Husum, MD, PhD
Tromsoe Mine Victim Resource Center
University Hospital North Norway
Norway

References

1. Nafissi N, Saghafinia M, Balochi K. Improving trauma care in rural Iran by training existing treatment chains. Rural and Remote Health 8:881. (Online) 2008. Available: www.rrh.org.au (Accessed 1 April 2010).

2. Husum H. Comment on: Improving trauma care in rural Iran by training existing treatment chains. Rural and Remote Health 9: 1226. (Online) 2009. Available: www.rrh.org.au (Accessed 1 April 2010).

3. Saghafinia M. Reply to comment on: Improving trauma care in rural Iran by training existing treatment chains. Rural and Remote Health 9: 1240. (Online) 2009. Available: www.rrh.org.au (Accessed 1 April 2010).

4. Altman DG. Practical statistics for medical research. London: Chapman & Hall, 1999: 282-288.

5. Zweig MH, Campbell G. Receiver-Operating Characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clinical Chemistry 1993; 39: 561-577.

6. Ross SE, Leipold C, Terregino C, O'Malley KF. Efficiacy of the motor component of the Glasgow Coma Scale in trauma triage. Journal of Trauma 1998; 45:42-44.

7. Meredith W, Rutledge R, Hansen AR, Oller DW, Thomason M, Cunningham P et al. Field triage of trauma patients based upon ability to follow commands: a study in 29,573 patients. Journal of Trauma 1995; 38:129-135.

8. Husum H, Gilbert M, Wisborg T, Heng YV, Murad M. Respiratory rate as prehospital triage tool in rural trauma. Journal of Trauma 2003; 55: 466-470.