| Abstract|| |
Background: Risk-adjusted mortality is widely used to benchmark trauma center care. Patients presenting with isolated hip fractures (IHFs) are usually excluded from these evaluations. However, there is no standardized definition of an IHF. We aimed to evaluate whether there is consensus on the definition of an IHF used as an exclusion criterion in studies evaluating the performance of trauma centers in terms of mortality. Materials and Methods: We conducted a systematic review of observational studies. We searched the electronic databases MEDLINE, EMBASE, BIOSIS, The Cochrane Library, CINAHL, TRIP Database, and PROQUEST for cohort studies that presented data on mortality to assess the performance of trauma centers and excluded IHF. A standardized, piloted data abstraction form was used to extract data on study settings, IHF definitions and methodological quality of included studies. Consensus was considered to be reached if more than 50% of studies used the same definition of IHF. Results: We identified 8,506 studies of which 11 were eligible for inclusion. Only two studies (18%) used the same definition of an IHF. Three (27%) used a definition based on Abbreviated Injury Scale (AIS) Codes and five (45%) on International Classification of Diseases (ICD) codes. Four (36%) studies had inclusion criteria based on age, five (45%) on secondary injuries, and four (36%) on the mechanism of injury. Eight studies (73%) had good overall methodological quality. Conclusions: We observed important heterogeneity in the definition of an IHF used as an exclusion criterion in studies evaluating the performance of trauma centers. Consensus on a standardized definition is needed to improve the validity of evaluations of the quality of trauma care.
Keywords: Benchmarking, consensus, definition, isolated hip fractures, mortality, systematic review, trauma
|How to cite this article:|
Tiao J, Moore L, Boutin A, Turgeon AF. Establishing consensus on the definition of an isolated hip fracture for trauma system performance evaluation: A systematic review. J Emerg Trauma Shock 2014;7:209-14
|How to cite this URL:|
Tiao J, Moore L, Boutin A, Turgeon AF. Establishing consensus on the definition of an isolated hip fracture for trauma system performance evaluation: A systematic review. J Emerg Trauma Shock [serial online] 2014 [cited 2022 Jan 18];7:209-14. Available from: https://www.onlinejets.org/text.asp?2014/7/3/209/136867
| Introduction|| |
Traumatic injuries represent the first cause of death in North Americans aged less than 45 years.  The financial impact of injuries in the United States was estimated at 80.2 billion in 2000, for a deficit of 326 billion dollars in productivity. , The evaluation of trauma center performance to improve the quality and efficiency of trauma care is therefore of crucial importance.
The evaluation of trauma center performance is widely based on estimates of risk-adjusted mortality.  The effectiveness and validity of performance evaluations depend partly on a clear, standardized definition of the target population. Patients with isolated hip fracture (IHF) are usually excluded from trauma performance evaluations, as such injuries are considered a marker of chronic disease rather than a traumatic injury.  IHF can be defined according to diagnostic codes (Abbreviated Injury Scale (AIS) or International Classification of Diseases (ICD)), the presence or severity of concomitant injuries, and additional criteria including age and injury mechanism. Lack of consensus on an appropriate definition may compromise the validity and comparability of trauma system performance evaluations. 
The aim of our study is to evaluate whether there is consensus on the definition of an IHF used as an exclusion criterion for trauma center performance evaluations based on patient mortality.
| Materials and methods|| |
We conducted a systematic review of observational studies designed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. We systematically searched MEDLINE, EMBASE, BIOSIS, the Cochrane Library, CINAHL, TRIP, and PROQUEST electronic databases.
We constructed our search strategy using combinations of keywords, MeSH (Medline) and Emtree (EMBASE) from three domains: i) Trauma, ii) performance/quality, and iii) mortality (see appendix 1) [Additional file 1]. The search strategy was first developed for Medline and then adapted for other databases. No restriction based on language or year was applied.
Both prospective and retrospective cohort studies evaluating the performance of acute care hospitals for the treatment of global trauma populations (i.e., not specific pathologies) using mortality were considered for inclusion. We then selected studies excluding patients with an IHF for the review. Performance evaluation was defined as a comparison of hospitals within a health care system, over time or to an external standard.
Duplicates were identified and eliminated using the EndNote software version X4 (Thomson Reuters, 2010) and manual screening. Two independent reviewers (JT and AB) evaluated citations identified in the systematic search for eligibility by screening titles, abstracts, and full publications. Disagreement on study eligibility was resolved by consensus and two other reviewers (LM and AFT) were involved when required. Inter-rater agreement was evaluated with kappa statistics on study eligibility. Articles written in a language other than English were translated.
Data was abstracted by two independent reviewers using a standardized data extraction form piloted on a sample of five representative articles. The data abstraction form was designed to capture information on study setting and design, the study population, definitions of IHF (including diagnostic codes, age, mechanism of injury, and additional injuries), and methodological quality. The latter was evaluated using the following six elements adapted from Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines,  the Cochrane Collaboration's tool for assessing risk of bias  and Downs and Black checklist: 
- Data quality assurance efforts reported,
- Measures of variation provided (i.e., standard errors or confidence intervals),
- risk adjustment performed,
- Appropriate treatment of missing data,
- Appropriate sample size, and
- Sensitivity analyses reported.
Appropriate treatment of missing data implied that the proportion of missing data were reported and if more than 10% of subjects had missing data, multiple imputation techniques, and/or sensitivity analyses were used.  Adequate sample size was considered to be respected if at least 100 patients per hospital were available for analysis or if not, analysis strategies designed for low-volume centers (e.g., shrinkage techniques) were used.  Disagreement between reviewers on abstracted data was resolved by consensus or if necessary, consultation with two other reviewers.
The definition used for an IHF was described in terms of ICD or AIS diagnostic codes and additional criteria including age, severity of concomitant injuries, and mechanism of injury. Consensus on the definition of IHF was considered to be reached if at least 50% of studies used the same definition according to Grading of Recommendations Assessment, Development, and Evaluation (GRADE) criteria.  The quality of study methodology (risk of bias) was described by the proportion of studies adhering to each of six quality criteria. A study was considered to be at low risk of bias if it adhered to at least four out of six criteria.
| Results|| |
The search strategy identified 8,506 references of which 58 were selected for final review because they evaluated hospital performance on general trauma populations using mortality data [Figure 1]. Of these, 11 (19%) reported that they excluded patients with IHF and were therefore included in our study. ,,,,,,,,,,
Included studies were conducted in Canada (n = 5) ,,,,, or in the United States (n = 6) ,,,,, [Table 1]. All were retrospective cohort studies published in English between 1993 and 2007. Despite the fact that all studies were performed in general trauma populations, inclusion/exclusion criteria varied across studies. Some studies used inclusion criteria based on traumatic injury; ,,, whereas, others added criteria based on injury severity, ,, transfer ,,,,, or diagnostic codes. ,,, Common exclusion criteria other than IHF included death on arrival and poisoning/suffocation/drowning.
Definition of IHFs
Three studies did not report any definition of an IHF ,, [Table 2]. Only two studies (18%) used the same definition of an IHF and these studies were conducted by the same first author. , Five studies used ICD codes ,,,, and three used AIS codes ,, to define an IHF. , Five studies also included a criterion based on secondary injuries, ,,,, four used a criterion based on mechanism of injury, ,,, and four studies used a criterion based on patient's age. ,,, Overall, one study used all four criteria (diagnostic code, secondary injury, mechanism, and age) to define IHF,  five studies used three criteria ,,,, and two studies used one criterion in their definition. ,
Methodological quality and risk of bias assessment
Eight studies (73%) were considered to be of good methodological quality, because they respected at least four out of six criteria ,,,,,,, [Table 3]. Nine studies reported variation measures for performance indices, ,,,,,,,, all studies used risk adjustment, and four studies reported using data quality assurance. ,,, Eight studies provided information on missing data and addressed this problem by using multiple imputation. ,,, However, three studies used single imputation techniques that do not account for the uncertainty surrounding missing values (leads to variance underestimation), , and four studies excluded variables with missing data from multivariable analyses. ,,, Finally, eight studies conducted sensitivity analyses ,,,,,,, and five studies reported having obtained research ethics board approval. ,,,,
| Discussion|| |
We did not observe consensus on the definition of IHF used as an exclusion criterion for trauma center performance evaluations based on mortality. Only two studies used the same definition of an IHF. , In addition, these two studies were based on a single diagnostic ICD code and were from the same group of researchers. Diagnostic codes (ICD or AIS) and additional criteria based on secondary injuries, mechanism of injury, and age all varied across studies. Overall, the methodological quality of included studies was high.
IHF represent up to 47% of injury admissions in patients aged 65 years or over and this proportion is increasing rapidly.  Patients with IHF significantly differ from other trauma patients for several reasons:
- IHF is frequently the consequence of a chronic disease rather than a trauma per se,
- They typically have higher resource use and worse outcomes despite lower injury severity, 
- They often present with important comorbidities,
- They are often treated outside the trauma system which leads to incomplete case capture in trauma registries, and
- According to the National Trauma Data Standard, IHF should be included in trauma registries. 
However, 48% of trauma centers that contribute to the National Trauma Data Bank exclude patients with IHF from their trauma registries.  Despite the inclusion of IHF in trauma registries, the question of whether they should be included in performance evaluations remains controversial.  In this systematic review, we observed that patients with IHF were excluded from 20% of studies evaluating the performance of trauma centers in terms of mortality. The influence of including/excluding IHF on trauma center performance results is well documented. In 2010, Gomez and colleagues reported that more than three-quarters (78%) of centers changed their rank after patients with IHF were excluded from trauma center performance analyses.  In addition, Bergeron and colleagues observed that the inclusion/exclusion of IHF from trauma populations had an important impact on the evaluation of patient outcome and resource use.  Therefore, if IHF are not excluded from performance evaluations, they should certainly be evaluated separately. If they are to be excluded or separated in trauma center benchmarking activities, identifying these cases according to a standard definition is of great importance. This study shows that at present, no consensus on such a definition exists. These results are consistent with previous studies that have observed variations in the definition of same-level falls and age cut-offs for IHF. , Indeed, a survey among trauma registrars reported that in 18 different states, six different definitions of same-level falls were used as trauma registry exclusion criteria with varying age cutoffs and ICD-9 codes.  Hospitals contributing to the National Trauma Data Bank also use different age cutoffs to define IHF. 
Our study has some limitations. First, despite the exhaustiveness of our search strategy and good inter-rater reliability on study inclusion, we might have missed some studies. Furthermore, we did not include studies that did not provide any information on exclusion or inclusion of patients with IHF. This may have led us to miss some definitions of IHF. However, this is unlikely to have affected the observed lack of consensus across studies. Second, several studies did not report information on methodological quality criteria. Lack of information was assumed to mean that they did not meet the criteria and may have led to an underestimation of methodological quality. However, adequate reporting of important information is a reflection of study quality. 
| Conclusion|| |
In this systematic review, we observed no consensus on the definition of IHF used as an exclusion criterion in studies evaluating the performance of trauma centers in terms of mortality. This lack of standardization may compromise the validity of trauma center benchmarking. It is thus important for future studies to assess the impact of the definition of IHF on the results of performance evaluations. In the event that significant changes are observed, an effort to reach expert consensus on a standardized definition of IHF must be made.
| References|| |
|1.||Center for Disease Control. Cost of injury. 2007. Available from: http://www.cdc.gov/ncipc/factsheets/Cost_of_Injury.htm. [Last accessed on 2013 June 6]. |
|2.||Sunnybrook Research Institute. The enormous human and financial cost of trauma. 2007. Available from: http://www.swri.ca/programs/trauma. [Last accessed on 2013 April 16]. |
|3.||Finkelstein EA, Corso PS, Miller TR. The incidence and economic burden of injury in the United States. Centers for Disease Control and Prevention. 2006. Available from: http://www.cdc.gov/ncipc/factsheets/CostBook/Economic_Burden_of_Injury.htm. [Last accessed on 2013 May 26]. |
|4.||Iezzoni L. Risk adjustment for measuring health care outcomes. 3 rd ed. Chicago: Health Administration Press; 2003. |
|5.||American College of Surgeons. National Trauma Data Bank report 2011. Available from: http://www.facs.org/trauma/ntdb.html [Last accessed on 2013 April 27]. |
|6.||von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP; STROBE Initiative. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: Guidelines for reporting observational studies. Ann Intern Med 2007;147:573-7. |
|7.||Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, et al., Cochrane Bias Methods Group, Cochrane Statistical Methods Group. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928. |
|8.||Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health 1998;52:377-84. |
|9.||Allison, PD. Missing Data Thousand Oaks, CA: Sage Publications;2001. |
|10.||Jaeschke R, Guyatt GH, Dellinger P, Schünemann H, Levy MM, Kunz R, et al., GRADE Working Group. Use of GRADE grid to reach decisions on clinical practice guidelines when consensus is elusive. BMJ 2008:337:a744. |
|11.||Bergeron E, Lavoie A, Belcaid A, Ratte S, Clas D. Should patients with Isolated hip fractures be included in trauma registries? J Trauma 2005;58:793-7. |
|12.||Bergeron E, Lavoie A, Moore L, Bamvita JM, Ratte S, Clas D. Paying the price of excluding patients from a trauma registry. J Trauma 2006;60:300-4. |
|13.||Cudnik MT, Newgard CD, Sayre MR, Steinberg SM. Level I versus Level II trauma centers: An outcomes-based assessment. J Trauma 2009;66:1321-6. |
|14.||Cudnik MT, Sayre MR, Hiestand B, Steinberg SM. Are all trauma centers created equally? A statewide analysis. Acad Emerg Med 2010;17:701-8. |
|15.||Gomez D, Haas B, Hemmila M, Pasquale M, Goble S, Neal M, et al. Hips can lie: Impact of excluding isolated hip fractures on external benchmarking of trauma center performance. J Trauma 2010;69:1037-41. |
|16.||Haas B, Gomez D, Hemmila MR, Nathens AB. Prevention of complications and successful rescue of patients with serious complications: Characteristics of high-performing trauma centers. J Trauma 2011;70:575-82. |
|17.||Haas B, Gomez D, Xiong W, Ahmed N, Nathens AB. External benchmarking of trauma center performance: Have we forgotten our elders? Ann Surg 2011;253:144-50. |
|18.||Mancuso C, Barnoski A, Tinnell C, Fallon W Jr. Using Trauma and Injury Severity Score (TRISS)-based analysis in the development of regional risk adjustment tools to trend quality in a voluntary trauma system: The experience of the Trauma Foundation of Northeast Ohio. J Trauma 2000;48:629-34. |
|19.||Moore L, Lavoie A, Camden S, Le Sage N, Sampalis JS, Bergeron E, et al. Statistical validation of the Glasgow Coma Score. J Trauma 2006;60: 1238-43. |
|20.||Moore L, Hanley JA, Turgeon AF, Lavoie A, Eric B. A new method for evaluating trauma centre outcome performance: TRAM-adjusted mortality estimates. Ann Surg 2010;251:952-8. |
|21.||Tinkoff GH, Reed JF 3rd, Megargel R, Alexander EL 3rd, Murphy S, Jones MS. Delaware's inclusive trauma system : Impact on mortality. J Trauma 2010;69:245-52. |
|22.||Moore L, Lavoie A, LeSage N, Abdous B, Bergeron E, Liberman M, et al. Statistical validation of the Revised Trauma Score. J Trauma 2006;60:305-11. |
|23.||Clark DE, DeLorenzo MA, Lucas FL, Wennberg DE. Epidemiology and short-term outcomes of injured medicare patients. J Am Geriatr Soc 2004;52:2023-30. |
|24.||American College of Surgeons. National Trauma Data Bank user manual. Version 7.2. 2009. Available from: http://www.facs.org/trauma/ntdb/pdf/usermanual72.pdf. [Last accessed on 2013 Sept 26]. |
|25.||Mann NC, Guice K, Cassidy L, Wright D, Koury J. Are statewide trauma registries comparable? Reaching for a national trauma dataset. Acad Emerg Med 2006;13:946-53. |
Department of Social and Preventive Medicine; Axe Santé des populations et pratiques optimales en santé (traumatologie-urgence-soins intensifs), Centre de Recherche du CHU de Québec - Hôpital de l'Enfant-Jésus, Université Laval, Québec
Source of Support: The Canadian Health Services Research
Foundation, the Fondation de Recherche en Santé du Québec
(project #RC2-1460-05), and the Canadian Health Services
Research Foundation (LM is a recipient of a new investigator
award)., Conflict of Interest: None
[Table 1], [Table 2], [Table 3]