Reference as: Padrnos L, Bui T, Pattee JJ, Whitmore EJ, Iqbal M, Lee S, Singarajah CU, Robbins RA. Analysis of overall level of evidence behind the Institute of Healthcare Improvement ventilator-associated pneumonia guidelines. Southwest J Pulm Crit Care 2011;3:40-8. (Click here for PDF version of manuscript)
Leslie Padrnos1,4(lpadrnos@email.arizona.edu)
Tony Bui1,4 (tony.bui@cox.net)
Justin J. Pattee2,4 (backageyard@gmail.com)
Elsa J. Whitmore2,4 (elsa_whitmore@hotmail.com)
Maaz Iqbal1,4 (maaziqbal@gmail.com)
Steven Lee3,4 (timmah2k@gmail.com)
Clement U. Singarajah2,4 (clement.singarajah@va.gov)
Richard A. Robbins1,4,5 (rickrobbins@cox.net)
1University of Arizona College of Medicine
2Midwestern University-Arizona College of Osteopathic Medicine
3Kirksville College of Osteopathic Medicine
4Phoenix VA Medical Center
5Phoenix Pulmonary and Critical Care Research and Education Foundation
None of the authors report any significant conflicts of interest.
Abstract
Background
Clinical practice guidelines are developed to assist in patient care but the evidence basis for many guidelines has recently been called into question.
Methods
We conducted a literature review using PubMed and analyzed the overall quality of evidence and made strength of recommendation behind 6 Institute of Health Care (IHI) guidelines for prevention of ventilator associated pneumonia (VAP). Quality of evidence was assessed by the American Thoracic Society levels of evidence (levels I through III) with addition of level IV when evidence existed that the guideline increased VAP. We also examined our own intensive care units (ICUs) for evidence of a correlation between guideline compliance and the development of VAP.
Results
None of the guidelines could be given more than a moderate recommendation. Only one of the guidelines (head of bed elevation) was graded at level II and could be given a moderate recommendation. One was graded at level IV (stress ulcer disease prophylaxis). The remainder were graded level III and given weak recommendations. In our ICUs compliance with the guidelines did not correlate with a reduction in VAP (p<0.05).
Conclusions
Most of the IHI guidelines are based on level III evidence. Data from our ICUs did not support guideline compliance as a method of reducing VAP. Until more data from well-designed controlled clinical trials become available, physicians should remain cautious when using current IHI VAP guidelines to direct patient care decisions or as an assessment of the quality of care.
Introduction
The growth of guideline publications addressing nearly every aspect of patient care has been remarkable. Over the past 30 years numerous medical regulatory organizations have been founded to improve the quality of care. Many of these organizations have developed medical regulatory guidelines with 6870 listed in the National Guideline Clearinghouse (1). Many of these guidelines were rapidly adopted by healthcare organizations as a method to improve care.
Interest has grown in critically appraising not only individual clinical practice guidelines but also entire guideline sets of different medical (sub)specialties based on their rapid proliferation and in many instances an overall lack of efficacy in improving care (2,3). We assessed the quality of evidence underlying recommendations from one medical regulatory organization, the Institute for Healthcare Improvement (IHI), regarding one set of guidelines, the ventilator associated pneumonia (VAP) guidelines or VAP bundles (4). This was done by senior medical students during a month long rotation in the Phoenix Veterans Administration ICU.
Methods
The study was approved by the Western Institutional Review Board.
Literature Search
In each instance PubMed was searched using VAP which was cross referenced with each component of the VAP bundle (as modified by the Veterans Administration) using the following MESH terms: 1. Elevation of the head of the bed; 2. Daily sedation vacation; 3. Daily readiness to wean or extubate; 4. Daily spontaneous breathing trial; 5. Peptic ulcer disease prophylaxis; and 6. Deep venous thrombosis prophylaxis. In addition, each individual component of the term was cross referenced with VAP. We also reviewed “Related citations” as listed on PubMed. Additional studies were identified using the “Related citations” in Pubmed from studies listed as supporting evidence on the IHI website and from the references of these studies.
Each study was assessed for appropriateness to the guideline. Studies were required to be prospective and controlled in design. Only studies demonstrating a reduction in VAP were considered, i.e., surrogate outcomes such as reduction in duration of mechanical ventilation were not considered.
The American Thoracic Society grading system was used to assess the underlying quality of evidence for the IHI VAP guidelines (5) (Table 1). Only evidence supporting a reduction in VAP was considered. We added category IV when there was literature evidence of potentially increasing VAP with the use of the recommendation. A consensus was reached in each case.
Table I. Levels of Evidence
|
Level of Evidence |
Definition |
|
Level I (high)
|
Evidence from well-conducted, randomized controlled trials.
|
|
Level II (moderate)
|
Evidence from well-designed, controlled trials without randomization (including cohort, patient series, and case-control Studies). Level II studies also include any large case series in which systematic analysis of disease patterns was conducted, as well as reports of data on new therapies that were not collected in a randomized fashion. |
|
Level III (low)
|
Evidence from case studies and expert opinion. In some instances, therapy recommendations come from antibiotic susceptibility data without clinical observations. |
|
Level IV |
No evidence of improvement with some evidence of an increase in a negative outcome. |
Guideline Compliance and VAP Incidence
We also assessed our ICUs for additional evidence of the effectiveness of the VAP bundle. Data was collected for a period of 50 months from January, 2007 through February, 2011. This was after the Veterans Administration requirements for VAP reporting and IHI compliance was instituted. Diagnosis and compliance were assessed by a single quality assurance nurse using a standardized protocol (6). Statistical analysis was done using a Pearson correlation coefficient with a two-tailed test. Significance was defined as p<0.05.
Results
Literature Review
Numbers of articles identified by PubMed search and used for grading the level of evidence and strength of recommendation are given in Table 2. Also included are the level of evidence and the strength of the recommendation.
Table 2.
|
Guideline |
Total Articles |
No. of Articles Used (references) |
Level of Evidence |
Strength of Recommendation |
|
Elevation of the head of the bed |
31 |
8 (7-14) |
II |
Moderate |
|
Daily sedation vacation |
66 |
4 (15-18) |
III |
Weak |
|
Daily readiness to wean or extubate |
47 |
3 (19-21) |
III |
Weak |
|
Daily spontaneous breathing trial |
29 |
1 (22) |
III |
Weak |
|
Peptic ulcer disease prophylaxis |
52 |
9 (23-29) |
IV |
Weak |
|
Deep venous thrombosis prophylaxis. |
14 |
2 (30-31) |
III |
Weak |
Head of Bed Elevation
A literature search identified 31 articles of which 8 were used in evaluating this guideline (7-14). However, only 2 specifically studied head of bed elevation with one supporting and another not supporting the intervention (7,8). Consequently it was graded as level II and the strength of recommendation was graded as moderate.
Daily Spontaneous Breathing Trial, Daily Readiness to Wean, and Daily Sedation Vacation
From 1-4 studies were identified for each of these interventions, however, none demonstrated a reduction in VAP. Consequently, it was judged that the evidence basis was level III and the strength of recommendation was graded as weak.
Stress Ulcer Disease Prophylaxis
We found no evidence that stress ulcer disease prophylaxis decreased VAP (23-29). There was some evidence that acid suppressive therapy increased pneumonia and VAP. Consequently, it was judged to be a level IV (possibly increasing VAP).
Deep Venous Thrombosis Prophylaxis
We could find no evidence that deep venous thrombosis prophylaxis decreased VAP (30,31).
Guideline Compliance and VAP Incidence
Beginning in the first quarter of fiscal year 2007 there was a significant decrease in the incidence of VAP in our hospital (33). This coincided with the requirement for the monitoring of VAP, compliance with the VAP bundles and our adoption of endotracheal aspiration with nonquantitative culture of the aspirate as opposed to bronchoalveolar lavage which had been out standard practice. We changed practices because bronchoalveolar lavage with quantitative cultures appeared to offer no improvement in clinical outcomes to endotracheal aspiration (34). In our medical and surgical ICUs, 5097 audits representing 5800 ventilator-days were assessed. Nineteen cases of VAP were identified with an average of 2.1 VAP infections/1000 ventilator-days. We assessed our surgical and medical ICUs, combined and separately, for a correlation between total bundle compliance and each component of the VAP bundle with VAP incidence (Appendices 1-3). There was no significant correlation between compliance with the bundles and VAP (p<0.05).
Discussion
This manuscript questions the validity of the VAP bundles as proposed by the IHI. We found that a systematic review of the literature revealed predominately weak evidence to support these guidelines. Only one guideline (head of bed elevation) was supported by a randomized trial (7), but an additional, larger trial showed no decrease in VAP (8). Furthermore, data from our own ICUs showed no evidence of IHI VAP guideline compliance with a reduction in VAP.
Head of bed elevation is a relatively simple and easy to perform intervention which may reduce VAP. Studies examining aspiration have shown a reduction in critical care patients with the head of bed elevation but it is unclear whether this translates into a reduction in VAP (36,37). Drakulovic et al. (7) reported a randomized controlled trial in 86 mechanically ventilated patients assigned to semi-recumbent or supine body position. The trial demonstrated that suspected cases of ventilator-associated pneumonia had an incidence of 34 percent while in the semi-recumbent position suspected cases had an incidence of 8 percent (p=0.003). However, another study in 221 subjects demonstrated that the target head elevation of 45 degrees was not achieved for 85% of the study time, and these patients more frequently changed position than supine-positioned patients (8). The achieved difference in treatment position (28 degrees vs. 10 degrees) did not prevent the development of ventilator-associated pneumonia. The other 5 articles identified either did not identify head of bed elevation directly or as part of a bundle. Most were a before and after design and not randomized. Therefore, it is difficult to draw any meaningful conclusions.
The IHI groups daily "sedation vacations" and assessing the patient’s “readiness to extubate.” The logic is that more rapid extubation leads to a reduction in VAP. Kress et al. (15) conducted a randomized controlled trial in 128 adult patients on mechanical ventilation, randomized to either daily interruption of sedation irrespective of clinical state or interruption at the clinician’s discretion. Daily interruption resulted in a reduction in the duration of mechanical ventilation from 7.3 days to 4.9 days (p=0.004). However, in a retrospective review of the data, the authors were unable to show a significant reduction in VAP (16).
Stress ulcer prophylaxis and deep venous thrombosis prophylaxis are routine in most ICUs. However, stress ulcer prophylaxis with enteral feeding is probably as effective as acid suppressive therapy and acid suppressive therapy may increase the incidence of VAP (38). Deep venous thrombosis prophylaxis has been shown to decrease the incidence of pulmonary emboli but not improve mortality (32). Although we use these interventions in our ICU, we would suggest that these would be more appropriate for recommendations rather than guidelines.
The diagnosis of VAP is difficult, requiring clinical judgment even in the presence of objective clinical criteria (6). The difficulty in diagnosis, along with the negative consequences for failure to follow the IHI guidelines, makes before and after comparisons of the incidence of VAP unreliable. Therefore, we sought evidence for the effectiveness of VAP prevention guidelines reasoning that the better the compliance with the guidelines, the lower the incidence of VAP. We were unable to show that improved VAP guideline compliance correlated with a reduced incidence of VAP.
The IHI guidelines would not meet the criteria outlined earlier in an editorial in the Southwest Journal of Pulmonary and Critical Care for a good guideline:
Our study has several limitations. No literature review is totally comprehensive. It is possible that studies relevant to the IHI VAP guidelines, especially those written in a foreign language, were not identified. Second, the Phoenix VA data may be underpowered to show a small beneficial effect despite having over 5000 patient audits. Third, as with other healthcare facilities, the VAP guidelines at our institution were mandated and monitored. The threat of negative consequences may have compromised the objective assessment of the data, likely invalidating a before and after comparison. Fourth, correlation between guideline compliance and VAP incidence is not a substitute for a randomized trial. Unfortunately, the later is not possible given that guideline compliance is mandated.
It is unclear why the IHI guidelines have received such wide acceptance given their weak evidence basis. Agencies involved in guideline writing should show restraint in guideline formulation based on opinion or weak or conflicting evidence. Only those interventions based on strong evidence which can make a real difference to patients should be designated as guidelines.
Acknowledgements
The authors acknowledge Janice Allen, MSN, RN who collected the VAP data reported from the Phoenix VA.
References
Appendices
Click here for Appendix 1. VAP rate in all ICUs