Search Journal-type in search term and press enter
Social Media-Follow Southwest Journal of Pulmonary and Critical Care on Facebook and Twitter

General Medicine

(Click on title to be directed to posting, most recent listed first)

Tacrolimus-Associated Diabetic Ketoacidosis: A Case Report and Literature 
   Review
Nursing Magnet Hospitals Have Better CMS Hospital Compare Ratings
Publish or Perish: Tools for Survival
Is Quality of Healthcare Improving in the US?
Survey Shows Support for the Hospital Executive Compensation Act
The Disruptive Administrator: Tread with Care
A Qualitative Systematic Review of the Professionalization of the 
   Vice Chair for Education
Nurse Practitioners' Substitution for Physicians
National Health Expenditures: The Past, Present, Future and Solutions
Credibility and (Dis)Use of Feedback to Inform Teaching : A Qualitative
   Case Study of Physician-Faculty Perspectives
Special Article: Physician Burnout-The Experience of Three Physicians
Brief Review: Dangers of the Electronic Medical Record
Finding a Mentor: The Complete Examination of an Online Academic 
   Matchmaking Tool for Physician-Faculty
Make Your Own Mistakes
Professionalism: Capacity, Empathy, Humility and Overall Attitude
Professionalism: Secondary Goals 
Professionalism: Definition and Qualities
Professionalism: Introduction
The Unfulfilled Promise of the Quality Movement
A Comparison Between Hospital Rankings and Outcomes Data
Profiles in Medical Courage: John Snow and the Courage of
   Conviction
Comparisons between Medicare Mortality, Readmission and 
   Complications
In Vitro Versus In Vivo Culture Sensitivities:
   An Unchecked Assumption?
Profiles in Medical Courage: Thomas Kummet and the Courage to
   Fight Bureaucracy
Profiles in Medical Courage: The Courage to Serve
   and Jamie Garcia
Profiles in Medical Courage: Women’s Rights and Sima Samar
Profiles in Medical Courage: Causation and Austin Bradford Hill
Profiles in Medical Courage: Evidence-Based 
   Medicine and Archie Cochrane
Profiles of Medical Courage: The Courage to Experiment and 
   Barry Marshall
Profiles in Medical Courage: Joseph Goldberger,
   the Sharecropper’s Plague, Science and Prejudice
Profiles in Medical Courage: Peter Wilmshurst,
   the Physician Fugitive
Correlation between Patient Outcomes and Clinical Costs
   in the VA Healthcare System
Profiles in Medical Courage: Of Mice, Maggots 
   and Steve Klotz
Profiles in Medical Courage: Michael Wilkins
   and the Willowbrook School
Relationship Between The Veterans Healthcare Administration
   Hospital Performance Measures And Outcomes 

 

Although the Southwest Journal of Pulmonary and Critical Care was started as a pulmonary/critical care/sleep journal, we have received and continue to receive submissions that are of general medical interest. For this reason, a new section entitled General Medicine was created on 3/14/12. Some articles were moved from pulmonary to this new section since it was felt they fit better into this category.

-------------------------------------------------------------------------------------

Friday
Apr272012

Profiles in Medical Courage: Peter Wilmshurst, the Physician Fugitive

"Nothing defines human beings better than their willingness to do irrational things in the pursuit of phenomenally unlikely payoffs." -Scott Adams, US humorist, in The Dilbert Principle (1996)

Peter Wilmshurst, a cardiologist in the UK, has taken on medical companies over their honesty not once, but twice. His exciting story illustrates how far companies will go to protect their quest for big bucks and the potential financial risk incurred by honest physicians who do clinical research.

A movie entitled “The Fugitive”, based on the popular 1960’s television series, was released in 1993. The movie and television series both tell the story of Dr. Richard Kimble wrongly accused of murdering his wife and Kimble’s search for the real killer, a one-armed man. In the movie version, Harrison Ford plays Kimble and discovers that the one-armed man, Frederick Sykes, is employed by a pharmaceutical company that is working on a new drug. Kimble had investigated the drug and discovered that it caused liver damage. He also discovers that his friend and associate, Dr. Charles Nichols, who is leading the drug's development, arranged a cover-up, and ordered Sykes to kill Kimble, with Kimble’s wife's death being incidental. This is exciting stuff worthy of a movie. Although the plot was plausible, most of us in medical research thought only barely plausible. Little did we realize that the extent some pharmaceutical companies would go to protect their patented products. After all, we are talking about big bucks here.

 

Peter Wilmshurst

Amrinone

In 1986, seven years before the release of “The Fugitive”, Dr. Peter Wilmshurst, a cardiologist in the United Kingdom, went to the Guardian Newspaper with a fantastic story regarding the development of a new drug.  Wilmshurst’s research was on heart failure. In the 1980’s there were few treatments to improve symptoms of heart failure patients and none to improve survival. Wilmshurst was offered the opportunity to do research on a promising new drug, amrinone, patented by Sterling-Winthrop. Research showed that the drug increased the strength of heart contraction in animals. To assess its effects in man, a group from Peter Bent Brigham Hospital and Harvard Medical School published in the New England Journal of Medicine the results on the hemodynamic responses to amrinone in 8 patients with heart failure immediately after amrinone infusion (1). Amrinone increased cardiac index and left ventricular end-diastolic pressure fell. No toxicity was observed. To those physicians who care for heart failure patients, this new therapy is exciting stuff. The New England Journal is probably the most influential medical journal in the world. The article came from the cardiology department at Harvard and one of the authors was Eugene Braunwald, the most well known cardiologist in the world at the time and head of medicine at Harvard. However, Wilmshurst research disagreed on the effects of amrinone with Braunwald’s findings. In a large series of experiments Wilmshurst group showed that amrinone did not affect contractility in patients with heart failure. Furthermore, life threatening side effects were frequent (2,3).

Wilmshurst reported his findings to Sterling-Winthrop. According to Wilmshurst, Sterling-Winthrop employees asked him to exclude the patients with a downward trend in contractility. When Wilmshurst refused he was threatened with litigation (4). Wilmshurst proceeded with publication of his results in abstracts and meeting presentations but his on-going research studies on amrinone ended when Sterling-Winthrop employees removed the drug stocks from the pharmacy (4).

A number of incidents occurred apparently to prevent presentation or discredit Wilmshurst’s research. One strange incident involved one of Wilmshurst’s colleagues, Alex Crowther, who was due to present Wilmshurst’s and his work on amrinone on the second day of a meeting in Luxembourg (4). When Crowther arrived he discovered that his talk had been rescheduled for the previous day. The organizers had received a forged letter that appeared to be from him asking for his talk to be brought forward a day.

When Wilmshurst presented his findings on side effects at another meeting, a Sterling-Winthrop employee stood up and said that he had made up the findings (4).  At other meetings, three professors of cardiology, each of who was a paid consultant to Sterling-Winthrop, made public statements that they had tried to replicate Wilmshurst’s findings and failed (4). At one point, Wilmshurst was asked to meet with representatives from Sterling-Winthrop and a different professor of cardiology. They assured Wilmshurst he was mistaken about the drug. The cardiologist said that he was aware of findings by other investigators and that these entirely refuted Wilmshurst’s data. He advised that Wilmshurst should not publish any more of his findings. He said that we would be found to be wrong and his reputation would be adversely affected.

The Netherlands Committee for the Evaluation of Medicines reviewed Wilmshurst’s paper on the side effects of amrinone (6). There were major discrepancies when compared with the clinical record cards submitted to the evaluation committee by Sterling-Winthrop compared to the findings published by Wilmshurst.  Wilmshurst showed that Sterling-Winthrop had sent the Netherlands Committee falsified clinical records on his patients with the information on adverse events deleted.

Because of these discrepancies, Wilmshurst contacted the UK Committee on Safety of Medicines (CSM) and discovered that Sterling-Winthrop had also failed to notify CSM of the side effects observed in his patients (4). Furthermore, Sterling-Winthrop had not obtained a Clinical Trials Certificate for oral amrinone, though they had for the amrinone injection (4). When Wilmshurst raised this with the Sterling-Winthrop, he was told that Sterling-Winthrop executives had told the UK Government that if the company was prosecuted it would close its large manufacturing plant in the UK. Wilmshurst also contacted the Association of the British Pharmaceutical Industry, the Faculty of Pharmaceutical Medicine of the Royal College of Physicians and the General Medical Council, none of whom were interested (4). Wilmshurst spoke to editors of medical journals, including BMJ, Lancet and Nature. Although none disputed the facts, all were afraid to take on a multinational pharmaceutical company with comparatively unlimited financial and legal resources.

While this was going on, Sterling-Winthrop told a hearing of the USA’s Food and Drugs Administration that there had been over 1400 serious adverse events in 1200 patients given amrinone in trials and they would cease trials and applications for product licenses worldwide. Yet Sterling-Winthrop continued to market the drug in several third world countries (4). It was at this time Wilmshurst contacted James Erlichman, a Guardian reporter. Erlichman’s paper covered the story on the front, back and the whole of an inside page of one issue and in follow-up stories in other issues (4). Sterling-Winthrop withdrew the oral version of the drug world wide in 1986 (4). This is exciting stuff and could have been the plot for a movie. For his efforts Wilmshurst was awarded the HealthWatch Award in 2003 (7).

MIST Trial

Wilmshurst’s experience may have served him well in knowing the lengths commercial companies will go to protect their patented products. In 2000 he published an interesting observation suggesting a relationship between the repair of patent foramen ovale (PFO) and the cure or reduction of migraines (8). PFO is a flap-like opening in the atrial septum of the heart, is the most common cause of a right to left shunt, and associated with an increased risk of migraine. As many as one person in four has a PFO (9). The research interested NMT Medical, a small, start-up medical device company based in Boston which manufactured a percutaneously inserted PFO closure device called STARFlex. NMT was planning a trial to see if their device would relief migraine in those with a PFO. The trial, eventually named the Migraine Intervention with STARFlex Technology (MIST) and Wilmshurst seemed the perfect partner for NMT. He had published in the area and was known as an ethical physician who had won the HealthWatch award (10). NMT estimated that a target population of 3.8 million migraine sufferers and the financial opportunity to be in excess of $15 billion (11). NMT was riding high with a stock price approaching $25/share. We are talking big bucks here and this must have been exciting stuff to NMT executives.

The trial ran from during 2005. However, who should conduct the final, post-procedure transthoracic echocardiograms, the implanting cardiologists or echocardiography specialists, was unclear. The echocardiograms would ensure that the STARFlex device had closed the shunts. Wilmshurst, who was not one of the trial’s implanting cardiologists, says he had argued from the outset for a central echocardiography core laboratory, however, NMT proceeded with having the implanting cardiologists read the echocardiograms. A compromise was reached. Wilmshurst would review the final echocardiograms. In contrast to the implanting cardiologists who interpreted the echos to indicate that only four of 65 patients had moderate or large residual shunts, Wilmshurst reported that more than a third had a significant residual shunt.

Wilmshurst’s results were potentially devastating to NMT with the efficacy of their device being called into question. NMT organized a second review which also identified over a third of the patients with residual shunt although most were classified as pulmonary rather than atrial shunts (10). When the paper was published in 2008, an accompanying editorial in Circulation stated “The lack of an established independent core laboratory for echocardiographic data analysis must haunt the trial investigators.” (13).

The results of the MIST trial were announced at the American College of Cardiology meeting in March, 2006. Although the study found “an approximate 37% reduction in migraine burden in those patients who received a STARFlex implant and a 17% reduction in those who received the sham procedure,” it had failed to reach a significant difference in its primary end point, eliminating migraines (12). Based on previous observational studies, including Wilmshurst’s, researchers had expected migraine to have stopped in 40% of the treatment group at six months compared with 15% in the control group (13).

Publication Dispute

Matters came to a head when Wilmshurst attended the Transcatheter Cardiovascular Therapeutics Conference in Washington, October 25-27, 2007. According to Wilmshurst, it had been agreed that he would make presentations about the trial at cardiac meetings and Andrew Dowson, the other PI on the MIST trial, would present the data at neurology meetings. However, Wilmshurst was surprised when he discovered Dowson was scheduled to present the MIST trial data at the cardiology meeting. In the course of giving a separate talk on patent foramen ovale and migraine at the same meeting, Wilmshurst mentioned the MIST trial. At the session was Shelley Wood, a writer for TheHeart.org, who realized there was disagreement about the rate of residual shunts and interviewed Wilmshurst. Her article published October 26 during the meeting quotes Wilmshurst as alleging that NMT lied about whether echocardiograms from the MIST patients have been independently reviewed, and massaged the data to portray its STARFlex device in the best possible light (15). The article also quotes an NMT spokesperson, who says Wilmshurst was "on a warpath" and was never really the co-principal investigator for MIST. Furthermore, the NMT spokesman claimed Wilmshurst had been dropped from the trial for committing protocol violations. Wilmshurst claims he committed no protocol violations. This must have been exciting stuff to those who relish a controversy in medicine.

 Libel Suit

Less than two weeks after returning from Washington, Wilmshurst received a letter from UK solicitors acting for NMT, accusing him of having made “seriously defamatory allegations” in the interview he had given to TheHeart.org (10). According to the solicitors, NMT “has a reputation in this jurisdiction which it cannot permit to be tarnished by [Wilmhurst’s] serious and unjustified libels upon it.” NMT sued Dr. Wilmshurst for libel in the UK during December 2007. Subsequently, three more suits were filed over the next few years.

NMT did not sue TheHeart.org or Shelley Wood who reported Wilmhurst’s comments. This may be because the suit would need to be filed in the US and American courts are highly protective of free speech rights under the US Constitution. However, the UK libel laws differ in several aspects from the US and have several provisions that favor the plaintiff (16):

  1. The onus of proof is on the defendant.
  2. Legal fees are usually considerably greater than the damages that might be awarded. In other words, even if the defendant wins he may have a considerable financial loss. 
  3. Lawyers fees are often not contingent on winning in the UK. 
  4. Trials are long often placing considerable financial hardship on the defendant.
  5.  There is no compensation for personal cost, such as time wasted and earnings lost by spending time dealing with the case.
  6. “Libel tourism” is permitted, so that wealthy foreign individuals or corporations are allowed to sue in the UK but are themselves immune to actions brought against them in the UK.

Wilmshurst’s libel case created some attention in the UK. Wilmshurst was asked to appear on BBC 4’s morning radio program (for which he was sued again by NMT) and even called to the office of the UK Minister of Justice to discuss his case. Several medical journals picked up on the case, most notably the British Medical Journal which published a 4 page review in January, 2010 (10). The perception was that an American corporation was using the UK courts to suppress a UK physician from telling the truth resulted in a call for libel reform in the UK (17). This is exciting stuff.

Circulation Correction

Over a year after publication of the MIST trial in August, 2009 Circulation published a 700 word correction (18). The correction notes that Dr. Peter Wilmshurst did not sign the Copyright Transfer Agreement because of an internal disagreement about the conduct of study. According to Wilmshurst the correction contains information regarding device embolization which had not been disclosed in the original paper, but there is no mention of several of Wilmshurst’s other concerns, including the alleged conflicts of interest of some authors of the paper and the two echocardiogram reviews (16).

CLOSURE I Trial

In January 2008, NMT announced it was closing its follow-up MIST trial in the US. Although the company denied that failure of the first trial was the reason for cancellation, they did note that the trial had become “an expensive endeavor with little likelihood of being completed in a reasonable timeframe.” (10). Costs of the trial were estimated at $14 million and instead NMT decided to focus its resources on a trial designed to evaluate the effectiveness of PFO closure by STARFlex as a treatment for stroke compared with existing medical therapy, the CLOSURE I trial.  However, like the MIST trial the results of the CLOSURE trial were disappointing (19). The rates of stroke or TIA were no different between those who had the STARFlex device inserted and anticoagulant therapy.  

Financial Fallout

The results of the CLOSURE I were obviously distressing financial news for NMT. NMT’s stock which had been as high as $25/share had decreased to $0.25/share after announcement of the CLOSURE I trial.

Apparently legal trials move as slowly in the UK as in the US. In the three years that had elapsed since filing of the initial the suit and the announcement of the CLOSURE I results, no legal action was taken, although legal expenses continued to accumulate (16). Wilmshurst had accrued over $150,000 in out of pocket expenses and was in danger of losing his house. The anticipated trial costs were estimated at over $5 million for each side and would have required Wilmshurst to be in court for about 6 months. Obviously this would be both financially and professionally ruinous to Wilmshurst. Predictably, NMT filed for bankruptcy in April, 2011 but not before filing the fourth libel action against Wilmshurst in March. The bankruptcy filing ended the legal actions against Wilmshurst. 

Wilmshurst’s Account

After the bankruptcy filing, Wilmshurst published a summary of his experience in Radical Statistics (16). In this article Wilmshurst points out that while the data supporting the STARFlex device was unfavorable, NMT’s website and annual report did not reflect the data. On a rotating banner on the website and in the annual report were the names and photographs of the three patients who had a STARFlex implant and who were free of migraine. Wilmshurst points out that an important question is how NMT came to make contact with the three patients.

Wilmshurst said “My experience suggests that corporations can use the English defamation laws to misrepresent the results of clinical research. A corporation can propagate a misleading version and can use the defamation laws to bully those who object into remaining silent.” (16). He goes on to say “The law courts are not the best way to determine scientific truth. Few judges and even fewer juries have the training to weigh scientific evidence. An adversarial system is not the appropriate way, particularly when it pits an ordinary individual with limited financial resources against expensive barristers employed by corporations with more money. Truth should not be decided by those with greatest wealth using bullying and threats to make a scientist retract what he or she knows is true.”

We should remember Peter Wilmshurst for his courage in standing up for the truth, not once but twice. However, given our system and the comparatively large financial resources of corporations, physicians do so at their own professional and financial peril. Some such as the UK not for profit group, Libel Reform, seek to change the UK libel laws to prevent incidents such as Wilmshurst’s.  Many physicians both in the UK and US likely agree that the exciting stuff experienced by Wilmshurst do not need to happen when a physician stands up for the truth. However, this is doubtful in the absence of real legal reform. After all, we are talking about big bucks here.  

References

  1. Benotti JR, Grossman W, Braunwald E, Davolos DD, Alousi AA. Hemodynamic assessment of amrinone. N Engl J Med 1978;299:1373-7.
  2. Wilmshurst PT, Walker JM, Fry CH, et al. Inotropic and vasodilator effects of amrinone on isolated human tissue. Cardiovasc Res 1984,18:302-9.
  3. Wilmshurst PT, Thompson DS, Juul SM, Dittrich HC, Dawson JR, Walker JM, Jenkins BS, Coltart DJ, Webb-Peploe MM. Effects of intracoronary and intravenous amrinone infusion in patients with cardiac failure and patients with near normal cardiac function. Br Heart J 1985;53:493-506.
  4. Erlichman J. Drug firm "made threats". Company tested heart drug with DHSS clearance. The Guardian 3rd November 1986; 1 and 6.
  5. B Massie, M Bourassa, R DiBianco, M Hess, M Konstam, M Likoff, M Packer. Long-term oral administration of amrinone for congestive heart failure: lack of efficacy in a multicenter controlled trial. Circulation 1985,71:963-971.
  6. Wilmshurst PT, Webb-Peploe MM. Side-effects of amrinone therapy. Br Heart J 1983;49:447-51.
  7. http://www.healthwatch-uk.org/awardwinners/peterwilmshurst.html
  8. Wilmshurst P, Nightingale S, Walsh K, Morrison W. Effect on migraine of closure of cardiac right-to-left shunts to prevent recurrence of decompression illness or stroke or for haemodynamic reasons. Lancet 2000;356:1648-51.
  9. Hagen PT, Scholz DG, Edwards WD. Incidence and size of patent foramen ovale during the first 10 decades of life: an autopsy study of 965 normal hearts. Mayo Clin Proc 1984;59:17-20.
  10. Gornall J. A very public break-up. BMJ 2010;34:180-3.
  11. PR Newswire. NMT Medical initiates patient enrollment in migraine headache study, expects earlier completion. Press release, January 7, 2005. http://www.advfn.com/news_NMT-Medical-Initiates-Patient-Enrollment-in-Migraine-Headache-Study-Expects-Ear_9896513.html
  12. Dowson A, Mullen MJ, Peatfield R, Muir K, Khan AA, Wells C, et al. Migraine intervention with STARFlex Technology (MIST) trial. Circulation 2008;117:1397-404.
  13. Carroll J. A controversial trial of migraine and patent foramen ovale closure. Circulation 2008;117:1358-60.
  14. Nugent AW, Britt A, Gauvreau K, Piercey GE, Lock JE, Jenkins KJ. Device closure rates of simple atrial septal defects optimized by the STARFlex device. J Am Coll Cardiol 2006;48:538-44.
  15. Wood S. Co-PI of MIST trial alleges data mismanagement, misinformation. TheHeart.org. October 26, 2007. http://www.theheart.org/article/821779.do
  16. Wilmshurst P. The effects of the libel laws on science-a personal experience. Rad Stats 2011;104:1-10. 
  17.  http://www.libelreform.org/
  18. Correction for Dowson et al. Circulation 2009; DOI: DOI: 10.1161/CIRCULATIONAHA.109.192626. Available at: http://circ.ahajournals.org
  19. Wood S. CLOSURE I: No overall benefit, no reduction in stroke or TIA with PFO closure. TheHeart.org. November 15, 2010. Available at: http://www.theheart.org/article/1149945.do
  20. Sample I. Setback for US company suing cardiologist Peter Wilmshurst for libel. The Guardian. December 1, 2010. Available at: http://www.guardian.co.uk/science/2010/dec/01/company-suing-peter-wilmshurst-libel
  21. http://www.libelreform.org/news/492-a-british-cardiologist-sued-by-an-american-medical-company-for-a-canadian-article

Acknowledgements

The author is indebted to Michael Garrett MD who suggested Dr. Wilmshurst as a subject for Profiles in Medical Courage and Peter Wilmshurst who reviewed the manuscript.

Referece as: Robbins RA. Profiles in medical courage: Peter Wilmshurst, the physician fugitive. Southwest J Pulm Crit Care 2012;4:134-41. (Click here for a PDF version of the posting)

 

Friday
Apr062012

Correlation between Patient Outcomes and Clinical Costs in the VA Healthcare System

Richard A. Robbins, M.D.1

Richard Gerkin, M.D.2

Clement U. Singarajah, M.D.1

1Phoenix Pulmonary and Critical Care Medicine Research and Education Foundation and 2Banner Good Samaritan Medical Center, Phoenix, AZ

 

Abstract

Introduction: Increased nursing staffing levels have previously been associated with improved patient outcomes.  However, the effects of physician staffing and other clinical care costs on clinical outcomes are unknown.

Methods: Databases from the Department of Veterans Affairs were searched for clinical outcome data including 30-day standardized mortality rate (SMR), observed minus expected length of stay (OMELOS) and readmission rate. These were correlated with costs including total, drug, lab, radiology, physician (MD), and registered nurse (RN), other clinical personnel costs and non-direct care costs.

Results: Relevant data were obtained from 105 medical centers. Higher total costs correlated with lower intensive care unit (ICU) SMR (r=-0.2779, p<0.05) but not acute care (hospital) SMR. Higher costs for lab, radiology, MD and other direct care staff costs and total direct care costs correlated with lower ICU and acute care SMR (p<0.05, all comparisons). Higher RN costs correlated only with ICU SMR. None of the clinical care costs correlated with ICU or acute care OMELOS with the exception of higher MD costs correlating with longer OMELOS. Higher clinical costs correlated with higher readmission rates (p<0.05, all comparisons). Nonclinical care costs (total costs minus direct clinical care costs) did not correlate with any outcome.

Conclusions: Monies spent on clinical care generally improve SMR. Monies spent on nonclinical care generally do not correlate with outcomes.

Introduction

Previous studies have demonstrated that decreased nurse staffing adversely affects patient outcomes including mortality in some studies (1-5). However, these studies have been criticized because studies are typically cross-sectional in design and do not account for differences in patients’ requirements for nursing care. Other observers have asked whether differences in mortality are linked not to nursing but to unmeasured variables correlated with nurse staffing (6-9). In this context, we correlate mortality with costs associated with other clinical expenditures including drug, lab, radiology, physician (MD), and other clinical personnel costs.

The observed minus the expected length of stay (OMELOS) and readmission rates are two outcome measures that are thought to measure quality of care. It is often assumed that increased OMELOS or readmission rates are associated with increased expenditures (10,11). However, data demonstrating this association are scant. Therefore, we also examined clinical care costs with OMELOS and readmission rates.

Methods

The study was approved by the Western IRB.  

Hospital level of care. For descriptive purposes, hospitals were grouped into levels of care. These are classified into 4 levels: highly complex (level 1); complex (level 2); moderate (level 3), and basic (level 4). In general, level 1 facilities and some level 2 facilities represent large urban, academic teaching medical centers.

Clinical outcomes. SMR and OMELOS were obtained from the Inpatient Evaluation Center (IPEC) for fiscal year 2009 (12). Because this is a restricted website, the data for publication were obtained by a Freedom of Information Act (FOIA) request. SMR was calculated as the observed number of patients admitted to an acute care ward or ICU who died within 30 days divided by the number of predicted deaths for the acute care ward or ICU. Admissions to a VA nursing home, rehabilitation or psychiatry ward were excluded. Observed minus expected length of stay (OMELOS) was determined by subtracting the observed length of stay minus the predicted length of stay for the acute care ward or ICU from the risk adjusted length of stay model (12). Readmission rate was expressed as a percentage of patients readmitted within 30 days.

Financial data. Financial data were obtained from the VSSC menu formerly known as the klf menu.  Because this is also a restricted website, the data for publication were also obtained by a Freedom of Information Act (FOIA) request. In each case, data were expressed as costs per unique in order to compare expenditures between groups. MD and RN costs reported on the VSSC menu were not expressed per unique but only per full time equivalent employee (FTE). To convert to MD or RN cost per unique, the costs per FTE were converted to MD or RN cost per unique as below (MD illustrated):

Similarly, all other direct care personnel costs/unique was calculated as below:

Direct care costs were calculated as the sum of drug, lab, x-ray, MD, RN, and other direct care personnel costs. Non-direct care costs were calculated as total costs minus direct care costs.

Correlation of Outcomes with Costs. Pearson correlation coefficient was used to determine the relationship between outcomes and costs. Significance was defined as p<0.05.

Results

Costs: The average cost per unique was $6058. Direct care costs accounted for 53% of the costs while non-direct costs accounted for 47% of the costs (Table 1 and Appendix 1).

Table 1. Average and percent of total costs/unique.

Hospital level. Data were available from 105 VA medical centers with acute care wards and 98 with ICUs. Consistent with previous data showing improved outcomes with larger medical centers, hospitals with higher levels of care (i.e. hospitals with lower level numbers) had decreased ICU SMR (Table 2). Higher levels of care also correlated with decreased ICU OMELOS and readmission rates (Table 2). For full data and other correlations see Appendix 1.

Table 2. Hospital level of care compared to outcomes. Lower hospital level numbers represent hospitals with higher levels of care.

 

*p<0.05

SMR. Increased total costs correlated with decreased intensive care unit (ICU) SMR (Table 3, r=-0.2779, p<0.05) but not acute care (hospital) SMR. Increased costs for lab, radiology, MD and other direct care staff costs and total direct care costs also correlated with decreased SMR for both ICU and acute care SMR (p<0.05, all comparisons). However, drug costs did not correlate with either acute care or ICU SMR. Increased RN costs correlated with improved ICU SMR but not acute care SMR. For full data and other correlations see Appendix 1.

Table 3. Correlation of SMR and costs.

*p<0.05

OMELOS. There was no correlation between SMR and OMELOS for either acute care (r= -0.0670) or ICU (r= -0.1553). There was no correlation between acute care or ICU OMELOS and clinical expenditures other than higher MD costs positively correlated with increased OMELOS (Table 4, p<0.05, both comparisons).

Table 4. Correlation of OMELOS and costs

*p<0.05

Readmission rate. There was no correlation between readmission rates and acute care SMR (r= -0.0074) or ICU SMR (r= 0.0463).Total and all clinical care costs directly correlated with readmission rates while non-direct clinical care costs did not (Table 5).

Table 5.Correlation of readmission rates and costs.

*p<0.05

Discussion

The data in this manuscript demonstrate that most clinical costs are correlated with a decreased or improved SMR Only MD costs correlate with OMELOS but all clinical costs directly correlate with increased readmission rates. However, non-direct care costs do not correlate with any clinical outcome.

A number of studies have examined nurse staffing.  Increased nurse staffing levels are associated with improved outcomes, including mortality in some studies (1-5). The data in the present manuscript confirm those observations in the ICU but not for acute care (hospital). However, these data also demonstrate that higher lab, X-ray and MD costs also correlate with improved SMR. Interestingly, the strongest correlation with both acute care and ICU mortality was MD costs. We speculate that these observations are potentially explained that with rare exception, nearly all physicians see patients in the VA system. The same is not true for nurses. A number of nurses are employed in non-patient care roles such as administration, billing, quality assurance, etc. It is unclear to what extent nurses without patient care responsibilities were included in the RN costs.

These data support that readmission rates are associated with higher costs but do not support that increased OMELOS is associated with higher costs implying that efforts to decrease OMELOS may be largely wasted since they do not correlate with costs or mortality. It is unclear whether the increased costs with readmissions are because readmissions lead to higher costs or the higher clinical care costs cause the higher readmissions, although the former seem more likely.

These data are derived from the VA, the Nation’s largest healthcare system. The VA system has unique features and actual amounts spent on direct and non-direct clinical care may differ from other healthcare systems. There may be aspects of administrative costs that are unique to the VA system, although it is very likely there is applicability of these findings to other healthcare systems. 

A major weakness of these data is that it is self reported. Data reported to central reporting agencies may be confusing with overlapping cost centers. Furthermore, personnel or other costs might be assigned to inappropriate cost centers in order to meet certain administrative goals. For example, 5 nurses and 1 PhD scientist were assigned to the pulmonary clinic at the Phoenix VA Medical Center while none performed any services in that clinic (Robbins RA, unpublished observations). These types of errors could lead to inaccurate or inappropriate conclusions after data analysis.

A second weakness is that the observational data reported in this manuscript are analyzed by correlation.  Correlation of decreased clinical care spending with increased mortality does not necessarily imply causation (13). For example, clinical costs are increased with readmission rates. However, readmission rates may also be higher with sicker patients who require readmission more frequently. The increased costs could simply represent the higher costs of caring for sicker patients.

A third weakness is that non-direct care costs are poorly defined by these databases. These costs likely include such essential services as support service personnel, building maintenance, food preparation, utilities, etc. but also include administrative costs. Which of these services account for variation in non-direct clinical costs is unknown. However, administrative efficiency is known to be poor and declining in the US, with increasing numbers of administrators leading to increasing administrative costs (14).

A number of strategies to control medical expenditures have been initiated, although these have almost invariably been directed at clinical costs. Programs designed to limit clinical expenditures such as utilization reviews of lab or X-ray expenditures or decreasing clinical MD or RN personnel have become frequent.  Even if costs are reduced, the present data imply that these programs may adversely affect patient mortality, suggesting that caution in limiting clinical expenses are needed. In addition, programs have been initiated to reduce both OMELOS and readmission rates. Since neither costs nor mortality correlate with OMELOS, these data imply that programs focusing on reducing OMELOS are unlikely to be successful in improving mortality or in reducing costs.

Non-direct patient care costs accounted for nearly half of the total healthcare costs in this study. It is unknown which cost centers account for variability in non-clinical areas. Since non-direct care costs do not correlate with outcomes, focus on administrative efficiency could be a reasonable performance measure to reduce costs. Such a performance measure has been developed by the Inpatient and Evaluation Center at the VA (15). This or similar measures should be available to policymakers to provide better care at lower costs and to incentivize administrators to adopt practices that lead to increased efficiency.

References

  1. Needleman J, Buerhaus P, Mattke S, Stewart M, Zelevinsky K. Nurse-staffing levels and the quality of care in hospitals. N Engl J Med 2002;346:1715-22.
  2. Aiken LH, Clarke SP, Sloane DM, Sochalski J, Silber JH. Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. JAMA 2002;288:1987-93.
  3. Aiken LH, Cimiotti JP, Sloane DM, Smith HL, Flynn L, Neff DF. Effects of nurse staffing and nurse education on patient deaths in hospitals with different nurse work environments. Med Care 2011;49:1047-53.
  4. Diya L, Van den Heede K, Sermeus W, Lesaffre E. The relationship between in-hospital mortality, readmission into the intensive care nursing unit and/or operating theatre and nurse staffing levels. J Adv Nurs 2011 Aug 25. doi: 10.1111/j.1365-2648.2011.05812.x. [Epub ahead of print]
  5. Cho SH, Hwang JH, Kim J. Nurse staffing and patient mortality in intensive care units. Nurs Res 2008;57:322-30.
  6. Volpp KG, Rosen AK, Rosenbaum PR, Romano PS, Even-Shoshan O, Canamucio A, Bellini L, Behringer T, Silber JH. Mortality among patients in VA hospitals in the first 2 years following ACGME resident duty hour reform. JAMA 2007;298:984-92.
  7. Lagu T, Rothberg MB, Nathanson BH, Pekow PS, Steingrub JS, Lindenauer PK. The relationship between hospital spending and mortality in patients with sepsis. Arch Intern Med 2011;171:292-9.
  8. Cleverley WO, Cleverley JO. Is there a cost associated with higher quality? Healthc Financ Manage 2011;65:96-102.
  9. Chen LM, Jha AK, Guterman S, Ridgway AB, Orav EJ, Epstein AM. Hospital cost of care, quality of care, and readmission rates: penny wise and pound foolish? Arch Intern Med 2010;170:340-6.
  10. Render ML, Almenoff P. The veterans health affairs experience in measuring and reporting inpatient mortality. In Mortality Measurement. February 2009. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/qual/mortality/VAMort.htm
  11. Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med;360:1418-28.
  12. Render ML, Kim HM, Deddens J, Sivaganesin S, Welsh DE, Bickel K, Freyberg R, Timmons S, Johnston J, Connors AF Jr, Wagner D, Hofer TP. Variation in outcomes in Veterans Affairs intensive care units with a computerized severity measure. Crit Care Med 2005;33:930-9.
  13. Aldrich J. Correlations genuine and spurious in Pearson and Yule. Statistical Science 1995;10:364-76.
  14. Woolhandler S, Campbell T, Himmelstein DU. Health care administration in the United States and Canada: micromanagement, macro costs. Int J Health Serv. 2004;34:65-78.
  15. Gao J, Moran E, Almenoff PL, Render ML, Campbell J, Jha AK. Variations in efficiency and the relationship to quality of care in the Veterans health system. Health Aff (Millwood) 2011;30:655-63.

Click here for Appendix 1.

Reference as: Robbins RA, Gerkin R, Singarajah CU. Correlation between patient outcomes and clinical costs in the va healthcare system. Southwest J Pulm Crit Care 2012;4:94-100. (Click here for a PDF version)

Friday
Mar302012

Profiles in Medical Courage: Of Mice, Maggots and Steve Klotz

“I never did give them hell. I just told the truth, and they thought it was hell.”-Harry S. Truman 

Mice and maggots bring to mind visions of filth and decay with an accompanying sense of sickening revulsion-hardly the impression you want associated with a hospital. However, an infestation of mice and maggots did occur in a hospital- not in medieval Europe as you might expect, but in 1998 at the Kansas City Veterans Administration (VA) Hospital. Although the mice and maggots are the attention grabbers, the story is worth repeating because it illustrates how dysfunctional modern hospitals can become and how Steve Klotz tried to affect change by speaking against the management that allowed the situation to occur.

To understand the story, we need to go back to 1995, ancient history to our fellows, residents and medical students but not so ancient to lots of us. Many Veterans Affairs (VA) hospitals were called Dean’s hospitals and had a special relationship with a local medical school (1). Each of these hospitals had a Dean’s committee, made up of officials from the VA and the local medical school. This committee approved physician hires and had a voice in most major decisions affecting the medical school faculty at the VA. The rationale for such an arrangement was the VA would not be able to hire high-quality faculty unless associated with a medical school where the faculty had an appointment. Overall this arrangement had served the VA well since shortly after World War II. The VA did get first rate faculty resulting in a level of care that could not be provided to Veterans by less qualified practioners.

However, not everyone was happy with the arrangement, particularly the hospital administrators. At that time, the administrators were in charge of the Medical Administration Service (MAS). This service supervised the business functions of the hospital (fiscal, human resources, purchasing, etc.) and several of the non-medical services (food preparation, janitorial services, security, etc.). The medical functions were headed by the chief of staff. The hospital director and the chief of staff had a shared and equal partnership between the hospital director and the chief of staff.

The administrators argued that the arrangement was disadvantageous to the VA in several ways. First, it gave the medical school, and therefore, the physicians too much voice in hospital operations. Second, physicians were often hired to fill medical school needs rather than VA needs. Third, the physician hires were often subspecialist and there was a move at the VA to emphasize primary care. Fourth, split responsibilities sometimes resulted in conflicts that were not easily resolved. Dr. Ken Kizer, then Under Secretary for Veterans Health Affairs in charge of all VA hospitals, was persuaded to dissolve the partnership and make the hospital director the ultimate authority at the hospitals in his Prescription for Change (2).

Against this background, Dr. Steve Klotz, an infectious disease specialist was consulted on two patients in the Kansas City VA ICU with nasal myasis (3). The first case occurred in July, 1998. The myasis was thought to be due to flies having direct access to the hospital through open windows during construction. However, after the second case in September, 1998, Klotz called his brother, John Klotz, an entomologist at the University of California, Riverside. He advised sending some of the flies and maggots to Nancy Hinkle, an entomologist with expertise on flies. She identified the flies and maggots as the green blowfly and explained to Klotz that these flies prefer to lay their eggs in mice carcasses. She suggested that the presence of green blowfly maggots suggested that the hospital had a mouse infestation.

In agreement with Dr. Hinkle’s speculation, the Kansas City VA was known to have a mouse problem preceding and coincident with the two cases of myasis (3). In response to this problem, warfarin-based mouse bait had been scattered throughout the hospital by a pest control contractor. However, this approach was largely unsuccessful. Numerous mice were observed during the daylight hours on all hospital floors. In some patient wards mice were being cared for as pets by the nursing personnel. Mice were so common in the building that they scampered over the feet of the associate hospital director during administrative morning report in the hospital director’s suite.

On learning of the egg-laying preferences of the blowfly, the warfarin baits and traps were replaced with live traps. The results of mouse captures showed the mice to be centered on the fourth floor of the hospital where the canteen was located (3). During an infection control inspection of the canteen and hospital canteen, inspectors discovered mouse carcasses in food storage rooms adjacent to the canteen on glue boards, mouse nests behind boxes on food shelves in the canteen, live mice trapped in a large wastebasket, and mouse droppings covering the floor of the canteen work room.

However, the above did not explain why there was a mouse problem.  All VA hospitals have canteens, many are in older buildings, and most do not have a mouse problem. It was apparent from the results of the live trappings and the canteen inspection that the mice seemed centered around the canteen (3). However, the real clue to the cause came when Klotz asked the janitors (S. Klotz, personal communication). They pointed out that a computer program had been purchased to indicate when rooms would be cleaned. The head of housekeeping was removed during downsizing along with the night time janitor who cleaned the canteen. However, the canteen and its storage rooms were not on the cleaning schedule. The janitors pointed out that these rooms had not been cleaned by housekeeping personnel for at least a year and every canteen employee was aware of the magnitude of the mouse problem (3).

Given that this was an interesting chain of epidemiological events, Klotz published the results of his investigations on March 25, 2002 (3). Action by VA Central Office was swift. Anthony Principi, Secretary of Veterans Affairs removed the regional network director, Patricia Crosetti, and ordered a full investigation. Principi would likely have removed the acting hospital director, Kent Hill, except that he was just hired. Hill had replaced the previous hospital director, Hugh Doran, who resigned after being filmed soliciting prostitution on “John TV”.  

Even prior to publication of the article, Klotz had phone calls from Hill and was told “not to talk to anyone” (4). The VISN headquarters in Kansas City closed a research lab with six employees that Klotz had formerly supervised. The employees were keyed out of the lab. Crosetti, the VISN director who had been removed, was heard to say ‘‘Klotz ain’t gonna work for the VA anymore” (4).

The publicity caused Congress to be involved and a field hearing was held at the Kansas City VA in June, 2002 (4). Klotz appeared as a witness and went first. Although he discussed the mice and maggot problem, he focused on five major root causes which he thought led to the incident:

  1. “The addition of an entire cadre of middle managers who embrace a business model of management. These managers have fiscal oversight in the clinical side of the organization and are neither sufficiently knowledgeable nor trained in the areas they supervise.
  2. The hospital director has more real power than the chief of staff. There is no equal partnership.
  3. A sundering of any meaningful relationship with local medical schools.
  4. Individuals in the organization with direct patient care, for example, physicians and nurses, have no meaningful influence in the organization of patient care.
  5. Supervisory positions are all too frequently held until retirement.”

To support his claims he showed that the number of doctors and nurses caring for the patients decreased in the VA while the number of support personnel increased (Table 1).

 

Yet the number of patients and expenditures had increased (Table 2).

When all of this was occurring, it appeared as if the possession of real credentials for a job position was grounds for immediate disqualification at the Kansas City VA according to Klotz (4). For example, “an engineer was given authority over pharmacy and housekeeping, disciplines for which he was untrained and had only superficial knowledge. Internists were placed in direct charge of subspecialty surgeons whose specific requirements often went unmet. A nonphysician was placed in charge of pathology and radiology”. The chief of pulmonary was asked to set a broken arm. When he refused, he was asked “You’re a doctor, aren’t you?” The position of Chief of Staff was eliminated because it was “obsolete”.  

Testimony was given by a representative from the office of inspector general (IG). He stated that most of the environmental problems identified during the April, 2002 inspection ordered by Principi fell into one of several categories: 1. An overall lack of cleanliness; 2 Failure to maintain equipment, furniture, utilities, hospital services; and 3. Inadequate pest control (4). Not mentioned is that 10 months prior to this report the IG had visited the Kansas City VA on a routine visit (5). Although rodent problems were identified by the employees, the IG’s recommended action was to remove the rodent traps from patient areas and the canteen.

Officials from the VA management testified next. Although there were multiple administrators that testified, particularly revealing were the testimonies of Doran, the former director of the Kansas City VA, and Robert Roswell, then Under Secretary of Veterans Health Affairs.

Doran went first. He stated that, “The unfortunate incident involving the maggots was handled expeditiously and appropriately by our staff.” This was the last time he mentioned any of his staff in a positive light. He went on to say that his first priority had been patient care and that he had initiated a number of construction projects towards this end. According to Doran, the problems arose from the Kansas City VA being an older building with an inadequate budget. Attacking Klotz he said, “There is absolutely no evidence to establish a relationship between the two nasal myasis cases and the alleged mouse problem. You have an obviously disgruntled former employee’s opinion who managed to get the article published.” Doran went on to tout his accomplishments at the Kansas City VA, particularly noting his Joint Commission on Healthcare Organization (JCAHO) scores. He noted that the JCAHO had recently inspected the hospital and found no problems. However, he failed to acknowledge the nurses, doctors and support personnel who were responsible for the success.

The ranking minority member, Dr. Bob Filner (D, CA) was unconvinced. By background, Dr. Filner is a former academic from the University of San Diego whose PhD is in the history of science. After some intense questioning, Filner chastised Doran saying, “The vocabulary used and the tone you use to defend yourself makes your testimony suspect in my eyes and it is contradictory to everything that we have heard over the years about problems here [Kansas City VA]. So I will tell you if you had to have me vote on who I was going to believe here, I would vote for the employees on the first line and I would have to say, you, sir, are the weakest link.”

Doran responded by attacking Dr. Filner. Saying that Filner’s personal attack was “unprofessional and totally uncalled for” and only done to embarrass him. He further accused Filner of “grandstanding for the cameras”.

Roswell, who had been confirmed as the Under Secretary for Veterans Healthcare Affairs only a few months earlier, went next. Like Doran, he also attacked Klotz noting that, “Despite the author’s assertions of a relationship between the rodents and the flies, there was (and is) no conclusive evidence that such a relationship existed”.  Again like Doran, Roswell went on to blame the age of the facility but did acknowledge the “lack of effective supervision and leadership in the housekeeping department…Due to the lack of knowledgeable leadership and supervision, the infrastructure within the housekeeping department eroded”.

Representative Dr. Filner was again skeptical. He asked Roswell, “What is it about a system that requires a publication of a significant problem to direct the resources where they need to go?” Roswell responded, “I think what we are dealing with is a situation where there were competing priorities, limited resources, ineffectual communication between various levels of management, and less than ideal monitoring.” Filner asked Roswell to assure him that the VA would not be retaliate against Klotz and was assured that the VA would not.

With that, the hearing and the controversy surrounding the cleanliness at the Kansas City VA ended. The Kansas City VA did receive a multi-million dollar facelift, but no changes occurred affecting the management problems that according to Klotz led to the incident.  Central office management became more concerned about employees publically speaking even through scientific publications.  A mandate was issued that all scientific manuscripts needed to be submitted to the local Research and Development Committee for review prior to publication (6).  

In the aftermath of the controversy Patricia Crosetti was proven right-Klotz does not work for the VA anymore. After receiving poor reviews on his Merit Review grant which he held for nearly 20 years, he left the VA to become a full time university professor. He is currently Chief of Infectious Disease at the University of Arizona. Ken Kizer left the VA when it became apparent his appointment would not be renewed by Congress. Roswell resigned from the VA a couple of years after these events in a controversy about a failed computer system. Patricia Crossetti, the VA regional director, was subsequently dismissed. Kent Hill became the permanent director of the Kansas City VA where he is today. Dr. Filner remains on the Veterans Affairs Committee and Hugh Doran remains retired.

Although Klotz’s 5 root causes of the mice and maggots incident have yet to result in substantial change, we should remember Klotz for his courage in speaking up and identifying the managerial problems that led to the infestation of mice and maggots.

Richard A. Robbins, M.D.*

References

  1. http://www1.va.gov/vhapublications/ViewPublication.asp?pub_ID=979 (accessed 3/18/2012).
  2. www.va.gov/HEALTHPOLICYPLANNING/rxweb.pdf (accessed 3/18/2012).
  3. Beckendorf R, Klotz SA, Hinkle N, Bartholomew W. Nasal myiasis in an intensive care unit linked to hospital-wide mouse infestation. Arch Intern Med 2002;162:638-40.
  4. http://veterans.house.gov/107th-congress-hearing-archives (accessed 3/18/2012).
  5. www.va.gov/oig/CAP/01-01515-40.pdf (accessed 3/18/2012).
  6. http://www.research.va.gov/resources/policies/pub_notice.cfm (accessed 3/26/2012).

*Dr. Robbins is a former employee of the Department of Veterans Affairs and was the Associate Chief of Staff for Research at the Southern Arizona VA when these events occurred in 2002.

Reference as: Robbins RA. Profiles in medical courage: of mice, maggots and Steve Klotz. Southwest J Pulm Crit Care 2012;4:71-7. (Click here for a PDF version of the manuscript)

Wednesday
Mar142012

Profiles in Medical Courage: Michael Wilkins and the Willowbrook School

“Never doubt that a small group of thoughtful, committed citizens can change the world. Indeed, it is the only thing that ever has. "- Margaret Mead

With this article we begin an intermittent series on physicians who displayed courage in trying to help their patients. Although there are many examples, hopefully what will be illustrated are examples of the lesser known doctors who identified problems and stood up to address them. Few remember the controversy surrounding the now closed Willowbrook School and Dr. Michael Wilkins’ involvement. However, Wilkins’ courage in advocating for change not only resulted in substantial improvement in conditions at the school but also led to the Civil Rights of Institutionalized Persons Act (CRIPA) of 1980.

Wilkins was originally from Kansas City and graduated from the University of Missouri School of Medicine in 1967 (1). He left Missouri to do his pediatric internship and complete his military obligation in the United States Public Health Service on Staten Island. There he became familiar with the Willowbrook State School, which at the time was the largest institution for the mentally retarded in the world. After completing his time in the Public Health Service, Wilkins was persuaded by his friend, Dr. Bill Bronston, to join him at Willowbrook as a full time physician.

The Willowbrook School already had a bad reputation when Wilkins arrived (2). Built in the late 1930’s, the campus was large with 38 buildings on 375 acres in the Willowbrook section of Staten Island. Designed to hold 4000 patients, it held 6000 when Wilkins arrived. As a result of the overcrowding and unsanitary conditions, virtually all children developed hepatitis, primarily hepatitis A. This led to a highly controversial medical study carried out from the mid-1950s through the 1970s by researchers Saul Krugman and Robert W. McCollum. Healthy children were intentionally inoculated, orally and by injection, with hepatitis virus, then monitored to gauge the effects of gamma globulin in combating the infection (2,3). Senator Robert Kennedy of New York, the younger brother of the slain President John Kennedy, had toured the institution in 1965 and proclaimed that individuals in the overcrowded facility were "living in filth and dirt, their clothing in rags, in rooms less comfortable and cheerful than the cages in which we put animals in a zoo" and offered a series of recommendations for improving conditions (4). However, the result was a public relations effort with the creation of a few “model” buildings (5). Visitors were escorted through these “model” buildings and the controversy subsided.

Wilkins was not assigned to one of the model buildings but to building 6. There he cared for about 70 severely mentally retarded clients along with 2 or 3 attendants. In a 2008 interview Wilkins said, “The first thing that assaults you when you walk in the building is the smell and that sets the tone for the whole experience. The smell is the smell of decay and a mixture of sweat and feces and lack of being cleansed …it permeated the building” (1). The clients spent their days in the day room. Few went to school. For the incontinent the attendants would get 6-8 clients together in the shower room and hose them to keep them clean and clean up after them between showers. The severely retarded would be on the floor in straightjackets so as not to scratch themselves or assault other patients. Some would rock on the floor, while others sat motionless. Each building had a supervisor, sometimes with a nursing degree, but in Wilkins’ building the supervisor had no nursing background. He was the person that would answer to the administration and carry out their orders.

Wilkins and Bronston began advocating for change especially in the wake of Governor Nelson Rockefeller’s budget cuts which decreased the number of employees from 3000 to 2000. Parents and friends were not allowed in the buildings so they held Sunday weenie roasts for the clients and their parents. There they would advocate for improvement in the conditions. This eventually led to a conference for the parents. The keynote speaker, an expert on mental retardation, described conditions at Willowbrook as “primitive” and “outdated” (1). After trying to organize the parents to advocate change, Wilkins was fired by the School’s administrator, Dr. Jack Hammond.

Now unemployed, and not having completed his residency, Wilkins fought back. He contacted his friend, a local WABC-TV New York newsman, Geraldo Rivera. Fortunately, when Wilkins was fired, Willowbrook had not asked for his key and he used it to let Rivera come to Willowbrook and document the conditions. The State of New York attempted to prevent the release of the resultant film citing patients’ privacy, but Rivera, an attorney by education, persuaded his producer to make the film public. The exposé, entitled “Willowbrook: The Last Disgrace”, garnered national attention and won a Peabody Award for Rivera. Rivera and Wilkins later appeared on the nationally televised Dick Cavett Show with the film.

As a result of the conditions, a class-action lawsuit was filed against the State of New York and a settlement in the case was reached mandating reforms. The publicity generated by the case was a major contributing factor to the passage of the Federal Civil Rights of Institutionalized Persons Act of 1980. This law enabled the US Department of Justice to protect the rights of those individuals who were in the care of state institutions, including jails and prisons, juvenile correctional facilities, public nursing homes, mental health facilities and institutions for individuals with intellectual disabilities. The law allows for the US Attorney General to intervene on behalf of institutionalized people whose rights may have been oppressed and ensure the safety of those individuals who may feel uncomfortable reporting issues of abuse in these government run institutions.

After leaving Willowbrook, Wilkins returned to Kansas City and completed his residency. For many years he ran a clinic in the inner city serving Kansas City’s poor. We should remember Dr. Michael Wilkins and how his act of courage led to improvement at Willowbrook and a law protecting all institutionalized individuals.

Richard A. Robbins, M.D.

Editor, Southwest Journal of Pulmonary and Critical Care

References

  1. library.albany.edu/speccoll/findaids/apap015/WILKINS.pdf CSEA interview with Dr. Michael Wilkins. Accessed February 21, 2012.
  2. http://en.wikipedia.org/wiki/Willowbrook_State_School Accessed February 21, 2012.
  3. Giles JP, McCollum RW, Berndtson LW Jr, Drugman S. Viral hepatitis — relation of Australia/sh antigen to the Willowbrook ms-2 strain. N Engl J Med 1969; 281:119-122.
  4. "Excerpts From Statement by Kennedy", The New York Times, September 10, 1965. Accessed February 21, 2012.
  5. library.albany.edu/speccoll/findaids/apap015/BRONSTON.pdf CSEA interview with Dr. William Bronston. Accessed February 21, 2012.

Reference as: Robbins RA. Profiles in medical courage: Michael Wilkins and the Willowbrook School. Southwest J Pulm Crit Care 2012;4:54-6. (Click here for a PDF version)

Wednesday
Mar142012

Relationship between the Veterans Healthcare Administration Hospital Performance Measures and Outcomes

Richard A. Robbins, M.D.1

Richard Gerkin, M.D.2

Clement U. Singarajah, M.D.1

1Phoenix Pulmonary and Critical Care Medicine Research and Education Foundation and 2Banner Good Samaritan Medical Center, Phoenix, AZ

Abstract

Health care organizations have been using performance measures to compare hospitals. However, it is unclear if compliance with these performance measures results in better healthcare outcomes. We examined compliance with acute myocardial infarction, congestive heart failure, pneumonia and surgical process of care measures with traditional outcome measures including mortality rates, morbidity rates, length of stay and readmission rates using the Veterans Healthcare Administration Quality and Safety report. Disappointingly, increased compliance with the performance measures was not correlated with better outcomes with the single exception of improved mortality with higher rates of compliance with echocardiography. We also evaluated the hospital level of care and found that higher levels of complexity of care correlated with the acute myocardial infarction performance measure, but not with the congestive heart failure, pneumonia, or surgical process of care performance measures. However, level of complexity of care strongly correlated with all cause mortality (p<0.001), surgical mortality (p=0.037) and surgical morbidity (p=0.01). These data demonstrate that compliance with the performance measures are not correlated with improved healthcare outcomes, and suggest that if measures are used to compare hospitals, different measures need to be developed.

Introduction

The Joint Commission recently released “Improving America’s Hospitals: The Joint Commission’s Annual Report on Quality and Safety 2011 (1). In this report the results of hospital compliance with the Joint Commission’s performance measures are listed. The Joint Commission announced not only is compliance improving but identified 405 hospitals as their “Top Performers on Key Quality Measures Program”. In a letter at the beginning of the report Mark Chassin, President of the Joint Commission, said “This program is designed to be an incentive for better performance on accountability measures and to support organizations in their quest to do better”.

However, there have been several criticisms of the report. First, many hospitals which were recognized as top hospitals by US News & World Report, HealthGrades Top 50 Hospitals, or Thomson Reuters Top Cardiovascular Hospitals were not included (2). Small community hospitals were overrepresented and large academic medical centers were underrepresented in the report. Chassin commented that this should be "a wake-up call to larger hospitals to put more resources into these programs…”. This is surprising since teaching hospitals, which are usually large, urban hospitals, have previously been reported to have lower risk-adjusted mortality rates and lengths of stay (3). Second, it has been pointed out that many of the performance measures are not or only weakly associated with traditional outcomes such as mortality (4-7). Therefore, we compared the compliance with the Joint Commission performance measures compared to mortality rates, morbidity rates, length of stay and readmissions using the Nation’s largest healthcare system, the Department of Veterans Affairs. The results demonstrate that compliance with performance measures are not correlated with improved outcomes.

Methods

The study was approved by the Western IRB.

Process Performance Measures. We evaluated hospital performance based on publicly available data from the 2010 VHA Facility Quality and Safety Report (9). These measures evaluate quality of care for acute myocardial infarction, congestive heart failure, pneumonia and surgical care improvement program (SCIP) during fiscal year 2009. For each of the measures, a hospital’s performance is calculated as the proportion of patients who received the indicated care out of all the patients who were eligible for the indicated care. The quality indicators are based on, and in most cases identical to those used for the Joint Commission’s Hospital Compare (acute myocardial infarction-Appendix 1; congestive heart failure-Appendix 2; pneumonia-Appendix 3, surgical quality-Appendix 4). Data were also available for each component of the congestive heart failure quality measure (see Appendix 2) which was evaluated independently.

Disease specific mortality. Hospital-specific, risk-standardized rates of mortality within 30 days of discharge are reported for patients hospitalized with a principal diagnosis of heart attack, heart failure, and pneumonia. For each condition, the risk-standardized (also known as "adjusted" or "risk-adjusted") hospital mortality rate are calculated using mathematical models that use administrative data to adjust for differences in patient characteristics that affect expected mortality rates (10).

Surgical morbidity and mortality. VA’s Surgical Quality Improvement Program (VASQIP) monitors major surgical procedures performed at VHA facilities and tracks risk adjusted surgical complications (morbidity) and mortality rates. Patient data are collected at each facility by a specially trained nurse and entered into the VA’s electronic health record: detailed preoperative patient characteristics including chart-abstracted medical conditions, functional status, recent laboratory tests, information about the surgical procedure performed, and 30-day outcomes data.

The VASQIP program analyzes these patient data using mathematical models to predict an individual patient’s expected outcome based on the patient’s preoperative characteristics and the type and nature of the surgical procedure. Overall patient outcomes for major surgical procedures are expressed by comparing observed rates of mortality and morbidity to the expected rates for those patients undergoing the procedure as observed-to-expected (O/E) ratios. For example, if, based on patient characteristics, a facility expected 5 deaths following major surgery, but only 4 patients died, the O/E ratio would be reported as 0.8.

Medical Surgical Length of Stay (LOS). These data are the VA hospital average length of stay for patients who were discharged from acute medicine or surgery bed sections. It does not include patients discharged from observation beds or discharged from other areas of the hospital such as mental health.

Readmission rates. A readmission was defined as a patient who has had a recent hospital stay and needs to re-enter the hospital again within 30 days. These rates are not adjusted for patient characteristics that affected expected admission rates, so comparisons among hospitals should be interpreted with caution.

CHF readmissions were reported separately. CHF readmission is defined by patients who had an initial hospitalization for CHF and were readmitted at least once to acute care in the hospital within 30 days following discharge for CHF.

Hospital level of care. For descriptive purposes, hospitals were grouped into levels of care. These are classified into 4 levels: highly complex (level 1); complex (level 2); moderate (level 3), and basic (level 4). In general, level 1 facilities and some level 2 facilities represent large urban, academic teaching medical centers.

Correlation with Outcomes. Pearson’s correlation coefficient was used to assess the correlation of compliance with the performance measures and outcomes. Significance was defined as p<0.05. For comparisons among hospital levels, ANOVA or Kruskall-Wallis testing was done, as appropriate.

Results

Disease specific and all cause mortality rates compared to performance measures. Hospital-specific, risk-standardized rates of mortality within 30 days of discharge for patients hospitalized with a principal diagnosis of heart attack, heart failure, and pneumonia were compared to performance measure compliance. There was no correlation (Table 1, p>0.05 all conditions) but with an increased incidence of pneumonia actually weakly correlating with higher compliance with the pneumonia performance measures (Table 1, p=0.0411). Furthermore, there was no correlation between all cause mortality and the average of the three compliance measures (Table 1, p>0.05). Because each table is large, only the correlation coefficients are presented in the text. The table data on which the correlations are based are given at the end of the manuscript. (N=the number of hospitals. NA=not available).

Table 1. Disease Specific Mortality Correlated with Performance Measure Compliance

Correlation Coefficients

r value

N

p value

Acute Myocardial Infarction Mortality and AMI Performance Measure

0.0266

103

0.7897

Congestive Heart Failure Mortality and CHF Performance Measure

0.0992

123

0.2752

Pneumonia Mortality and Pneumonia Performance Measure

0.1844

123

0.0411

All Cause Mortality vs. Average of Performance Measures

0.1118

122

0.2202

Each component of the congestive heart failure performance measure was evaluated individually. Performance of echocardiography correlated with improved mortality (Table 2, p=0.0496) but there was no correlation with use of a angiotensin converting enzyme inhibitor (ACEI) or angiotensin receptor blocker (ARB) at discharge, discharge instructions, nor smoking cessation advice (Table 2, p>0.05 all comparisons).

Table 2. Heart Failure Mortality Correlated with Compliance to Individual Heart Failure Performance Measures

Correlation Coefficients

r value

N

p

ACEI or ARB

-0.1007

112

0.2908

Smoking Cessation

0.0651

112

0.4953

Discharge Instructions

0.1411

111

0.1396

Echocardiography

-0.1860

112

0.0496

Surgical mortality and morbidity rates compared to surgical performance measures. There was no correlation between compliance with the surgical care improvement program (SCIP) and surgical mortality or morbidity (Table 3, p>0.05 both comparisons).

Table 3. Surgical Care Improvement Program (SCIP) Compliance Correlated with Observed/Expected (O/E) Morbidity/Mortality

Correlation Coefficients

r value

N

p value

O/E Mortality

0.0943

99

0.3530

O/E Morbidity

0.0031

99

0.9757

Length of Stay. None of the performance measures correlated with medical-surgical length of stay (Table 4, p>0.05 all comparisons).

Table 4. Length of Stay (LOS) Correlated with Performance Measure Compliance

Correlation Coefficients

r value

N

p value

LOS compared to AMI

0.1047

103

0.2926

LOS compared to CHF

-0.0178

123

0.8451

LOS compared to Pneumonia

-0.1679

123

0.0634

LOS compared to SCIP

-0.0404

106

0.6809

LOS compared to Average

0.0028

123

0.9755

Readmission rates. There was no correlation between all cause readmission rates and the acute myocardial infarction, congestive heart failure, pneumonia or surgical performance measures (Table 5, p>0.05 all comparisons). There was no correlation between heart failure readmission rate and the heart failure performance measure (data not shown, r=0.1525, p=0.0921).

Table 5. Readmission Rate Correlated with Performance Measure Compliance

Correlation Coefficients

 

r value

N

p

AMI

0.1688

103

0.0883

CHF

0.1505

123

0.0966

Pneumonia

0.0581

123

0.5233

Average

0.1281

122

0.1597

Hospital level of care. Acute myocardial infarction performance measures inversely correlated with the hospital level of care, i.e., the higher the hospital complexity level, the better the compliance (Table 6, p=0.004). However, there was no correlation between congestive heart failure, pneumonia, surgical care improvement program or the average of the measures and the hospital level of care (Table 6).

Table 6. Hospital Level Correlated with Performance Measure Compliance

ANOVA

N

p

Acute Myocardial Infarction (AMI)

103

0.004

Congestive Heart Failure (CHF)

120

0.782

Community Acquired Pneumonia

120

0.296

Surgical Care Improvement Program (SCIP)

106

0.801

Average of Process of Care Measures

120

0.285

There was no correlation between the level of hospital care and the acute myocardial infarction, congestive heart failure, nor pneumonia mortality (Table 7, p>0.05 all comparisons). However, there was a strong correlation between all cause morality (p<0.001) and a correlation between surgical Observed/Expected mortality (Table 7, p=0.037) and surgical Observed/Expected morbidity (p=0.010).

Table 7. Hospital Level Correlated with Mortality and Surgical Morbidity

ANOVA

N

p

Acute Myocardial Infarction (AMI) Mortality

103

0.835

Congestive Heart Failure (CHF) Mortality

120

0.493

Pneumonia Mortality

120

0.547

All Cause Mortality

106

<0.001

Surgical O/E Mortality

99

0.037

Surgical O/E Morbidity

99

0.010

Discussion

These data from the Nation’s largest healthcare system demonstrate that increasing compliance of the performance measures prescribed by the Joint Commission does not affect disease specific mortality, all cause mortality, surgical mortality, surgical morbidity, length of stay or readmissions with the single exception of improved mortality correlating with increased compliance with performance of echocardiography. In contrast to the Joint Commission’s list of top hospitals which found smaller and rural hospitals to be overrepresented, we found that only the acute myocardial infarction performance measure correlated with a higher level of hospital care which represents mostly large, urban hospitals. We did find that all cause mortality and surgical morbidity highly correlated with the level of care. This would appear to differ from the Joint Commission’s list of top hospitals which tended to be small and rural, since VA hospitals with higher levels of care largely represent large urban, academic teaching medical centers.

There are multiple possible reasons for the lack of correlation between the performance measures and outcomes. Many of the outcomes are evidence based but several are not. For example, there are no randomized, multi-center studies evaluating the efficacy of discharge instructions, smoking cessation advice and pneumococcal vaccination. Studies with discharge instructions are retrospective, observational studies and have largely not shown improved outcomes (11,12). Several meta-analyses have failed to demonstrate the efficacy of pneumococcal vaccine in adults (13-15). Advice to quit smoking without follow up support or pharmacologic intervention has not been shown to lower smoking cessation rates (16). Mandating ineffective interventions such as these would not be expected to have a positive effect on outcomes. However, this is where most of the improvement in performance measure outcome has occurred (2).

Most of the interventions are grouped or bundled. Lack of compliance with any one of the bundle is taken as noncompliance with the whole. However, if the only difference between hospitals is noncompliance with an ineffective performance measure, there would not be any expected improvement in outcomes.

Many of the strongly evidence-based outcomes have very high compliance, usually exceeding 95% (9). It is possible that small improvements of 1 or 2% in effective performance measures might have too small an impact on outcomes to be detected even in large databases such as the Veterans Administration which examined 485,774 acute medical/surgical discharges in 2009.

The performance measures appear to avoid highly technical or costly interventions and often avoid interventions which have been shown positively affect outcomes. For example, beta blockers and spironolactone have been shown to be effective in heart failure but are not included in the congestive heart failure performance measures (17,18). Furthermore, carvedilol has been shown to be superior to metoprolol in improving survival (19). Why the performance measures include use of an angiotensin converting enzyme inhibitor or angiotensin receptor blocker but not carvedilol and spironolactone is unclear.

Some of the performance measures may have caused inadvertent harm. For example, administration of antibiotics within 4 hours to patients with pneumonia was a previous performance measure. However, studies showed that this performance measurement led to administration of antibiotics in many patients who proved not to have pneumonia or another infectious disease, and a systematic review concluded that “evidence from observational studies fails to confirm decreased mortality with early administration of antibiotics in stable patients with [community acquired pneumonia]” (20-22). The time has since been changed to 6 hours, but it is unclear if that it is any better than the initial 4 hour timing used (7).

We did not confirm the Joint Commission’s findings that the top hospitals are overrepresented by small, rural hospitals. We found no correlation between hospital level of complexity of care and performance measure compliance with the exception of acute myocardial infarction which was higher in hospitals with higher complexities of care. Although we found no correlation of the performance measures with any outcome measures, we did find a strong correlation between the hospital level of complexity of care and overall survival and surgical morbidity with the hospitals having the higher level of complexity having improved survival and decreased surgical morbidity. This would seem consistent with concept that volume of care correlates with outcomes.

It seems surprising that initiation of performance measures seem to go through such little scrutiny. In a 2005 editorial Angus and Abraham (23) addressed the issue of when there is sufficient evidence for a concept to be widely applied as a guideline or performance measure. Comparing guidelines to evaluation of novel pharmacologic therapies, they point out that promising phase II studies are insufficient for regulatory approval. Instead, one, and usually two, large multicenter phase III trials are necessary to confirm reliability. The same principle is echoed in evidence-based medicine, where grade A recommendations are based on two or more large, positive, randomized, and multicenter trials. This seems a reasonable suggestion. Perhaps what is needed is an independent Federal or private agency to review and approve performance measures, and as Angus and Abraham suggest, require at least two randomized, multicenter trials before implementation

The data presented in this manuscript do not support the usefulness of increasing compliance with the Veterans Administration’s (or the Joint Commission’s) performance measures in improving outcomes such as mortality, morbidity, length of stay or readmission rates. Until compliance with the performance measures results in improved outcomes, investment to improve these performance measures seems to be a poor utilization of resources. It suggests that oversight of regulatory agencies is needed in developing and implementing performance measures. If performance measures are to be used, new, clinically meaningful measures that correlate with outcomes need to be developed.

References

  1. Available at: http://www.jointcommission.org/accreditation/hospitals.aspx (accessed 9-25-11).
  2. Available at: http://www.forbes.com/sites/davidwhelan/2011/09/20/is-the-joint-commission-list-of-top-hospitals-worth-heeding/ (accessed 9-25-11).
  3. Rosenthal GE, Harper DL, Quinn LM. Severity-adjusted mortality and length of stay in teaching and nonteaching hospitals. JAMA 1997;278:485-90.
  4. Werner RM, Bradlow ET. Relationship between Medicare's hospital compare performance measures and mortality rates. JAMA 2006;296:2694-702.
  5. Peterson ED, Roe MT, Mulgund J, DeLong ER, Lytle BL, Brindis RG, Smith SC Jr, Pollack CV Jr, Newby LK, Harrington RA, Gibler WB, Ohman EM. Association between hospital process performance and outcomes among patients with acute coronary syndromes. JAMA 2006;295:1912-20.
  6. Fonarow GC, Yancy CW, Heywood JT; ADHERE Scientific Advisory Committee, Study Group, and Investigators. Adherence to heart failure quality-of-care indicators in US hosptials: analysis of the ADHERE Registry. Arch Int Med 2005;165: 1469-77.
  7. Wachter RM, Flanders SA, Fee C, Pronovost PJ. Public reporting of antibiotic timing in patients with pneumonia: lessons from a flawed performance measure. Ann Intern Med 2008;149:29-32.
  8. Stulberg JJ, Delaney CP, Neuhauser DV, Aron DC, Fu P, Koroukian SM. Adherence to surgical care improvement project measures and the association with postoperative infections. JAMA. 2010;303:2479-85.
  9. Available at: http://www.va.gov/health/docs/HospitalReportCard2010.pdf (accessed 9-28-11).
  10. Ross JS, Maynard C, Krumholz HM, Sun H, Rumsfeld JS, Normand SL, Wang Y, Fihn SD. Use of administrative claims models to assess 30 day mortality among Veterans Health Administration hospitals. Medical Care 2010; 48: 652-658.
  11. VanSuch M, Naessens JM, Stroebel RJ, Huddleston JM, Williams AR. Effect of discharge instructions on readmission of hospitalised patients with heart failure: do all of the Joint Commission on Accreditation of Healthcare Organizations heart failure core measures reflect better care? Qual Saf Health Care 2006;15:414-7.
  12. Fonarow GC, Abraham WT, Albert NM, Stough WG, Gheorghiade M, Greenberg BH, O'Connor CM, Pieper K, Sun JL, Yancy C, Young JB; OPTIMIZE-HF Investigators and Hospitals. Association between performance measures and clinical outcomes for patients hospitalized with heart failure. JAMA 2007;297:61-70.
  13. Fine MJ, Smith MA, Carson CA, Meffe F, Sankey SS, Weissfeld LA, Detsky AS, Kapoor WN. Efficacy of pneumococcal vaccination in adults. A meta-analysis of randomized controlled trials. Arch Int Med 1994;154:2666-77.
  14. Dear K, Holden J, Andrews R, Tatham D. Vaccines for preventing pneumococcal infection in adults. Cochrane Database Sys Rev 2003:CD000422.
  15. Huss A, Scott P, Stuck AE, Trotter C, Egger M. Efficacy of pneumococcal vaccination in adults: a meta-analysis. CMAJ 2009;180:48-58.
  16. Rigotti NA, Munafo MR, Stead LF. Smoking cessation interventions for hospitalized smokers: A systematic review. Arch Intern Med 2008;168:1950-1960.
  17. Gottlieb SS, McCarter RJ, Vogel RA. Effect of beta-blockade on mortality among high-risk and low-risk patients after myocardial infarction. N Engl J Med 1998;339:489-97.
  18. Pitt B, Zannad F, Remme WJ, Cody R, Castaigne A, Perez A, Palensky J, Wittes J for the Randomized Aldactone Evaluation Study Investigators. The effect of spironolactone on morbidity and mortality in patients with severe heart failure. N Engl J Med 1999;341:709-17.
  19. Poole-Wilson PA, Swedberg K, Cleland JG, Di Lenarda A, Hanrath P, Komajda M, Lubsen J, Lutiger B, Metra M, Remme WJ, Torp-Pedersen C, Scherhag A, Skene A. Carvedilol Or Metoprolol European Trial Investigators. Comparison of carvedilol and metoprolol on clinical outcomes in patients with chronic heart failure in the Carvedilol or Metoprolol European Trial (COMET): randomised controlled trial. Lancet. 2003;362:7-13.
  20. Kanwar M, Brar N, Khatib R, Fakih MG. Misdiagnosis of community acquired pneumonia and inappropriate utilization of antibiotics: side effects of the 4-h antibiotic administration rule. Chest 2007;131:1865-9.
  21. Welker JA, Huston M, McCue JD. Antibiotic timing and errors in diagnosing pneumonia. Arch Intern Med 2008;168:351-6.
  22. Yu KT, Wyer PC. Evidence-based emergency medicine/critically appraised topic. Evidence behind the 4-hour rule for initiation of antibiotic therapy in community-acquired pneumonia. Ann Emerg Med 2008;51:651-62.
  23. Angus DC, Abraham E. Intensive insulin therapy in critical illness: when is the evidence enough? Am J Resp Crit Care 2005;172:1358-9

Click here for Excel version of Table 1

Click here for Excel version of Table 2

Click here for Excel version of Table 3

Click here for Excel version of Table 4

Click here for Excel version of Table 5

Click here for Excel version of Table 6

Click here for Excel version of Table 7

Click here for Appendix 1

Click here for Appendix 2

Click here for Appendix 3

Click here for Appendix 4

Reference as: Robbins RA, Gerkin R, Singarajah CU. Relationship between the Veterans Healthcare Administration Hospital Performance Measures and Outcomes. Southwest J Pulm Crit Care 2011;3:92-133. (Click here for PDF version of manuscript)

Page 1 ... 3 4 5 6 7