FDA Asks Hospitals to Report Safety Glitches in Digital Health Systems

A theme of my writings on this blog and on my teaching site for the past decade has been mismanagement of healthcare information technology by an industry and people who have been operating for many years far beyond their qualifications and competencies. Technology requiring the highest levels of biomedical-IT cross disciplinary expertise has most commonly been designed, managed, implemented, led and defended by amateurs [see note 1].

As a result of this mismanagement, the technology is not without its perils. The problem is that this industry suffers from constricted information flows for a number of reasons, and we do not know the magnitude of the perils. Even speculation is difficult since data is scarce. The technology remains experimental.

After a commitment of tens of billions of economic recovery act funds to roll out this technology nationally, the Federal government has finally taken notice:

FDA Asks Hospitals to Report Safety Glitches in Digital Health Systems
In Letter to 350 Health Centers, Agency Signals Growing Concern

By Fred Schulte and Emma Schwartz
Huffington Post Investigative Fund
3:24 pm | 11 Mar 2010

Concern over safety risks posed by health information technology has led the Food and Drug Administration to step up scrutiny of the products, including digital medical records systems on which the government plans to spend billions of dollars in coming years.

The FDA last month asked a network of 350 hospitals it set up across the country to report data on potential hazards from a range of computer-assisted medical devices, according to an agency document obtained by the Huffington Post Investigative Fund.

This should have occurred a decade ago, but better late than never (of course, this action will not help people already harmed by this technology such as these and these).

The FDA action comes as federal officials forge ahead with plans to use as much as $27 billion in economic stimulus money to replace paper patient records with digital ones. The Obama administration wants to create a digital health file for every American by 2014, saying the conversion will save money and improve the quality of health care.

From an analysis that I concur with from the Heartland Institute:

... Proponents of this spending rely heavily on a short RAND Corporation analysis from 2005 that predicted $77 billion in annual savings and improved outcomes. RAND estimated “implementation would cost around $8 billion per year, assuming adoption by 90 percent of hospitals and doctors offices over 15 years.” It said, “The benefits can include dramatic efficiency savings, greatly increased safety, and health benefits.”

Unfortunately, RAND assumed an error-free system that is quickly and enthusiastically adopted by virtually the entire health care system. That might happen, but it is an absolute best-case scenario. [See my aforementioned HIT teaching site on why a "best case scenario" is a near impossibility - ed.] Even then, instead of “dramatic savings,” the $77 billion hoped-for savings amounted to a mere 4.5 percent of total costs, placed at $1.7 trillion by RAND.

Far more likely is that every penny of the $20 billion will be wasted on systems that don’t work and can never be implemented. [As per my Feb. 18, 2009 Letter to the Editor in the Wall Street Journal - ed.] That was the outcome of federal attempts to upgrade technology at the IRS, the FBI, and the air traffic control system. And these are all relatively simple enterprises involving single federal agencies. Health IT is vastly more complex and must include hundreds of thousands of private organizations that have invested in legacy systems that work reasonably well and are as varied as there are providers.

Back to the Huffington Post Investigative Fund article:

... Reports that hospitals send to the FDA are to be posted on an agency Web site. The FDA maintains it has the authority to regulate the technology, but has not taken steps to do so, leaving the industry largely to police itself. Reporting of problems is voluntary and most manufacturers have not done so.

I think it accurate to say reporting is nearly nonexistent, yet I regularly hear stories from colleagues, former students, and others that make my hair stand on end. These people are afraid to speak out publicly, lest their careers be threatened by hospital sham peer review, vendor lawsuits, or other forms of retaliation against 'whistleblowers' (see an example here).

... In the letter sent to hospitals last month, Marilyn Flack, of the FDA’s Center for Devices and Radiological Health, said use of digital medical equipment “continues to grow and affect patient care and safety.” The FDA “is exploring problems….that may affect patient safety,” she wrote.

The agency plans to collect reports using its Medical Product Surveillance Network, called MedSun. It is asking hospitals to note a wide range of problems involving electronic health records and other computerized systems for hospital laboratories, pharmacies and anesthesia and radiology devices, including hand-held ones.

Here is the FDA letter. Click to enlarge:

FDA letter on health IT safety reporting, page 1 (click to enlarge)

FDA letter on health IT safety reporting, page 2 (click to enlarge)

I note that this letter is remarkable, a real breakthrough. It is also remarkable that this 'breakthrough' had to wait until 2010, not 1990 or 2000, and only after the government has pushed hard to spread this technology nationally without knowing the flip side. The expression "ready, fire, aim" comes to mind.

Some of the 'glitches' were mentioned:

... She also cited an example of a software package used in a hospital emergency room in which lab tests “ordered for one patient returned the results for another.”

In another case, a hospital in the MedSun network reported an operating room software product that often “locked up” during surgery, without alerting anyone that “data entry had ceased.”

According to the FDA letter: “At the end of the surgical procedure, surgical procedure notes were incomplete—compromising the accuracy of the data as nurses had to manually re-enter from memory many of the surgical notes.”

In a third case cited by Flack, a radiology workstation became “extremely slow, delaying procedures and causing X-ray techs to subject patients to repeat X-rays.” The cause was determined to be a software glitch that happened when too many characters were entered in.

These 'glitches' largely come from poor design, engineering, implementation and support and don't even include mission hostile user experiences from software that is 'working correctly.'

... Arthur Bartosch, director of Biomedical Engineering Services at Westchester Medical Center in New York, agreed [that in the past, cinical engineering staff would not have been involved in health IT safety reporting]. He said he planned to circulate the FDA’s alert to the hospital’s chief medical information officer and they would “probably” create a task force to figure out how to document any problems.

It is pathognomonic of IT irrational exuberance and special accommodation given to this technology (greased by ignorance, money, politics and other pathologies among healthcare and hospital leadership juntas) that such 'task forces' are not as common as health IT itself.

... “The fundamental problem we have here is we’re dealing with an industry that really isn’t used to a transparent reporting of problems,” [Paul Egerman, co-chair of a government panel looking at the safety of health information technology] said.

[Isn't used to transparent reporting? I would have said "hostile to the extreme" regarding that kind of transparency - ed.]

Egerman, whose advisory panel expects to make its findings public in April, said government officials have a long way to go in fully understanding the potential hazards of adopting the new technology.

I noted a "hit" from EOP.gov (Executive Office of the President) on my ten year old HIT teaching site just yesterday. I believe it is the first. Welcome to Medical Informatics 101, Mr. President.

“Are there much more serious problems that we would know about if we had the data?,” he said. “These are all reasonable questions to ask.”

I injured my career in the past for asking just those questions, having been badgered to stop asking them essentially to the point of constructive discharge in my former CMIO role.

Let's see if the change of culture is real, or just talk.

It's not that safety reporting and transparency are anything new or special. From the Joint Commission in their August 2009 Sentinel Events Alert #43:

Existing Joint Commission requirements:
The Leadership chapter in the standards manual addresses leadership and safety, specifically relating to the organization's governing body, the chief executive and senior managers, and medical and clinical staff leaders.

The standards specifically require that these three leadership groups create a culture of safety (11) by creating an atmosphere of trust and fairness that encourages reporting of risks and adverse events, by allocating the resources necessary to support safety, by discussing and reporting safety issues and indicators, and by developing plans to assure and improve safety performance, especially in relation to high-risk or problem-prone processes. Other issues covered in the standards are: the implementation of important systems within the organization that support safety; the organization's safety program for reporting adverse events and near misses; and the design or modification of processes to support safety.

And:

Suggested Actions (item #2):
Institute an organization-wide policy of transparency that sheds light on all adverse events and patient safety issues within the organization, thereby creating an environment where it is safe for everyone to talk about real and potential organizational vulnerabilities and to support each other in an effort to report vulnerabilities and failures without fear of reprisal. (8,9)

Finally, considering that the average salary offered to a hospital "Director of Informatics" is a penurious $90-120K per annum with respect to the true expertise required for top roles (don't just take my word for it, take ONC's), and that 'lowball hiring' in terms of expertise is pervasive, I predict that if robust error reporting takes place, the next few years in HIT will be a wild ride indeed.

-- SS

[1] I use the term "amateur" in the same sense that I am a telecommunications amateur, not a professional. Even though I hold the highest license class possible, the Extra, I would not even dream of leading a large telecom project.

On ONC's "Proposed Establishment of Certification Programs for Health Information Technology"

The Office of the National Coordinator for Health Information Technology of HHS (the Department of Health and Human Services) has issued a proposed rule "RIN 0991-AB59 Proposed Establishment of Certification Programs for Health Information Technology." The proposed rule is available in PDF at this link and more information is available from ONC itself at this link.

I have written a response to the proposed rule that will be sent as a public comment to the Federal eRulemaking Portal (http://www.regulations.gov/search/Regs/home.html).

I reproduce my response below:

Mar. 9, 2010

Re: RIN 0991-AB59, "Proposed Establishment of Certification Programs for Health Information Technology" (http://www.federalregister.gov/OFRUpload/OFRData/2010-04991_PI.pdf):

Dear HHS/ONC:

I believe the deadlines driving establishment of a certification program for health IT as proposed in RIN 0991-AB59, as well as for achieving “meaningful use of healthcare IT” and for onset of medicare penalties for “non adopters”, will result in diffusion of healthcare IT that, in the words of the Jan. 2009 National Research Council report on health IT “will not be sufficient to achieve medical leaders' vision of health care in the 21st century and may even set back the cause” (http://www8.nationalacademies.org/onpinews/newsitem.aspx?RecordID=12572).

I believe the national health IT system that will result will be injurious to patients at an unacceptably high level as well.

I am a physician and Yale-trained medical informatician and have been writing about the challenges of healthcare IT since the late 1990’s. My bio is at http://www.ischool.drexel.edu/faculty/ssilverstein/biography.htm and my teaching site on HIT difficulties is at http://www.ischool.drexel.edu/faculty/ssilverstein/failurecases/ . I also write on Medical Informatics and HIT for the Healthcare Renewal blog of the Foundation for Integrity and Responsibility in Medicine (FIRM), a 501(c)(3) policy think tank, at http://hcrenewal.blogspot.com.

I have labored over the past decade to steer health IT efforts away from known and predictable paths of difficulty, failure and adverse consequences based on medical science, the science of Medical Informatics, ethical considerations, and the experience of other nations with HIT. I am writing to you to express serious concerns about ONC’s HIT Certification Program NPRM (http://www.federalregister.gov/OFRUpload/OFRData/2010-04991_PI.pdf).

In effect, the NPRM calls for healthcare IT to receive a special governmental accommodation, apparently in part due to politically-decided, and certainly non-scientifically derived timelines. The special accommodations are in the areas of certification, post-market surveillance and inadequate use of existing regulatory expertise over safety-critical IT by agencies with specific domain expertise in that undertaking, thus “reinventing the wheel.”

(On the non-scientific nature of the timelines, see, for example, “Predicting the Adoption of Electronic Health Records by Physicians: When Will Health Care be Paperless?”, Ford et. al, JAMIA 2006 13: 106-112, http://jamia.bmj.com/content/13/1/106.full.pdf.) [note: also see addendum below - ed.]

First and foremost, the term “safety” itself appears in the RIN 0991-AB59 proposal text only four times, and not in the context of strong provisions to safeguard patients from adverse consequences of healthcare IT. This in and of itself is, quite frankly, of great concern, especially in the context of known HIT safety issues.

For instance, FDA’s testimony at ONC’s HIT Policy Committee Adoption/Certification Workgroup meeting on HIT safety (Feb. 25, 2010) itself revealed known patient injuries and deaths related to healthcare IT difficulties.

Even more importantly than the fact of these HIT-related adverse events, however, was the revelation that the true extent of these adverse events is unknown. As FDA’s Jeffrey Shuren, MD, JD expressed it, the data he provided is likely “just the tip of the iceberg.” This supports the contention that the technology is still in an experimental phase, rather than being tried and true.

A growing body of literature supports that view (e.g., see “2009 a pivotal year in HIT” at http://www.ischool.drexel.edu/faculty/ssilverstein/failurecases/?loc=cases&sloc=2009).

It also seems that unscientifically arrived at timelines (i.e., the politically-decided timelines for HIT adoption and achievement of “meaningful use”) that ignore the experimental nature of healthcare IT – that it is not yet “ready for prime time” in a national rollout - are promoting a rush to a superficial certification and surveillance process.

This is alien to the science, culture and ethical obligations of medicine and its practitioners.

The latter process, surveillance, is apparently intended to merely surveil continued conformance of HIT to agreed-upon standards, not patient safety as in the pharmaceutical and tangible-medical device postmarketing surveillance process.

I consider HIT a medical device that is virtual in nature, but a medical device nonetheless, a position the EU is steering towards. See "The Medical Products Agency’s Working Group on Medical Information Systems: Project summary" (available in English translation in PDF at http://www.lakemedelsverket.se/upload/foretag/medicinteknik/en/Medical-Information-Systems-Report_2009-06-18.pdf).

While I believe the NPRM proposal is a step up from the former certification roles envisioned by CCHIT and HIMSS, the proposal still lacks the rigor I have called for in many of my writings about HIT over the past decade.

On the formation of new “ONC-Approved Accreditors” (ONC-AA’s) for certification, this is a special accommodation for the HIT industry that appears to inexplicably place that sector in a favored position compared to the pharmaceutical, medical device and other industries that utilize safety-critical IT.

The FDA, for example, has significant expertise in validating and regulating IT in the pharmaceutical and medical device industries, including that used in clinical trials which bear similarities to HIT used in the delivery sector. For instance, see "General Principles of Software Validation; Final Guidance for Industry and FDA Staff" at http://www.fda.gov/downloads/RegulatoryInformation/Guidances/ucm126955.pdf.

This document opens with the statement:

• This guidance outlines general validation principles that the Food and Drug Administration (FDA) considers to be applicable to the validation of medical device software or the validation of software used to design, develop, or manufacture medical devices.

As yet another example, NASA has published a document “Certification Processes for Safety-Critical and Mission Critical Aerospace Software” (http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20040014965_2004000657.pdf). This document begins:

• Since safety-critical aerospace software is prevalent and important to human life, what is the rationale behind certification of such software? In other words, how do engineers know when a new software product works properly and is safe to fly? In the United States, software must undergo a certification process described in various standards by various regulatory bodies including NASA and the Requirements and Technical Concepts for Aviation (RTCA) which is enforced by the Federal Aviation Administration (FAA).

• How do researchers know which standards apply to their software? Each NASA center and the FAA have unique certification processes for different types of software. For example, there are special processes for the Space Shuttle and different processes for the Space Station. Any software that flies onboard an aircraft in FAA airspace must adhere to special FAA certification processes. There are also different processes depending upon whether the software is safety- or mission-critical or falls into another category. The UK and Europe have similar certification processes.

HHS should not be creating new, potentially (likely?) amateur organizations and bureaucracies overseeing these new virtual medical devices that will have variable (or no) experience in software validation, certification, regulation, postmarketing safety surveillance, etc. Rather, HHS should be leveraging existing governmental expertise in certifying, validating and regulating mission critical IT.

Further, what is to protect these new bureaucracies from being staffed by those with conflicts of interest with the industry whose products they are purported to certify and surveil? At the very least, existing federal agencies have policies on such conflicts.

Of note, we have a prime example of what can occur due to politically-mediated rushing of healthcare IT – that of the UK’s National Programme for Healthcare IT (NPfIT).

British PM Tony Blair repeatedly sought to shorten the timetable for the NHS national IT programme in a move that would have brought results for patients in time for a general election in 2005 (see http://www.computerweekly.com/Articles/2008/02/18/229447/secret-downing-street-papers-reveal-tony-blair-rushed-nhs.htm.) The result was predictable. A summary of the UK’s House of Commons, Public Accounts Committee’s 2009 report on near-disastrous problems in their £12.7 billion national EMR program is at http://www.publications.parliament.uk/pa/cm200809/cmselect/cmpubacc/153/15304.htm. From that summary:

• “Recent progress in deploying the new care records systems has been very disappointing …The Programme is not providing value for money at present because there have been few successful deployments of the Millennium system and none of Lorenzo in any Acute Trust … Despite our previous recommendation, the estimate of £3.6 billion for the Programme's local costs remains unreliable … Little clinical functionality has been deployed to date, with the result that the expectations of clinical staff have not been met … Patients and doctors have understandable concerns about data security."

And so forth.

Further, from the UK National Audit Office Executive Summary of 16 May 2008 (http://www.nao.org.uk/publications/0708/the_national_programme_for_it.aspx):

• At the outset of the Programme, the aim was for implementation of the systems to be complete and for every patient to have an electronic care record by 2010, although the timetable from 2006 was described as tentative. While some parts of the Programme are complete or well advanced, the original timescales for the Care Records Service – one of the key components of the Programme – have not been met.

We ignore the UK experience at our peril, an experience in a medical environment smaller and far more government-controlled than our own.

Finally, I call attention below to the actual ONC NPRM passages from which my concerns arise on “time constraints” leading to a rushed and superficial certification program (which I believe is frankly cavalier and irresponsible considering the stakes involved).

I believe that a rushed National Program for HIT in the United States will suffer the same fate as the aforementioned National Programme for IT in the UK, and perhaps even a worse fate as the UK’s socialized medicine system is certainly a smaller, more homogeneous and more controllable testbed environment for experimenting with HIT.

In summary, I believe the current approach to Healthcare IT certification is inadequate, in large part due to time constraints set upon the effort that are themselves artificially rushed and inadequate. I believe much more significant leveraging of existing biomedical and mission critical IT certification/validation expertise is essential, and that patient safety, not continuing adherence to existing standards should be a primary concern of post-implementation surveillance.

Thank you for considering these views.

I believe rushing health IT, and burying our heads in the sand about the predictable and demonstrated repercussions of doing so as outlined above and on this and other websites, is a very bad idea.

Making like an ostrich on national-scale healthcare IT is a very bad idea.

I have written about FDA myself and not always in complementary terms (e.g., here, here), but my concern is that the creation of multiple new potentially amateur organizations does not bode well for HIT, either.

The key to successful HIT certification, validation and patient safety is 1) leveraging the needed expertise but 2) without industry conflict of interest and 3) without the pathologies of the HIT 'ecosystem' and culture spoiling the environment (see my aforementioned website on HIT difficulties for more on that topic, as well as the HIT ecosystem essay at that site).

Perhaps a new federal HHS subunit is a potential solution - a Clinical Computing Administration (CCA) with regulatory teeth.

The oversight of hundreds of billions of dollars of technology and the patients the technology itself affects calls for a quite serious approach to these issues, in my view.

-- SS

March 9, 2010 Addendum:

On rushing national health IT programs - unknown to me when I wrote the post above, this article just appeared in the British Press:

Patients' medical records go online without consent
Telegraph.co.UK
By Kate Devlin, Medical Correspondent
Published: 10:20PM GMT 09 Mar 2010

Those who do not wish to have their details on the £11 billion computer system are supposed to be able to opt out by informing health authorities.

But doctors have accused the Government of rushing the project through, meaning that patients have had their details uploaded to the database before they have had a chance to object.

... Hamish Meldrum, [the British Medical Association] chairman, writes: "The breakneck speed with which this programme is being implemented is of huge concern ... "If the process continues to be rushed, not only will the rights of patients be damaged, but the limited confidence of the public and the medical profession in NHS IT will be further eroded."

... Norman Lamb, the Liberal Democrat health spokesman, said: "The Government needs to end its obsession with massive central databases. "The NHS IT scheme has been a disastrous waste of money and the national programme should be abandoned."

Read the whole thing.

-- SS