Subject category:
Knowledge, Information and Communication Systems Management
Published by:
IBS Center for Management Research
Length: 25 pages
Data source: Published sources
Topics:
International business machines corporation; IBM Watson Health; Memorial Sloan Kettering Cancer Center; University of Texas MD Anderson Cancer Center; Jeopardy!; DeepBlue; DeepQA; Oncology expert advisor; Advanced analytics; Cloud computing infrastructure; Clinical research; Drug discovery; Genomic sequencing capabilities
Abstract
The case discusses the artificial intelligence (AI) failure at American multinational technology company International Business Machines' (IBM) AI software Watson for Oncology (Watson). In 2012, IBM partnered with New York-based cancer treatment center and research organization Memorial Sloan Kettering Cancer Center (MSK) to develop Watson as AI software that could provide medical professionals with improved access to up-to-date and comprehensive cancer data and practices regardless of where a patient was living or the physician was practising. In addition to this, in 2013, IBM partnered with University of Texas MD Anderson Cancer Center (MD Anderson) to develop an AI tool called Oncology Expert Advisor (OEA) with the stated mission of eradicating cancer. IBM touted Watson as a project that could be better at diagnosing cancer and giving treatment recommendations to patients than human doctors due to its cognitive computing ability through which it could stay current with the huge volume of new findings published every day on cancer research and care. Despite Watson's Natural Language Processing (NLP) ability that enabled it to read and gain insights from unstructured data such as doctors' notes, clinical summaries, etc., it was not able to interpret data as human doctors could. A 2017 investigation carried out on Watson by news website STAT revealed how IBM's AI software could not live up to the hype created around it by the company. STAT reported that internal documents from IBM revealed that Watson had recommended 'unsafe and incorrect' cancer treatments. There were no reports of patients being harmed since the recommendations were not used on real cancer patients. But the documents obtained by STAT showed that doctors who tried using Watson to help them design treatment complained that the AI software was not ready for practicing medicine. To add to IBM's troubles, a 2017 audit carried out by the University of Texas showed that MD Anderson was using old data to train its OEA. The same year, i.e. in 2017, the cancer center closed down its project with IBM after spending US$62 million, attributing the closure to delays, cost overruns, procurement problems, and IBM's Research Team shifting the focus of the project from one form of cancer to another. Analysts at American market research company Forrester Research Inc. (Forrester) revealed that data quality was among the biggest challenges faced by AI projects, which often led to failures. In February 2021, American business-focused international newspaper the Wall Street Journal (WSJ) reported that IBM was considering selling Watson Health since the business was not profitable. Experts felt that AI tools in healthcare lacked the ability to diagnose a patient like a physician could and it would require huge amounts of information to treat patients especially cancer, which was complex and unique to every patient. With data quality and domain expertise being some of the major reasons for the failure of AI projects, is it worth it for companies such as IBM to invest in AI and Machine Learning (ML) tools in healthcare like Watson for Oncology without proper data preparation? What are the roadblocks standing in the way of IBM finding success in using AI and ML in a bid to drive clinical research and drug discovery? Going forward, how should technology companies make AI and ML a truly transformative force in healthcare?
Time period
The events covered by this case took place in 2012-2021.Geographical setting
Region:
Americas
Country:
United States
Featured company
International Business Machines Corporation
Employees:
10000+
Turnover:
USD 73 billion
Type:
Public company
Industry:
Technology & communications
About
Abstract
The case discusses the artificial intelligence (AI) failure at American multinational technology company International Business Machines' (IBM) AI software Watson for Oncology (Watson). In 2012, IBM partnered with New York-based cancer treatment center and research organization Memorial Sloan Kettering Cancer Center (MSK) to develop Watson as AI software that could provide medical professionals with improved access to up-to-date and comprehensive cancer data and practices regardless of where a patient was living or the physician was practising. In addition to this, in 2013, IBM partnered with University of Texas MD Anderson Cancer Center (MD Anderson) to develop an AI tool called Oncology Expert Advisor (OEA) with the stated mission of eradicating cancer. IBM touted Watson as a project that could be better at diagnosing cancer and giving treatment recommendations to patients than human doctors due to its cognitive computing ability through which it could stay current with the huge volume of new findings published every day on cancer research and care. Despite Watson's Natural Language Processing (NLP) ability that enabled it to read and gain insights from unstructured data such as doctors' notes, clinical summaries, etc., it was not able to interpret data as human doctors could. A 2017 investigation carried out on Watson by news website STAT revealed how IBM's AI software could not live up to the hype created around it by the company. STAT reported that internal documents from IBM revealed that Watson had recommended 'unsafe and incorrect' cancer treatments. There were no reports of patients being harmed since the recommendations were not used on real cancer patients. But the documents obtained by STAT showed that doctors who tried using Watson to help them design treatment complained that the AI software was not ready for practicing medicine. To add to IBM's troubles, a 2017 audit carried out by the University of Texas showed that MD Anderson was using old data to train its OEA. The same year, i.e. in 2017, the cancer center closed down its project with IBM after spending US$62 million, attributing the closure to delays, cost overruns, procurement problems, and IBM's Research Team shifting the focus of the project from one form of cancer to another. Analysts at American market research company Forrester Research Inc. (Forrester) revealed that data quality was among the biggest challenges faced by AI projects, which often led to failures. In February 2021, American business-focused international newspaper the Wall Street Journal (WSJ) reported that IBM was considering selling Watson Health since the business was not profitable. Experts felt that AI tools in healthcare lacked the ability to diagnose a patient like a physician could and it would require huge amounts of information to treat patients especially cancer, which was complex and unique to every patient. With data quality and domain expertise being some of the major reasons for the failure of AI projects, is it worth it for companies such as IBM to invest in AI and Machine Learning (ML) tools in healthcare like Watson for Oncology without proper data preparation? What are the roadblocks standing in the way of IBM finding success in using AI and ML in a bid to drive clinical research and drug discovery? Going forward, how should technology companies make AI and ML a truly transformative force in healthcare?
Settings
Time period
The events covered by this case took place in 2012-2021.Geographical setting
Region:
Americas
Country:
United States
Featured company
International Business Machines Corporation
Employees:
10000+
Turnover:
USD 73 billion
Type:
Public company
Industry:
Technology & communications