Introduction

IBM Watson™ for Oncology, trained by Memorial Sloan Kettering Cancer Center (MSKCC) uses certain attributes found in a patient’s case to identify potential treatment options for physicians to consider when making patient care decisions. The treatment options are generally consistent with National Comprehensive Cancer Network (“NCCN”) guidelines and are supported by MSKCC-curated literature reflecting MSKCC experience and expertise. Links are provided to the MSKCC-curated literature supporting each treatment option, as well as supplemental information from published medical literature, clinical trials, and the manufacturers’ prescribing information for oncology drugs. Watson™ for Oncology does not diagnose a patient. The treatment options are suggestions only for physicians to consider when planning treatment options and do not replace an oncologist’s expert judgment.

Intended Use

IBM Watson™ for Oncology, trained by oncologists at the Memorial Sloan Kettering Cancer Center (“MSKCC”), is designed to help cancer-treating physicians make evidence‐informed cancer patient care decisions based on up‐to‐date knowledge.

It features Dynamic patient summaries built from:

  • structured clinical data
  • unstructured clinical data (English only)
  • evidence based cancer treatment options, based on the patient profile weighed against US consensus guidelines, relevant US literature, and the client’s expertise

Table of Contents

 

Treatment Recommendation FAQs

Q. I do not agree with some of Watson™ for Oncology’s recommendations. Can local guidelines or practices be incorporated into Watson™ for Oncology?

Currently, Watson™ for Oncology is only trained by Memorial Sloan Kettering Cancer Center (MSKCC). However, we have a localization service using the Admin Console that can be used to localize specific treatments.  Using this localization feature, you can make modifications to dosing details, treatment rationale, attribute names, and other aspects of the system.

The MSKCC treatment recommendations, however, cannot be altered, since that is how Watson™ for Oncology was trained.

Q. Can I select a treatment option in the For Consideration orange zone or Not recommended red zone?

Yes. Clinical opinions vary, especially in complex cases. Watson™ for Oncology does not consider all patient factors, such as personal preferences or financial situation, which can influence a treatment decision.

Note: Watson for Oncology’s recommendations are provided in the context of a MSKCC practice in New York, which might not always be the most appropriate course of action for your situation in another part of the world. Recommendations that are identified by Watson™ for Oncology are only suggestions, and do not replace the expert judgment of a cancer-treating physician.

Q. For some treatments, Watson’s rationale and evidence appears lacking or not appropriate.

The display of rationale, data, and statistics is continually being improved. These improvements will be included in future releases throughout the year. Should you encounter inappropriate data, print the case details and email a Watson Health clinical adoption team member for feedback.

Q. I noticed some cancer treatment recommendations are not in the National Comprehensive Cancer Network (NCCN) guidelines. Why is this occurring?

Watson for Oncology’s cancer treatment recommendations are generally consistent with NCCN guidelines, but it is possible that Watson for Oncology’s recommendations include a cancer treatment that is not incorporated into the NCCN guidelines.  These treatments, chosen by MSK as possible treatment options, are typically based on expert opinion or recommendations and/or peer-reviewed literature.In addition, Watson for Oncology’s recommendations are updated frequently, and might be updated before the NCCN guidelines have been updated.  It’s also important to note that the system is trained by MSKCC, and not always in exact agreement with NCCN guidelines.

Q. Does Watson for Oncology provide recommendations for alternative therapies such as traditional medicine, herbal medicine, vitamins, and steroids?

No. Watson for Oncology does not support regimens that are considered alternative therapies.

Q. Can Watson for Oncology compare more than 2 treatments at one time?

No, Watson for Oncology can only compare two treatments at one time. Trying to simultaneously compare three or more treatments on one screen would be hard to read, and is not currently supported by our user interface (UI) design.

Q. How are recommendations ranked in Watson™ for Oncology and how is it better than the base NCCN guidelines?

The NCCN guidelines consider a very small set of patient attributes when compared to Watson for Oncology.

For example, in breast cancer, when trying to decide between an aggressive chemotherapy, such as AC followed by Taxane, versus a less aggressive chemotherapy, such as CMF, Watson for Oncology considers more patient attributes, more evidence, and the expert guidance of MSK than do the NCCN Guidelines. The following factors are considered:

  • patient age
  • performance status
  • comorbidities
  • special concerns about toxicities for heart, liver, kidney, etc.
  • multifocality
  • burden of disease
  • HER2 status
  • ER/PR status
  • Prior treatments/surgeries
  • Margin status
  • Tumor size and histology
  • Lymph node status
  • Oncotype test (recurrence score)

These and other factors are included in the trained model of Watson for Oncology, and are scored to return the answer that best reflects MSK training and the most current expert guidance. Watson™ for Oncology also returns supporting evidence and rationale so that you are informed of the reasons for a recommendation.

 

Evidence, Literature, Corpus

Q. How often are textbooks, journals, and publications updated?

The evidence corpus is continuously updated and new versions of Watson for Oncology are released each month.

Q. What does “MSK curated literature” mean?

When there are specific papers identified by MSK as being of very high importance for a treatment option provided by Watson™ for Oncology, these papers are highlighted as having been curated by MSK oncologists.  Since these papers have been curated by MSK oncologists, they appear more prominently in the User Interface than other papers found by the Watson for Oncology literature search, and are provided alongside the treatment options.

Q. Are ESMO guidelines included in Watson for Oncology?

No, not at this time.

Q. Where can I find the sources of literature ingested by Watson for Oncology?

Select Notices in the navigation banner, and then select Table of Contents to view the list or corpus of materials ingested by Watson for Oncology.

 

Patient Cases and Attributes

Q. Watson for Oncology’s NLP does not utilize certain attributes such as smoking history or prior therapies, in my clinical notes. Why?

The accuracy and scope of concept detection from unstructured text is continually being improved.  However, there are patient attributes that are only useful for demographics reporting and diagnosis, and Watson for Oncology does not utilize this information.  Watson for Oncology focuses on treatment recommendations and utilizes the attributes that influence patient treatment.

Q. Can I share my case with another colleague?

Yes, you can share your cases. The default setting allows you to view only your cases. You can share a case by selecting the Duplicate icon that is displayed next to the case. In the window that is displayed, select the Share check box.

Note: all Watson for Oncology users in your organization can view the shared case.  After a case is shared, each user can create their own copy of the case and make modifications to it, but this does not modify the original shared case.

Q. How do I send a patient case to a fellow clinician?

There are two ways to send a patient case to another clinician:

  1. Save the case as a PDF file and email the file to your colleague,
  2. Share the case by using the Sharing feature to duplicate the case in Watson for Oncology. This is described in more detail in a previous FAQ. However, your colleague needs a Watson for Oncology user account in order to login and view the shared case.

Q. If I delete a patient case, is the data removed from the database completely?

No, the data remains in the database, but you can no longer view the patient case data using the Watson for Oncology application. Data is stored for audit trail purposes and for reporting and metrics.  Watson for Oncology also has the capability to have patient cases older than a specified threshold deleted completely.  In this scenario, the patient cases are removed from the database and not recoverable.

Lastly, if a Watson for Oncology customer wishes to have their institution’s entire patient data removed, in the event of their usage of the product coming to an end, then all patient data undergoes a hard delete and cannot be recovered.

Q. Can Watson track a patient over time and  provide longitudinal data?

Watson for Oncology is not designed like an Electronic Medical Record (EMR) system. Each patient case in Watson for Oncology can be modified over time and only reflects the latest attribute values that were saved. The Watson for Oncology user interface is not designed to display changes in history or treatment over time like an EMR system. Use your EMR system to store changes in patient history, investigations,and treatment decisions. You can print or attach a PDF copy of Watson for Oncology’s recommendations, with patient attribute values, in your EMR system or medical record folders.

 

Providing Feedback

Q. I have suggestions for improvements. How can I provide this feedback to IBM?

You can enter feedback in Watson for Oncology by selecting the Feedback link. Alternatively, you can also collate a suggestion list and provide this information to an IBM Watson Health clinical adoption member when he or she visits your organization.

Q. How can a clinician communicate the need for functionality that is not currently supported?

You can use the same feedback process as listed above, or you can contact your IBM clinical adoption team.

Q. What is the difference in Comment and Feedback?

The Comment function on the patient banner is typically used to share information about the patient with other colleagues, or to record personal thoughts about a case.  IBM never sees these patient comments.

The Feedback function on the navigation bar is used to share feedback about Watson for Oncology with IBM.

 

Product Version Updates

Q. How frequently does IBM update Watson for Oncology?

IBM will inform you when a new version/release is available for your organization. This typically happens every 1-2 months, but may be moving toward a release every 3 months.

Q. What can we expect for each new version/release?

New releases can contain a variety of updates, including the addition of new treatment options, literature, cancer types, and changes to existing data attributes and functions. New releases can also contain usability improvements to the product and changes in the visual design.

Q. Is the version/release update process complicated, requiring technical effort from the client?

Minimal technical effort is required from you. Our support team informs you of a short system down time, which is typically after regular business hours,that is required to install the updates in our data centers. You can again access the system after the installation of the updates is complete. IBM suggests that your lead clinicians or champion users perform the following actions after the updates are complete:

  • Test: Enter some test case information in your Test system environment to ensure that everything is working correctly.
  • Communicate: Inform all your users of the updates.
  • Provide additional training, if needed: Additional user training might be required for new functionality.

Q. How do I know what version of Watson for Oncology I am using? Where do I find the latest information about Watson for Oncology’s supported cancer types and coverage?

On the navigation bar, select Notices, and then select Product Notices. The version information, latest supported cases, and cancer types and coverage are displayed.

 

Artificial Intelligence (AI) and Machine Learning

Q. Does Watson for Oncology learn from my usage patterns and adapt to my practices?

Watson™ for Oncology does not directly learn from your usage patterns, because Watson™ for Oncology is currently only trained by MSKCC. However, feedback is collected through the Watson™ for Oncology application and through contact with the IBM clinical adoption team, which meets regularly with MSKCC.  This feedback is used to improve and train Watson™ for Oncology.

Q. Is Watson for Oncology a robot?

No

Q. Will Watson for Oncology replace oncologists?

No, Watson for Oncology does not replace oncologists or their expert opinions. Recommendations identified by Watson™ for Oncology are only suggestions and do not replace an oncologist’s expert judgment.  Based on the oncologist’s own experience, courses of treatment that are not identified by Watson™ for Oncology and that should be considered might exist.  Watson™ for Oncology should only be used to assist qualified clinicians who are practicing within their areas of competence.Watson for Oncology is extremely valuable in presenting options to consider, and by raising questions that might not have been considered by an oncologist.

It can also be used as a training tool for entering hypothetical cases, to see the recommendations provided by Watson.

Q. Does Watson for Oncology diagnose cancer?

Watson for Oncology does not diagnose cancer. Watson for Oncology should be used for patients with a confirmed diagnosis of cancer.

 

Data, Interfaces, and Security

Q. I understand Watson™ for Oncology is a cloud environment that hosts data from other organizations. Can other clients view my organization’s data?

No, each client’s data is strictly separated from other client data. No sharing is permitted.

Q. How long will IBM store my organization’s data? Will my data be completely erased after my contract ends?

Yes, all data is clearly identified with the organization ID that created it.  When a contract ends, all data associated with that organization is erased. Furthermore, there is also an option to have data older than a given threshold to be completely erased.  When this happens, the data can no longer be accessed from the database and will be overwritten.

Q. Is there a backup for Watson for Oncology’s database?

Yes

Q. If there is power failure in a data center, does Watson for Oncology have redundancy?

Yes

Q. Is it a requirement for Watson for Oncology to interface with an electronic medical record (EMR) system?

No. There is no requirement for Watson for Oncology to interface with an EMR system. However, having Watson for Oncology interface with an EMR system can reduce data entry workload by eliminating the need to enter basic patient data, including age, gender, and lab results.

Q. How is data fed or sent to Watson for Oncology?

There are patient data interfaces that can be used to programmatically push patient data into the Watson for Oncology patient list.  These are Application Programming Interfaces, or APIs.  These API calls can be configured to either analyze the data (i.e. Ask Watson) or to just store the patient data to make it available the next time the patient list is reviewed. There is, of course, the ability to create a case from scratch, in which the data is entered into Watson by the clinician or assistant using the system interactively.  This method is useful when there is no Electronic Medical Record (EMR), or when there is a patient (or hypothetical patient) that does not yet exist in an EMR.

 

Admin Console

Q. How can my organization generate usage reports every month?

Watson for Oncology has an Admin Console to access aggregated usage data and statistics. If you have administrator authority, you can generate reports on demand and download the data in different formats, including Microsoft Excel.

 

Localization

Q. Can we localize to just one cancer type?

Yes. Using the Admin Console, you can choose to localize the dosing, attributes, treatments, and rationale for a single cancer type, or for all cancer types.

Q. Is there a plan to translate the UI to languages other than English?

Yes.  Not all literature and evidence can be translated, but the translation of the UI, including attributes, options, buttons, and banners, into other languages is in progress. Contact your IBM Customer Support Representative (CSR) for a list of supported languages.