OpenAI, Color Health partner to transform cancer care

Color Health is working with OpenAI to pioneer a new way to accelerate cancer patients' treatment access. Their new copilot application uses GPT-4o to identify missing diagnostics and create tailored workup plans, enabling healthcare providers to make evidence-based decisions about cancer screening and treatment.

How does it work?

Color Health uses OpenAI's APIs to integrate patient medical data with clinical knowledge. The outcome is a copilot application that creates customized, comprehensive treatment plans for providers to review and use in their patient care.

The copilot application's output is analyzed by a clinician at every step and, if need be, modified before being presented to the patient. It works as follows:

  1. It extracts, processes, and normalizes patient information, such as family history and individual risk factors, along with clinical guidelines and data from trusted sources. The Color team was particularly impressed with GPT-4o's ability to extract and normalize information that was buried within pages of inconsistently structured and phrased information, often in different formats, such as with PDFs or clinical notes.
  2. Using this data, it answers key questions like, "What screenings should the patient be doing?" to identify missing diagnostics and generate a personalized screening plan. It also generates documentation required to complete any diagnostic workups, such as medical necessity documents and insurance pre-authorizations.
  3. The clinician-in-the-loop evaluates the output, which includes source information. The clinician can edit the copilot's output, which also helps refine future iterations.
  4. Once the clinician-in-the-loop is satisfied with the result, they can add the information to the patient's existing treatment plan.

Why does it matter?

Screening, diagnosis, and treatment for cancer is notoriously complex and time-consuming. And every delay makes a difference: patients whose treatments are delayed by just four weeks face a 6-13% higher mortality risk.

Screening needs are also often highly individualized. More than a third of Color's patients, for example, require earlier, different screening approaches based on individual risk factors not addressed by standard guidelines.

Beyond screening, diagnostic workups create more challenges. Documenting and performing a single patient's diagnostic workup can take weeks, with most patients arriving at their first oncology appointment without a complete workup.

Color Health's partnership with OpenAI

Color began working with OpenAI in 2023, to use AI to improve cancer patient care and health equity. With the challenges of cancer screening, diagnosis, and treatment in mind, Color was looking for a solution that could:

  • Interpret inconsistently-formatted patient data
  • Analyze dense healthcare guidelines
  • Protect patient data privacy
  • Support clinician-in-the-loop workflow design to ensure patient safety
  • Integrate with electronic health records (EHRs) and core hospital systems

During the initial exploration, Color set up its approach for rapid experimentation, including testing the performance of GPT-4 and GPT-4o in complex tasks such as extracting information from PDFs of clinical guidelines for cancer diagnosis. These PDFs are often hundreds of pages of complicated diagrams that outline care paths based on diagnostic workup. Together, OpenAI and Color developed a method of asking GPT-4 Vision to describe screenshots of these diagrams that was most effective in maintaining output accuracy.

OpenAI also helped guide the Color team to prototype clinical workflows using the standard ChatGPT interface and generate sample cases using a custom GPT — gaining effective proof of concept before committing to extensive engineering resources.

With OpenAI's expert guidance, powerful models, and HIPAA-compliant data protection standards, Color was able to focus on deconstructing complex medical decision-making, refining prompts, and designing clinician-in-the-loop workflows to create the initial version of the copilot.

For example, OpenAI engineers guided Color to use retrieval-augmented generation (RAG) instead of model fine-tuning to increase output quality and rewrite clinical documentation for ChatGPT to process it more easily. Ultimately, after experimenting, Color selected OpenAI as its AI solutions provider, with GPT-4o at the core of its cutting-edge copilot application.

Testing the new tool

To measure the impact of this tool, Color is partnering with the University of California, San Francisco Helen Diller Family Comprehensive Cancer Center (UCSF HDFCCC). For the initial implementation, Color and UCSF will conduct a retrospective evaluation, followed by a targeted rollout. Based on the evaluation, the potential of the copilot to be integrated into clinical workflows for all new cancer cases at UCSF is high.

Color is taking a measured approach in rolling out the copilot, and has started an initial phase-in for its own clinicians, applying the tool to a limited number of cases. These cases receive several layers of quality assurance:

  • Healthcare providers using the copilot are able to identify 4x more missing labs, imaging, or biopsy and pathology results than those without the copilot.
  • Using the copilot, it takes on average 5 minutes for clinicians to analyze patient records and identify gaps. Without the copilot, data is fragmented and can lead to weeks of delay.

Through the second half of 2024, Color intends to use the copilot application to provide AI-generated personalized care plans, with physician oversight, for over 200,000 patients.

On the record

"Color's vision is to make cancer expertise accessible at the point and time when it can have the greatest impact on a patient's healthcare decisions," said Othman Laraki, CEO of Color Health. "As a healthcare company, technology that improves access and equity has to go hand-in-hand with technology that supports patient safety and privacy. OpenAI's HIPAA-compliant data protection standards are key."

"UCSF is a leader in implementing cutting-edge technology to improve patient care," said Dr. Alan Ashworth, PhD, FRS, President of the UCSF HDFCCC. "Patients frequently come to primary oncologists with incomplete diagnostic workups, and the time it takes to collate and accurately identify the completion of those workups prevents providers from working at the top of their license. We are interested in tools that can improve the efficiency and accuracy of pre-visit charting and avoid costly delays in treatment initiation for cancer patients at UCSF."

"The idea of combining AI technologies with digitally-enabled clinical workflows to expedite that process would be a positive advancement for all parties involved - the patient and their clinicians, as well as the payer covering the cost of treatment," added Dr. Karen Knudsen, CEO of the American Cancer Society.

The context

Color Health has been working to improve access to healthcare for a decade, serving more than 7 million patients since it was founded. In 2023, it partnered with the American Cancer Society to help employers and health plans take control of cancer — the second most common cause of death in the United States and the leading driver of American healthcare costs.


💡Did you know?

You can take your DHArab experience to the next level with our Premium Membership.
👉 Click here to learn more