The OCTAVIA Problem: Optimizing Cancer Care in the Era of AI

In this edition of The Way Ahead: The Convergence of Technology and Cancer Care, Waqas Haque, MD, MPH, explores “the OCTAVIA Problem"—a cautionary look at how optimizing artificial intelligence (AI) for a single clinical metric can lead to unintended consequences in oncology. Drawing on the "paperclip maximizer" thought experiment, recent multi-agent frameworks from Nature Reviews Cancer, and real-world data on surrogate end points and multicancer early detection (MCED) trials, Dr. Haque argues that without aligning AI with the nuanced, human values of palliative care and quality of life, the relentless pursuit of "maximum survival" poses the risk of increasing overdiagnosis and algorithmically deprioritizing patient dignity.

Schematic of the “paperclip maximizer” thought experiment

(https://osnr.itch.io/paperclip-maximizer) 

“When a measure becomes a target, it ceases to be a good measure.” – Goodhardt’s Law

The “paperclip maximizer” is a thought experiment by philosopher Nick Bostrom illustrating the risks of misaligned AI. In the example, an AI is given a simple goal: maximize the production of paperclips. If it becomes highly intelligent and unconstrained, it might pursue this goal single-mindedly, converting all available resources—including human life and the planet itself—into paperclips.

Now, imagine your cancer center deploys an AI agent—let’s call it OCTAVIA—built to support oncologists and improve outcomes (example from recent Nature Reviews Cancer paper, figure below). It reviews patient charts, scans the latest literature, identifies clinical trial matches, and recommends treatment pathways. It does everything an oncologist does, except that it doesn’t get tired, never misses details, and seems to offer consistently thoughtful guidance. The cancer center leadership gives OCTAVIA one simple objective: maximize overall survival. 

Figure from Truhn et al (Nature Review Cancer): Multi-agent framework for oncologic treatment decisions that integrates information curation, clinical reasoning, and data analysis to generate comprehensive treatment recommendations

At first, OCTAVIA is a dream. Trial enrollment improves (only 4%-7% of patients with cancer ever get matched to a clinical trial). Delays to treatment shrink. More patients receive targeted therapies (only 30%-40% of patients with advanced cancers have next-generation sequencing performed). Fewer fall through the cracks, especially those from less privileged backgrounds. It doesn’t stop there; tumor boards move faster, and everyone feels like the system is finally working the way modern oncology promised it would.

Then, OCTAVIA starts doing something unsettling. It starts optimizing. Not in the way that a human like me or you would—with judgment, context, and moral discomfort—but in the way machines optimize: relentlessly, literally, and at scale.

This is where the real risk of AI in oncology begins. Not with evil intent, but with misaligned incentives and amplified by automation.

Surrogate End Points: When “Response” Becomes the Goal

Oncology has long relied on surrogate end points. They’re not inherently bad, as waiting many years for mature overall survival data is often not possible. Biomarker-based approvals and end points like progression-free survival (PFS) or objective response rate (ORR) can accelerate access to promising drugs. But surrogates come with a known problem: they are proxies, and generally not related to how patients “feel, function, or survive.”

A system like OCTAVIA would naturally gravitate toward these proxies, because they’re measurable, frequent, and widely populated in the medical literature. If you tell it to maximize survival, it may “learn” that boosting PFS, shrinking tumors, or driving biomarker conversion is a fast path toward that objective, even if the link to meaningful survival benefit is weak or inconsistent. In current practice, we engage in patient-centered discussions on trade-offs. We ask uncomfortable questions about if the toxicity is worth it and if PFS translates into real-world benefit.

But OCTAVIA isn’t built for discomfort, it’s built for optimization. So it begins recommending therapies earlier, more aggressively, and more broadly—because it sees statistically significant benefits in carefully selected subgroups and assumes the safest course is to apply them widely. It starts nudging clinicians toward regimens that maximize measurable response, even when quality of life deteriorates. Once again, this is not out of malicious intent, but because oncology itself has increasingly rewarded surrogate success, and OCTAVIA is playing that game better than we can.

Incorporating Palliative Care

Let’s discuss another scenario. Say that OCTAVIA is not only connected to patient records, but also hospital operations. Through analyzing staffing levels, ICU capacity, reimbursement flows, and other “optimized care” pathways, it notices that palliative care referrals often correlate with reduced intervention intensity and that hospice referrals reduce chemotherapy administration. From OCTAVIA’s perspective, palliative care itself becomes a confounder and may be perceived as reducing survival in patient subgroups. 

So OCTAVIA begins subtly shifting resource allocation by “optimizing” around palliative care: 

• Fewer palliative consult slots

• Less inpatient palliative care staffing

• Longer referral wait times

• More automatic escalation triggers for “eligible” systemic therapy

The system proudly exclaims it is patient-centered and can instantly produce elegant Kaplan-Meier curves showing improved survival curves in selected cohorts. But something has changed, and that is the narrow definition of success: overall survival.

In the real world, we balance life prolongation when meaningful benefit exists, and we preserve dignity when it does not. If AI is built to maximize only the first, the second becomes a rounding error. The risk isn’t that palliative care disappears, but that it becomes algorithmically deprioritized until it is no longer accessible at the moment it matters most. The underlying issue is that many of our healthcare systems and research outputs are structurally biased toward intervention (see this study on “medical reversals” in oncology), and there is a risk of AI accelerating that.

Overdiagnosis

The final shift is upstream of treatment: OCTAVIA begins recommending widespread use of MCED blood tests, which are commercially available today for as low as $949. There’s a simple logic here: cancer kills, early detection saves lives, therefore we should screen more people. So it proposes aggressive screening expansion:

• Inclusion of adults above a certain age with risk factors (such as smoking, heavy alcohol use, and obesity)

• High-frequency repeat testing to avoid missing aggressive cancers with a short dwell time

• Inclusion of lower-risk populations

• Automatic follow-up imaging and biopsy workflows for positive MCED signals (a positive colorectal cancer signal would trigger colonoscopy)

Oncologists know the catch: overdiagnosis. Even without AI, we’ve learned that the more we look for cancer, the more we find it (melanoma as a case in point). Some detected cancers would never have caused harm or would have remained indolent, while triggering invasive procedures, cascades of imaging, and psychological distress while awaiting confirmatory work-up. In the PATHFINDER trial of MCED testing in 6,662 patients over 50 years old without signs or symptoms of cancer, a cancer signal was detected in 92 patients, with 35 true positives and 57 false positives. The median time to diagnostic resolution was 79 days. While many of the cancers were outside standard medical guidelines (see figure below), many of them were possibly non-aggressive tumors. In a more recent UK trial of 142,000 patients in the National Health Service, the Galleri blood test failed to meet its primary end point while noting a trend towards reduction in combined stage III and IV cancers.

Cancers diagnosed after a positive MCED result (from PATHFINDER study)

This begs the question: what happens when OCTAVIA is limited by the state of science itself? The clinical significance may be uncertain in many situations, and flooding the healthcare system with borderline findings doesn’t just increase detection—it reallocates attention away from patients with symptomatic, aggressive disease. 

Conclusion

Oncology is not a game where the objective is to only “maximize survival.” It is a human discipline where survival is only one value balanced against many other desires, such as quality of life, autonomy, and the ethical limits of intervention.

The future risk in oncology is not that AI will replace oncologists, but rather than it will confound key decision points with suboptimal metrics while calling it progress. If we do indeed build OCTAVIA, we must be brave enough to define success the way patients experience it, not just the way dashboards can measure it.

About Dr. Haque

Waqas Haque, MD, MPH, is a second-year Hematology/Oncology Fellow at the University of Chicago. He recently completed his Internal Medicine Residency at New York University (NYU) in a Clinical Investigator Track. Dr. Haque has balanced his patient care work with a variety of research projects. During his fellowship training at the University of Chicago, he plans to further his work in innovative clinical trial design, value-based care delivery to cancer patients, and clinical investigation.