Building Trust in Real-World Evidence With AI and High-Quality Data

Artificial intelligence (AI) is rapidly transforming oncology research, but its impact depends on high-quality data. As researchers increasingly rely on real-world evidence (RWE) to complement clinical trials, ensuring that data is accurate, complete, and transparent has become just as important as developing new AI tools. In this episode of Exploring AI in Oncology, Waqas Haque, MD, MPH, Hematology/Oncology Fellow at the University of Chicago, speaks with Saamir Pasha, Senior Biostatistician at Ontada, about two complementary areas of innovation—the establishment of rigorous frameworks to assess the quality of oncology real-world data, and the exploration into how generative AI can accelerate complex biostatistical workflows while maintaining scientific rigor and regulatory transparency. Together, these efforts illustrate how AI is poised to enhance—not replace—the expertise required to generate reliable evidence in oncology.

Why Data Quality Comes Before AI

Cancer care generates enormous amounts of clinical data, but assembling that information into a research-ready dataset is far from straightforward. Oncology encompasses hundreds of cancer types, diverse patient populations, evolving treatment strategies, biomarker-driven therapies, genomic testing, pathology reports, imaging, and longitudinal treatment histories. Much of this information exists across multiple healthcare systems and is often split between structured electronic health record (EHR) fields and unstructured clinical notes.

"The challenge isn't simply collecting data," Mr. Pasha explains. "It's making sure the information is complete, traceable, and fit for the research question you're trying to answer." This complexity has led regulators to develop formal frameworks for evaluating real-world data quality. Ontada's work builds upon initiatives from both the FDA and the European Medicines Agency (EMA) to create a comprehensive approach for assessing oncology datasets before they are used in research.

Three Pillars of Reliable Real-World Evidence

The framework is centered on three core dimensions. First is relevance—whether the data actually contains the information researchers need. Ontada's oncology platform includes more than 250 standardized variables spanning over 70 cancer types, with additional information extracted from pathology reports, genomic sequencing results, and physician documentation. Just as importantly, the dataset represents approximately four million patients treated across roughly 700 community oncology clinics throughout the United States, providing broad representation for many research questions.

Second is reliability. Even extensive datasets are only valuable if researchers can trust them. Reliability involves several critical elements, including:

• Traceability from the original clinical source through the final analytic dataset

• Documentation of how information is harmonized across multiple data sources

• Clear provenance showing who handled or reviewed the data

• Completeness of key clinical variables such as medications, diagnoses, and treatments

Maintaining an audit trail throughout the data pipeline helps ensure that analyses can be reproduced and validated.

Third is external validity. Researchers must determine whether findings generalize beyond the dataset itself. As part of its validation efforts, Ontada compared pancreatic cancer overall survival estimates derived from its real-world data against the National Death Index, considered the gold standard for mortality information. Kaplan-Meier survival analyses produced nearly identical results, supporting the reliability of the dataset for survival research. These validation exercises strengthen confidence that real-world findings can meaningfully inform clinical research and regulatory decision-making.

Bringing Generative AI into Biostatistical Workflows 

While AI has attracted considerable attention for applications such as medical imaging and clinical documentation, Mr. Pasha sees another promising opportunity: accelerating the complex statistical workflows required to emulate clinical trials using observational data.

Trial emulation is a resource-intensive process that requires researchers to translate clinical trial protocols into real-world datasets, identify appropriate patient cohorts, define endpoints, write statistical code, perform analyses, and generate reports. Generative AI can assist throughout this process. Rather than manually translating every inclusion criterion into database queries, researchers can provide AI with structured context, including the original clinical trial protocol, information about the underlying database, data dictionaries, and study objectives. The AI then generates an initial cohort definition and analytic framework that researchers refine and validate. "It gives us a strong starting point instead of building everything from scratch," Mr. Pasha says.

Human Oversight Remains Essential 

Importantly, Ontada is not pursuing fully autonomous AI. Instead, researchers maintain continuous oversight throughout the workflow. One strategy involves creating progressive summaries at major milestones. At each stage, the AI documents, what tasks have been completed, key decisions that were made, challenges encountered, and remaining uncertainties. These checkpoints allow biostatisticians to verify that the AI's reasoning remains scientifically appropriate before moving forward.

Human expertise also becomes essential when AI encounters limitations in the available data. For example, some clinical trial endpoints, such as adverse events, are often incompletely captured in routine EHR data. Rather than forcing an inappropriate analysis, researchers evaluate alternative endpoints and explicitly document these limitations. Transparency, Mr. Pasha emphasizes, is preferable to overstating what the data can support.

Reducing Bias Through Structured Validation

AI can accelerate analyses, but it cannot eliminate the need for careful epidemiologic thinking.

Before conducting any study, researchers establish expectations regarding patient demographics, disease characteristics, and clinical variables. They then compare the AI-generated cohorts against these predefined benchmarks to identify unexpected imbalances or implausible results. This process helps ensure that biases are detected early rather than inadvertently propagated through downstream analyses. Researchers also remain transparent about known limitations of real-world datasets, acknowledging where certain outcomes cannot be measured reliably.

Regulatory Transparency Will Shape AI Adoption 

As AI becomes increasingly integrated into evidence generation, regulatory agencies are placing growing emphasis on transparency. Rather than viewing AI outputs as a "black box," regulators want researchers to document which AI models were used, how prompts were structured, what decisions were made by AI versus human investigators, and how outputs were reviewed and validated.

Maintaining structured prompting strategies and documenting decision making throughout the workflow helps satisfy these expectations while making analyses easier to reproduce. According to Mr. Pasha, transparency is ultimately more important than whether AI was involved. "Having a structured prompting for each component of your analysis is key so that you can trace those decisions that were made and show the thought process that you took. Along with the AI, that helps us steer those conversations into something more productive.”

Looking Ahead: AI as an Accelerator, Not a Replacement

Clinicians often imagine a future where they can simply ask an EHR a clinical question and instantly receive reliable, trial-quality evidence. While that vision remains some distance away, Mr. Pasha believes that AI will steadily move oncology research closer to that goal. Today's research workflows involve substantial manual effort, from linking disparate data sources to writing code and validating outputs. As AI learns these repetitive tasks, researchers will spend less time on implementation and more time interpreting results, designing better studies, and answering clinically meaningful questions. 

Equally important, AI enables organizations to preserve institutional knowledge. Instead of rebuilding workflows for each new project, teams can reuse validated approaches across disease areas and study designs, improving consistency while accelerating future analyses. "The work becomes less about starting from scratch every time," Mr. Pasha says. "We're building on what we've already learned." 

Key Takeaways 

• The future of AI in oncology extends well beyond clinical decision support.

• AI has the potential to transform how researchers generate real-world evidence by streamlining complex statistical workflows while reinforcing—not replacing—the scientific principles of transparency, validation, and rigorous data quality.

• High-quality real-world data provides the foundation, human expertise provides the oversight, and AI provides the acceleration.

• Together, these three elements may enable oncology researchers to deliver reliable evidence more quickly, ultimately helping translate real-world clinical experience into insights that improve cancer care.

About the Speakers

Waqas Haque, MD, MPH, is a second-year Hematology/Oncology Fellow at the University of Chicago. He recently completed his Internal Medicine Residency at New York University (NYU) in a Clinical Investigator Track. Dr. Haque has balanced his patient care work with a variety of research projects. During his fellowship training at the University of Chicago, he plans to further his work in innovative clinical trial design, value-based care delivery to cancer patients, and clinical investigation.

Saamir Pasha, MPH, is a Senior Biostatistician and Data Scientist at Ontada. He has extensive experience across various healthcare and research organizations, specializing in real-world studies, study protocol preparation, and statistical analysis plans. His previous roles include Senior Data Scientist at Tempus Labs, where he coordinated epidemiologic studies and led data quality monitoring, and Research Associate II at IQVIA, where he focused on COVID-19 surveillance for major sports leagues. He holds a Master of Public Health in Epidemiology from the University of Toronto.