The Hard Parts of AI-Assisted Science Are Not Technical

The Allen Institute for AI has published a recap of an August 27 workshop it hosted with clinicians and researchers from the Paul G. Allen Research Center at Providence Swedish Cancer Institute, marking an expansion of the two organizations' collaboration. The write-up is unusual for a corporate research blog in that it spends most of its length on obstacles rather than capabilities. The five it names are worth reading closely, because they describe where automated discovery tends to fail in practice rather than where it demonstrates well.

Who was in the room

The session brought together oncologists and computational researchers rather than an audience of AI specialists. Kelly Paulson, who leads the Center for Immuno-Oncology and serves as medical director for melanoma and cutaneous oncology, participated alongside Sasha Stanton, a medical breast oncologist who leads the Cancer Immunoprevention Laboratory. From the computational side, Ai2 senior research scientist Bodhisattwa Prasad Majumder joined Kyle Travaglini of the Allen Institute for Brain Health Accelerator, Abraham Flaxman of the University of Washington's health metrics sciences faculty, Stephen Salerno of the WashU Bursky School of Public Health, and Hoifung Poon, chief AI officer at Recursion.

The system under discussion was AutoDiscovery, an Ai2 tool demonstrated during the workshop that surfaces candidate hypotheses from data. The recap reports that in use, distinguishing its useful output from its useless output required domain expertise — the tool produced findings that were statistically surprising but biologically nonsensical, a category only a specialist can reliably identify.

That observation frames everything else in the write-up.

Five obstacles

Scientific taste. Majumder introduced the term to describe the judgment that separates a trivial result from a meaningful one. This is the least tractable item on the list and arguably the most important. A statistical discovery engine optimizes for something measurable — correlation strength, effect size, unexpectedness relative to a null model. None of those correspond to scientific significance. A researcher who has spent fifteen years on tumor immunology knows within seconds that a particular association is an artifact of how the samples were collected. Encoding that into a system is not a data problem; it is closer to an apprenticeship problem, and nobody has a clean method for it.

Steering. Research changes direction. New evidence arrives, a hypothesis collapses, a collaborator's result reframes the question. The workshop participants identified the need for systems that can absorb such shifts without starting over or being retrained from scratch. This is a real architectural gap. Most current discovery pipelines are configured once and run; the interactive loop where a scientist says "not that direction, this one" and the system genuinely reorients is not a solved interface problem.

What to delegate. Poon drew a line between productivity gains and creativity gains. Automating routine tasks produces the first kind, and the cost of an error is usually recoverable. Generating novel hypotheses produces the second kind, and there the output cannot simply be accepted — it has to be verified, which consumes the expert time the automation was supposed to save. This is the economics that determines whether AI-assisted science pays for itself. If checking a machine-generated hypothesis takes as long as forming one, the system has moved work rather than reduced it.

Amplification of bad practice. Salerno's framing was that AI acts as an amplifier rather than an equalizer. A rigorous methodology scaled up produces more rigorous results; a flawed set of assumptions scaled up produces more flawed results, faster and with more apparent authority. This inverts a common assumption about AI in research — that it democratizes access to sophisticated analysis. It may instead widen the gap between groups with strong methodological culture and groups without one, because the tool multiplies whatever it is pointed at.

Integration with the bench. Travaglini and Poon discussed connecting systems directly to laboratory instruments and building "virtual tissue" models as intermediate steps toward broader biological simulation. This is the most concrete engineering agenda in the recap and also the furthest from delivery. Closing the loop between a computational hypothesis and a physical experiment is where AI-assisted discovery stops being literature synthesis and starts being science, but it requires instrument integration, experimental design, and validation infrastructure that mostly does not exist yet.

Why this framing is useful

Paulson's summary line from the session — that this is research, not search, and that discovery has to be paired with verification — is the throughline. The distinction matters because the two activities have different failure modes. A search engine that returns a bad result wastes a minute. A discovery engine that returns a plausible bad hypothesis can consume months of laboratory time and a grant cycle before anyone notices.

The recap is a discussion summary, not a study. It reports no benchmark numbers, no trial, no measurement of whether AutoDiscovery improved anyone's research throughput. Its value is as a field report on where practitioners think the bottlenecks sit, and the honest read is that the bottlenecks named are largely not model-capability bottlenecks. Better models will not confer scientific taste. They will not automatically make verification cheaper. They will, if Salerno is right, make methodological weakness more consequential rather than less.

That is a less comfortable message than the one usually attached to AI-for-science announcements, and it is more consistent with how the field has actually progressed. The tools that have stuck in computational biology — sequence alignment, structure prediction, image segmentation — succeeded because they automated a well-specified task with a verifiable output. Hypothesis generation has neither property. The output is open-ended and verification is expensive, which is precisely the combination the workshop participants identified as difficult.

The collaboration between Ai2 and Providence Swedish is structured to put the tools in front of practicing oncologists rather than benchmark curators, which is the right test. Whether it produces anything beyond a better-calibrated understanding of the limits is the open question.

What to watch

• Whether AutoDiscovery or similar tools produce a hypothesis that is independently validated at the bench, which is the only result that would settle the productivity-versus-creativity question.
• Attempts to formalize "scientific taste" — whether through expert preference data, domain constraints, or human-in-the-loop ranking — and whether any of them generalize beyond a single subfield.
• Progress on instrument integration and virtual tissue models, the most falsifiable item on the list.
• Whether the amplification concern shows up empirically, for example as a rise in retracted or non-replicating findings from AI-assisted pipelines.
• How the Ai2 and Providence Swedish collaboration reports results going forward — whether in measurable research outcomes or only in tool releases.