The Immortal Killifish Problem
Why scientific AI needs dynamic objectives
I recently asked Opus to “cure aging.” Disappointingly, it did not succeed. If only Fable hadn’t rejected the prompt.
This was not entirely fair. It is roughly akin to walking into a room of physicists and asking them to “solve physics,” then complaining when nobody hands over a theory of quantum gravity. Nevertheless, Opus had an answer:
The most promising path to curing aging may be an automated fish farm.
The fish were killifish, small vertebrates with conveniently short lifespans. Opus proposed automating their husbandry, imaging, and phenotyping so researchers could quickly test aging interventions against functional outcomes. It considered senolytics, partial reprogramming, and aging clocks, but decided that the primary bottleneck was not a lack of interventions. It was a lack of fast, reliable ways to know whether those interventions worked.
This is an excellent answer to my stupid prompt.
Instead of proposing yet another drug, the model targeted the experimental loop itself. Accelerating that loop can be tremendously valuable.
Unfortunately, fish remain fish.
A facility capable of measuring every aspect of a killifish could transform aging research without, by itself, curing human aging. It operationalizes one valuable slice of a much larger purpose.
“Cure aging” is not a particularly coherent research task. To act on it, we must translate it into narrower, measurable objectives. Each translation embeds a hypothesis: that progress on the objective will advance the larger purpose. The larger the scientific ambition, the harder it is to choose a measurable objective that remains coupled to it under optimization. A fixed-objective AI will continue optimizing that proxy even after gains stop transferring to the goal.
This essay is about building systems that pursue a purpose without mistaking their initial proxy for the world itself.
Beating the Wrong Objective
For decades, cardiologists had observed that patients with frequent premature ventricular beats after a heart attack were more likely to die suddenly. The causal story was persuasive: damaged cardiac tissue produced abnormal electrical activity, which might trigger a lethal rhythm. Suppress that activity, and more patients should survive. Simple enough.
Encainide and flecainide were remarkably good at this task. The Cardiac Arrhythmia Suppression Trial (CAST)1 went further: it enrolled only patients whose ventricular ectopy could first be suppressed by the drugs. This was a perfect setup: a measurable risk marker, a plausible mechanism, and an intervention that moved the marker in the desired direction.
Then the patients taking the drugs began dying.
After an average of ten months, 43 patients receiving encainide or flecainide had died from arrhythmia, compared with 16 receiving placebo. The treatment arms were stopped early. This, as far as trials go, is a very bad result.
CAST is not a story about foolish cardiologists. The evidence was suggestive, the mechanism plausible, and the drugs genuinely optimized the chosen measure. It is a story about a proxy moving in precisely the desired direction while the outcome it represented moved catastrophically in the opposite direction.
Are proxies useless, then?
Of course not. Proxies buy iterations. Testing every treatment against mortality is impossibly slow, not to mention dangerous. A proxy filters the search before we commit to testing the outcome we ultimately care about. But choosing one is itself a hypothesis, and in CAST that hypothesis failed.
Science is full of useful proxies. Reporter fluorescence stands in for gene expression. Animal phenotypes — or, more recently, organoids and human cell models — provide early evidence about possible human effects. Agreement with experimentally determined structures gave protein-prediction methods a common target and helped organize progress toward AlphaFold. These reductions make thousands of cheap iterations possible before a few expensive ideas are tested against the outcomes we ultimately care about.
The important question, therefore, is not whether to use proxies. It is how much authority to give them. How closely are they coupled to the purpose? What do they obscure? How quickly can we detect when they diverge? And can new evidence cause the system to replace them?
AI systems are becoming extraordinarily good at optimizing proxies. What they cannot guarantee is that the underlying hypothesis connecting a proxy to its purpose is true. A perfect optimizer could have ranked encainide and flecainide as spectacular candidates: they genuinely suppressed ventricular ectopy. Better optimization would not have repaired the mismatch.
Fixed evaluators create bounded worlds. An AI may discover astonishing things inside them, but the score cannot value what it was never built to see. Science extends beyond that world.
The danger is not that the optimizer fails. It’s that it beats the wrong objective.
A Grand World
Leonard Savage, one of the founders of modern decision theory, distinguished between the grand world of lived reality and the small worlds we construct to reason about it2.
“Small” was an unfortunate word choice. What he meant was not easy, or small in absolute size. It is closer to closure.
Go is a small world. It has more possible board states than atoms in the universe, and by golly, it’s hard (I just started learning). But it is closed. The board has edges, the rules are fixed, and the win condition is unambiguous. Every relevant event happens inside the game.
AI loves closure. It doesn’t need to worry about proxies, and it receives cheap, perfect feedback. Many tasks in games, programming, and math provide rich environments for this: vast search space, fixed rules, and a resulting score matching what we care about.
Science has basically none of that.
Science exists in the grand world. The relevant variables are not supplied in advance. New instruments reveal phenomena that our previous representations could not express. Measurements are partial and noisy. Science is hard in a very different way from games, coding, or math.
To compute over science, we must manufacture closure. We need to carve small world slices out of the grand world.
We decide what to measure, how to encode it, which interventions are available, and which outputs count as progress. We wrap all of this in a model and ask it to start searching. This box we use to enclose the scientific world allows us to make scalable computational progress. It is also not the world.
Stuffing Reality into a Box
Some scientific boxes can contain much of the reality relevant to their task. Take optimizing a reaction’s yield in a controlled reactor. You can specify practically every input that counts and measure the target directly (though chemistry retains the right to surprise us).
Let’s move to proteins. AlphaFold carved out a narrow but extraordinarily useful problem: given an amino-acid sequence, predict an experimentally determined three-dimensional structure3. Dynamics, conformational ensembles, binding partners, cellular context, and function remained partly outside its box. The model did not need to contain all of protein biology to transform it.
Move from proteins to a cell and the possible omissions explode. A cell coordinates transcription, translation, metabolism, signaling, mechanics, electrical gradients, and spatial organization across time. It also responds to its environment and to other cells, which are inconsiderate enough to have their own internal states.
This provides us with a useful principle:
The more of reality a model claims to stand in for, the more ways an omitted variable can change what its score means.
Consider virtual cells. One prominent approach effectively represents a cell as a bag of RNA. Not a spatially organized bag, either — just a very large collection of transcript counts.
This is partly a scientific bet about how to best represent a cell and partly a consequence of what we can measure at scale. RNA is an informative readout of cellular state, but it is also one for which we possess enormous high-throughput datasets. Both mattered.
The most prominent task these models perform is this: given a transcriptomic state and a perturbation — usually genetic — predict the resulting transcriptomic state. The Arc Institute’s 2025 Virtual Cell Challenge4 standardized this into a benchmark the field can converge on.
Carving out this slice may produce an AlphaFold moment for virtual cells. Accurate perturbation prediction could help prioritize experiments, reveal regulatory relationships, and search enormous intervention spaces before entering the lab. This is all genuinely exciting.
However, at the end of the day, it is a model of transcriptomic response, not a cell. Every model reflects a series of choices:
Measurement: Single-cell RNA sequencing using current sequencers.
Representation: Primarily encoded as counts.
Action: Primarily genetic perturbations.
Evaluation: Accuracy of transcriptomic count predictions.
These choices are what make the model optimizable. They also bound the possible improvement.
Perhaps the measurements are insufficient, and we should add proteomics and spatial data. Perhaps the representation is poor. Perhaps we should expand the action space to multi-gene perturbations, drug interventions, or bioelectrical stimulation. Or perhaps transcriptomic state prediction is simply a bad proxy for what we care about, and we need to change the evaluator itself.
When progress fails to transfer, we have two options: optimize more within the box, or look inside a different box. Current AI systems almost always choose the first. I don't disagree that optimization within the box is critically important. But I claim that to dramatically accelerate science, a system must also be capable of recognizing the second.
Up, Down, and All Around
Let’s return to my extremely reasonable request to “cure aging”:
As we descend, objectives become narrower, feedback becomes faster, and optimization gradients become stronger. Each node is an operationalization of the goal; each arrow is a transfer hypothesis that may fail. At the top is the purpose. At the bottom is something we can actually optimize.
Most optimization occurs near the bottom. But the transfer hypotheses connecting it to the larger purpose must be tested periodically. When gains stop transferring upward, the system must revise the path rather than optimize the proxy harder. The CAST researchers did not look at rising mortality and say “we really need a better arrhythmia suppressor.”
And as Opus showed, the movement is not only up and down. Opus made a sideways move: rather than working on an intervention, it proposed accelerating the rate of discovery. Another system might search for senolytics instead of reprogramming factors, or target a different functional endpoint entirely.
Our initial goal generates a web of possible operationalizations. A scientific AI must learn to crawl through that web.
The Web is Under Construction
Unfortunately, science does not make the web easy to build. It is full of holes. Not just holes — disconnected strands, floating fibers, and sections that snap the moment you put weight on them. Even a perfect navigator cannot reach a destination that no strand connects to.
This is where basic science enters.
Everything discussed so far has concerned goal-directed research5: begin with a purpose, operationalize it, and search for a path. But the scientific web is not merely difficult to navigate. It is incomplete. Our models are small worlds carved from the grand one, and the grand world will always contain phenomena that our current web cannot represent.
In the 1960s, Thomas Brock and Hudson Freeze were studying a basic question: can organisms survive in extreme heat? Their curiosity — and stubbornness — led them to discover Thermus aquaticus, a bacterium living in Yellowstone’s hot springs6.
Huge discovery. Time to rejoice!
Actually, it turns out that very few people care about the dwellings of bacteria. For more than a decade, this remained a fairly obscure discovery.
Years later, polymerase chain reaction (PCR) encountered a problem. Each amplification cycle required heating DNA to separate the strands, but that heat also destroyed the polymerase, making the process unscalable and uneconomical. But every organism needs polymerases. Aha! An organism that reproduces in extreme heat should possess one capable of surviving extreme heat.
That observation led researchers to the DNA polymerase from T. aquaticus — Taq polymerase — which helped make PCR practical and automatable, transforming molecular biology.
Modern science is full of discoveries that follow this pattern. Basic research finds a strange organism, mechanism, or physical effect without knowing what future problem it will solve. Years later, somebody discovers where the dangling strand connects7.
But we can also run the connection in reverse, reasoning outward from a bottleneck:
Goal: Automate PCR and make it scalable.
Bottleneck: Polymerase is destroyed during heating.
Missing Capability: A heat-resistant polymerase.
Empirical Test: Measure polymerase activity and fidelity after repeated thermal cycles.
Source Hypothesis: Organisms adapted to extreme heat.
I call this a frontier assay: an empirical test for a capability relevant to some task that no known candidates satisfy. This is not equivalent to basic science, which adds unexpected elements to our web. Frontier assays let us deliberately spin a missing strand once we know where we want to go.
Generating an assay is only the beginning. There will be many — we have a lot of holes — and infinitely many capabilities we’d love to exist. The system must prioritize which gaps justify exploration, weighing time, cost, importance, and probability of success.
That leaves an allocation question I’ll return to: how should a scientific system divide effort between extending the web and optimizing its navigation of the strands that already exist?
Architectural Sketch
Most scientific AI systems follow a simple paradigm:
Objective → Candidates → Experiments → Score → Update ↺
This loop is extraordinarily powerful. Closing it tightly enough, it produces systems that are superb at optimizing their assigned objectives. We should build these systems wherever we can. Call this the inner loop.
In its standard form, though, the inner loop contains no mechanism for revising the objective itself. It can search for better AAVs without asking whether AAVs remain the right delivery platform. In science, where every tractable model is carved from the grand world, we also need a system that can reconsider the small world it is optimizing.
That is the outer loop:
Frame → Optimize → Audit → Keep or Revise → Inner Loop
Stitched together:
Our system is built!
Well, in roughly the same sense that drawing a rectangle labeled FUSION REACTOR solves the energy crisis. The devil is in the details.
A complete specification deserves its own paper and would be a major research project in its own right. Many of the components exist individually, but we cannot yet build a reliable outer loop and frame revision system. What follows is a sketch of the main capabilities and components such a system would require.
Frames
First, the system begins with a user goal: say, create a safe, durable, redosable method for delivering a gene therapy payload to human heart tissue.
It should not attempt to map every conceivable research path. Instead it maintains a small working graph of active programs and plausible alternatives, expanding only when evidence warrants.
Each active program has an explicit frame: what the system measures, how those measurements are represented, which interventions it can perform, its current model of the relevant world, and the evaluator it uses to recognize progress.
Nodes contain frames. The arrows between them are transfer hypotheses: claims that improving one objective will advance the objective above it. Each arrow carries supporting evidence, an uncertainty estimate, a proposed domain of validity, and a plan for testing whether progress actually travels upward.
Evaluators
Evaluator construction is the hardest architectural component.
An evaluator must specify what cases are tested, what outputs are required, how independent reference evidence will be produced, how performance is measured, and — critically — how success on that measure is expected to transfer to the parent purpose. A useful evaluator is measurable, difficult to game, scientifically consequential, and connected upward by a testable hypothesis.
At any point, the system should optimize at the cheapest, fastest level of the chain supported by transfer evidence strong enough for the stakes. That is where the inner loop operates. Scheduled and triggered audits then test whether gains continue to propagate upward. If confidence weakens, the system moves to a higher-fidelity evaluator or revises the proxy.
One close precedent is SAGA8, which uses a bi-level architecture: an outer loop proposes scientific objectives and converts them into executable scoring functions, while an inner loop optimizes candidates against those objectives. The proposal extends this pattern by also constructing or locating independent reference evidence, auditing transfer to the parent purpose, revising other components of the frame, and allocating research across competing frames.
If no adequate evaluator exists for a required capability, the system constructs a frontier assay. That assay adds a new node to the web and becomes the evaluator for a new inner loop searching across organisms, proteins, and engineered variants.
Optimize and Audit
Once a frame and evaluator exist, the inner loop does what it does best: optimize. Most optimization can occur in silico, using existing data and learned surrogates. Empirical experimentation can help resolve particular uncertainties and test specific candidates where no data exist. Experiments may be executed by software, automated equipment, cloud laboratories, or human scientists — what matters is that the data are reliable and comparable across the campaign.
Most of the time, the outer loop should leave a productive campaign alone. Constantly questioning the frame would waste computing resources, laboratory capacity, and optimization attention. But two kinds of audits are needed:
Triggered audits, activated by surrogate prediction divergence, plateaus in inner loop optimization, and diversity collapse.
Scheduled audits, which periodically stress-test transfer even when the current proxy appears healthy.
When an audit finds a problem, the crucial step is diagnosis, because the three failure modes have different remedies:
Insufficient search → Continue the inner loop.
Inadequate frame → Change the frame.
Missing capability → Construct a frontier assay and expand the web.
Allocation
When a frame appears inadequate, the outer loop generates a small number of challengers. Each challenger should make predictions that allow it to be distinguished cheaply from the current frame and from the other alternatives.
Frames need not be accepted or rejected outright. The system can maintain a limited portfolio: productive frames receive most of the resources, while plausible challengers receive enough to generate discriminating evidence. The exact method for allocating resources across that portfolio — bandits, portfolio optimization, and quality-diversity methods9— all supply relevant ingredients, though combining different timescales, costs, and definitions of success remains an open problem.
Memory
Finally, the system needs persistent scientific memory. It must retain not only experimental results, but also negative results, the assumptions behind each arrow, uncertainty, experimental conditions, and the reasons frames were introduced, promoted, or abandoned.
Because negative results and the reasoning behind research decisions are often recorded poorly10, persistent memory may be one of the cheapest available improvements. Store it externally as an evidence graph to start. Continual learning that folds it into weights without erasing prior knowledge is the eventual goal, and is an unsolved problem.
Overview
Many pieces of this architecture already exist separately: active learning, Bayesian optimization, lab automation, causal inference, scientific knowledge graphs, hypothesis-generating agents, and systems for scientific memory11. The unsolved problem is integrating them into a system that can recognize the limits of its current formulation, propose meaningful alternatives, and test whether changing the question produces progress toward the original purpose.
Early versions don’t need to automate the entire outer loop. They may begin as tools that identify suspicious proxy behavior, recommend transfer audits, propose challenger frames, or help scientists preserve the assumptions behind a research program.
The architecture is therefore less a blueprint than a research agenda. Its central claim is not that we already know how to automate scientific judgment. It is that as optimization accelerates, the processes by which science questions its measurements, revises its models, and redirects its search will need to become more explicit, systematic, and scalable.
Auditing the Auditor
An essay about evaluators should not exempt itself from evaluation. Here I present possible objections to this framework and my thoughts about each.
“Aren’t you still just measuring proxies?”
Yes. Many scientific goals cannot be measured directly, cheaply, or quickly. This architecture does not eliminate proxies; it makes them explicit, testable, and provisional.
The system repeatedly audits a cheap working proxy against higher-fidelity evidence closer to the purpose and subject to different failure modes. Each audit can strengthen or weaken the transfer hypothesis, promote a more demanding evaluator, or force revision of the proxy itself.
While never producing certainty, the architecture provides evidence-responsive confidence by being optimized through revision and validated through chained transfer checks.
“You’re going to reward hack your way to success.”
This is perhaps the biggest engineering hurdle to overcome and a legitimate worry. A system allowed to revise its own success criteria may favor revisions that make success easier to demonstrate. If it can influence both the evaluator and the audit, it may manufacture evidence that appears to justify the revision.
The system includes several possible safeguards.
The ultimate purpose and safety constraints should be revisable only through an authorized external process. Evaluator changes and their supporting evidence should remain inspectable, while audits should use blinded holdouts or verification methods distinct from the evaluator. Separating proposal, evaluation, and approval roles can further reduce correlated reward hacking.
None of these safeguards guarantees alignment. Evaluator revision should therefore remain the system’s most constrained, observable, and externally governed capability.
“Scale dissolves the loop distinction.”
The argument essentially goes like this: if we can create a sufficiently capable inner loop, given the ability to change the assay or representation, doesn’t that just produce the outer loop for free? We just optimize over a bigger search space.
A flat optimizer operating over every candidate, representation, assay, and scientific frame would face an enormous heterogeneous search space with sparse and delayed feedback. The proposed architecture factors that search into levels. It supplies local credit assignment, preserves evidence about why frames changed, allocates different budgets to optimization and reframing, and places additional verification around changes to the evaluator itself.
In that sense, the outer loop provides a curriculum. It lets the system make progress inside relatively tractable small worlds while gradually testing whether those worlds remain connected to the larger purpose.
In the long run, a sufficiently general optimizer may internalize these functions, in which case this architecture would become scaffolding. Until then, explicitly separating optimization from frame revision may provide better credit assignment, verification, and control than simply enlarging the search space.
“Humans already do this.”
They do.
Scientists choose measurements, construct assays, question models, abandon failed programs, and redirect research toward better questions. Funders and institutions allocate resources across competing frames through a mixture of evidence, judgment, incentives, and social processes. A system that optimized only the inner loop while leaving these decisions to humans could already be tremendously valuable. Versions of that approach are an active area of research.
The question is whether the human outer loop can continue to operate at the cadence of an increasingly automated inner loop. If candidate generation and experimentation accelerate dramatically, human framing, review, and resource allocation may become bottlenecks.
That outcome is not inevitable. Human institutions can also become faster, more parallel, and better supported by AI. Human judgment contains tacit knowledge, values, and contextual understanding that may be difficult to formalize. The first useful outer-loop systems may therefore advise human scientists rather than replace them: identifying proxy failures, proposing challenger frames, and surfacing experiments that distinguish among them.
Additionally, if scientific framing can be converted into a general, feedback-driven process, it may also improve with scale. That is a hypothesis for the proposed system to test, not a conclusion we can assume in advance.
A Worked Example: Gene Delivery
Sometimes the easiest way to understand a system is to run through an example. Let’s return to our goal of safe, durable, redosable delivery to human heart tissue. One path through the research web might look like this:
The inner loop begins by optimizing AAV capsids in an animal model. It starts in silico, where it generates candidates, predicts their properties, and sends a handful for experimentation. Cardiac expression steadily improves.
Ding! A scheduled transfer audit is here.
A diverse candidate set is now tested more extensively for biodistribution, toxicity, immune response, durability, and repeat dosing. Oh no! The top-performing capsid for expression produces a strong immune response.
The system has discovered a failure in the transfer hypothesis: maximizing expression does not necessarily produce safe, durable delivery. Its confidence in that connection falls. The evaluator is revised, previously tested candidates are rescored, and a different leading candidate returns to the inner loop.
After further optimization, redosing and toxicity remain persistent problems across all AAV candidates. The audit catches this and recommends testing different delivery modalities. The outer loop allocates an exploratory budget to LNPs and EVs, evaluating all three platforms against the same targets.
Suppose an LNP substantially outperforms the alternatives. Evidence supporting the LNP frame raises its fitness, so it receives more experimental resources. LNP optimization becomes the primary inner loop.
Optimization continues to show improvement. We slowly make our way up the chain through progressively more demanding transfer tests: human cardiac cells, organoids, and, eventually, clinical trials. At each stage, the system reevaluates the candidates, advancing them only if they pass the evaluations. Finally, it returns a winner.
Congratulations! We optimized a narrow slice of gene delivery.
But the coolest part isn’t that we found a successful LNP. To me, the best aspect of this system is that it can take its original target of AAVs in an animal model, reformulate its proxy, change its delivery platform, optimize the new system, and demonstrate the results in the target system.
It didn’t just find a better answer; it learned to change the question.
The World Beyond the Pond
Humans currently provide the outer loop in science. We notice when a model has become inadequate, build new assays, collect different data, and redirect research toward better questions. But as inner loop optimization accelerates, this informal process risks becoming the bottleneck. It is also, I suspect, highly fallible.
I’m not proposing to eliminate proxies. I’m saying we must make them provisional. Have a human purpose at the top, aggressively optimize at the bottom, and revise the frame when the grand world refuses to cooperate.
An automated killifish farm may genuinely be a brilliant step toward curing aging. We should build the pond. We should optimize it. But if it eventually produces immortal killifish while teaching us nothing about human longevity, the correct response is not to optimize the fish more aggressively.
It is time to leave the pond.
Savage’s small and grand worlds: The Foundations of Statistics, pp. 84–90.
Arc Institute: Virtual Cell Challenge
This essay doesn’t explore it much, but I went down a rabbit hole in artificial life and open-endedness research. Here are two papers to inspire you to start your journey into this world!
Artificial Life: Flow Lenia and open-ended evolution
Open-ended RL: Enhanced POET
Quality Diversity Algorithms: A New Frontier for Evolutionary Computation
No documentation of negative results is probably the most frequent frustration I encounter that feels as though there should be an easy fix. Well, here’s a possible one.
A paper similar in flavor to this essay on the need to give epistemic autonomy in autonomous science: Socratic Agents
AI-use Disclosure:
Portions of this text have been drafted with AI. The core idea, structure, and examples are my own.
Brilliant piece, Ben. Your critique of the "bag of RNA" proxy is the raw nerve.
If we expect an AI to execute your Outer Loop frame revision, it cannot rely on discrete, dead-cell snapshots. It needs the continuous physical derivatives (dx/dt).
I’m currently building the hardware evaluator for this. Imagine localized edge-compute boxes deployed into wet-labs to ingest, for example, electrophysiology and optical data simultaneously. By hardware-syncing these streams at the Linux kernel level and feeding them directly into continuous-time State Space Models natively on the bench, we stop forcing continuous biology into discrete AI architectures.
I recently open-sourced the infrastructural requirements to make this happen via the Biology Data Center (BDC) Standards: https://github.com/continuity-labs-io/Biology-Data-Center-Standards I see this edge pipeline as the literal, physical "Frontier Assay" your essay is calling for.
AI is annoyingly adept at p-hacking proxy metrics. You ask it to reason through a chess game, it might write a chess engine in python. You ask it why tests are failing and it re-writes the tests. Always trying to get a little peak at the answer sheet, devaluing the proxy.
For those reasons I totally agree that evaluator revision should stay the most constrained and externally governed. Our main job will be to make sure all these ‘discoveries’ and ‘breakthroughs’ amount to tangible progress.
(That is of course unless AI gets good at that too.)