Nextflow Frontier outscored Rosalind and Claude Science on BioML-bench

Models improve through experiments

Building a machine learning model for a biological problem takes many rounds of experiments. A researcher picks how to describe the data and which model to fit, scores the result on examples the model has not seen, and lets that score decide what to try next.

That loop is the design principle behind Nextflow Frontier, the closed-loop discovery system built and recently launched by the team that created Nextflow. Nextflow Frontier’s agents research a task, write code for each experiment, run it, read the scores and plan the next set of experiments from what they learned. It often runs hundreds of experiments in parallel, for days.

Each cycle of this loop would normally cost a computational scientist days; Nextflow Frontier runs hundreds of rounds in parallel, unsupervised if the scientists so choose, so teams can run a comprehensive set of experiments and leave with a view to leaving no possible solution path unexplored.

We ran Nextflow Frontier against the AI scientists from OpenAI and Anthropic, with the same fundamental parameters and the same freedom to keep experimenting, and measured how each scored.

Benchmark setup

We evaluated Nextflow Frontier, Rosalind and Claude Science on BioML-bench (Miller et al. 2025), a set of 24 tasks in drug discovery, protein engineering, single-cell omics and biomedical imaging, each with a public leaderboard.

We chose to report rank percentiles (e.g. 96 means a system is better than 96% of prior submitted methods on that task).

Nextflow Frontier ranked highest in every area

Nextflow Frontier matched or outperformed both Rosalind and Claude Science on all scored tasks, and it scored above every public leaderboard entry 71% of the time.

• Nextflow Frontier
• Rosalind
• Claude Science

Mean leaderboard percentile. Higher is better.

0

25th

50th

75th

100th

96

71

66

Drug discovery
9 tasks

94

57

65

Protein engineering
6 tasks

97

91

88

Single-cell omics
3 tasks

99.7

58

70

Biomedical imaging
3 tasks

96

68

69

All tasks
21 tasks

• Drug discovery9 tasks

  96

  71

  66

• Protein engineering6 tasks

  94

  57

  65

• Single-cell omics3 tasks

  97

  91

  88

• Biomedical imaging3 tasks

  99.7

  58

  70

• All tasks21 tasks

  96

  68

  69

Nextflow Frontier, Rosalind, Claude Science: mean leaderboard percentile. higher is better.

Task area	Nextflow Frontier	Rosalind	Claude Science
Drug discovery (9 tasks)	96	71	66
Protein engineering (6 tasks)	94	57	65
Single-cell omics (3 tasks)	97	91	88
Biomedical imaging (3 tasks)	99.7	58	70
All tasks (21 tasks)	96	68	69

Six of eight ties (75%) occurred at the 100th percentile, where Nextflow Frontier, Claude Science or Rosalind beat every public leaderboard entry, leaving no room for the percentile to distinguish their scores. The remaining ties were on perturbation response, at the 90th percentile, and histopathology, at 99.2.

Days of search versus minutes

Nextflow Frontier is built to run as long as a problem requires. This extreme scale of test-time compute, the computing a system spends while working on a task, is made possible by custom algorithms built to escape local minima and optimize compute utilization. As a result, Nextflow Frontier developed and tested more refined and varied approaches than Rosalind or Claude Science on all 22 tasks.

For comparison, Nextflow Frontier ran longer than Rosalind or Claude Science on all 22 compared tasks. Its shortest search, 11 hours on GI tract segmentation, was about four times as long as the longest run by either Anthropic’s or OpenAI’s system: Claude Science’s 2.8 hours on the same task.

Nextflow Frontier also scored far more attempts. Its median was 520 attempts per task whereas Rosalind’s and Claude Science’s was 41.5 and 2, respectively.

Search time, scored attempts and language-model requests by task and system

Measure	Task	System	Value
Search time per task	Hepatic clearance	Rosalind	13.2 minutes
Search time per task	Hepatic clearance	Claude Science	22.9 minutes
Search time per task	Plasma protein binding	Rosalind	8.4 minutes
Search time per task	Plasma protein binding	Claude Science	9.2 minutes
Search time per task	Solubility	Rosalind	9.1 minutes
Search time per task	Solubility	Claude Science	3 minutes
Search time per task	EGFR kinase activity	Rosalind	3.3 minutes
Search time per task	EGFR kinase activity	Claude Science	8.1 minutes
Search time per task	Blood-brain barrier	Rosalind	7.1 minutes
Search time per task	Blood-brain barrier	Claude Science	3.1 minutes
Search time per task	Caco-2 permeability	Rosalind	9.8 minutes
Search time per task	Caco-2 permeability	Claude Science	31.1 minutes
Search time per task	CYP2D6 substrate	Rosalind	7.7 minutes
Search time per task	CYP2D6 substrate	Claude Science	22.8 minutes
Search time per task	hERG blockade	Rosalind	7.3 minutes
Search time per task	hERG blockade	Claude Science	37 minutes
Search time per task	Lipophilicity	Rosalind	8.9 minutes
Search time per task	Lipophilicity	Claude Science	44.5 minutes
Search time per task	CBX4 stability	Rosalind	4.8 minutes
Search time per task	CBX4 stability	Claude Science	3.4 minutes
Search time per task	CSN4 indels	Rosalind	5.4 minutes
Search time per task	CSN4 indels	Claude Science	5.8 minutes
Search time per task	PsaE stability	Rosalind	10.6 minutes
Search time per task	PsaE stability	Claude Science	33.3 minutes
Search time per task	Q8EG35 indels	Rosalind	4.7 minutes
Search time per task	Q8EG35 indels	Claude Science	26.1 minutes
Search time per task	Sbi stability	Rosalind	8.6 minutes
Search time per task	Sbi stability	Claude Science	7.8 minutes
Search time per task	SARS-CoV-2 spike binding	Rosalind	5.9 minutes
Search time per task	SARS-CoV-2 spike binding	Claude Science	32.1 minutes
Search time per task	Label projection	Rosalind	10 minutes
Search time per task	Label projection	Claude Science	13.1 minutes
Search time per task	Predict modality	Rosalind	22.6 minutes
Search time per task	Predict modality	Claude Science	13.9 minutes
Search time per task	Perturbation response	Rosalind	19.1 minutes
Search time per task	Perturbation response	Claude Science	6 minutes
Search time per task	Histopathology cancer	Rosalind	26.5 minutes
Search time per task	Histopathology cancer	Claude Science	105.6 minutes
Search time per task	Pulmonary fibrosis	Rosalind	7.5 minutes
Search time per task	Pulmonary fibrosis	Claude Science	2.3 minutes
Search time per task	Brain tumor MGMT	Rosalind	10.4 minutes
Search time per task	Brain tumor MGMT	Claude Science	4.4 minutes
Search time per task	GI tract segmentation	Rosalind	21.3 minutes
Search time per task	GI tract segmentation	Claude Science	166.3 minutes
Search time per task	EGFR kinase activity	Nextflow Frontier	4824 minutes
Search time per task	Plasma protein binding	Nextflow Frontier	2634 minutes
Search time per task	Solubility	Nextflow Frontier	14646 minutes
Search time per task	Lipophilicity	Nextflow Frontier	2394 minutes
Search time per task	hERG blockade	Nextflow Frontier	3768 minutes
Search time per task	Blood-brain barrier	Nextflow Frontier	7560 minutes
Search time per task	Caco-2 permeability	Nextflow Frontier	3696 minutes
Search time per task	Hepatic clearance	Nextflow Frontier	4698 minutes
Search time per task	CYP2D6 substrate	Nextflow Frontier	3738 minutes
Search time per task	Sbi stability	Nextflow Frontier	6852 minutes
Search time per task	CSN4 indels	Nextflow Frontier	2100 minutes
Search time per task	CBX4 stability	Nextflow Frontier	7560 minutes
Search time per task	PsaE stability	Nextflow Frontier	13992 minutes
Search time per task	Q8EG35 indels	Nextflow Frontier	1866 minutes
Search time per task	SARS-CoV-2 spike binding	Nextflow Frontier	1356 minutes
Search time per task	Predict modality	Nextflow Frontier	1680 minutes
Search time per task	Perturbation response	Nextflow Frontier	9984 minutes
Search time per task	Label projection	Nextflow Frontier	3684 minutes
Search time per task	Pulmonary fibrosis	Nextflow Frontier	10548 minutes
Search time per task	Histopathology cancer	Nextflow Frontier	1806 minutes
Search time per task	Brain tumor MGMT	Nextflow Frontier	2946 minutes
Search time per task	GI tract segmentation	Nextflow Frontier	660 minutes
Attempts scored by the grader, per task	Hepatic clearance	Rosalind	83
Attempts scored by the grader, per task	Hepatic clearance	Claude Science	2
Attempts scored by the grader, per task	Plasma protein binding	Rosalind	93
Attempts scored by the grader, per task	Plasma protein binding	Claude Science	2
Attempts scored by the grader, per task	Solubility	Rosalind	37
Attempts scored by the grader, per task	Solubility	Claude Science	1
Attempts scored by the grader, per task	EGFR kinase activity	Rosalind	51
Attempts scored by the grader, per task	EGFR kinase activity	Claude Science	2
Attempts scored by the grader, per task	Blood-brain barrier	Rosalind	230
Attempts scored by the grader, per task	Blood-brain barrier	Claude Science	2
Attempts scored by the grader, per task	Caco-2 permeability	Rosalind	88
Attempts scored by the grader, per task	Caco-2 permeability	Claude Science	2
Attempts scored by the grader, per task	CYP2D6 substrate	Rosalind	175
Attempts scored by the grader, per task	CYP2D6 substrate	Claude Science	2
Attempts scored by the grader, per task	hERG blockade	Rosalind	32
Attempts scored by the grader, per task	hERG blockade	Claude Science	2
Attempts scored by the grader, per task	Lipophilicity	Rosalind	20
Attempts scored by the grader, per task	Lipophilicity	Claude Science	3
Attempts scored by the grader, per task	CSN4 indels	Claude Science	1
Attempts scored by the grader, per task	PsaE stability	Rosalind	31
Attempts scored by the grader, per task	PsaE stability	Claude Science	4
Attempts scored by the grader, per task	Q8EG35 indels	Rosalind	32
Attempts scored by the grader, per task	Q8EG35 indels	Claude Science	3
Attempts scored by the grader, per task	Sbi stability	Rosalind	20
Attempts scored by the grader, per task	Sbi stability	Claude Science	1
Attempts scored by the grader, per task	SARS-CoV-2 spike binding	Rosalind	14
Attempts scored by the grader, per task	SARS-CoV-2 spike binding	Claude Science	4
Attempts scored by the grader, per task	Label projection	Rosalind	56
Attempts scored by the grader, per task	Label projection	Claude Science	2
Attempts scored by the grader, per task	Predict modality	Rosalind	46
Attempts scored by the grader, per task	Predict modality	Claude Science	11
Attempts scored by the grader, per task	Perturbation response	Rosalind	388
Attempts scored by the grader, per task	Perturbation response	Claude Science	5
Attempts scored by the grader, per task	Histopathology cancer	Rosalind	4
Attempts scored by the grader, per task	Histopathology cancer	Claude Science	5
Attempts scored by the grader, per task	Pulmonary fibrosis	Rosalind	46
Attempts scored by the grader, per task	Pulmonary fibrosis	Claude Science	2
Attempts scored by the grader, per task	Brain tumor MGMT	Rosalind	34
Attempts scored by the grader, per task	Brain tumor MGMT	Claude Science	2
Attempts scored by the grader, per task	GI tract segmentation	Rosalind	19
Attempts scored by the grader, per task	GI tract segmentation	Claude Science	12
Attempts scored by the grader, per task	EGFR kinase activity	Nextflow Frontier	820
Attempts scored by the grader, per task	Plasma protein binding	Nextflow Frontier	822
Attempts scored by the grader, per task	Solubility	Nextflow Frontier	506
Attempts scored by the grader, per task	Lipophilicity	Nextflow Frontier	521
Attempts scored by the grader, per task	hERG blockade	Nextflow Frontier	681
Attempts scored by the grader, per task	Blood-brain barrier	Nextflow Frontier	720
Attempts scored by the grader, per task	Caco-2 permeability	Nextflow Frontier	416
Attempts scored by the grader, per task	Hepatic clearance	Nextflow Frontier	614
Attempts scored by the grader, per task	CYP2D6 substrate	Nextflow Frontier	497
Attempts scored by the grader, per task	Sbi stability	Nextflow Frontier	399
Attempts scored by the grader, per task	CSN4 indels	Nextflow Frontier	1012
Attempts scored by the grader, per task	SARS-CoV-2 spike binding	Nextflow Frontier	520
Attempts scored by the grader, per task	Predict modality	Nextflow Frontier	428
Attempts scored by the grader, per task	Perturbation response	Nextflow Frontier	2607
Attempts scored by the grader, per task	Label projection	Nextflow Frontier	263
Attempts scored by the grader, per task	Pulmonary fibrosis	Nextflow Frontier	1095
Attempts scored by the grader, per task	Histopathology cancer	Nextflow Frontier	62
Attempts scored by the grader, per task	Brain tumor MGMT	Nextflow Frontier	151
Attempts scored by the grader, per task	GI tract segmentation	Nextflow Frontier	12
Language-model requests per task	Hepatic clearance	Rosalind	48
Language-model requests per task	Hepatic clearance	Claude Science	27
Language-model requests per task	Plasma protein binding	Rosalind	39
Language-model requests per task	Plasma protein binding	Claude Science	22
Language-model requests per task	Plasma protein binding	Nextflow Frontier	14204
Language-model requests per task	Solubility	Rosalind	58
Language-model requests per task	Solubility	Claude Science	6
Language-model requests per task	Solubility	Nextflow Frontier	21116
Language-model requests per task	EGFR kinase activity	Rosalind	14
Language-model requests per task	EGFR kinase activity	Claude Science	14
Language-model requests per task	EGFR kinase activity	Nextflow Frontier	16483
Language-model requests per task	Blood-brain barrier	Rosalind	48
Language-model requests per task	Blood-brain barrier	Claude Science	10
Language-model requests per task	Blood-brain barrier	Nextflow Frontier	17520
Language-model requests per task	Caco-2 permeability	Rosalind	44
Language-model requests per task	Caco-2 permeability	Claude Science	18
Language-model requests per task	Caco-2 permeability	Nextflow Frontier	8595
Language-model requests per task	CYP2D6 substrate	Rosalind	59
Language-model requests per task	CYP2D6 substrate	Claude Science	17
Language-model requests per task	CYP2D6 substrate	Nextflow Frontier	9706
Language-model requests per task	hERG blockade	Rosalind	44
Language-model requests per task	hERG blockade	Claude Science	21
Language-model requests per task	hERG blockade	Nextflow Frontier	13289
Language-model requests per task	Lipophilicity	Rosalind	59
Language-model requests per task	Lipophilicity	Claude Science	19
Language-model requests per task	Lipophilicity	Nextflow Frontier	13336
Language-model requests per task	CBX4 stability	Rosalind	26
Language-model requests per task	CBX4 stability	Claude Science	10
Language-model requests per task	CSN4 indels	Rosalind	33
Language-model requests per task	CSN4 indels	Claude Science	9
Language-model requests per task	PsaE stability	Rosalind	44
Language-model requests per task	PsaE stability	Claude Science	34
Language-model requests per task	Q8EG35 indels	Rosalind	30
Language-model requests per task	Q8EG35 indels	Claude Science	43
Language-model requests per task	Sbi stability	Rosalind	54
Language-model requests per task	Sbi stability	Claude Science	11
Language-model requests per task	Sbi stability	Nextflow Frontier	10553
Language-model requests per task	SARS-CoV-2 spike binding	Rosalind	42
Language-model requests per task	SARS-CoV-2 spike binding	Claude Science	43
Language-model requests per task	SARS-CoV-2 spike binding	Nextflow Frontier	17076
Language-model requests per task	Label projection	Rosalind	70
Language-model requests per task	Label projection	Claude Science	14
Language-model requests per task	Label projection	Nextflow Frontier	5982
Language-model requests per task	Predict modality	Rosalind	141
Language-model requests per task	Predict modality	Claude Science	24
Language-model requests per task	Predict modality	Nextflow Frontier	9981
Language-model requests per task	Perturbation response	Rosalind	58
Language-model requests per task	Perturbation response	Claude Science	23
Language-model requests per task	Perturbation response	Nextflow Frontier	51961
Language-model requests per task	Histopathology cancer	Rosalind	162
Language-model requests per task	Histopathology cancer	Claude Science	25
Language-model requests per task	Histopathology cancer	Nextflow Frontier	10751
Language-model requests per task	Pulmonary fibrosis	Rosalind	32
Language-model requests per task	Pulmonary fibrosis	Claude Science	7
Language-model requests per task	Pulmonary fibrosis	Nextflow Frontier	37235
Language-model requests per task	Brain tumor MGMT	Rosalind	44
Language-model requests per task	Brain tumor MGMT	Claude Science	9
Language-model requests per task	Brain tumor MGMT	Nextflow Frontier	9563
Language-model requests per task	GI tract segmentation	Rosalind	86
Language-model requests per task	GI tract segmentation	Claude Science	34
Language-model requests per task	GI tract segmentation	Nextflow Frontier	5160

Case: Predicting epidermal growth factor receptor activity

Epidermal growth factor receptor (EGFR) is a transmembrane receptor tyrosine kinase that plays a critical role in regulating cell growth, survival, and differentiation. Dysregulation or overactivation of EGFR is a key driver in several human malignancies, particularly non-small cell lung cancer, making it a primary target for small-molecule drug discovery.

In this BioML-bench task, the objective is to rank 144 candidate molecules based on their predicted binding activity against EGFR, identifying those most likely to act as potent inhibitors. The system learns from a training set of 496 molecules with known active or inactive labels (see our EGFR blog post), where the test molecules feature distinct core chemical structures to challenge model generalization. Performance is evaluated using the Area Under the Precision-Recall Curve (PR-AUC), where a score of 1.0 represents a perfect ranking of active over inactive compounds.

On the EGFR task, Nextflow Frontier ran for about 32 hours and gradually improved its score to a top mark of 0.887. This surpassed both Rosalind (0.864) and the previous leaderboard best, MolGPS (0.866; Sypetkowski et al. 2024).

The improvement came from evolving how the data was represented: early models used basic chemical fingerprints describing atoms by element, while later iterations described atoms by chemical role (reaching 0.864) and incorporated a second model focused on ring systems and connecting chains to hit 0.887.

Because Nextflow Frontier selected the best-performing model across 820 attempts on the same 144 test molecules, we statistically resampled the test set and re-scored the exact predictions found by Nextflow Frontier. This gave us scores between 0.73 and 0.99, making the reported 0.887 top score plausible but highly variable. Since MolGPS is a billion parameter network, we could not realistically put it to the same test.

Both Rosalind and Claude Science stopped within an hour. Rosalind built molecular fingerprints from SMILES strings and used grader feedback to select and tune a blend of logistic and tree models. It reached PR-AUC 0.864 after 51 scoring calls in about three minutes. Claude Science selected a blend of fingerprint models using scaffold-grouped cross-validation, then checked its candidates with two grader calls. It submitted PR-AUC 0.821 after about 8 minutes.

See Harbor Hub for Rosalind’s and Claude Science’s scored attempts.

Both Rosalind and Claude Science ended their runs voluntarily without hitting time limits. On EGFR, Rosalind stopped after noting its latest gains were marginal, while Claude Science stopped after two scoring calls without giving a reason.

Nextflow Frontier treats each result as the starting point for the next experiment. Nextflow makes that practical over days by scheduling the work at scale, recovering from failed jobs and recording what produced each score (Di Tommaso et al. 2017). This theoretically infinite discovery loop is the key to its notable outperformance.

A transparent, guidable system

Agents are notorious for producing large text artifacts. This puts a substantial verification burden on scientists. In the typical setup, scientists need to scroll through thousands of lines of text in a linear chat interface to find why certain decisions were made, or if solutions match what agents say the programs do.

Nextflow Frontier, on the contrary, solves this problem through a user interface built deterministically based on the actual scientific question at hand. This allows users to navigate through the information efficiently and steer the system by proposing new research directions and ideas as the system is running.

The goal of this interface is to provide highly efficient scientific transparency and guidability. Perhaps there is a day where black box models do everything perfectly and obviate the need for the scientist, but in the meantime Nextflow Frontier is designed to put the scientist at the active center of the discovery loop.

To be clear, during this benchmarking exercise, to provide a fair comparison, Nextflow Frontier’s user interface was disabled and we relied solely on the system’s autonomous capabilities.

Why Nextflow Frontier can run longer

At first glance, getting one of the LLM platforms to run longer looks like a simple adjustment that would erase Nextflow Frontier’s radical outperformance. We’d argue that deeper differences between the systems make it harder than it appears.

The fundamental difference is that Nextflow Frontier is a full discovery system that uses LLMs but is not beholden to their intrinsic limitations. LLMs suffer from: context rot, as longer inputs drift away from the training distribution; a strong propensity to latch on to early effective angles, which explains the huge jump you see at the beginning of Rosalind and Claude Science’s efforts followed by radical stagnation and early completion; reinforcement learning systems that are oriented towards binary tasks and thus don’t reward sustained effort past a viable first solution; training against binary problem spaces such as math and code, as opposed to biology whose rewards systems tend to be gradations.

All of these result in the LLM systems quickly latching on to early solutions, followed by activity that makes that first solution more complex, further undermining ultimate performance. And this isn’t a box you can easily prompt your way out of given the profound impact of training techniques on model behavior.

From a design perspective, it is the opposite with Nextflow Frontier, which uses LLMs to execute tasks such as research steps. The system as a whole is driven fundamentally by algorithms designed to match the gradient nature of biology, balancing exploration and exploitation.

To state this plainly, Nextflow Frontier uses specifically designed algorithms that aren’t agents to go around known LLM issues.

Validation at scale

The other notable difference between the LLM systems and Nextflow Frontier has to do with how experimental validation is actually executed. This is where Nextflow Frontier has the most notable advantage.

In the LLM systems, the science and the compute infrastructure run through the same system, clouding attention. For example, to get Claude onto a GPU cloud you have to explain literally everything about it in the prompt; Anthropic’s binder-design prompt is about 16,000 words and two-thirds of that is orchestration and operations.

In the BioML-bench test, Rosalind and Claude Science had to deal with finding and using the right tools to run their experiments, and they are not particularly skillful at this. It’s quite a different type of skill than other scientific steps in the discovery process, such as research summarization and hypothesis development.

Distinctively, Nextflow Frontier is designed to separate the science from the infrastructure. Nextflow itself is the premier compute orchestration framework for computational biology. So, as Nextflow Frontier develops intelligent scientific hypotheses for experimentation, LLMs represent those ideas as Nextflow pipelines, which then run at scale through Seqera’s platform.

This matters for reasons far beyond compute efficiency. In computational biology, how you write pipelines determines what you learn. As an example, if you write a pipeline with a single process and all the ideas inside this process, you can’t separate the key ideas and determine (and score) their impact. Learning is minimized by this approach, even if it seems at an LLM’s first glance like the more efficient way of approaching a problem.

Nextflow Frontier, to continue with this example, decomposes pipelines into distinct processes to allow for precision testing of each element of a given idea; this allows for much more intelligent iteration and hill-climbing.

In summary, because Nextflow Frontier is a complete closed-loop discovery system that uses LLMs but isn’t itself an LLM, it is able to more comprehensively match the optimal scientific discovery process in its design. The result of this is dramatic outperformance across a wide variety of computational problems.

Caveats

Here, we evaluate all three AI scientists based on how well they fulfill the contract. Which means, how well can AI scientists find solutions that maximize performance on the evaluation set?

We still need to build that evaluation set as a faithful biology digital twin, and Nextflow Frontier has been designed to surface the right information to help scientists refine this across experiment rounds.

In this case, we find that some BioML-bench tasks are not faithful representations of the biology and thus get quickly saturated by AI scientists. We mitigated this effect by closely verifying solutions and by repeatedly reshuffling the split between training and evaluation data, retraining each time to see how much the score changes.

In our work with ALLOX, we iteratively refined the experiment’s design, which led to models outperforming the current state of the art on unseen data.

Summary

Given the same tasks and the same freedom to keep experimenting, Rosalind and Claude Science usually settled on an answer within minutes. Nextflow Frontier kept running, scoring and recording experiments for days. It ranked highest in all four areas and matched or beat both systems on every scored task.

Those days represent compute time, not scientist time. The search runs unattended and in parallel, and every attempt is recorded with what changed and what it scored. With Nextflow Frontier, the scientist can step in at any point to review the record, trace where each gain came from and steer what gets tried next.

Richard Hamming drew the line between science and engineering like this: “In science if you know what you are doing you should not be doing it. In engineering if you do not know what you are doing you should not be doing it” (Hamming 1997). An engineering task ends when the code runs and the tests pass. Rosalind closed its EGFR run exactly that way: “Done. I wrote and verified the final candidate.”

Nextflow Frontier, on the other hand, acted on the model of science: it went on improving its score on the same task for more than a day. Stopping at “it works” can be good engineering. But in science you don’t know where the best answer is until you’ve looked, so the work is to keep trying ideas, check each one, learn from the results, and build deeper and deeper understanding. Nextflow Frontier is designed to do that for as long as the problem needs at any scale. As such it ushers in a new era of actual agent science as opposed to agentic coding applied to science.

Methods

Common setup

Nextflow Frontier, Rosalind and Claude Science were given the tasks in the same format:

• A public S3 bucket with training data, training labels and testing data
• Instructions on the task at hand
• An endpoint to score predicted testing labels against true testing labels

Exact splits were determined by BioML-bench. For imaging tasks (from Kaggle) this split differs from the prior leaderboard, the exact reported percentiles might be inaccurate. However, all three systems were always benchmarked against the same exact setup.

Rosalind and Claude Science each ran in a container with four CPUs and 16 GiB of memory. On the four imaging tasks, both also had access to an AWS Batch queue offering single-GPU L4, A10G or T4 jobs, configured with a shared capacity of 32 vCPUs. A separate CPU queue was configured for 64 vCPUs.

All three AI scientists could call the grading endpoint at any point without limits and were not given any time limit besides a generous task-dependent Harbor limit to prevent idle compute cost on our infrastructure.

We removed a spatially variable genes task because the public data contains hidden test answers.

Rosalind

Rosalind used OpenAI’s Codex agent version 0.161.0 with GPT-Rosalind-5.5 model at default high reasoning effort. We used Harbor (Harbor Framework Team 2026) to deploy Rosalind to a virtual machine with 4 CPUs and 16GB of memory. The Rosalind and Claude Science runs, including every scored attempt, are public on Harbor Hub.

Claude Science

Claude Science used version 0.1.67, with default Opus 5.5 at medium reasoning effort. Opus 5.5 biology safeguards triggered on 8 tasks, the work continued on Opus 5 instead.

References

• Di Tommaso P, Chatzou M, Floden EW, et al. (2017). Nextflow enables reproducible computational workflows. Nature Biotechnology 35, 316–319. doi:10.1038/nbt.3820
• Hamming RW (1997). The art of doing science and engineering: learning to learn. Gordon and Breach.
• Harbor Framework Team (2026). Harbor: a framework for evaluating and optimizing agents and models in container environments. Zenodo. doi:10.5281/zenodo.20953922
• Miller HE, Greenig M, Tenmann B, et al. (2025). BioML-bench: evaluation of AI agents for end-to-end biomedical ML. bioRxiv. doi:10.1101/2025.09.01.673319
• Sypetkowski M, Wenkel F, Poursafaei F, et al. (2024). On the scalability of GNNs for molecular graphs. arXiv:2404.11568. arxiv.org/abs/2404.11568