Stanford's AI Research Team Included a Dedicated Skeptic. It Worked.

In late 2024, a Stanford and Chan Zuckerberg Biohub team gave a group of AI agents a genuinely hard, open-ended research problem: design new antibody-like proteins (nanobodies) that bind to current SARS-CoV-2 variants the way older nanobodies bind to the original strain. Not a benchmark question with a known answer — a real research question nobody had solved yet. The AI team designed 92 candidates. Researchers then actually synthesized and tested them in a wet lab. More than 90% expressed and folded correctly, and two showed genuinely useful binding profiles against variants that didn't exist when older antibodies were designed.

The team wasn't one model. It was a cast of specialists.

Kyle Swanson, Wesley Wu, Nash Bulaong, John Pak, and James Zou built what they call the Virtual Lab: a Principal Investigator agent that assembles and directs a team of scientist agents — an immunologist, a machine learning specialist, a computational biologist — each running the same underlying model (GPT-4o) but given a genuinely different title, expertise, and goal. Critically, the team also includes a Scientific Critic agent whose entire job is to find problems with what the other agents propose.

"We find it useful to create an explicit critic agent to catch errors and oversights from the other agents and to give critical feedback on answers provided by the other agents."

The research runs through meetings, not single prompts: team meetings where all the agents discuss a question and the PI synthesizes their input round by round, and individual meetings where one specialist works on a specific task while the critic pushes back on each draft before a final answer is accepted. A dedicated dissenting voice isn't a side effect of the design — it's a deliberately engineered role.

The result: a real research problem nobody had graded in advance

The team's computational pipeline — combining the protein language model ESM, AlphaFold-Multimer, and the modeling software Rosetta — was itself something the agents worked out collaboratively, not a fixed tool they were handed. Applied to nanobodies against SARS-CoV-2's more recent JN.1 and KP.3 variants:

• 92 mutant nanobodies were designed and experimentally validated in a real lab
• Over 90% were successfully expressed and soluble — a basic but non-trivial bar for a designed protein
• Two candidates showed improved binding to the newer variants while keeping strong binding to the original strain

There's no benchmark score to compare this against, because no benchmark existed for this exact problem before the agents solved it. That's the point: this wasn't multiple-choice accuracy, it was real molecules that either bind or don't, checked in a real lab.

The person who built it also predicted it

James Zou, one of the paper's authors, made a specific forecast in Stanford HAI's predictions for 2025, citing this exact project as the example:

"In 2025, we will see a significant shift from relying on individual AI models to using systems where multiple AI agents of diverse expertise work together […] more reliable and effective than a single model."

Stanford HAI's Russ Altman made a similar prediction independently, describing a "general contractor" LLM that subcontracts pieces of a problem to "expert" LLMs. Diyi Yang, also at Stanford, framed the underlying goal directly: "Identifying the best ways for AI and humans to work together to achieve collective intelligence will become increasingly important."

What this means in practice

The detail worth taking seriously isn't "more AI agents are good" in the abstract — it's that the team that actually produced a validated real-world result deliberately built in a role whose entire job was disagreement. Not a model second-guessing itself, but a distinct agent instructed to find what the others got wrong, on every single round, before anything was accepted. That's the same principle behind giving a Brainstorming Room genuine skeptic and premise-challenger roles rather than letting every participant simply agree and build on whatever came first.

See our Best Practices guide for how Parley's own role system works, and our pieces on multi-agent debate and self-preference bias for the research explaining exactly why a dedicated dissenting voice catches what a single model, agreeing only with itself, cannot.

Sources

• Swanson, K., Wu, W., Bulaong, N. L., Pak, J. E., & Zou, J. (2024, preprint; 2025, Nature). The Virtual Lab of AI agents designs new SARS-CoV-2 nanobodies with experimental validation. https://doi.org/10.1038/s41586-025-09442-9 (preprint: bioRxiv:2024.11.11.623004).
• Stanford HAI (December 2024). Predictions for AI in 2025: Collaborative Agents, AI Skepticism, and New Risks.