1
The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
arxiv.orgOne of the grand challenges of artificial general intelligence is developing agents capable of conducting scientific research and discovering new knowledge. While frontier models have already been used as aides to human scientists, e.g. for brainstorming ideas, writing code, or prediction tasks, they still conduct only a small part of the scientific process. This paper presents the first comprehensive framework for fully automatic scientific discovery, enabling frontier large language models to perform research independently and communicate their findings. We introduce The AI Scientist, which generates novel research ideas, writes code, executes experiments, visualizes results, describes its findings by writing a full scientific paper, and then runs a simulated review process for evaluation. In principle, this process can be repeated to iteratively develop ideas in an open-ended fashion, acting like the human scientific community. We demonstrate its versatility by applying it to three distinct subfields of machine learning: diffusion modeling, transformer-based language modeling, and learning dynamics. Each idea is implemented and developed into a full paper at a cost of less than $15 per paper. To evaluate the generated papers, we design and validate an automated reviewer, which we show achieves near-human performance in evaluating paper scores. The AI Scientist can produce papers that exceed the acceptance threshold at a top machine learning conference as judged by our automated reviewer. This approach signifies the beginning of a new era in scientific discovery in machine learning: bringing the transformative benefits of AI agents to the entire research process of AI itself, and taking us closer to a world where endless affordable creativity and innovation can be unleashed on the world's most challenging problems. Our code is open-sourced at https://github.com/SakanaAI/AI-Scientist
AI doesn’t grok anything. It doesn’t have any capability of understanding at all. It’s a Markov chain on steroids.
Did you read the paper? Or at least have an llm explain it?
I read the abstract, and the connection to your title is a mystery. Are you using “grock” as in “transcendental understanding” or as Musk’s branded AI?
No c, just grok, originally from Stranger in a Strange Land. But a more technical definition is provided and expanded upon in the paper. Mystery easily dispelled!
Thanks for clarifying, now please refer to the poster’s original statement:
AI doesn’t grok anything. It doesn’t have any capability of understanding at all. It’s a Markov chain on steroids.
Oh okay so they’re just redefining words that are already well-defined so they can make fancy claims.
Well-defined for casual use is very different than well-defined for scholarly research. It’s standard practice to take colloquial vocab and more narrowly define it for use within a scientific discipline. Sometimes different disciplines will narrowly define the same word two different ways, which makes interdisciplinary communication pretty funny.
No. It’s not standard at all, especially when the goal is overtly misleading.
Maybe one or both disciplines is promoting bullshit.
In that case I refer you to u/catloaf 's post. A machine cannot grock, not at any speed.
…is how generative-AI haters redefine terms and move the goalposts to fight their cognitive dissonance.
Imagine believing that AI-haters are the ones who redefine terms and move goalposts to fight their cognitive dissonance.