Motivation

Recently, sakana ai announced the AI Scientist. This is a system that can handle everything from idea generation to experiment execution and paper writing in one seamless process. This research is very important in the context of "AI that researches," which I'm interested in.

https://github.com/SakanaAI/AI-Scientist

Personally, I believe the automation of AI research is crucial because it allows AI to conduct research on the automation of AI or AI research itself, potentially leading to self-improvement through its research results.

                                       [Slide from “Possibilities of AI research conducted by AI”](<https://www.canva.com/design/DAGG8QNKblg/ma1d7m8Qi-1cLhDpBIs5qA/view?utm_content=DAGG8QNKblg&utm_campaign=designshare&utm_medium=link&utm_source=editor>) 

Ideally, this could result in exponential performance improvements. In June 2024, it was predicted in situational awareness that the trigger for the intelligence explosion would be automated AI research.

                                                                 [from Situational Awareness by Leopold Aschenbrenner](<https://situational-awareness.ai/from-agi-to-superintelligence/>)

What I tried

So, I'm planning to attempt using the AI Scientist to research the AI Scientist itself. As a first step, I gave it the seed idea of "improving the idea generation process" and had the AI Scientist generate a paper. Simply put, it's like creating a template/experiment.py file.

Result

Here are the results. This paper is a proposal summarized by the AI Scientist itself, suggesting an improvement to its own idea generation process to consider ethical points.

ethical_idea_generation.pdf

Due to time and cost constraints, I significantly simplified the idea generation process and used GPT-4, so the content of the paper itself is quite poor. However, as you can see from the abstract, it's evident that the automated research system can write papers about improving automated research itself.

After running it, I've also gotten a grasp of the key points of the AI Scientist, so from the next time onwards, I hope I can improve the generated paper quality. I’ll share it again if high-quality results are obtained.