He says the path to AGI is AI generating content and self-assessing it, like AlphaZero playing itself.
"the path to AGI is something like that is self generated content where it effectively plays"
How will AI overcome the data wall? (asked on X)
Q957 answers2023-2024asked on 𝕏
Short answer
He says at a certain point AI runs out of human-created data.
All 7 answers, oldest to newest
He says at a certain point AI runs out of human-created data.
"At a certain point you kind of run out of human created data."
He said yeah. It’s a little sad that you can fit the text of every book ever written by humans on one hard drive (sigh). Synthetic data will exceed that by a zillion.
"Yeah. It’s a little sad that you can fit the text of every book ever written by humans on one hard drive (sigh). Synthetic data will exceed that by a zillion."
Asked My god. It can do basic math, he said that part doesn’t matter. Synthetic data generation and accurate evaluation, ie recursive self-improvement, is more likely.
"That part doesn’t matter. Synthetic data generation and accurate evaluation, ie recursive self-improvement, is more likely."
He said vector sum of humans on team, usable compute and unique access to data define AI competitiveness. It is remarkable how quickly we run out of human-created data. Reality itself and synthetic data ftw.
"Vector sum of humans on team, usable compute and unique access to data define AI competitiveness. It is remarkable how quickly we run out of human-created data. Reality itself and synthetic data ftw."
He said two sources of data scale infinitely: synthetic data, which has an “is it true?” problem and real-world video, which does not.
"Two sources of data scale infinitely: synthetic data, which has an “is it true?” problem and real-world video, which does not."
Asked LLMs are on track to consume the very last bit of the internet’s human written knowledg..., he said synthetic, validated against reality by humanoid robots
"Synthetic, validated against reality by humanoid robots"