| Not researched |
Factual |
2025 |
Every time an AI benchmark is run, it turns out somebody has gamed it.
Factual · 2025 |
AI capabilities |
| Not checkable |
Not checkable |
2025 |
So I can’t personally vouch for the work, because I haven’t gone through it.
Not checkable · 2025 |
|
| Not checkable |
Not checkable |
2025 |
I’m not that fond, by the way, of the “people break too” thing.
Not checkable · 2025 |
|
| Not researched |
Factual |
2025 |
LLMs are trained on chess rules and books about chess yet still make illegal moves.
Factual · 2025 |
AI capabilities |
| Not checkable |
Not checkable |
2025 |
I do think we could do better.
Not checkable · 2025 |
|
| Not checkable |
Not checkable |
2025 |
You know, AGI is not going to be magic, right?
Not checkable · 2025 |
|
| Resolved true |
Factual |
2025 |
Bolting extra components onto an LLM and scaling it up has so far not produced human-level AGI.
Factual · 2025 |
AI capabilities |
| Not checkable |
Not checkable |
2025 |
Yeah, that’s an optimism that needs to be based in a plan, which we don’t have.
Not checkable · 2025 |
|
| Not checkable |
Not checkable |
2025 |
I have a phrase I like, which is that AI, current AI, is morally and technically inadequate.
Not checkable · 2025 |
|
| Not researched |
Factual |
2025 |
LLMs grasp nouns but not verbs, and are limited in what aspects of language they abstract.
Factual · 2025 |
AI capabilities |
| Not researched |
Factual |
2025 |
Animals store the values of variables and update them over time.
Factual · 2025 |
Science and research methods |
| Not researched |
Factual |
2025 |
Current AI systems are not reliable.
Factual · 2025 |
AI capabilities |
| Not checkable |
Not checkable |
2025 |
You shouldn’t be surprised.
Not checkable · 2025 |
|
| Unresolvable |
Prediction |
Dec 2024 |
With medium confidence, technical moat will continue to be elusive in 2025, with convergence on broadly similar models across the US and China and some European systems catching up to roughly the same place.
Prediction · Dec 2024 |
Economics and markets |
| Unresolvable |
Prediction |
Dec 2024 |
Few companies and even fewer consumers will adopt OpenAI's o3 at wide scale in 2025 because of price and robustness concerns.
Prediction · Dec 2024 |
Economics and markets |
| Not researched |
Factual |
9 Nov 2024 |
Gary Marcus has warned about foundational limits of neural networks since his 2001 book The Algebraic Mind, where he first described hallucinations.
Factual · 9 Nov 2024 |
Their own work or organisation |
| Not researched |
Factual |
9 Nov 2024 |
Leading AI developers' LLMs are converging to roughly the same level of capability, leaving no company with a competitive moat.
Factual · 9 Nov 2024 |
Economics and markets |
| Not checkable |
Not checkable |
9 Nov 2024 |
As I have always warned, that’s just a fantasy.
Not checkable · 9 Nov 2024 |
|
| Unresolvable |
Prediction |
9 Nov 2024 |
The US's focus on LLMs for AI may prove to be a massive mistake.
Prediction · 9 Nov 2024 |
AI policy and governance |
| Unresolvable |
Prediction |
9 Nov 2024 |
LLMs will become a commodity and price wars will keep revenue low.
Prediction · 9 Nov 2024 |
Economics and markets |
| Not researched |
Factual |
9 Nov 2024 |
Scaling of AI models is running out of steam, with improvements from adding data and compute diminishing.
Factual · 9 Nov 2024 |
AI timelines and forecasting |
| Unresolvable |
Prediction |
9 Nov 2024 |
LLMs will not disappear, but their economics will likely never make sense because training costs rise with scaling.
Prediction · 9 Nov 2024 |
AI, the economy and work |
| Not checkable |
Not checkable |
9 Nov 2024 |
There is no principled solution to hallucinations in systems that traffic only in the statistics of language without explicit…
Not checkable · 9 Nov 2024 |
|
| Unresolvable |
Prediction |
9 Nov 2024 |
Given chip costs, profits from LLMs will be elusive.
Prediction · 9 Nov 2024 |
Economics and markets |
| Unresolvable |
Prediction |
9 Nov 2024 |
The AI financial bubble may burst quickly once people realize the situation, and Nvidia's valuation may take a hit.
Prediction · 9 Nov 2024 |
Economics and markets |
| Not checkable |
Not checkable |
9 Nov 2024 |
I’m glad that the market is finally recognizing that what I’ve been saying is true.
Not checkable · 9 Nov 2024 |
|
| Unresolvable |
Factual |
9 Nov 2024 |
Melanie Mitchell, Subbarao Kambhampati, Emily Bender, Ernie Davis and others have long emphasized that LLMs have principled limits.
Factual · 9 Nov 2024 |
Their own work or organisation |
| Not checkable |
Not checkable |
9 Nov 2024 |
The thing is, in the long term, science isn’t majority rule. In the end, the truth generally outs.
Not checkable · 9 Nov 2024 |
|
| Resolved true |
Factual |
9 Nov 2024 |
Gary Marcus has for years warned that AI scaling by adding data and compute without architectural changes would not continue forever.
Factual · 9 Nov 2024 |
Their own work or organisation |
| Unresolvable |
Prediction |
Apr 2024 |
Reliable, trustworthy AI is achievable, but may require going back to the drawing board.
Prediction · Apr 2024 |
AI safety and alignment |
| Unresolvable |
Prediction |
Apr 2024 |
If GenAI enthusiasm dwindles and valuations plummet, AI and LLMs will not disappear and will remain tools for statistical approximation.
Prediction · Apr 2024 |
Economics and markets |
| Unresolvable |
Prediction |
Apr 2024 |
LLMs' place may be smaller, and LLMs on their own may never live up to the 2023 expectations.
Prediction · Apr 2024 |
AI timelines and forecasting |
| Unresolvable |
Factual |
Apr 2024 |
Gary Marcus says that in April, when he first saw what he called direct empirical evidence that "this moment had come", he wrote a passage that he says he still stands by.
Factual · Apr 2024 |
Their own work or organisation |
| Not researched |
Factual |
31 Mar 2024 |
Because everyone is building similar models, generative AI companies have no moat and compete heavily on price.
Factual · 31 Mar 2024 |
Economics and markets |
| Not checkable |
Not checkable |
31 Mar 2024 |
I am not yet ready to claim victory.
Not checkable · 31 Mar 2024 |
|
| Unresolvable |
Prediction |
31 Mar 2024 |
If OpenAI releases a truly mindblowing GPT-5 in 2024, the generative AI bubble will not burst soon.
Prediction · 31 Mar 2024 |
Economics and markets |
| Not researched |
Factual |
31 Mar 2024 |
Larger AI models have higher internal costs.
Factual · 31 Mar 2024 |
Economics and markets |
| Not checkable |
Not checkable |
31 Mar 2024 |
The more hype, the bigger the fall, if expectations aren’t met.
Not checkable · 31 Mar 2024 |
|
| Not researched |
Factual |
31 Mar 2024 |
Income from AI coding assistants is not large enough to cover the costs of chips, legal and other expenses.
Factual · 31 Mar 2024 |
Economics and markets |
| Not checkable |
Not checkable |
31 Mar 2024 |
Other problems are modest in size.
Not checkable · 31 Mar 2024 |
|
| Not checkable |
Not checkable |
31 Mar 2024 |
The entire industry is based on hype, and on the specific hope that the kinds of problems we saw again and again with GPT-2, GPT-3, and…
Not checkable · 31 Mar 2024 |
|
| Mostly true |
Prediction |
31 Mar 2024 |
Marcus doubted that a mindblowing GPT model would be released in 2024.
Prediction · 31 Mar 2024 |
AI timelines and forecasting |
| Mostly true |
Factual |
31 Mar 2024 |
In early August 2023 Gary Marcus wrote a piece titled 'What if Generative AI turned out to be a Dud', which was picked up by the Financial Times and other outlets.
Factual · 31 Mar 2024 |
Their own work or organisation |
| Mostly true |
Factual |
31 Mar 2024 |
Current-sized LLMs are expensive and not reliable enough for many jobs.
Factual · 31 Mar 2024 |
Economics and markets |
| Not checkable |
Not checkable |
31 Mar 2024 |
You have software that isn’t making much money, isn’t secure, and is keeping a lot of people up at night.
Not checkable · 31 Mar 2024 |
|
| Not checkable |
Not checkable |
31 Mar 2024 |
To be honest, I am not confident Generative AI as currently envisioned will ever come together.
Not checkable · 31 Mar 2024 |
|
| Not checkable |
Not checkable |
31 Mar 2024 |
But as time goes on, I increasingly believe that call was correct.
Not checkable · 31 Mar 2024 |
|
| Unresolvable |
Prediction |
31 Mar 2024 |
If OpenAI releases a truly mindblowing GPT-5 in 2024, hype around generative AI will increase further.
Prediction · 31 Mar 2024 |
Economics and markets |
| Unresolvable |
Factual |
31 Mar 2024 |
LLMs themselves cannot look things up and reason over what they find.
Factual · 31 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
31 Mar 2024 |
Sam Altman hinted in his recent Lex Fridman interview that nothing worthy of the GPT-5 name will be released in 2024.
Factual · 31 Mar 2024 |
AI timelines and forecasting |
| Not checkable |
Not checkable |
31 Mar 2024 |
Find some huge problem(s) that they can charge a lot for.
Not checkable · 31 Mar 2024 |
|
| Not researched |
Factual |
31 Mar 2024 |
Demis Hassabis, recently knighted, has started warning that AI hype is a risk and appears to be trying to forestall it.
Factual · 31 Mar 2024 |
AI policy and governance |
| Not checkable |
Not checkable |
31 Mar 2024 |
the dud idea itself has become quite mainstream.
Not checkable · 31 Mar 2024 |
|
| Unresolvable |
Prediction |
31 Mar 2024 |
Marcus said he would be truly astonished if hallucinations in large language models were brought down to human expert levels by the end of 2024.
Prediction · 31 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
31 Mar 2024 |
Core problems of generative AI such as hallucinations are not going away.
Factual · 31 Mar 2024 |
AI capabilities |
| Unresolvable |
Prediction |
31 Mar 2024 |
If OpenAI releases a truly mindblowing GPT-5 in 2024, generative AI will live for another day, perhaps imploding only later when people realize there is no killer app to justify the increasingly high costs.
Prediction · 31 Mar 2024 |
Economics and markets |
| Mostly true |
Factual |
31 Mar 2024 |
Hallucinations are inherent to how LLMs make predictions.
Factual · 31 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
31 Mar 2024 |
Sam Altman said in a recent interview that GPT-4 'kinda sucks'.
Factual · 31 Mar 2024 |
AI capabilities |
| Unresolvable |
Prediction |
31 Mar 2024 |
If no company releases a true quantum leap addressing reliability, hallucination, data leakage and security by the end of 2024, the generative AI bubble may start to pop by around March 2025.
Prediction · 31 Mar 2024 |
Economics and markets |
| Not checkable |
Not checkable |
31 Mar 2024 |
Meanwhile hype itself is becoming a kind of risk factor.
Not checkable · 31 Mar 2024 |
|
| Not researched |
Factual |
31 Mar 2024 |
Optimistic predictions made by Reid Hoffman on September 7, 2023 about generative AI have not so far come true.
Factual · 31 Mar 2024 |
AI timelines and forecasting |
| Mostly true |
Factual |
31 Mar 2024 |
Marcus cited a statistic from a Wall Street Journal piece, "$50B in, $3B out", and said it was obviously not sustainable.
Factual · 31 Mar 2024 |
Economics and markets |
| Unresolvable |
Factual |
31 Mar 2024 |
All the big AI players are converging on systems roughly equivalent to GPT-4, all flawed in essentially the same ways.
Factual · 31 Mar 2024 |
Economics and markets |
| Mostly true |
Prediction |
31 Mar 2024 |
GPT-5 is likely to be even more expensive to train and operate than GPT-4.
Prediction · 31 Mar 2024 |
Economics and markets |
| Not researched |
Factual |
31 Mar 2024 |
Customers have not been thrilled with Microsoft's Copilot.
Factual · 31 Mar 2024 |
Economics and markets |
| Not researched |
Factual |
31 Mar 2024 |
Generative search is not reliable enough and does not yet have a clear business model.
Factual · 31 Mar 2024 |
Economics and markets |
| Unresolvable |
Prediction |
31 Mar 2024 |
Generative AI will still exist and find some use, but valuations and excitement may dissipate, returning only once genuine advances are made.
Prediction · 31 Mar 2024 |
Economics and markets |
| Not checkable |
Not checkable |
31 Mar 2024 |
Thus far, the closest thing to a killer app is probably coding assistace
Not checkable · 31 Mar 2024 |
|
| Not researched |
Factual |
31 Mar 2024 |
Generative AI revenue so far has been about 0.01% of US GDP.
Factual · 31 Mar 2024 |
Economics and markets |
| Not researched |
Factual |
31 Mar 2024 |
So far most uses of LLMs have been for testing rather than production-level systems.
Factual · 31 Mar 2024 |
Economics and markets |
| Not checkable |
Not checkable |
31 Mar 2024 |
Some new facts have come to light
Not checkable · 31 Mar 2024 |
|
| Resolved true |
Factual |
Feb 2024 |
After Sora's February 2024 announcement, Marcus predicted its physics troubles would persist, and they have.
Factual · Feb 2024 |
Their own work or organisation |
| Not researched |
Factual |
26 Oct 2023 |
A few days before 26 October 2023, Bengio, Hinton and others came out in favor of much of what Sam Altman and Gary Marcus advocated at the US Senate in May 2023.
Factual · 26 Oct 2023 |
AI policy and governance |
| Mostly true |
Factual |
26 Oct 2023 |
Some people, such as Marc Andreessen, are denying that there is any AI risk, long term or short term, and trying to turn whether to pursue AI safety into a debate.
Factual · 26 Oct 2023 |
AI policy and governance |
| Not checkable |
Not checkable |
26 Oct 2023 |
the collective sum of those efforts hasn’t yielded all that much.
Not checkable · 26 Oct 2023 |
|
| Mixed |
Prediction |
26 Oct 2023 |
With more than 60 elections around the world in 2024, the chance that at least one would be swayed by deepfakes is high.
Prediction · 26 Oct 2023 |
Other global risks |
| Not checkable |
Not checkable |
26 Oct 2023 |
Don’t listen to them; enormous money is at stake.
Not checkable · 26 Oct 2023 |
|
| Not researched |
Factual |
26 Oct 2023 |
Gary Marcus first proposed a CERN for AI in 2017.
Factual · 26 Oct 2023 |
Their own work or organisation |
| Not checkable |
Not checkable |
26 Oct 2023 |
Either way, it’s surely a taste of things to come.
Not checkable · 26 Oct 2023 |
|
| Resolved true |
Factual |
26 Oct 2023 |
Bengio et al.'s proposal states that research breakthroughs are needed to solve some of today's technical challenges in creating AI with safe and ethical objectives.
Factual · 26 Oct 2023 |
AI safety and alignment |
| Not researched |
Factual |
26 Oct 2023 |
There is currently no way to guarantee that large language models will reliably stick to facts.
Factual · 26 Oct 2023 |
AI capabilities |
| Not checkable |
Not checkable |
26 Oct 2023 |
Many of the arguments I originally gave for a CERN for AI still hold.
Not checkable · 26 Oct 2023 |
|
| Not checkable |
Not checkable |
26 Oct 2023 |
I don’t know when we will get to such machines, but I do know that we don’t have tons of controls over current AI
Not checkable · 26 Oct 2023 |
|
| Not checkable |
Not checkable |
26 Oct 2023 |
We can either be like Prometheus or Epimetheus, addressing foreseeable risks before they come, or picking up the pieces afterwards.
Not checkable · 26 Oct 2023 |
|
| Not checkable |
Not checkable |
26 Oct 2023 |
Few other ideas are even on the table.
Not checkable · 26 Oct 2023 |
|
| Not checkable |
Not checkable |
26 Oct 2023 |
I think those are legitimate things to worry about
Not checkable · 26 Oct 2023 |
|
| Not checkable |
Not checkable |
26 Oct 2023 |
Anthropic’s “Constitutional AI” idea seems to me to be clever and the best of the bunch, but still inadequate, riddled with the usual…
Not checkable · 26 Oct 2023 |
|
| Not checkable |
Not checkable |
26 Oct 2023 |
We need to stop worrying (just) about Skynet and robots taking over the world, and think a lot more about what criminals, including…
Not checkable · 26 Oct 2023 |
|
| Not researched |
Factual |
26 Oct 2023 |
The UK AI Summit appears to be focusing primarily on long-term AI risk, the risk of future machines going beyond human control.
Factual · 26 Oct 2023 |
AI policy and governance |
| Not checkable |
Not checkable |
26 Oct 2023 |
Is AI going to kill us all? I don’t know, and you don’t either… Although a lot of the literature equates artificial intelligence risk with…
Not checkable · 26 Oct 2023 |
|
| Not researched |
Factual |
26 Oct 2023 |
Current AI has no reliable way to anticipate the consequences of actions it might recommend.
Factual · 26 Oct 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
26 Oct 2023 |
But every day brings us closer to a world in which AI causes real harm.
Not checkable · 26 Oct 2023 |
|
| Not checkable |
Not checkable |
26 Oct 2023 |
The real issue is control.
Not checkable · 26 Oct 2023 |
|
| Not checkable |
Not checkable |
26 Oct 2023 |
Asimov’s laws themselves are insufficient, but if we can’t get the First, how can we believe we are even on track for AI safety at all?
Not checkable · 26 Oct 2023 |
|
| Mostly true |
Factual |
26 Oct 2023 |
A newly published technique shows that current LLMs can be jailbroken quickly and automatically.
Factual · 26 Oct 2023 |
AI safety and alignment |
| Resolved true |
Factual |
26 Oct 2023 |
There is currently no way to guarantee that large language models won't be jailbroken into saying unethical things.
Factual · 26 Oct 2023 |
AI safety and alignment |
| Mostly true |
Factual |
26 Oct 2023 |
Gary Marcus emphasized a CERN for AI and global AI governance in April 2023 in his TED talk and in The Economist.
Factual · 26 Oct 2023 |
Their own work or organisation |
| Not checkable |
Not checkable |
26 Oct 2023 |
Count me among those who continue to favor a CERN for AI safety and Global Safety, so long as we balance the portfolio of risks we are…
Not checkable · 26 Oct 2023 |
|
| Not checkable |
Not checkable |
26 Oct 2023 |
We need to consider both current AI risk and AGI risk. Period.
Not checkable · 26 Oct 2023 |
|
| Not researched |
Factual |
Apr 2023 |
In April 2023 Gary Marcus and Anka Reuel proposed a global, neutral, non-profit International Agency for AI (IAAI).
Factual · Apr 2023 |
Their own work or organisation |