| Not checkable |
Not checkable |
30 Mar 2023 |
But I don’t think that legitimate concerns about the hypey-ness of letter in anyway undermines the real need to rein in the systems are…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Much (not all) of the pushback seemed off base to me
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I personally haven't changed; I still think that LLMs are unreliable, and still think that they are a very poor basis for factuality.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
But that doesn't mean that they don't have the potential to rip apart our social fabric
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
This doesn’t mean we are doomed, but we do need to think hard, and quickly, about what proper measures look like, just as we have done for…
Not checkable · 30 Mar 2023 |
|
| Unresolvable |
Factual |
30 Mar 2023 |
Gary Marcus wrote a long Twitter thread on the morning of 2023-03-30 responding to criticism of the FLI letter.
Factual · 30 Mar 2023 |
Their own work or organisation |
| Not researched |
Factual |
30 Mar 2023 |
The FLI open letter did not call for a ban on GPT-4.
Factual · 30 Mar 2023 |
Their own work or organisation |
| Not checkable |
Not checkable |
30 Mar 2023 |
One criticism do I agree with is this
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I am with Stability AI founder Emad Mostaque in his read on how most people actually working on this stuff see things:
Not checkable · 30 Mar 2023 |
|
| Not researched |
Factual |
30 Mar 2023 |
Most signers of the FLI open letter, including Gary Marcus and Yoshua Bengio, have no connection to the Future of Life Institute.
Factual · 30 Mar 2023 |
Their own work or organisation |
| Not checkable |
Not checkable |
30 Mar 2023 |
The real news here is not that Elon Musk signed it but that so many people who are not natural allies (eg Bengio and l myself, famous for…
Not checkable · 30 Mar 2023 |
|
| Not researched |
Factual |
30 Mar 2023 |
The letter Marcus was discussing did not call for a ban on most AI research, only for a brief pause on one very specific project involving a technology with known risks and no known solutions.
Factual · 30 Mar 2023 |
Their own work or organisation |
| Not checkable |
Not checkable |
30 Mar 2023 |
A lot of the attacks on the letter focused on who sponsored it, not who signed it.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Sooo many of the attacks on the proposed 6 month ban on training super large #LLMs miss the point.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Please put your own suggestions in the comments below.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
It is *not* fine to do nothing.
Not checkable · 30 Mar 2023 |
|
| Resolved true |
Factual |
30 Mar 2023 |
The FLI open letter called for more AI research.
Factual · 30 Mar 2023 |
Their own work or organisation |
| Not checkable |
Not checkable |
30 Mar 2023 |
The letter should by judged by what it says, not who wrote it.
Not checkable · 30 Mar 2023 |
|
| Not researched |
Factual |
30 Mar 2023 |
Virtually everyone, including people at OpenAI, has acknowledged serious AI risks, but as of March 2023 few tangible steps to mitigate them have been taken by government or industry.
Factual · 30 Mar 2023 |
AI policy and governance |
| Not checkable |
Not checkable |
30 Mar 2023 |
I don't think they are close to AGI.
Not checkable · 30 Mar 2023 |
|
| Delivered on time |
Promise |
30 Mar 2023 |
Gary Marcus will speak at TED about short-term and long-term AI risk and what to do about it, in about two weeks from 2023-03-30.
Promise · 30 Mar 2023 |
Their own work or organisation |
| Resolved true |
Factual |
30 Mar 2023 |
The FLI open letter did not call for a ban on AI or for a permanent ban.
Factual · 30 Mar 2023 |
Their own work or organisation |
| Not checkable |
Not checkable |
30 Mar 2023 |
It is perfectly fine to propose a different alternative, but most of the critiques of the letter have not.
Not checkable · 30 Mar 2023 |
|
| Mostly true |
Factual |
30 Mar 2023 |
As of March 2023, none of the top AI labs is remotely transparent, AI governance basically doesn't exist, and there are basically no safeguards in place.
Factual · 30 Mar 2023 |
AI policy and governance |
| Not checkable |
Not checkable |
30 Mar 2023 |
As I said yesterday to The New York Times, “We have a perfect storm of corporate irresponsibility, widespread adoption, lack of regulation…
Not checkable · 30 Mar 2023 |
|
| Unresolvable |
Factual |
30 Mar 2023 |
Gary Marcus signed the Future of Life Institute open letter and promoted it publicly.
Factual · 30 Mar 2023 |
Their own work or organisation |
| Not checkable |
Not checkable |
30 Mar 2023 |
Not everyone who signed the letter is principally concerned with long-term risk; many of us who signed are worried at least as much about…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
The idea (which I have actually heard expressed) that AI should be exempt from regulation is absurd.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
To my mind, doing nothing is truly the most foolish option.
Not checkable · 30 Mar 2023 |
|
| Resolved true |
Factual |
24 Mar 2023 |
There have already been reports of contamination in GPT-4's training data.
Factual · 24 Mar 2023 |
AI capabilities |
| Not checkable |
Not checkable |
24 Mar 2023 |
The strengths and weaknesses of GPT-4 are qualitatively the same as before.
Not checkable · 24 Mar 2023 |
|
| Not checkable |
Not checkable |
24 Mar 2023 |
They reify the practice of substituting press releases for science and the practice of discussing models with entirely undisclosed…
Not checkable · 24 Mar 2023 |
|
| Not checkable |
Not checkable |
24 Mar 2023 |
I don’t expect Coca Cola to present its secret formula.
Not checkable · 24 Mar 2023 |
|
| Mostly true |
Factual |
24 Mar 2023 |
The OpenAI and Microsoft GPT-4 papers reveal nothing about the model's architecture or training set.
Factual · 24 Mar 2023 |
AI capabilities |
| Not checkable |
Not checkable |
24 Mar 2023 |
It’s a silly claim, given that it is entirely open to interpretation
Not checkable · 24 Mar 2023 |
|
| Not checkable |
Not checkable |
24 Mar 2023 |
Perhaps all this would all be fine if the companies weren’t pretending to be contributors to science, formatting their work as science with…
Not checkable · 24 Mar 2023 |
|
| Mostly true |
Factual |
24 Mar 2023 |
Citing Ernie Davis, Marcus said OpenAI had begun to incorporate user experiments into its models' training corpus.
Factual · 24 Mar 2023 |
AI capabilities |
| Resolved true |
Factual |
24 Mar 2023 |
GPT-4 has not solved hallucinations, reliability or planning on complex tasks, as the paper's authors acknowledge for planning.
Factual · 24 Mar 2023 |
AI capabilities |
| Not checkable |
Not checkable |
24 Mar 2023 |
Microsoft and OpenAI are producing and deploying products with potentially enormous risks at mass scale, and making extravagant, unreviewed…
Not checkable · 24 Mar 2023 |
|
| Not checkable |
Not checkable |
24 Mar 2023 |
Microsoft and OpenAI are rolling out extraordinarily powerful yet unreliable systems with multiple disclosed risks and no clear measure…
Not checkable · 24 Mar 2023 |
|
| Not checkable |
Not checkable |
24 Mar 2023 |
But nor do I plan to give them scientific credibility for alleged advances that we know nothing about.
Not checkable · 24 Mar 2023 |
|
| Mostly true |
Factual |
24 Mar 2023 |
Gary Marcus said that on 2023-03-23 Microsoft had put out what he called a press release masquerading as science, which claimed that GPT-4 was an early (yet still incomplete) version of an AGI system.
Factual · 24 Mar 2023 |
AI capabilities |
| Not checkable |
Not checkable |
24 Mar 2023 |
if Coca Cola wants to keep secrets, that’s fine; it’s not particularly in the public interest to know the exact formula.
Not checkable · 24 Mar 2023 |
|
| Not checkable |
Not checkable |
24 Mar 2023 |
By excluding the scientific community from any serious insight into the design and function of these models, Microsoft and OpenAI are…
Not checkable · 24 Mar 2023 |
|
| Not checkable |
Not checkable |
24 Mar 2023 |
if we don’t get it, we must contemplate shutting these projects down.
Not checkable · 24 Mar 2023 |
|
| Not checkable |
Not checkable |
24 Mar 2023 |
This cannot and should not stand.
Not checkable · 24 Mar 2023 |
|
| Not researched |
Factual |
24 Mar 2023 |
OpenAI CEO Sam Altman has recently publicly expressed fears about where AI development is going.
Factual · 24 Mar 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
24 Mar 2023 |
That claim would never survive serious scientific peer review.
Not checkable · 24 Mar 2023 |
|
| Not checkable |
Not checkable |
24 Mar 2023 |
You would archive the message without reading further.
Not checkable · 24 Mar 2023 |
|
| Not checkable |
Not checkable |
24 Mar 2023 |
literally cannot be tested with serious scrutiny, because the scientific community has no access to the training data.
Not checkable · 24 Mar 2023 |
|
| Not checkable |
Not checkable |
24 Mar 2023 |
We must demand transparency
Not checkable · 24 Mar 2023 |
|
| Not checkable |
Not checkable |
24 Mar 2023 |
As I have said before, I don’t really think GPT-4 has much to do with AGI.
Not checkable · 24 Mar 2023 |
|
| Not checkable |
Not checkable |
24 Mar 2023 |
At some point, we would want public hearings.
Not checkable · 24 Mar 2023 |
|
| Not researched |
Factual |
Feb 2023 |
Marcus's February 2023 warning that generative AI would corrupt the internet increasingly appears correct.
Factual · Feb 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
25 Dec 2022 |
I have never seen anything like this much buzz.
Not checkable · 25 Dec 2022 |
|
| Mostly true |
Prediction |
25 Dec 2022 |
GPT-4 will still be reckless and hard to control, making a significant number of stupid errors.
Prediction · 25 Dec 2022 |
AI capabilities |
| Mostly true |
Prediction |
25 Dec 2022 |
GPT-4's natural language output still won't be reliably usable as input to downstream programs such as databases or virtual assistants.
Prediction · 25 Dec 2022 |
AI capabilities |
| Mostly true |
Factual |
25 Dec 2022 |
ChatGPT regularly makes things up, and a fair bit of what it says is untrue.
Factual · 25 Dec 2022 |
AI capabilities |
| Not checkable |
Not checkable |
25 Dec 2022 |
But will it solve the problems we have seen before? I am not so sure.
Not checkable · 25 Dec 2022 |
|
| Unresolvable |
Prediction |
25 Dec 2022 |
Trustworthy AI will need genuinely new architectures incorporating explicit knowledge and world models at their core.
Prediction · 25 Dec 2022 |
AI capabilities |
| Unresolvable |
Factual |
25 Dec 2022 |
ChatGPT is frequently retuned based on feedback, but weeks later Marcus still keeps being sent tweets showing its blunders.
Factual · 25 Dec 2022 |
AI capabilities |
| Unresolvable |
Prediction |
25 Dec 2022 |
When GPT-4 comes out it will totally eclipse ChatGPT, and it is a safe bet that even more people will be talking about it.
Prediction · 25 Dec 2022 |
AI capabilities |
| Resolved true |
Prediction |
25 Dec 2022 |
GPT-4 will definitely seem smarter than its predecessors, but its internal architecture will remain problematic.
Prediction · 25 Dec 2022 |
AI capabilities |
| Not checkable |
Not checkable |
25 Dec 2022 |
If all seven predictions prove correct, I hope that the field will finally realize that it is time to move on.
Not checkable · 25 Dec 2022 |
|
| Mostly true |
Prediction |
25 Dec 2022 |
GPT-4 will not be trustworthy and complete enough to give reliable medical advice.
Prediction · 25 Dec 2022 |
AI capabilities |
| Mostly true |
Factual |
25 Dec 2022 |
ChatGPT might sound like the Star Trek computer, but for now it still cannot be trusted.
Factual · 25 Dec 2022 |
AI capabilities |
| Unresolvable |
Prediction |
25 Dec 2022 |
Marcus fully expected GPT-4 to be the shiniest model so far, but said that this does not mean it is a critical step on the optimal path to AI that can be trusted.
Prediction · 25 Dec 2022 |
AI capabilities |
| Mostly true |
Prediction |
25 Dec 2022 |
GPT-4 will be a very large, powerful system (a 'monster').
Prediction · 25 Dec 2022 |
AI capabilities |
| Mostly true |
Prediction |
25 Dec 2022 |
With GPT-4, fluent hallucinations will still be common and easily induced, escalating the risk of LLMs being used to create misinformation.
Prediction · 25 Dec 2022 |
AI capabilities |
| Mostly true |
Prediction |
25 Dec 2022 |
GPT-4 may be better at faking term papers, but if it follows the same playbook as its predecessors, it still won't really understand the world, and its flaws will eventually show.
Prediction · 25 Dec 2022 |
AI capabilities |
| Not yet due |
Prediction |
25 Dec 2022 |
Within a decade, maybe much less, AI's focus will shift from pure scaling of large language models to integrating them with other techniques.
Prediction · 25 Dec 2022 |
AI capabilities |
| Unresolvable |
Factual |
25 Dec 2022 |
GPT-4 is architecturally essentially the same as GPT-3, according to rumors.
Factual · 25 Dec 2022 |
AI capabilities |
| Unresolvable |
Prediction |
25 Dec 2022 |
GPT-4 by itself won't be a general-purpose AGI; without external aids it won't beat Cicero in Diplomacy, drive a car reliably, or reliably guide a robot like Optimus.
Prediction · 25 Dec 2022 |
AI capabilities |
| Mostly true |
Prediction |
25 Dec 2022 |
GPT-4 will blow people's minds when released.
Prediction · 25 Dec 2022 |
AI capabilities |
| Own records only |
Factual |
25 Dec 2022 |
Marcus knows several people who have tried GPT-4, and all were impressed.
Factual · 25 Dec 2022 |
AI capabilities |
| Mostly true |
Prediction |
25 Dec 2022 |
Marcus suspected that GPT-4's release would follow a pattern of immense buzz, then scientific scrutiny, then recognition that many problems remain.
Prediction · 25 Dec 2022 |
AI capabilities |
| Mostly true |
Prediction |
25 Dec 2022 |
If GPT-4 is architecturally essentially the same as GPT-3, it can be expected to be unable to construct internal models of how the world works and, in consequence, unable to understand things at an abstract level.
Prediction · 25 Dec 2022 |
AI capabilities |
| Mostly true |
Prediction |
25 Dec 2022 |
AI alignment between human wants and machine behavior will remain a critical unsolved problem, with GPT-4 unable to reliably follow shared human values.
Prediction · 25 Dec 2022 |
AI safety and alignment |
| Mostly true |
Factual |
25 Dec 2022 |
ChatGPT can write about anything but is easily confused.
Factual · 25 Dec 2022 |
AI capabilities |
| Unresolvable |
Prediction |
25 Dec 2022 |
GPT-4's reasoning about the physical, psychological and mathematical world will still be unreliable.
Prediction · 25 Dec 2022 |
AI capabilities |
| Mostly true |
Prediction |
25 Dec 2022 |
GPT-4 will remain a pastiche generator and useful for brainstorming and first drafts, but not trustworthy general intelligence.
Prediction · 25 Dec 2022 |
AI capabilities |
| Mostly true |
Factual |
Dec 2022 |
AI-generated misinformation became increasingly widespread, as Marcus warned in December 2022, but its effects have so far been less than he feared.
Factual · Dec 2022 |
AI safety and alignment |
| Not researched |
Factual |
Dec 2022 |
Marcus's December 2022 prediction of chatbot-associated deaths was confirmed in 2023, with another apparent incident in 2024.
Factual · Dec 2022 |
AI safety and alignment |
| Mostly true |
Factual |
Mar 2022 |
Gary Marcus's March 2022 article argued pure scaling would not solve hallucinations or abstraction and concluded there are serious holes in the scaling argument.
Factual · Mar 2022 |
Their own work or organisation |
| Resolved true |
Factual |
Mar 2022 |
In March 2022 Gary Marcus published an article arguing that deep learning is hitting a wall.
Factual · Mar 2022 |
Their own work or organisation |
| Not researched |
Factual |
2022 |
Marcus's predictions since 2022 that pure LLM scaling would run out appear to have been confirmed in November and December 2024 by comments from industry insiders.
Factual · 2022 |
AI timelines and forecasting |
| Not checkable |
Not checkable |
29 Dec 2021 |
No current AI is remotely close to understanding the everyday physical or psychological world
Not checkable · 29 Dec 2021 |
|
| Not checkable |
Not checkable |
29 Dec 2021 |
clueless statistical approximators.
Not checkable · 29 Dec 2021 |
|
| Not checkable |
Not checkable |
29 Dec 2021 |
these systems are long way from understanding basic linguistic like negation
Not checkable · 29 Dec 2021 |
|
| Not checkable |
Not checkable |
29 Dec 2021 |
Game on!
Not checkable · 29 Dec 2021 |
|
| Unresolvable |
Prediction |
29 Dec 2021 |
Current AI, being an approximation to intelligence rather than the real thing, will never really be trustworthy.
Prediction · 29 Dec 2021 |
AI safety and alignment |
| Not checkable |
Not checkable |
28 Dec 2021 |
seriously lacking in AI
Not checkable · 28 Dec 2021 |
|
| Not checkable |
Not checkable |
26 Dec 2021 |
This is not "awesome"; it is deeply confused.
Not checkable · 26 Dec 2021 |
|
| Not checkable |
Not checkable |
23 Dec 2021 |
Good thread (and review) on neurosymbolic programming, very much in line with The Algebraic Mind, which I hope will get the attention it…
Not checkable · 23 Dec 2021 |
|
| Not checkable |
Not checkable |
22 Dec 2021 |
we are calling for a respect for what nature has to teach us, and for humility in accepting what we have not yet discovered
Not checkable · 22 Dec 2021 |
|
| Not checkable |
Not checkable |
22 Dec 2021 |
For those who have forgotten their biology
Not checkable · 22 Dec 2021 |
|
| Not checkable |
Not checkable |
20 Dec 2021 |
Trainable deep networks accumulate statistics between features; they don’t induce operations over variables or structured representations…
Not checkable · 20 Dec 2021 |
|
| Unresolvable |
Factual |
20 Dec 2021 |
Gary Marcus wrote the 2001 book The Algebraic Mind, and its predictions have so far held true.
Factual · 20 Dec 2021 |
Their own work or organisation |
| Not checkable |
Not checkable |
19 Dec 2021 |
It's not all about the data.
Not checkable · 19 Dec 2021 |
|
| Not checkable |
Not checkable |
19 Dec 2021 |
the deeper problem, per The Algebraic Mind, remains lack of adequate substrate for structured representations, individuals & abstract…
Not checkable · 19 Dec 2021 |
|