Facebook has deployed XLM-R, a model based on its RoBERTa architecture, to improve hate speech classifiers in multiple languages across Facebook and Instagram.
Mostly true True in substance, wrong in a detail.
Factual: Statements about the present or the past, checked against the record: did they hold up?
The second judge would not resolve this claim; the verdict shown is the first judge's.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
True if Facebook/Meta records or credible reports from on or before 2021-03-04 show that XLM-R, a model built on RoBERTa, had been deployed in production to improve hate speech classifiers in more than one language on both Facebook and Instagram. Mostly true if it was deployed for hate speech in multiple languages but a minor detail differs, such as the deployment being confirmed on only one platform or XLM-R being described somewhat differently in relation to RoBERTa. Mixed if it was deployed but for a materially different purpose or in only one language. Mostly false if XLM-R was only tested or announced as planned for this use, not deployed, by that date. False if there is evidence it was not deployed for hate speech classification at all.
Set before the research began.
Evidence (7)
-
E1 Community Standards Enforcement Report, Fourth Quarter 2020 Meta (about.fb.com) · 11 February 2021
“26.9 million pieces of hate speech content, up from 22.1 million in Q3 due in part to updates in our technology in Arabic, Spanish and Portuguese”
-
E2 facebook, inc. community standards enforcement report, q4 ... Meta (about.fb.com, press call transcript) · 11 February 2021
“We have a new architecture called Linformer, which we put into production last October that allows AI models to train on larger and more complex pieces of text.”
-
E3 Facebook details AI advances in catching misinformation and hate speech | TechCrunch TechCrunch · 19 November 2020
“One improvement is in the language analysis systems Facebook employs to detect things like hate speech.”
-
E4 FACEBOOK, INC. COMMUNITY STANDARDS ENFORCEMENT REPORT, Q4 2021 (heading as printed on the page; the page has no separate title) Facebook, Inc. · 11 February 2021 Own record
“We also saw progress in the way our systems operate in multiple languages thanks largely to improvements in Spanish and Arabic, the number of pieces of hate speech content that were taken down increased from 26.9 million up from 22.1 million in the previous quarter.”
-
E5 [PDF] FACEBOOK, INC. November 19, 2020 11:30 a.m. ET Operator Facebook, Inc. (Community Standards Enforcement Report Q3 2020 call transcript) · 19 November 2020 Own record
“In November of 2019, we announced new state of the art research, XLM-R, and a year later, it's in production helping us detect harmful content in multiple languages across Facebook and Instagram.”
-
E6 Facebook hopes its new AI moderation tools can further counter hate speech - Engadget Engadget · 19 November 2020
“In addition to Facebook's existing array of tools including semi-supervised self-learning models and XLM-R, the company unveiled and implemented a pair of new technologies.”
-
E7 AI gets better every day. Here’s what that means for stopping hate speech Meta AI (Facebook AI blog) · 11 February 2021 Own record
“Thanks largely to improvements in the way our AI tools can detect violating content in widely spoken languages like Spanish and Arabic, the amount of hate speech content that was taken down reached 26.9 million, up from 22.1 million in the previous quarter.”
Reasoning
Facebook's official transcript of its November 2020 Community Standards Enforcement Report call says XLM-R was "in production helping us detect harmful content in multiple languages across Facebook and Instagram" (E5). Engadget, an independent outlet, reported the same day that XLM-R was among Facebook's existing tools for countering hate speech (E6). Both items are dated before 2021-03-04. However, E5 says "harmful content" rather than hate speech classifiers specifically, and none of the evidence describes XLM-R's link to RoBERTa, so minor details are not fully confirmed.
Cite this claim
Yann LeCun, “Facebook has deployed XLM-R, a model based on its RoBERTa architecture, to improve hate speech classifiers in multiple languages across Facebook and Instagram.” (said 4 March 2021). Verdict: mostly true. Reliability API, claim c_433dd1ad9a5c, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_433dd1ad9a5c
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Yann LeCun
-
Mostly true
“For starters, transformers are equivariant to permutations.”
Transformers are equivariant to permutations, as ConvNets are equivariant to translations.
-
Mostly true
“If it were the case, we would have AI systems that could teach themselves to drive a car in 20 hours of practice, like any 17 year-old.”
-
Mostly true
“The architecture is good, the architecture of the encoder is good, but the fact that you train the system to reconstruct images does not lead it to produce to long, good generic features of images.”
-
Mostly true
“LLMs can do none of those or they can only do them in a very primitive way and they don’t really understand the physical world.”
-
Mostly true
“No, there’s one thing that autoregressive LLMs or that LLMs in general, not just the autoregressive one, but including the bird style bidirectional ones, are exploiting and its self supervised…”
Verdicts are made automatically; how verdicts are made.