Fact Checkers Accuracy Called Into Question
“are you, you as a fact checker accurate?”
Summary
While newer versions of ChatGPT generate factual answers more reliably than earlier models, they still produce hallucinations and incorrect information at notable rates. Therefore, the output should be verified and cannot be assumed completely accurate.
Sources 59 searched
- Fact Checking and Evaluation - Generative AI - Research Guides at Santa Clara University
According to OpenAi, there are three limitations you should know about ChatGPT: ChatGPT may create "hallucinations" - incorrect or misleading information that may sound convincing. Sometimes, these are facts that it makes up, but they can also be false citations or quotes.
- Fact checking is always needed - Student Guide to ChatGPT - LibGuides at University of Arizona
Scholarly sources as grounding There are AI systems that combine language models with scholarly sources. These models use scholarly papers as a source of grounding, which makes them more factual.
- Fact checking is always needed - Student Guide to Generative AI - Guides at Baylor University
According to OpenAI, it's "40% more likely to produce factual responses than GPT-3.5 on our internal evaluations." But it's still not perfect. So verification of the output is still needed. ChatGPT often makes up fictional sources One area where ChatGPT usually gives fictional answers is when ...
- How to Fact-Check ChatGPT Responses (The Honest Way) | QWE AI Academy
ChatGPT's hallucination rate hits 45% in some tests. Learn the 3-step verification method researchers use (not just 'check sources') plus the hidden setting that changes everything.
- Fact checking is always needed - Student Guide to Generative Artificial Intelligence - Research Guides at Modesto Junior College Library
Learn to use generative AI (like ChatGPT) ethically and effectively
- Fact-checking AI with Lateral Reading - Using AI Tools in Research - Research Guides at Texas A&M University-Corpus Christi
Here's how to fact-check something you got from ChatGPT or a similar tool:
- What does AI get wrong? - Artificial Intelligence (AI) and Information Literacy - Research Guides at University of Maryland Libraries
This means that these footnotes are not where the AI sourced its information. (Also note that the sources on the right are all either not peer-reviewed or not relevant. Plus, artsy.net, history.com, and certainly theprouditalian.com are not reliable enough for you to source from in your assignments.) This matters because an important part of determining a human author’s credibility is seeing what sources they draw on for their argument. You can go to these sources to fact-check the information they provide, and you can look at their sources as a whole to get insight into the author’s process, potentially revealing a flawed or biased way of information-gathering.
- Fact-checking AI with Lateral Reading - Artificial Intelligence (AI) and Information Literacy - Research Guides at University of Maryland Libraries
Of course, many of you know that Jim Henson completed his undergraduate at UMD, not Ohio State, which illustrates a critical distinction to be made about AI; it will take what you provide it and try to answer your question as best it can, but it will NOT fact check you or spot incorrect assumptions in the prompt you give it.
- AI is so good now, do I really still need to fact-check? – Learning Technologies
What it cannot do reliably is judge the quality of sources it finds on its own. It does not know whether something comes from a respected journal or from a random blog. And even when it provides links, it can struggle to show which parts it relied on or how accurate those sources are.
This fact check is free and donation-based. $1 powers ~30 fact-checks.
