Generative AI & Research

Think of ChatGPT as a more sophisticated version of predictive text when you're in iMessage or another messaging app. It's not concerned with what's true--only with what is the most likely next word or string of words. It's important to remember that ChatGPT doesn't have any built-in fact-checking capabilities to date. In fact, the first time you use ChatGPT, you may see a pop-up window with a message encouraging you to "check your facts."

This is less common now than it was even a few months ago, but Generative AI tools can completely make up people, events, and even sources like scholarly articles. It can give you an inaccurate or misleading answer to your prompt. What you're more likely to see, though, is that these tools might mix fact and fiction--which means it's up to you to figure out which is which. 

The good news about fact-checking information from Generative AI tools is that you do it exactly the same way you would for any other information source. Think about how you might decide whether a TikTok video is accurate or if you can trust the information you're finding on Wikipedia. You probably already do something called lateral reading, which involves double-checking information against other sources.

• Watch: how to read laterally (2 min TikTok video)
• Watch: "Sort Fact from Fiction Online with Lateral Reading" (4 min YouTube video)

The TikTok video linked above focuses on where information is coming from and the source's reputation. When your source is ChatGPT, you can't really do this. However, you can use the exact same technique to try to confirm the information the tool has produced. You'll need to break the information down into smaller chunks since you might be dealing with a mix of fact and fiction, but you follow the same basic steps in every situation. 

The steps below are taken from the University of Maryland Libraries' guide to AI:

1. Break down the information. Take a look at the response and see if you can isolate specific, searchable claims. 
2. Then it’s lateral reading time! Open a new tab and look for supporting pieces of information. Here are some good sources to start with:
   • When searching for specific pieces of information: Google results or Wikipedia
   • When seeing if something exists: Google Scholar, WorldCat, or Wikipedia
   • Tip: Some things to watch out for – is the AI putting correct information in the wrong context? Is it attributing a fake article to a real author?
3. Next, think deeper about what assumptions are being made here
   • What did your prompt assume?
   • What did the AI assume?
   • Who would know things about this topic? Would they have a different perspective than what the AI is offering? Where could you check to find out?
4. Finally, make a judgment call. What here is true, what is misleading, and what is factually incorrect? Can you re-prompt the AI to try and fix some of these errors? Can you dive deeper into one of the sources you found while fact-checking? Remember, you’re repeating this process for each of the claims the AI made – go back to your list from the first step and keep going!

University of Maryland Libraries. (2023, October 18). Assess content. Artificial Intelligence (AI) and Information Literacy. https://lib.guides.umd.edu/c.php?g=1340355&p=9880575

In the example linked below, we asked ChatGPT 3.5 to write a five-paragraph essay about the experience of an actual female student at Dartmouth College during the early days of coeducation, including specific information about any challenges she might have faced. 

Can you use lateral reading to find out what year Sarah Thompson graduated and what her major was?