AI and Information Credibility

This classroom activity was developed and used in an entry-level Biotechnology course to help students evaluate the credibility of content generated by six different generative AI tools: Gemini, ChatGPT, Poe, Claude, Consensus, and Google's AI Overview. Working in groups, students review the text of a chat the librarian generated. They evaluate the credibility of the text, paying special attention to purpose, objectivity and bias, accuracy of content, and newness (if sources are provided by the tool). This activity could be modified to be done with any topic or prompt.

The librarian used the same prompt to generate responses from the tools in advance so that students would not be required to make accounts to use any of the tools. 

Activity instructions:

Each group will be given / assigned one generative AI tools. On 25 Sept. 2024, each tool was fed the prompt: “Provide a summary of how biotechnology can be used with plants to make plants virus resistant. Please provide evidence and sources when applicable. The audience is college students in an entry-level biotechnology course.” (Two tools had modified prompts.)

Each group will evaluate the credibility of the text generated by their assigned tool. Pay special attention to:

• Purpose - Why does this information exist in the way it does?
• Verifiability (Accuracy) - Can you verify this information? Is the generated summary factual? Is there any way for you to know where the information originally came from?
  • If it provides references or suggested resources, check them against Google Scholar. Do they exist? Or were they hallucinated?
• Newness - If it provides references or suggested resources, is the currency of those sources appropriate?

Please be prepared to share your thoughts on these four factors for your assigned tool.

Attribution:

This activity is built upon a modified form of the "P.R.O.V.E.N. Source Evaluation Process" by Ellen Carey. CC BY-NC-SA 3.0.