AI in the Classroom

What is Generative Artificial Intelligence?

While artificial intelligence has been present in software and web applications for years, the terms AI or Generative AI have recently been used widely to discuss large language model (LLM) programs which simulate human-like chat responses, such as ChatGPT, Bing Chat, Claude 2, and Google Bard, and image generation tools like Midjourney and DALL-E. LLM tools in particular have received significant media attention since December 2022 for their increasing ability to write passable college essays. By mimicking human writing and reasoning, LLM tools have the potential to shift many existing educational paradigms, both for students and teachers.

One important thing to note when understanding LLM tools is that they do not understand what they are composing. Instead, the tool is guessing the most likely next word that follows the previous word, and the most likely word after that. Warner (2022) describes the inner workings of ChatGPT in layman's terms, saying,

I cannot emphasize this enough: ChatGPT is not generating meaning. It is arranging word patterns. I could tell GPT to add in an anomaly... and it would introduce it into the text without a comment about being anomalous. It is not entirely unlike the old saw about a million monkeys banging on a typewriter for along enough, that one of them would produce the works of Shakespeare through random chance, except this difference is, ChatGPT has been trained on a data set that eliminates all the gibberish.

Haven (2007) describes the neuroscience behind why our brains follow language and syntax rules in stories. LLM tools have no concept of language or syntax rules. The phrase "Colorless green ideas sleep furiously." would not be uttered by a human because it is meaningless (Chomsky, 1991, as cited in Haven, 2007, p. 62); in contrast, these words would not be strung together by an LLM because the probability of each word appearing after the next is so low.

Access to LLM programs is constantly evolving, but in general each requires users to create an account in order to use the program, and most include some form of monetization or access limitation. For instance, ChatGPT charges a monthly premium for users to access the more powerful ChatGPT 4.0, while free accounts can only use GPT 3.5. Claude 2 limits access to a handful of prompts per day, with the ability to use the program more by paying a monthly subscription. These access constraints will continue to change as LLM tools are developed and as companies acquire and seek to monetize them.

What are the limitations of LLM Tools?

The most important limitation to understand about these tools is that, since they do not understand what they are composing and instead chain related words together, they are prone to factual inaccuracies, to the point that they will sometimes produce demonstrably false information in response to user prompts. LLM tools will seem to show great confidence in their fabricated information, a tendency that has been labelled as “hallucinations”.

Part of the reason for this is that most LLMs are currently not connected to the internet and so they cannot accurately fact-check their responses. ChatGPT 3.5, for instance, was trained on information that stops in 2021, so it will not have access to information for recent than that. This may change, however, as the tools are continually developed; Bing Chat uses the language model used by ChatGPT, but also searches Bing to find active information on the topics it references.

Nevertheless, since even LLM tools that are connected to the internet hallucinate, we advise that users do not look to LLM tools as a way to search for or compose factual information. These tools should not be considered “The New Google”. Instead, LLM tools can be used for brainstorming, or as a conversational partner in talking through your thoughts, or producing language drafts that can then be filtered and checked by a human for accuracy and appropriateness.

Finally, LLM developers generally try to add intentional limitations around the kinds of content produced by their tool. For instance, ChatGPT will not answer questions that may be harmful to humans, like those that promote violence, racism, homophobia, etc. (Sharkland, 2022). The moderation is not completely effective, so OpenAI does ask users to report potential harms in addition to bugs (OpenAI, 2022).

On the Horizon

These tools are constantly evolving and tech companies are continuing to invest in refining their models and incorporating them into tools we already use.

• Microsoft will be rolling out Copilot in November to Microsoft Office products, which will incorporate ChatGPT functionality within office productivity applications.
• Google continues to update its LLM Bard with improved functionality.

 What does the availability of generative AI mean for higher education?

Answers to this question of course vary widely among educators, with some seeing it as a threat, some as an equalizer, some as a tool, some as all of those and more. Regardless of how you answer this question, though generative AI is readily available to most students, and most have already used it in a variety of capacities: to generate ideas and outlines for papers; to summarize or rephrase information; to get an immediate, responsive explanation for a complex topic; and, yes, sometimes as a search engine or as a way to quickly have it do their homework for them.

In response, some educators have encouraged the adoption of AI detection tools. These tools generally have a low rate of accuracy, and have displayed a pattern of bias against non-native English writers (Liang, 2023). Others have attempted to assign projects using AI that purposely highlight the factual inaccuracies and hallucinations that the tools can produce to encourage students to stop using the tools altogether. While it is important to understand the limitations of these tools in regards to output accuracy, both of these approaches take an avoidant approach to AI. Instead of avoiding the technology, Ethan Mollick (2023) argues that students will be using these tools regardless of our opinion of them, and that we are uniquely positioned as educators to help them navigate this new environment:

Students will cheat with AI. But they also will begin to integrate AI into everything they do, raising new questions for educators. Students will want to understand why they are doing assignments that seem obsolete thanks to AI. They will want to use AI as a learning companion, a co-author, or a teammate. They will want to accomplish more than they did before, and also want answers about what AI means for their future learning paths. Schools will need to decide how to respond to this flood of questions.

In addition to grappling with how to help students understand these tools and use them responsibly and ethically, educators must also contend with how generative AI can impact their assessment practices. LLM tools are excellent at producing passable five-paragraph essays and other shorter papers, especially when the topics are formulaic or are concerned with widely-studied materials. Essay exam responses, especially those meant to test knowledge gained in a particular subject, are also easy to outsource to LLM tools. These assessment strategies have previously been decent indicators that, among other things, students had spent time devoted to understanding the material or crafting a well-reasoned argument, and in doing so, were gaining skills critical to a successful education. Bioethicist Nita Farahany discusses how the availability of generative AI might impact these traditional assessment tasks, saying

Teachers… [need] to be thinking about, okay, what are the fundamental skills of reasoning and critical thinking and empathy and emotional intelligence and mental agility that we think are essential and that we have been teaching all along but we have been teaching by tasks that now can be outsourced? And then how do we shift our teaching to be able to teach those skills? (Ward, 2023, 1:08:16)

While this is admittedly a large task, this guide aims to provide you with several resources to start thinking through how we can address and incorporate AI in our teaching, including ideas for bringing AI into your classroom in instructional activities, and how you can adjust your assignments to make them less likely to be quickly outsourced.

Additional Resources

Sample Syllabus Statement

The use of generative AI (e.g. ChatGPT, Bing, Bard [LaMDA], copilot, AlphaCode, DALL-E, Midjourney, etc) on any assignment without specific permission is a form of academic dishonesty. Such tools prevent you from engaging with important concepts and topics. As a professor, I want to know what you think, how you developed your ideas, and how you see your ideas interacting with others. Please talk with me if you have questions.

Note: If AI Tools Are Allowed

The Sentient Syllabus Project recognizes that students will begin to use AI in their work. In addition to laying out guidelines for how to ethically use AI in an academic course, this syllabus redefines achievement and pushes students to produce work that surpasses work produced by AI. (Note that this syllabus language assumes that most students will use AI somewhere in their writing process.)

References

Haven, K. (2007). Story proof: The science behind the startling power of story. Libraries Unlimited.

Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023). GPT detectors are biased against non-native English writers. Patterns, 4(7). https://doi.org/10.1016/j.patter.2023.100779 

Mollick, E. (2023, July 1). The homework apocalypse. The Homework Apocalypse - by Ethan Mollick. https://www.oneusefulthing.org/p/the-homework-apocalypse?utm_source=profile&utm_medium=reader2

OpenAI. (2022, November 30). ChatGPT: Optimizing language models for dialogue. Retrieved January 5, 2023, from https://openai.com/blog/chatgpt/

Sharkland, S. (2022, December 22). Why everyone's obsessed with ChatGPT, a mind-blowing AI chatbot. CNET. Retrieved January 5, 2023, from https://www.cnet.com/tech/computing/why-everyones-obsessed-with-chatgpt-a-mind-blowing-ai-chatbot/

Steipe, B. (2023). The Sentient Syllabus Project. http://sentientsyllabus.org

Ward, A. (Host). (2023, August 2). Neurotechnology (AI + BRAIN TECH) with Dr. Nita Farahany [Audio Podcast Episode]. In Ologies. https://www.alieward.com/ologies/neurotechnology 

Warner, J. (2022, December 11). ChatGPT can't kill anything worth preserving: If an algorithm is the death of high school English, maybe that's an okay thing. The Biblioracle Recommends. Retrieved on January 11, 2023, from https://biblioracle.substack.com/p/chatgpt-cant-kill-anything-worth