Evolving generative AI (AI) tools present both opportunities and tensions related to learning and academic integrity. Instructors can shape how students use AI in their courses. Some instructors permit, support, or encourage students to use AI, while others avoid the risk of misuse by reverting to handwritten, in-person exams and assessments. However, these in-person practices present accessibility challenges for students. In response, instructors may turn to AI content detectors as part of banning AI. However, tracking AI use with a detection tool is practically impossible and introduces ethical and workload challenges (Berdahl, 2024).
Limitations of AI Detectors
Currently, AI detection tools are not reliable or accurate (Berdahl, 2024; Leechuy, 2024). Recent research by Elkhatat et al. (2023) showed that AI detectors are inconsistent and produced false positives. Even OpenAI, the organization responsible for ChatGPT, discontinued their own AI detection software because of its poor accuracy (Nelson, 2023). “Unlike plagiarism tools, generative AI tools do not give a clear cut result. The percentage likelihood of AI generated content is less accurate than plagiarism detection, more open to interpretation, and therefore requires more consideration on the educator’s part “ (Furze, 2024).
Ethan Mollick writes in Co-Intelligence: Living and Working with AI, “there is no way to detect whether or not a piece of text is AI-generated. A couple of rounds of prompting remove the ability of any detection system to identify AI writing. … Unless you are doing in-class assignments, there is no accurate way of detecting whether work is human-created”.
AI detection has high error rates and can lead to instructors to falsely accuse students of misconduct (Edwards, 2023; Fowler, 2023). Students at several universities say they have been falsely accused of misconduct, with these accusations delaying graduation and negatively impacting their wellbeing. Monitoring student use of generative AI places instructors in the difficult position of questioning the integrity of student work. As Elizabeth Steere writes, “an accusation based on a false positive can irrevocably damage trust between an instructor and student.” Furthermore, AI detectors privilege English as a first language students, students with access to paid accounts, and those who are more digitally literate (Furze, 2024).
At VIU, the use of AI detection tools is discouraged due to concerns about effectiveness, accuracy, bias, privacy, and potential infringement on intellectual property. At this time, no AI detection tools have undergone a VIU Privacy Impact Access (PIA).
What can you to do instead?
Join us for a 20 Minute Tea session to discuss adapting assessments.
In the 20 Minute Tea series we share a teaching strategy in just 20 minutes. It’s a quick way to learn new strategies and leave with inspiration for your classroom, with discussion available afterwards for those who want to linger a bit longer. In this session we will share a strategy to evaluate and consider changes to your assessment given the evolving capabilities of GenAI tools.
Attribution
Berdahl, L. (2024, December 11). Focusing on GenAI detection is a no-win approach for instructors. University Affairs. https://universityaffairs.ca/career-advice/focusing-on-genai-detection-is-a-no-win-approach-for-instructors/
Edwards, B. (2023, July 14). Why AI detectors think the US Constitution was written by AI. Ars Technica. https://arstechnica.com/information-technology/2023/07/why-ai-detectors-think-the-us-constitution-was-written-by-ai
Elkhatat, A. M., Elsaid, K., & Almeer, S. (2023). Evaluating the efficacy of AI content detection tools in differentiating between human and ai-generated text. International Journal for Educational Integrity, 19(1). https://doi.org/10.1007/s40979-023-00140-5
Fowler, G. A. (2023, April 14). We tested a new ChatGPT-detector for teachers. It flagged an innocent student. The Washington Post. https://www.washingtonpost.com/technology/2023/04/01/chatgpt-cheating-detection-turnitin
Furze, L (2024, May 1). AI detection in education is a dead end. https://leonfurze.com/2024/04/09/ai-detection-in-education-is-a-dead-end/
Leechuy, J. (2024, March 14). How reliable are AI detectors? claims vs. reality. The Blogsmith. https://www.theblogsmith.com/blog/how-reliable-are-ai-detectors/#:~:text=That%20said%2C%20AI%20detectors%20can,different%20results%20from%20one%20another
Mollick, E. (2024). Co-intelligence: Living and working with ai. Portfolio/Penguin. Co-Intelligence: Living and Working with AI
Nelson, J. (2023, July 24). OpenAI quietly shuts down its AI detection tool. Decrypt. https://decrypt.co/149826/openai-quietly-shutters-its-ai-detection-tool
Office of Teaching, Learning, and Technology – The University of Iowa. (2024, September 30). The case against AI detectors. https://teach.its.uiowa.edu/news/2024/09/case-against-ai-detectors#:~:text=Issues%20surrounding%20AI%20detectors%20also,original%20thinking%20and%20academic%20integrity.
Steere, E. (October 18, 2023). Faculty should know the tools students use to beat AI detection (opinion). Inside Higher Ed | Higher Education News, Events and Jobs. https://www.insidehighered.com/opinion/career-advice/teaching/2023/10/18/faculty-should-know-tools-students-use-beat-ai-detection