Artificial Intelligence
Texas A&M University Artificial Intelligence

Challenges with AI Detection Tools

Although AI detection tools are available, the university strongly advises against sole reliance on these tools due to the following limitations:

Inaccuracy
AI detection software often produces unreliable results, generating both false positives and false negatives. For instance, some tools have erroneously flagged the U.S. Constitution as AI-generated content. 1,2,4,6,7,12,18,19
Bias
Research has highlighted biases in some AI detection tools, particularly against non-native English speakers. This results in higher false positive rates for their work, unfairly penalizing these students.
Ease of Circumvention
Many detection methods can be easily bypassed through simple techniques such as paraphrasing or use of other tools to make subtle changes, rendering them less effective.
Rapid Evolution of AI
AI technologies evolve at a pace that detection tools struggle to match, diminishing their reliability in identifying newer, more sophisticated AI-generated content as well as increasing number of available large language models.

Emphasizing Human-Centered Learning

Rather than focusing on detection, Texas A&M University advocates for a human-centered approach to education. This strategy emphasizes critical thinking, academic integrity, and meaningful engagement with course material.

Recommendations for Instructors

The Center for Teaching Excellence, Texas A&M System, and other university partners offer a wide variety of opportunities to learn about AI including through peer engagement with other faculty and staff.

Learn More

Establish Clear Guidelines

Clearly articulate policies on the use of AI tools in your course syllabus and during class discussions. Provide examples of acceptable and unacceptable uses.

Encourage Transparency

Foster open conversations with students about AI technologies. Address their questions, explain your policies, and highlight the importance of ethical AI use.

Design Engaging Assignments

Create assignments that capture students’ interest, encourage deeper exploration of topics, and allow flexibility in project formats and themes.

Incorporate Inclusive Assessment

Use diverse assessment methods that cater to various learning styles and minimize biases against any student group.

Prioritize Understanding Over Detection

Focus on assessing students' comprehension of subject matter through discussions, probing questions, and reflective assignments.

Why Caution is Necessary

AI detection tools, while widely marketed, remain inconsistent in accuracy and reliability. They analyze linguistic patterns to determine if text is AI-generated, yet studies reveal frequent false positives, particularly affecting non-native English writers and those with unique writing styles. This raises concerns about fairness and equitable assessment as well as the potential impact on student outcomes stemming from unwarranted accusations of academic misconduct. Moreover, as AI-generated content grows more sophisticated, detection tools face challenges in keeping pace. 2,8,9,14,20

References
  1. Bowen, J. A., & Watson, C. E. (2024, April 30). Is it time to turn off Turnitin? Times Higher Education. https://www.timeshighereducation.com/campus/it-time-turn-turnitin
  2. Cal State LA. (n.d.). Teaching & Learning with AI: AI Detection Tools – Consideration and Limitations. https://www.calstatela.edu/cetl/teaching-learning-ai#3
  3. Chechitelli, A. (2023, May 23). AI writing detection update from Turnitin’s Chief Product Officer. https://www.turnitin.com/blog/ai-writing-detection-update-from-turnitins-chief-product-officer
  4. Coldewey, D. (2023, July 25). OpenAI scuttles AI-written text detector over ‘low rate of accuracy’. TechCrunch. https://techcrunch.com/2023/07/25/openai-scuttles-ai-written-text-detector-over-low-rate-of-accuracy
  5. Council of Writing Program Administrators. (2019). Defining and Avoiding Plagiarism: The WPA Statement on Best Practices. https://wpacouncil.org/aws/CWPA/pt/sd/news_article/272555/_PARENT/layout_details/false
  6. De Vynck, G. (2023, May 30). ChatGPT ‘hallucinates.’ Some researchers worry it isn’t fixable. Washington Post. https://www.washingtonpost.com/technology/2023/05/30/ai-chatbots-chatgpt-bard-trustworthy
  7. Edwards, B. (2023, July 14). Why AI detectors think the US Constitution was written by AI. Ars Technica. https://arstechnica.com/information-technology/2023/07/why-ai-detectors-think-the-us-constitution-was-written-by-ai/3
  8. Fowler, G. A. (2023, April 14). We tested a new ChatGPT-detector for teachers. It flagged an innocent student. Washington Post. https://www.washingtonpost.com/technology/2023/04/01/chatgpt-cheating-detection-turnitin
  9. Klee, M. (2023, June 6). She was falsely accused of cheating with AI — and she won’t be the last. Rolling Stone. https://www.rollingstone.com/culture/culture-features/student-accused-ai-cheating-turnitin-1234747351
  10. Kumar, R., & Mindzak, M. (2024). Who wrote this? Detecting artificial intelligence-generated text from human-written text. Canadian Perspectives on Academic Integrity, 7(1), 1-9. https://doi.org/10.55016/ojs/cpai.v7i1.77675
  11. McAdoo, T. (2023, April 7). How to cite ChatGPT. APA Style Blog. https://apastyle.apa.org/blog/how-to-cite-chatgpt
  12. MIT Sloan Teaching & Learning Technologies. (2023). AI detectors don't work. Here's what to do instead. MIT Sloan EdTech. https://mitsloanedtech.mit.edu/ai/teach/ai-detectors-don’t-work/
  13. MLA Style Center. (2023, March 17). How do I cite generative AI in MLA style?https://style.mla.org/citing-generative-ai/
  14. Myers, A. (2023, May 15). AI-detectors biased against non-native English writers. Stanford University Institute for Human-Centered Artificial Intelligence. https://hai.stanford.edu/news/ai-detectors-biased-against-non-native-english-writers
  15. Sands, L. (2023, April 6). ChatGPT falsely told voters their mayor was jailed for bribery. He may sue. Washington Post. https://www.washingtonpost.com/technology/2023/04/06/chatgpt-australia-mayor-lawsuit-lies
  16. Sweetland Center for Writing, University of Michigan. (n.d.). Beyond Plagiarism: Best Practices for the Responsible Use of Sources. https://sites.lsa.umich.edu/beyondplagiarism/
  17. University of Kansas, Center for Teaching Excellence. (n.d.). Adapting your course to artificial intelligence. https://cte.ku.edu/adapting-classes-artificial-intelligence-era
  18. University of Kansas Center for Teaching Excellence. (n.d.). Careful use of AI detectors. https://cte.ku.edu/careful-use-ai-detectors
  19. University of Maryland College of Computer, Mathematical, and Natural Sciences. (2023). Is AI-generated content actually detectable? https://cmns.umd.edu/news-events/news/ai-generated-content-actually-detectable
  20. Young, J. R. (2024, April 4). Can using a grammar checker set off AI-detection software? EdSurge. https://www.edsurge.com/news/2024-04-04-can-using-a-grammar-checker-set-off-ai-detection-software

Disclosure: ChatGPT and NotebookLM were used to help curate this content.