Identifying and Responding to Unpermitted GenAI Use

Whether you prohibit GenAI use entirely or place just a few limitations, you know that restricting GenAI use is easier said than done. Discussions of GenAI detection often feel like a cat-and-mouse game, with technologies meant to evade easy detection emerging as quickly as instructors identify strategies.

Students may misuse GenAI in several ways that result in academic integrity violations, including using GenAI in ways prohibited by course policies or assignment directions, failing to attribute their GenAI use adequately, or using GenAI in ways that lead to other academic integrity violations, such as misrepresenting sources or fabricating citations. 

As we’ve previously discussed, some solutions that might seem initially appealing, like AI detectors and hidden-text traps, pose practical and ethical hurdles that lead us to recommend against their use. So what should you do as an instructor if you’re reading a paper and suspect GenAI misuse?

Step One: Determine If Academic Misconduct Occurred

When you suspect a violation of your AI policy or other course policies, the question you need to answer is whether it is “more likely than not” that the student engaged in misconduct. This “preponderance of evidence” standard is the one outlined in University policy and used by entities like the Office of Community Standards. Misconduct does not need to be proven beyond a reasonable doubt, and no “AI detector” is considered entirely foolproof.

Look for Textual Indications of GenAI Use

Your own knowledge of the student and your course can provide among the most powerful signals that GenAI use—or other forms of source misuse such as patchwriting—may be impacting a piece of writing. You might notice features like these:

  • Course concepts explained in generic ways that don’t align with your course materials
  • Lack of apparent knowledge of or connection to class content or discussions
  • Inclusion of high-level material not covered in your course
  • Major differences in voice or level of sophistication from students’ prior work
  • Marked shifts in tone, voice, or level of sophistication throughout a single piece of writing

Additionally, while AI companies are working quickly to address hallucinations, they remain a natural and inevitable consequence of the text-generation process. Hallucinations in AI-generated academic writing can include:

  • Fabricated sources—sources that are made up or cited with fictitious or inaccurate information (e.g., a fake DOI or DOI of an unrelated article)
  • Fabricated quotes—quotes that do not exist, are not worded correctly, or are cited incorrectly based on the edition of the text used by the student
  • Fabricated data
  • Misuse of words, like “significant,” “parameter,” or “culture,” that have specific meaning in an academic field that is distinct from a more general use that appears most frequently in training data

Over time, readers have also noted stylistic features that occur at higher rates in AI-generated text than human-written text:

  • Broad, sweeping claims backed by shallow or trite analysis
  • Highly specific claims not backed up by evidence or sources
  • Tone that is very confident or even feels like a sales pitch
  • High frequency of structures that appear more frequently in GenAI output than in typical human text, such as “It’s not just X—it’s Y,” overuse of em-dashes, and use of emojis for emphasis (Wikipedia editors have constructed an extensive list of common GenAI text features)
  • Lingering user-directed language that frequently appears in GenAI responses (“You’re right!”, “Would you like me to …”)

Ultimately, reading GenAI output is the best way to hone your sense of these features. If you are open to using GenAI tools, consider inputting portions of your specific assignment a few different times to several GenAI models, with varied prompting strategies, to get a sense of the text a GenAI tool might generate in response to your assignment.

Keep in mind, however, that many textual features frequently found in GenAI writing also appear in human texts for a variety of reasons. For example, students grappling with a new concept may produce papers that make overbroad claims, summarize without analysis or synthesis, or misuse field-specific terminology. Unintentional misrepresentation of sources may result from reading comprehension challenges, confusion among sources, or borrowing citations from other sources without fact-checking. A student might be taking other courses with overlapping content or reading independently, leading to a different framing of a concept than you presented in class. 

Linguistic features like em-dashes or “not just X, but Y” phrasing appear frequently in GenAI text precisely because they are fixtures of academic writing on which GenAI models are trained. Students using a thesaurus tool too liberally or awkwardly attempting to adopt an unfamiliar register may end up with confusing language choices that read like the output of a text spinner. 

Parsing the distinction between human and AI text is doubly complex if you allow some kinds of GenAI use (e.g. for translation or grammar correction). These processes may introduce some of the features of GenAI text described above, but don’t necessarily indicate that misconduct occurred. 

Certainly, some GenAI blunders, such as fabricated citations or incriminating inclusions (e.g., “As a GenAI model, I cannot offer a personal example of the bystander effect”), are obvious signs of AI use. But in the vast majority of cases, textual features alone are not sufficient evidence of GenAI misuse to substantiate an allegation. Instead, they are signals to dig deeper. 

Review Search Results, Annotations, Notes and Drafts

Many instructors ask students to submit evidence of their writing process, including drafts or version history for documents, annotated copies of sources, and search logs that capture research practices. Reviewing these documents can confirm that students developed their own ideas and writing, or may reveal unexplained gaps that raise questions about GenAI use:

  • Sources appear without a traceable search process.
  • Information in the writing is not accounted for in sources the student cites.
  • Process begins with a developed draft rather than iterative notes and brainstorming.
  • Large, unexplained differences between drafts, or between in-class research/drafting and later work outside of class.
  • Large chunks copy/pasted into a draft without a clear source.
  • Unnatural typing patterns (linear writing with no revisions, deletions, and/or errors).
  • Last-minute addition of errors or targeted revision of textual features associated with GenAI use.
     

Like textual features themselves, evidence from notes and drafts can have plausible explanations other than GenAI (mis)use and therefore is not damning evidence of GenAI use in isolation. Rather, they are another feature of the text you can discuss with students and use to make the case that, more likely than not, a student has used GenAI inappropriately.

Step Two: Discuss Your Observations With the Student Writer

The most informative way to investigate cases of suspected GenAI misuse is to talk with students directly. You can often learn a great deal just by asking students to share more about how they completed the assignment. Consider posing questions like these and documenting the student’s responses:

  • What was your writing/research process for this assignment? Can you share and walk me through your notes, drafts, etc.?  
  • What was your favorite part of this paper to write? What parts of the assignment were most challenging?
  • What made you decide to write about this topic/argue this position? How did you come up with these ideas?
  • How did your thinking shift as you read sources? As you wrote your paper?
  • Which aspect of your argument or piece of evidence do you think is most compelling?
  • Can you summarize your argument for me? How would you explain it to someone unfamiliar with the topic?
  • You brought up [a particular theory or term we didn’t discuss in class]. Can you explain it to me? How did you learn about it? Why did you opt to use it in this paper?
  • Why did you decide to structure the paper the way you did? What alternatives did you consider?
  • How did you find your sources? What sources did you consider but decide not to include? Do you have any notes or annotations on the sources you read?
  • What was the most useful source you found? Tell me about it.
  • This sounds very different from some of your other work to me. What do you think? Why might that be?
  • Did you use any GenAI tools, like ChatGPT or Grammarly? How did you use them?
  • Did you use any translation tools like Google Translate? How did you use them?

Many students, particularly those who are multilingual and/or neurodiverse, are more able to articulate their ideas in writing than in speaking. You shouldn’t expect that students will be able to perfectly recreate their arguments or ideas verbally, especially without referencing their text. However, students with almost no knowledge of a portion of their text, or who can’t describe their writing process, likely over-relied on some external source such as a GenAI tool.

Of course, opening a conversation with students about their writing isn’t just helpful in illuminating GenAI misuse. It also allows for broader discussions of students’ writing processes, surfaces possible misconceptions (e.g., “I just used Grammarly, not AI”), and provides an opportunity to clarify your expectations.

Involve TAs and Graders

In some courses, TAs and graders read most of the writing students produce and have more opportunities to interact with students in small group settings. This positions them well to notice changes in students’ writing that may indicate overreliance on or misuse of GenAI. While instructors of record need to be involved in decisions to assign penalties for academic misconduct, TAs can absolutely be responsible for having conversations with students about suspected GenAI use, making this strategy much more feasible in large courses. The instructor of record should give TAs some guidance on navigating the conversation and remind them to carefully document the student’s responses.

Avoid Sneaking and Traps

While technological solutions purporting to identify AI text or hidden-text traps to catch unwitting AI users may seem initially appealing, we recommend against them because of the practical and ethical hurdles they introduce. Learn more in “The Prohibition/Detection Arms Race: Escalation is Unwise, but So Is Surrender.”

Step Three: Apply Appropriate Penalties for Unpermitted GenAI Use

Determining an Appropriate Penalty

If you determine that it is more likely than not that academic misconduct occurred, your next step is to determine an appropriate penalty that aligns with the course’s GenAI policy. Consequences can range from requiring the student to redo the assignment, to reduced or no credit for the assignment, to a lowered or failing grade in the course. The Office of Community Standards provides this guidance for instructors on selecting an appropriate academic penalty. Generally, we recommend consequences be scaled to the severity of the violation and reflect your assessment of whether the plagiarism was intentional or related to a misunderstanding of policy. 

Follow Your Campus’s Reporting Procedures

When you assign a penalty, you must report the suspected misconduct and the penalty you chose to the Office of Community Standards (on the Twin Cities campus) or other campus entity dictated by policy (for other UMN campuses). Reporting suspected academic dishonesty to the Office of Community Standards provides students with due process, including formal notification of penalties and the opportunity to schedule a hearing to contest the accusations. For this reason, University policy dictates that any instance of academic dishonesty where a penalty is applied must be reported. You can learn more about the Student Conduct Process. Note that it is possible to file a joint report for suspected misconduct by multiple students, so long as you provide adequate evidence that each individual included in the report engaged in misconduct.

Notably, requiring students to address GenAI use in the context of a required revision completed by all students in the course is not an academic penalty, and therefore does not need to be reported. (Yet another benefit of revision!) However, if the student loses points on the draft for GenAI use, or does not meet your expectations for revision and thus faces a consequence later, an OCS report is required.

Some instructors may mistakenly believe they are doing a student a favor by not “reporting to the University” and merely assigning a grade penalty. But by circumventing University processes, instructors deny students the opportunity to dispute the penalty AND open themselves up to dramatically escalating conflict. For a first report, students can work with academic integrity staff to take steps to clear their record; broader university penalties are reserved for repeat offenses.

TAs can also be involved in reporting academic misconduct. With the instructor of record's approval, TAs can take the lead on filing reports in cases of suspected academic misconduct. In fact, this is preferable in cases where TAs have the most first-hand knowledge of the case. Involving TAs in the reporting process reduces the instructor's burden, enables TAs to share evidence they encountered, and helps prepare them to teach courses independently in the future.

Continually Adapting Assignments

High rates of suspected GenAI misuse are undoubtedly frustrating, but they are also useful data for you about the ways students are responding to your policies and assignments. As you move through the semester, take notes about when and where you observe students over-relying on GenAI tools. These are excellent places to introduce targeted strategies for deterring AI misuse.